Description
🐛 Bug
I manually replaced batch_norm with group_norm in inception_v3, but there is still a problem: AttributeError: 'Parameter' object has no attribute 'grad_sample'.
And this is the trace:
Traceback (most recent call last):
File "e:\jupyter\LearnTorch\deep learning with privacy\cifar10_inception3_dp.py", line 139, in <module>
main()
File "e:\jupyter\LearnTorch\deep learning with privacy\cifar10_inception3_dp.py", line 126, in main
train(model, DEVICE, test_loader, optimizer, epoch)
File "e:\jupyter\LearnTorch\deep learning with privacy\cifar10_inception3_dp.py", line 65, in train
optimizer.step()
File "F:\Anaconda3\envs\pytorch_gpu\lib\site-packages\opacus\privacy_engine.py", line 197, in dp_step
self.privacy_engine.step()
File "F:\Anaconda3\envs\pytorch_gpu\lib\site-packages\opacus\privacy_engine.py", line 280, in step
self.clipper.clip_and_accumulate()
File "F:\Anaconda3\envs\pytorch_gpu\lib\site-packages\opacus\per_sample_gradient_clip.py", line 182, in clip_and_accumulate
flat=not self.norm_clipper.is_per_layer,
File "F:\Anaconda3\envs\pytorch_gpu\lib\site-packages\opacus\utils\tensor_utils.py", line 38, in calc_sample_norms
norms = [param.view(len(param), -1).norm(2, dim=-1) for name, param in named_params]
File "F:\Anaconda3\envs\pytorch_gpu\lib\site-packages\opacus\utils\tensor_utils.py", line 38, in <listcomp>
norms = [param.view(len(param), -1).norm(2, dim=-1) for name, param in named_params]
File "F:\Anaconda3\envs\pytorch_gpu\lib\site-packages\opacus\per_sample_gradient_clip.py", line 263, in <genexpr>
if p.requires_grad
AttributeError: 'Parameter' object has no attribute 'grad_sample'
github repository
Sorry, I haven't used coLab.
here is my code: https://github.com/mr-lz/inception_v3_dp/tree/master , you can run it with python cifar10_inception3_dp.py
Environment
PyTorch version: 1.6.0
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A
OS: Microsoft Windows 10 家庭版
GCC version: (x86_64-posix-seh-rev0, Built by MinGW-W64 project) 8.1.0
Clang version: Could not collect
CMake version: Could not collect
Python version: 3.6 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: 10.2.89
GPU models and configuration: GPU 0: GeForce GTX 960M
Nvidia driver version: 451.67
cuDNN version: F:\Program Files\NVDIA GPU Computing Toolkit\CUDA\v10.2\bin\cudnn_ops_train64_8.dll
HIP runtime version: N/A
MIOpen runtime version: N/A
Versions of relevant libraries:
[pip3] numpy==1.19.1
[pip3] torch==1.6.0
[pip3] torchcsprng==0.1.2
[pip3] torchtext==0.7.0
[pip3] torchvision==0.7.0
[conda] blas 1.0 mkl https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
[conda] captum 0.3.0 0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch
[conda] cpuonly 1.0 0 pytorch
[conda] cudatoolkit 10.2.89 h74a9793_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
[conda] mkl 2020.2 256 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
[conda] mkl-service 2.3.0 py36hb782905_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
[conda] mkl_fft 1.1.0 py36h45dec08_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
[conda] mkl_random 1.1.1 py36h47e9c7a_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
[conda] numpy 1.19.1 py36h5510c5b_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
[conda] numpy-base 1.19.1 py36ha3acd2a_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
[conda] pytorch 1.6.0 py3.6_cuda102_cudnn7_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch
[conda] torchcsprng 0.1.2 py36_cpu [cpuonly] pytorch
[conda] torchtext 0.7.0 py36 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch
[conda] torchvision 0.7.0 py36_cu102 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch