I initialize the random weights with torch.randn. The dimensions is (1,channale,1,1) as given in the hint.
Next I generate the noise shape which has the dimensions of (1,channels, w,h)
Lastly i multiply the weights with the noise in the forward.
below is the snippet will remove it later.
I get the following error in unit test
assert torch.abs((inject_noise(fake_images) - fake_images).std(1)).mean() < 1e-4
No I am loosing my mind over this.
The shape you are using for the noise in the forward function is incorrect. It needs to be the mirror image of the shape you used in the __init__ section.
One other note is that in the forward() section, I specified the device of the random values to match that of the input image.
Thank you for pointing out the device. I updated it by mistake . Changed it back to take the image.device.
I did try with noise shape
noise_shape = (1, image.shape[1],1, 1)
However with both I get the same result.
I have the following shapes for noise and the weight
torch.Size([1, 3000, 1, 1]) noise shape
torch.Size([1, 3000, 1, 1]) weigth shape
torch.Size([20, 3000, 10, 10]) image
I think you are misinterpreting my comment about the noise shape. For the shape in the “init” section, all dimensions are 1 except the channel dimension. For the noise shape in forward, the channel dimension is 1 and all the others agree with the image dimensions. Which dimension is the channel dimension?
Ok. But then it results in two matrices which are not compatible in size.
torch.Size([20, 1, 10, 10]) noise shape
torch.Size([1, 3000, 1, 1]) weigth shape
torch.Size([20, 3000, 10, 10]) image
If I use the image shape noise_shape = (image.shape[0], 1,image.shape[2],image.shape[3])
I get the following error RuntimeError: The size of tensor a (3000) must match the size of tensor b (10) at non-singleton dimension 1
Actually I worked it out. Thanks for the help. figured out the correct dimension shape. It works now. Thank you for the guidance.
It’s good to hear that you found the solution! Thanks for confirming.