Hello, I’ve been having a rough time implementing forward_propagation_with_dropout, and the case is, my implementation doesn’t pass Test 2. I made sure that i used correct random function (np.random.rand). Please help.
4 Likes
Did you check to make sure you did that at both of the layers with dropout? And remembered to divide the output by keep_prob? Also note that they tell us to make the mask such that it does not treat each sample the same, right? In other words you don’t generate a column of randoms and then duplicate it: you create the random matrix directly in the final shape that you actually want.
2 Likes
Thank for your quick reply. I worked my problem out. It was just me dropping neurons with probability higher than keep_prob (D1 > keep_prob), however in lecture we used to drop neurons with probability lower than keep_prob (D1 < keep_prob).
5 Likes
That’s great to hear that you found the solution under your own power! Thanks for confirming
1 Like
hi, @anteii I got the same problem as you have encountered. Can you tell me how did you rectify the error?
1 Like
Hello, @madhav2381. After you sampled random vector D1, you should convert entries in it to 0 or 1. As i wrote earlier, my mistake was converting it with formula D1 > keep_prob. I solved this by simply using correct formula wich is D1 < keep_prob. And just in case, check whether you are using right numpy function for sampling random vector or not.
3 Likes
@anteii Thanks I used the wrong NumPy function for creating D1, D2 masks.
5 Likes
Yes, that’s the other popular mistake: using “randn” instead of “rand”.
11 Likes
I´m getting the same A3 as you but I´m having (D1 < keep_prob). Did you have an additional error?
{moderator edit - solution code removed}
This is my code…cannot find the problem. Any hints?
1 Like
Ok solved it randn as mentioned by @paulinpaloalto was it haha
1 Like
I followed the advice listed above, and I still get an error. I’m using np.random.rand not ‘randn’ , comparing D1 < keep_prob and dividing A1 by keep_prob. Maybe you can advise anything, @paulinpaloalto ?
Here’s the output
1 Like
That all sounds correct to me. The only other thing I can think of is that you also need to apply dropout at layer 2. If you’ve done that in a similar way, then we’ll probably need to graduate to the “in case of emergency, break glass” method. Check your DMs.
1 Like
One other thing I can think of is that just typing new code into a function cell and then calling the function again does nothing: it just runs the old code again. You actually need to click “Shift-Enter” to execute the function cell itself in order to get the new code added to the runtime image. So maybe your code is actually correct, but that’s not what you are really running.
The other way to make sure things are consistent is to do “Kernel → Restart and Clear Output” followed by “Cell → Run All”.
1 Like
The other error is to set the dropout “mask” to be a column vector, so that all samples are treated the same in each minibatch. The matrix multiply uses “broadcasting” in that case. But it turns out that is not what they intend here. The random mask should be the size of the activation matrix, which means that each sample is handled differently.
It is a legitimate question which approach makes more sense. There have been some interesting discussions of that in the past with some actual experimental results which seem to show that (at least in the particular case here) the two methods are pretty much equivalent.
1 Like
Hi all,
I believe I have followed the advice above and I am also getting an error.
I have done: making sure I use rand not randn
Making sure that the Dx < keep_prob
Diving Ax by keep_prob
The code looks the same to me in both layers.
The code and output is below, is there anything else I am missing?
{moderator edit - solution code removed}
1 Like
Hmmm, I would say that your code looks correct, but with essentially the same code, I get different results on that test cell:
A3 = [[0.36974721 0.00305176 0.04565099 0.49683389 0.36974721]]
All tests passed.
One theory would be that you changed the code, but did not actually click “Shift - Enter” on the function cell itself. If you just call it again from the test cell without actually executing the function cell again, it just runs the old code. Or try “Kernel → Restart and Clear Output” followed by “Cell → Run All” and see if you still fail the test.
1 Like
Hey paulinpaloalto,
Thank you so much for you swift reply. I have tried restarting the Kernel to no avail.
I have tried to delete my jupyter notebook and get the “updated version” and rewrote my code, I have gotten a different set of results which is still incorrect of:
A3 = [[0.36974721 0.01459639 0.09212225 0.49683389 0.36974721]] 1 Tests passed 1 Tests failed
Which looks closer to your numbers, I don’t believe I have changed the “np.random.seed(1)” but could that be causing the issue?
1 Like
If you changed the seed, I would expect you to get a different value, but it’s also set to 1 in the standard template code and in my code. So there must be something else going on. It’s probably time for the “in case of emergency, break glass” method. Please check your DMs.
1 Like
Ok, we worked this out: it turns out that there is one place on the LHS of an assignment statement where the variable name is misspelled. It was intended to be A1, but it was actually Al (A followed by lower case letter ell). It’s really really hard to see with the default font that the notebooks use.
1 Like