Hi,
The assignment for Week 3 is using TensorFlow framework. I want python hard-coded implementations using Numpy (similar as previous assignments) for the concepts learned in Week 3 such as: hyper-parameter scaling and sampling, batch norm and running average. The purpose is to get more understanding around the concepts and compare my implementation with it.
Thanks in advance
It’s admirable that you are doing your own implementations of the various concepts that Professor Ng teaches us in Course 2. The W3 assignment introduces us to using TF to implement things that we’ve previously built ourselves directly in numpy, e.g. linear functions, forward propagation, computing the loss and minibatch gradient descent. So I think everything in the assignment is actually something we have already constructed in numpy.
The assignment does not use the other techniques you mention like batch norm and hyperparameter selection and moving averages. For batch norm and moving averages, Professor Ng has written out the math formulas for us, so it shouldn’t be that difficult to translate those into numpy. There are no “official” implementations in the course for those, but there are a couple ways to check your implementations:
You could write your versions in numpy and then write a second version using TF and feed the same data to both and compare the answers.
TF is actually open source, so you could look at their implementations for batch norm, for example.
We aren’t supposed to share the code for the solutions to the assignments, but if you’re building something that isn’t one of the assignments I guess you could post the code here and we could discuss it.
If I were in your position, I’d start with the first method I suggested above. If the results match in a number of test cases, then you’re done. If you encounter any problems getting your “hand-coded” version to match the TF results, then we could discuss here.