Need help to create a Neural Networks class from scratch

Course Q&A Deep Learning Specialization Neural Networks and Deep Learning
1

I built a custom class to implement neural networks from scratch, and while the models show promise (with loss decreasing on each iteration), I’ve encountered a few challenges:

  1. Regression: Despite initializing weights using Xavier’s method, the network struggles with exploding/vanishing weights during training. Check test_regression.ipynb file

  2. Classification: While the weights remain stable, the network frequently predicts only two classes (0 or 3) in a 4-class classification problem, leading to a significant class imbalance in the output. Check test_classification.ipynb file

2

I glanced at your code very briefly.

It appears to me that your optimize() function is handling the layers in the wrong order. Backpropagation should proceed from the output layer to the input. Your code appears to go from the input to the output.

That’s all I will be able to reply on this topic.

3

Thanks for your reply @TMosh.
but I don’t think the order of updation matters here as I have already computed and stored all the grads as self.grads when I called backpropagation().
optimization() is just used to update the weights using these calculated gradients.