Derivative of Relu in output layer

Hello sir @paulinpaloalto I hope you are doing well.
Below is my first NN model from scratch (regression problem). It did a poor job. In training, some points fit but some are extremely poor. It seems both high bias, and high variance. How can I improve it? This graph shows training (not dev/test).

1 Like