Hi.
I am trying to implement my own neural network model using numpy and a csv dataset and pandas
Now im running to these problems
When i train my model without changing anything somtimes it works well and accuracy is ok but sometimes its not it changes and switches between these two sates and sumtimes prediction gets nan; and i tried various hyperparameters but the results still the same
Sometimes cost during trainig decreases ok and sometimes with those same hyperparameters statys the same for each batch and its almost constant
:

I haven’t had a chance yet to look at your model, but up front, what do you mean by ‘sometimes’ ? Are you changing your training set/hyperparameters each time (which if you seek consistency at least, obviously you shouldn’t) ?

Thank you
I tried your aprpach and chose better hyperparameters and know its a bit more stable but now it generaly overfits so i need to implement regularization to see if it gets better

Look for the numerical stability of all the formulas in your implementation, specially sigmoid and log related formulas. I had this problem of getting nan outputs due to this.