Implementation of Neural Networks from Scratch

Should we implement Neural Networks/Machine Learning from scratch or use pre built algorithms which require less code?

Thank You

2 Likes

It is a great question. Please see this earlier thread for a discussion on that.

There’s a rather wonderful framework called ‘jax’ now, which lies somewhere in between Numpy and frameworks like Tensorflow. It replicates exactly much of the Numpy syntax and functions, and adds superpowers : automatic differentiation, jacobian-vector products, TF-like graph compilation for faster execution, execution on TPUs and GPUs and much more

‘JAX is NumPy on the CPU, GPU, and TPU, with great automatic differentiation for high-performance machine learning research’

https://jax.readthedocs.io/en/latest/notebooks/quickstart.html

So it gives a way of implementing NNs that is more flexible than Tensorflow, and I think it’s popular with at least some NN researchers - e.g. see the material on neural net training dynamics at CSC2541 Winter 2021