If a computer scientist created the basic concept of machine learning more than 60 years ago, then why did it take so long for AI to become a major technological advancement? This supports the theory that says humans have always knew how to create artificial intelligence (AI), but we haven’t had the resources to do so until recently (High processing power & big database to train the machines)
Computing resources is the key!
Yes, Neural Networks have been around for quite a long time, but as Gent says we didn’t have the compute resources to train a really large and power Neural Network until after the year 2000. Since then the power of NNs has exploded. Once that started to happen with events like AlexNet winning the ImageNet challenge in 2012, a lot more creative thought got put into designing networks and some new ideas like Transformers and Attention Models were developed in the last 10 years that also contributed to the current amazing increase in capabilities. So it’s the combination of Moore’s Law plus more human intelligence being applied, once it became clear what the potential really is. The other contributing factor of course is the explosion of the Internet since the early 1990s bringing with it vast quantities of data that can be used as training data.
BTW Prof Ng discusses all this in the first week of lectures in DLS Course 1 and probably in other places as well. He was there and a major contributor to everything that’s happened in the last 20 years, so he can tell the stories from a firsthand perspective. He also includes optional interviews with a number of the other major contributors along with the lectures in the DLS courses. So you can get a lot more sense of how this all played out by listening to those optional interviews and watching his lectures in Week 1. Look for the topics with the title “Heroes of Deep Learning”.
joining the interesting discussion This is a great article from Bill Gates about the age of AI, touching many of the ideas you shared.