Why traditional Machine Learning algorithms fail to produce accurate predictions compared to Deep Learning algorithms?

[PRABHAT_VERMA]

As we increase the amount of data the performance of the ML model decreases why do DL models take so massive amounts of data?

[Christian_Simonis]

Hi there,

one reason is that DNNs w/ advanced architectures (like transformers but also architecture w/ convolutional | pooling layers) are designed to perform well on very large datasets and also process highly unstructured data like pictures or videos in a scalable way: basically the more data, the merrier!

Compared to classic ML models, DNNs possess less structure and can learn more complex and abstract relationships given sufficient amount of data.

This thread could be interesting to you since it is about a similar topic:

Isn't anything but neural networks obsolete?

For guidance of the process along developing AI models the CRISP-DM model is worth to mention, see also this example as some practical guidance. Feel free to take a closer look at it since it is some standard for data mining since several years even though it might not be perfect. With respect to NN as possible “gold standard” in addition to the previous discussion above, I recommend to take a look at the No-free-lunch-theorem: In statistics, the term has been used to describe the tradeoffs o…

Best regards
Christian

[PRABHAT_VERMA]

thanks, for the guidance

[Christian_Simonis]

Sure it’s my pleasure. If you have any further questions, pls. do not hesitate to contact me.

Happy learning!

Best
Christian