So I understand the concept of neural networks in the extent presented in MLS Course W1 & W2 videos I understand the usefulness of the convolutional layer because you pick which features that get evaluated in each unit engineering a new feature that might improve predictions in the output layer.
Such as presented in 1st example of the course: predicting if the product will be a top seller we take the input and engineer affordability, percived quality and awareness which allows us to make a better prediction in the output layer.
However, I’m not sure what is happening in the background of a Dense layer and its point. The presenter states that each unit in a Dense layer is a simple logistic unit within itself. From the example above or from the example of predicting an image we get N number of inputs and we declare a Dense layer with 25 units and a second Dense layer of 15 units.
What are we actually doing here? How does adding extra units and layers improve our model ? Wouldn’t a single layer with 1 unit be the same thing? Like the weights from gradient descent for each unit in the layer would be about the same. We are not engineering a new feature from this so what’s actually happening here ?