Selecting Feature

Course Q&A Machine Learning Specialization Advanced Learning Algorithms
1

How to select the feature in the hidden layer
(i.e) how does the algorithm predict by it’s own and how does it select the function

2

Hi @Sivaananth_NM great question

Selecting features in the hidden layers of a neural network, is largely determined by the training process. The features are not explicitly selected by a programmer or user; rather, they are learned from the data. Here’s how it works:

  1. Initialization: Initially, the weights of the neural network are set randomly.

  2. Forward Propagation: During training, input data is fed into the network, and it passes through the hidden layers. Each layer applies a set of learned features (weights and biases) to the input data. These operations typically involve linear transformations (like matrix multiplication) followed by non-linear activations (like ReLU, sigmoid, etc.).

  3. Loss Calculation: The output of the network is compared to the desired output, and the difference is calculated using a loss function.

  4. Backpropagation: This is where the learning happens. The gradient of the loss function is calculated with respect to each weight in the network. This tells us how much a small change in each weight would decrease or increase the loss.

  5. Weight Update: The weights are then adjusted slightly in the direction that reduces the loss, using an optimization algorithm like gradient descent.

  6. Iteration: This process is repeated for many iterations, over many examples from the training data.

During this process, the network ‘learns’ which features are important for making accurate predictions. The features in the hidden layers are essentially complex patterns that the network has found to be useful for minimizing the loss function on the training data. These features become increasingly abstract and high-level as you move deeper into the network.

The selection of these features is an emergent property of the training process—it’s not something that is directly controlled, but rather something that develops as a result of the network’s structure and the data it is trained on.

I hope this helps