I was wondering in case of Inception Network, we are using multiple types of layers with different hyperparameters because we don’t know which one is the best so we use them all, Is it possible for the network to learn same features on different layers used.
For example if we are using 3x3 and 5x5 filters what prevent both of them from learning similar features as they have the same input so one of them is partially not adding any new information?
Thanks in advance.
In an Inception Network, filters of different sizes (e.g., 3x3 and 5x5) are designed to capture features at different scales, such as fine details versus broader patterns. While it’s possible for them to learn similar features, the optimization process typically drives them to focus on complementary information to reduce redundancy. If overlap occurs, the network adjusts weights so that each filter contributes uniquely to improving performance.
Hope it helps! feel free to ask if you need further assitance.
1 Like
Hi @Alireza_Saei, thanks for the reply
Can you please clarify how optimization can drive them to focus on complementry information?
You’re welcome, sure!
Optimization in NNs works by minimizing the overall loss function, which is influenced by all the layers and filters. If two filters (for example 33 and 55) were learning redundant features, their effects to improving the loss would be similar and their overall utility reduces.
The gradients during backpropagation naturally adjust the weights so that each filter learns distinct features that maximize the network’s ability to reduce loss. This makes sure that the filters complement each other by capturing different aspects of the input that leads the model to a better performance.
Hope it helps!
1 Like
Im starting to get it, let me make sure Im getting it right, so we can think of it as we are trying to minimize the loss per feature, and if the two filters are learning the same feature so minimizing loss in one is enough and when optimizing the second filter the loss doesnt change, so it tries to learn a different useful feature in that filter instead, Am I right?
You’re getting the idea, and you’re almost there!
You can think of it this way: when two filters start learning the same feature, their contributions to minimizing the loss overlap, so the second filter’s optimization adds less value. During backpropagation, the gradients will adjust the weights of both filters, nudging them toward learning distinct features that better reduce the loss overall.
NOTE: This doesn’t happen explicitly per filter but is a natural outcome of optimizing the network’s total loss. Over time, the filters evolve to capture complementary information.
Hope it helps! Let me know if you have more questions.
2 Likes
@Alireza_Saei
Thank you so much for this amazing clarification.
1 Like
You’re absolutely welcome! happy to help 
1 Like