Hi Noah @noahred16,

Having one local minimum only means there is only one set of weight values that corresponds to a minimum cost. However, among the weights values they can be different.

The reason why we will end up with different weights is exactly because we initialize them differently at the beginning, please refer to this post for a simple explanation and within which there is another link to some maths and an example for explaining why weights get diversified.

Cheers,

Raymond