Hi Joe @ybakos,
Your parabola has to be a function of w in order for you to calculate the solution of w following that quadratic way. If you even attempt to write a neural network’s cost function in terms of w, you will see that in most cases it is not a quadratic equation of w.
However, there is a special case that it is a quadratic of w, which is a simple linear regression (e.g. y=wx) together with mean squared error cost. In this case you can calculate w analytically with a closed form solution called “the normal equation” (please google this name for how it looks like).
Next time, when you have a neural network that is not a simple linear regression, try to write out the cost function as a function of w to see if it is a parabola in w.
Raymond