Creating polynomial features

I am trying to define my function that creates polynomial features given X and the degree of polynomial desired. To start with, this is the general equation for linear regression with two (2) features.
Screenshot (196)

Raising the features to a power of 2, I should have this equation:
Screenshot (197)

Usually, further expanding the equation should give me the result below.
Screenshot (202)

If I am to write my function, should I make it work as in the figure above? or I should get rid of the co-efficient of x_1*x_2 as in the figure below?
Screenshot (203)

1 Like

Hello @Basit_Kareem,

It usually goes like this:

degree 1: y = w_1x_1 + w_2x_2 + b

degree 2: y = w_1x_1 + w_2x_2 + w_3x_1^2 + w_4x_2^2 + w_5x_1x_2 + b

Your degree 1 expression y=\alpha(x_1 + x_2) + \beta forces my w_1 to be equal to w_2 and consequently we are losing one degree of freedom (from two trainable weights w_1 w_2 to only one trainable weight \alpha.

What do you think?

Raymond

Well, I intended to indicate that α is a vector with the same length as the number of features to be used for fitting the model.

Now, you know where you have this term w_5*(x_1) (x_2). Since the feature x_1x_2 would have been created before introducing weight w_5, while creating x_1*x_2, was it multiplied by the coefficient 2 as I would have done had I been performing a manual mathematical operation

1 Like

I see. Let’s keep all notation correct. If your \alpha is a vector and \vec{\alpha} = \begin{bmatrix} w_1 & w_2 \end{bmatrix}, then your y = \alpha (x_1 + x_2) + \beta should have been written as y = \vec{\alpha} \cdot \vec{x} + \beta where \vec{x}=\begin{bmatrix} x_1\\ x_2 \end{bmatrix}

Then I guess you won’t be able to ask the same question under the correct notation? What do you think?

On the other hand, I think a good point in your first post is how you can derive the second-order terms by taking the square of (x_1 + x_2) if that’s your goal. In that case, we don’t need any coefficients because the trainable weights will do their jobs. The only two things that I am not sure about your first post is:

  1. Generally speaking, a second order polynomial includes also the first order terms
  2. A “stretchable” \alpha sounds like an unusal notation

Cheers,
Raymond

You are right. I even tried inspecting your code to see if I could do the same. I guess I would have to spend hours trying to learn how MathJax works to do that.

Since you already understood my question, you clarified my confusion.

Thanks.

Yes, I was planning on concatenating the results of all orders together later. I just need to figure out the calculation step first.

I would try to see if I can get the arrow indication (I don’t know if there is a name for it) on it then.

OK. This is my feature vector for a first-order polynomial: \vec{x} = \begin{bmatrix} x_1 \\ x_2 \\ \end{bmatrix}

Second-order polynomial: \vec{x} = \begin{bmatrix} x_1 \\ x_2 \\ x_1^2 \\ x_2^2 \\ x_1x_2 \end{bmatrix}

The latex code for them are:

$\vec{x} = \begin{bmatrix}
x_1 \\
x_2 \\
\end{bmatrix}$

and

$\vec{x} = \begin{bmatrix}
x_1 \\
x_2 \\
x_1^2 \\
x_2^2 \\
x_1x_2
\end{bmatrix}$

Raymond

Oh! you mean the alpha α notation seems unusual? I would stick with using w then. Thanks.

PS: I found the arrow superscript under accents in MSWord.

\alpha is fine. But

is unusual. The correct way is to write down both \alpha and x as vectors and use a dot product between them, as I did in my earlier reply.

Wow. Thanks for this. I will look more into TeX formatting then.

You are welcome, @Basit_Kareem!