Incorrect Matrices X & W in section "Evaluate Cost Function with regularization" C3_W2_Collaborative_RecSys_Assignment

In cell “Evaluate cost function with regularization”, Matrix X is changed from shape (5,3) to shape (4778, 100) and this is the reason for incorrect Cost Function with Regularization. The following is the output of this cell showing Matrix X with the two different shapes. Matrix W is also being changed from shape (4, 3) to shape (444, 100). This output is displayed before calculating the Cost Function. You can see the differences of Matrix X and Matrix W that the project displays and what I displayed. I have my own code that is equivalent to the suggested code and the output from both codes are exactly the same (incorrect).

Evaluate cost function with regularization

J = cofi_cost_func(X_r, W_r, b_r, Y_r, R_r, 1.5);
print(‘\nX: {} \n{}’.format(np.shape(X), X))
print(‘\nW: {} \n{}’.format(np.shape(W), W))

here, I calculate the Cost Function with Regularization<<<
Using MY Code and the suggested code<<<
the Cost Function is incorrect due to Matrix X and Matrix W being incorrect<<<
print(f"Cost (with regularization): {J:0.2f}")

R: (5, 4) X: (5, 3) W: (4, 3) b: (1, 4) Y: (5, 4)

R:
[[0. 0. 0. 0.]
[0. 0. 0. 0.]
[0. 0. 0. 0.]
[0. 0. 0. 0.]
[1. 0. 0. 0.]]

X:
[[-0.03328682 1.1667464 -0.5064895 ]
[ 0.40405777 2.2664917 -0.7428241 ]
[ 0.02347802 0.42810223 -0.4263811 ]
[ 0.5112806 1.5187973 1.602747 ]
[-2.0565143 0.45851684 0.6193708 ]]

W:
[[ 0.47935075 0.4766997 0.48794055]
[ 0.03604829 -0.38190877 0.2958575 ]
[ 0.05561632 -0.05899394 -0.2452817 ]
[ 0.4550811 0.15987432 -0.4527412 ]]

b:
[[0.23609531 0.35653394 0.12819964 0.3155024 ]]

y:
[[0. 0. 0. 0.]
[0. 0. 0. 0.]
[0. 0. 0. 0.]
[0. 0. 0. 0.]
[5. 0. 0. 0.]]

X: (4778, 100)
<tf.Variable ‘X:0’ shape=(4778, 100) dtype=float64, numpy=
array([[ 0.07582331, -0.03427632, -0.01609749, …, 0.06658309,
0.01564235, -0.02251503],
[-0.16276721, 0.01704462, -0.07876187, …, -0.02995775,
0.04415855, -0.06698893],
[-0.04948705, 0.02500312, -0.00641614, …, 0.07086587,
0.07787514, -0.05801108],
…,
[-0.00107412, -0.00191295, 0.00067007, …, -0.00181859,
0.00155094, -0.00386389],
[-0.00111467, -0.00183998, 0.00066644, …, -0.00174597,
0.00156552, -0.00400675],
[-0.0011723 , -0.00237138, 0.00059307, …, -0.00177517,
0.00166738, -0.0045013 ]])>

W: (444, 100)
<tf.Variable ‘W:0’ shape=(444, 100) dtype=float64, numpy=
array([[-0.16305114, -0.04327508, 0.1389465 , …, 0.03278738,
0.06512779, -0.15487556],
[ 0.09429301, -0.00534318, 0.0066646 , …, -0.03065442,
-0.01911861, 0.03692493],
[ 0.11140035, -0.02719898, 0.29152023, …, -0.11738583,
0.10447387, 0.20986626],
…,
[ 0.2001063 , -0.05523927, 0.05234348, …, -0.15164598,
0.04851693, -0.08608987],
[-0.36511943, 0.14320772, -0.59280517, …, -0.03249258,
-0.57793591, -0.13744584],
[ 0.35865834, -0.06980807, 0.13946261, …, -0.16272426,
-0.36644676, -0.00784709]])>
Cost (with regularization): 3657.99
Expected Output:

28.09

Thanks for your help in advance.

Hello @Joaquin_Ramirez,

Let’s focus on these 3 lines that you shared

J = cofi_cost_func(X_r, W_r, b_r, Y_r, R_r, 1.5);
print('\nX: {}   \n{}'.format(np.shape(X), X))
print('\nW: {}   \n{}'.format(np.shape(W), W))

You are printing the shapes of X and W, but X_r and W_r are used in J = cofi_cost_func(X_r, W_r, b_r, Y_r, R_r, 1.5).

I think it looks to you the shapes have changed because you are not printing the variables that are actually being used??

We are expected to use X_r and W_r.

Cheers,
Raymond

1 Like

Hello @Joaquin_Ramirez,

Maybe you don’t need further hints, but since it may take some time exchanging messages here forth and back…

  1. I suppose you have got the previous test correct, which is, without regularization, the cost is 13.67.

  2. Now we know what is additional when regularization is activated (by setting a positive lambda).
    image

  3. Apparently, we need some squared x and squared w when regularization is in.

  4. Now, your first post says you were getting 3657.99, which means, those squared x and squared w were contributing 3657.99 - 13.67 = 3644.32 which is pretty large

  5. However, you have also printed those W_r and X_r out in your post, and they are nothing close to be able to contribute that much loss. There are just less than 30 different numbers and most of them are not even larger than 1.

  6. Therefore, it must be something in your code that is activated by a non-zero lambda.

That’s all I can share and I hope at least my above logic will be motivative enough and my point 6 will give you somewhere to start your debugging work with.

Cheers,
Raymond

1 Like

Hello Raymond,
Thanks for your help. I was confused with two cells: 1. cell without regularization and 2. cell with regularization. In the first, I didn’t include the regularization code and in the second I included the regularization code. Looking back at the way this project lab is structured, cell #1 should really include the Cost Function code plus the regularization code; and with Lambda_ = 0, this lambda pretty much eliminated the regularization.
Thanks, again. You’re a lot of help.
Joaquin Ramirez