A possible Mistake in TripletLossFn Grader #Week3

Course Q&A Natural Language Processing NLP with Sequence Models
1

Hi,

I am facing an issue with the last assignment [Programming Assignment: Question Duplicates, C3W2_Assignment] in the TripletLossFn section.

My implementation is wrong according to the grader:

Failed test case: Got a wrong triplet loss for inputs v1: [[0.26726124 0.53452248 0.80178373]
 [0.5178918  0.57543534 0.63297887]], v2: [[ 0.26726124  0.53452248  0.80178373]
 [-0.5178918  -0.57543534 -0.63297887]].
Expected:
0.7035077,
but got:
2.4070153719293854.

However, after I do plug this input to the supplied notebook for calculating the triple loss, the notebook agreed with my implementation:


v1 = np.array([[0.26726124, 0.53452248, 0.80178373],[0.5178918 , 0.57543534, 0.63297887]])
v2 = np.array([[ 0.26726124,  0.53452248,  0.80178373],[-0.5178918 , -0.57543534, -0.63297887]])


sim_hardcoded = v1 @ v2.T

sim = sim_hardcoded


b = sim.shape[0]

print("-- Inputs --")
print("sim :")
print(sim)
print("shape :", sim.shape, "\n")


sim_ap = tf.linalg.diag_part(sim) 
print("sim_ap :")
print(tf.linalg.diag(sim_ap), "\n")

sim_an = sim - tf.linalg.diag(sim_ap)
print("sim_an :")
print(sim_an, "\n")

print("-- Outputs --")

mean_neg = tf.math.reduce_sum(sim_an, axis=1) / (b - 1)
print("mean_neg :")
print(mean_neg, "\n")

mask_1 = tf.eye(b) == 1            
mask_2 = sim_an > tf.expand_dims(sim_ap, 1)  
mask = tf.cast(mask_1 | mask_2, tf.float64)
sim_an_masked = sim_an - 2.0*mask

closest_neg = tf.math.reduce_max(sim_an_masked, axis=1)
print("closest_neg :")
print(closest_neg, "\n")

alpha = 0.25


l_1 = tf.maximum(mean_neg - sim_ap + alpha, 0)
print(f"Loss 1: {l_1}\n")

l_2 = tf.maximum(closest_neg - sim_ap + alpha, 0)
print(f"Loss 2: {l_2}\n")

l_full = l_1 + l_2

cost = tf.math.reduce_sum(l_full)

print("-- Outputs --")
print("Loss full :")
print(l_full, "\n")
print("Cost :", "{:.3f}".format(cost))

Ouput:

-- Inputs --
sim :
[[ 1.         -0.95350769]
 [ 0.95350769 -1.        ]]
shape : (2, 2) 

sim_ap :
tf.Tensor(
[[ 1.  0.]
 [ 0. -1.]], shape=(2, 2), dtype=float64) 

sim_an :
tf.Tensor(
[[ 0.         -0.95350769]
 [ 0.95350769  0.        ]], shape=(2, 2), dtype=float64) 

-- Outputs --
mean_neg :
tf.Tensor([-0.95350769  0.95350769], shape=(2,), dtype=float64) 

closest_neg :
tf.Tensor([-0.95350769 -1.04649231], shape=(2,), dtype=float64) 

Loss 1: [0.         2.20350769]

Loss 2: [0.         0.20350769]

-- Outputs --
Loss full :
tf.Tensor([0.         2.40701537], shape=(2,), dtype=float64) 

Cost : 2.407
2

If the grader says your code doesn’t work correctly, I recommend you believe it.

3

Exactly. The point is that there are lots of ways you can pass the tests in one of the notebooks, but fail the grader. Basically it means that you have written your code in a way that is not general: it works with one test case, but not with a different test. Examples of ways to create that situation would be hard-coding assumptions about the shapes of the inputs or referencing global variables from the local scope of your function and those variables are either not present in the grader’s context or happen to have different values.

So the first step is to let go of the notion that your code is correct and the bug must be in the grader. :nerd_face:

5

Passing the tests in the notebook does not prove your code is perfect.

The notebook and the grader use different tests.