I first started to change the value of batch_size inside gradient_descent() function. I expected the values of the cost to be different than the expected cost values ( iters: 10 cost: 11.714748…) corresponding to a value of 128 for batch_size. But it was not the case. After investigation, I found that the compute_cost() function always returned the same values independently of the value of batch_size. I found the reason is that the get_batches() function is flawed. Here is a corrected version:
def get_batches(data, word2Ind, V, C, batch_size):
batch_x =
batch_y =
for x, y in get_vectors(data, word2Ind, V, C):
“”"
while len(batch_x) < batch_size:
batch_x.append(x) # same value (x) added to batch_x multiple (batch_size) times
batch_y.append(y) # same value (y) added to batch_y multiple (batch_size) times
else:
yield np.array(batch_x).T, np.array(batch_y).T
batch_x =
batch_y =
“”"
batch_x.append(x)
batch_y.append(y)
if len(batch_x) == batch_size:
yield np.array(batch_x).T, np.array(batch_y).T
batch_x =
batch_y =