Week 2 : How is Negative Sampling a bunch of Binary problems?

Course Q&A Deep Learning Specialization Sequence Models
1

During the lecture in the Negative Sampling to find the Word Embeddings, Andrew called it as a multiple binary classification problems rather than single softmax problem.
image

As per Andrew the above image represents Training dataset. But for me it seems like it is single binary problem i.e., given the “Context” and “Word” our target is either 1 or 0
image

I realized I was right after seeing Andrew righting formula that was mentioned above.

But while drawing its neural network representation, Andrew said it was multiple binary class problem but the formula written above says otherwise.

So is the formula correct or neural network representation part posted below which doesn’t make any sense to me?

image
image1103×262 34.6 KB

2

Both are correct. The “multiple binary classification”, aka “multi-label classification” (instead of multi-class), applies the formula (sigmoid) on each output neuron. In negative sampling, we only care (calculate training loss) about K+1 neurons, and ignore other neurons at each iteration.

1 Like
3

Hi, I don’t fully get this, could you explain using other words?

4

@Awong. You have replied on a thread that has been cold for four years.

I recommend you start a new thread for your question.