C3W2 Assignment worked but prediction not working (AI TensorFlow Developer Professional Certificate)

Course: DeepLearning.AI TensorFlow Developer Professional Certificate

Course assignment: C3W2_Assignment

Trained model output :
Epoch 29/30
56/56 [==============================] - 0s 2ms/step - loss: 0.0270 - accuracy: 0.9994 - val_loss: 0.2176 - val_accuracy: 0.9213
Epoch 30/30
56/56 [==============================] - 0s 2ms/step - loss: 0.0246 - accuracy: 1.0000 - val_loss: 0.2170 - val_accuracy: 0.9213

I cleared the assignment but when I try predict for the same text which is trained by model. Label should be “tech” , instead it is showing as “sport” as per below prediction output. Can someone confirm.

[[3.4719217e-03 3.3709027e-03 6.4150634e-05 9.9195898e-01 1.1340454e-03]]

Here are labels: {‘politics’, ‘entertainment’, ‘sport’, ‘business’, ‘tech’}

Code to predict: (this sentence is part of bbc news which is shown in assignment and trained)

sentence = [“tv future in the hands of viewers with home theatre systems plasma high-definition tvs and digital video recorders moving into the living room the way people watch tv will be radically different in five years time. that is according to an expert panel which gathered at the annual consumer electronics show in las vegas to discuss how these new technologies will impact one of our favourite pastimes. with the us leading the trend programmes and other content will be delivered to viewers via home networks through cable satellite telecoms companies and broadband service providers to front rooms and portable devices. one of the most talked-about technologies of ces has been digital and personal video recorders (dvr and pvr). these set-top boxes like the us s tivo and the uk s sky+ system allow people to record store play pause and forward wind tv programmes when they want. essentially the technology allows for much more personalised tv. they are also being built-in to high-definition tv sets which are big business in japan and the us but slower to take off in europe because of the lack of high-definition programming. not only can people forward wind through adverts they can also forget about abiding by network and channel schedules putting together their own a-la-carte entertainment. but some us networks and cable and satellite companies are worried about what it means for them in terms of advertising revenues as well as brand identity and viewer loyalty to channels. although the us leads in this technology at the moment it is also a concern that is being raised in europe particularly with the growing uptake of services like sky+. what happens here today we will see in nine months to a years time in the uk adam hume the bbc broadcast s futurologist told the bbc news website. for the likes of the bbc there are no issues of lost advertising revenue yet. it is a more pressing issue at the moment for commercial uk broadcasters but brand loyalty is important for everyone. we will be talking more about content brands rather than network brands said tim hanlon from brand communications firm starcom mediavest. the reality is that with broadband connections anybody can be the producer of content. he added: the challenge now is that it is hard to promote a programme with so much choice. what this means said stacey jolna senior vice president of tv guide tv group is that the way people find the content they want to watch has to be simplified for tv viewers. it means that networks in us terms or channels could take a leaf out of google s book and be the search engine of the future instead of the scheduler to help people find what they want to watch. this kind of channel model might work for the younger ipod generation which is used to taking control of their gadgets and what they play on them. but it might not suit everyone the panel recognised. older generations are more comfortable with familiar schedules and channel brands because they know what they are getting. they perhaps do not want so much of the choice put into their hands mr hanlon suggested. on the other end you have the kids just out of diapers who are pushing buttons already - everything is possible and available to them said mr hanlon. ultimately the consumer will tell the market they want. of the 50 000 new gadgets and technologies being showcased at ces many of them are about enhancing the tv-watching experience. high-definition tv sets are everywhere and many new models of lcd (liquid crystal display) tvs have been launched with dvr capability built into them instead of being external boxes. one such example launched at the show is humax s 26-inch lcd tv with an 80-hour tivo dvr and dvd recorder. one of the us s biggest satellite tv companies directtv has even launched its own branded dvr at the show with 100-hours of recording capability instant replay and a search function. the set can pause and rewind tv for up to 90 hours. and microsoft chief bill gates announced in his pre-show keynote speech a partnership with tivo called tivotogo which means people can play recorded programmes on windows pcs and mobile devices. all these reflect the increasing trend of freeing up multimedia so that people can watch what they want when they want.”]

seq_pad_sentence = seq_and_pad(sentence, tokenizer, PADDING, MAXLEN)
prediction = model.predict(seq_pad_sentence)
print(prediction)

Hmm, there is probably a mistake in the labeled training set, it happens, most trained sets have mistakes.

Thanks @gent.spah for the reply.

Irrespective of labelling wrongly done in training set, model should learn the same wrong mapping right?
What change I need to do , to make it work?

I believe labels are ordered in alphabetical.
0th index - business
1st index - entertainment
2nd index - politics
3rd index - sport
4th index - tech

From prediction output , it should be “sport” … right?
[[3.4719217e-03 3.3709027e-03 6.4150634e-05 9.9195898e-01 1.1340454e-03]]

I dont remember this assignment but given that an NLP model can learn from a variety of text sources, it could give the wrong mapping, but it could also give the correct mapping because other examples with the same label and similar texts, overwhelm the learning process. So it instead gives the correct prediction as is in the labels of the most and same examples.

What can you change to make it work?
Correct the training dataset, even manually if not possible otherwise. You may have some mistakes in the training set that is acceptable as long as they are a very little percentage of that class, but one should make sure the majority of the labels >90% per class are correct (depending on class size). There are exceptional situations of course.

Hi @Kamavaram . I would like to add something to @gent.spah response.
The labels are in a string format. You have to assign an ID to relate it. You can do that with the function tokenizer.

# Convert labels to sequences
label_seq = label_tokenizer.texts_to_sequences(split_labels)
  
# Convert sequences to a NumPy array. Don't forget to subtract one from every entry in the collection!
label_seq_np = np.array(label_seq) - 1

Now that your labels are tokenizers, you can print the first 5 in both format string and id and match them.

train_label_seq = tokenize_labels(labels, train_labels)
print(f"First 5 labels of the training set should look like this:\n{train_label_seq[:5]}\n")

# Test the functions
print(f"The first 5 labels are {train_labels[:5]}")

Now, you can relate the order; I tested it, and the order that I have is like this:

The first five labels are ['tech,' 'business,' 'sport,' 'sport,' 'entertainment']

The First five labels of the training set should look like this:
[[3],[1], [0], [0], [4]]

That means that the order is

{
  0:'sport',
  1:'business',
  2:'politics',
  3:'tech',
  4:'entertainment'
}

You can test it to see your order.

I hope this helps. :hugs:

1 Like

Perfect @adonaivera … I did not realize and stuck my mind that we are already doing this as part of assignment. So, my output of prediction is correct which is “tech” at index 3.

Till now I realised this course is not worthy but you saved with the response :smile:. It would be good if course and assignment have predictions as well which was there in C1 and C2 .

1 Like

Hello , I do that but still give me error :
TypeError: unsupported operand type(s) for -: ‘list’ and ‘int’

Hi @Amar_Ismail
Can you share your notebook for DM, please. :hugs: