Unable to Exclude OOV Tokens

Course Q&A TensorFlow Developer Professional Natural Language Processing in TensorFlow
2

As a hint, these are the arguments according to the Tensorflow documentation:

tf.keras.layers.StringLookup(
    max_tokens=None,
    num_oov_indices=1,
    mask_token=None,
    oov_token='[UNK]',
    vocabulary=None,
    idf_weights=None,
    invert=False,
    output_mode='int',
    pad_to_max_tokens=False,
    sparse=False,
    encoding='utf-8',
    name=None,
    **kwargs
)

In order to exclude OOV tokens, your argument of interest should be num_oov_indices. Hope this helps.

4 Likes