LAB 3 Doubt in the dataset filtering part

Hi so could anyone please give me an explaination for the dataset filtering part in LAB 3 where they use the filter function to get the dialogues which fall in a certain range.
If possible could you share that line of code in your explaination.
Thanks in Advance

# Filter the dialogues of length between input_min_text_length and input_max_text_length characters.
dataset = dataset.filter(lambda x: len(x["dialogue"]) > input_min_text_length and len(x["dialogue"]) <= input_max_text_length, batched=False)

You mean this one right.

You need to dig a bit deeper to see how the, dataset = load_dataset(dataset_name, split=“train”), this function works (search in google - from datasets import load_dataset) and then its method filter.

The lambda function just compares that x length falls within the range.

1 Like

Thank you very much