05_Training_lab_student tokenize_and_split_data fuction

in 05_Training_lab_student, there is a function call:
train_dataset, test_dataset = tokenize_and_split_data(training_config, tokenizer)
how can I see inside the “tokenize_and_split_data” function and implement it by myself?

1 Like

There are many ways to look at source code of a function.
Use either the inspect module or ?? in a jupyter notebook to view function definition.
Example below shows how to look at the source code of encode function inside base64 module.

3 Likes

thanks for your guidance
I also found this solution helpful:

%load utilities

this way helped me to see all functions and variables that were defined in utilities package

1 Like