Lab 3 - Getting "Cannot import name 'top_k_top_p_filtering' from 'transformers'" error

Course Q&A Generative AI with Large Language Models GenAI with LLMs Resources
1

I’m trying to do the first import step after the “pip install” step.

I get something like the below and the rest of steps give an error implying things are not defined.

Much TIA.

=====================
2025-04-17 03:51:00.316992: I tensorflow[/core/platform/cpu_feature_guard.cc:210](removed link)] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. To enable the following instructions: SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.

--------------------------------------------------------------------------- ImportError Traceback (most recent call last) Cell In[3], line 6 3 from peft import PeftModel, PeftConfig, LoraConfig, TaskType 5 # trl: Transformer Reinforcement Learning library ----> 6 from trl import PPOTrainer, PPOConfig, AutoModelForSeq2SeqLMWithValueHead 7 from trl import create_reference_model 8 from trl.core import LengthSampler File [/opt/conda/lib/python3.12/site-packages/trl/init.py:5](link removed) 1 # flake8: noqa 3 version = “0.4.2.dev0” ----> 5 from .core import set_seed 6 from .import_utils import is_peft_available 7 from .models import ( 8 AutoModelForCausalLMWithValueHead, 9 AutoModelForSeq2SeqLMWithValueHead, 10 PreTrainedModelWrapper, 11 create_reference_model, 12 ) File [/opt/conda/lib/python3.12/site-packages/trl/core.py:23](link removed) 21 import torch.nn.functional as F 22 from torch.nn.utils.rnn import pad_sequence —> 23 from transformers import top_k_top_p_filtering 26 try: 27 from collections.abc import Mapping ImportError: cannot import name ‘top_k_top_p_filtering’ from ‘transformers’ ([/opt/conda/lib/python3.12/site-packages/transformers/init.py]

2

Hi @MasNish

There is a version mismatch between the transformers library and trl.Because, trl is trying to import top_k_top_p_filtering from transformers, but that function may have been moved or deprecated in the version you have installed. Make sure your transformers library is aligned with what trl expects. Don’t forget to restart kernel or runtime after that!

Hope it helps! Feel free to ask if you need further assistance.

3

seems like you didn’t download all the data(metadata) which comes with the lab assignment.

Your error is pointing to any of module packages which is provided with lab assignment is not present in your system.

So check File==>Open, download all the files, next check requirements.txt file which provides the module version required you to run the codes locally.

4

Hi everyone. I’ve escalated this issue to our partners. In the meantime, please manually edit the pip install block and use 4.38.2 for the transformers package instead of 4.27.2. That worked for the labs I tested. Remember to restart the kernel before importing. Still waiting for updates. Thank you, and sorry for the inconvenience.

5

Hi everyone! I’m closing this thread so we can have one topic regarding this installation issue. It would be easier for everyone to manage feedback and monitor updates as we work on this. Please refer to the topic here. Thank you!