FINE TUNING LLMs - How to import BasicModelRunner in colab

Short Course Q&A Finetuning Large Language Models
1

Not able to run this command in google collab “from llama import BasicModelRunner” it generates an error “cannot import name ‘BasicModelRunner’ from ‘llama’ (/usr/local/lib/python3.10/dist-packages/llama/init.py)”
Please advise how to import BasicModelRunner

3 Likes
2

!pip install --upgrade --force-reinstall --ignore-installed lamini

Try this first and don’t forget to restart the runtime before you import.

1 Like
3

Thanks Saurav I missed that was the required dependency.

I wish there was a commented hint like #!pip install on top of the notebooks… or a dependency file

4

Hi all,
I am having the same problem but installing lamini (and restarting) does not resolve the error in Colab.

----> 7 from llama import BasicModelRunner
8 from transformers import AutoTokenizer, AutoModelForCausalLM
9 from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

ModuleNotFoundError: No module named ‘llama’

I have the following pip installs:
!pip install datasets
!pip install transformers
!pip install --upgrade --force-reinstall --ignore-installed lamini
!pip install jsonlines

What am I missing?

Many thanks for any help!
Best,
Perusha

5

Ah nevermind… it is deprecated.
Using from lamini import Lamini as suggested.

6

Did anybody able to import BasicModelRunner?

I am stuck there

7

…guess it’s updated

instead you can do inference by

from lamini import Lamini
llm = Lamini(model_name='meta-llama/Llama-2-7b-hf')

# "Tell me how to train my dog to sit"

print(llm.generate("Tell me how to train my dog to sit"))
8

I checked the source code:

from lamini.runners.base_runner import BaseRunner


class BasicModelRunner(BaseRunner):
    """This class is deprecated, use lamini.BaseRunner or lamini.Lamini instead."""

However,
[tool.setuptools]
packages = [
“lamini”,
“lamini.classify”,
“lamini.error”,
“lamini.api”,
“lamini.api.utils”,
“lamini.generation”,
“lamini.evaluators”,
“lamini.evaluators.custom”,
“lamini.evaluators.custom.datasets”,
“lamini.evaluators.helm”,
“lamini.evaluators.utils”,
“lamini.index”,
]
The runners package seems not to be included through the build process, that’s why you cannot even for from lamini.runners import BaseRunner