Week 2: Instruction fine-tuning

Course Q&A Generative AI with Large Language Models
,
1

Generative AI with Large Language Models
Week 2: Instruction fine-tuning

How can we achieve full fine-tuning, i.e., ensure that ALL the weights got updated?

Even if we perform once again the pre-training on new amount of massive data, how do we ensure that ALL weights got updated?
Is full fine-tuning only a theoretical concept unless we provide greatly enough data that will enable ALL weights to get modified?

Another question. How does the LLM separate the “main instruction” and “examples” from the rest of the content in the prompt?
There were few examples like: “Summarize the text…” or “Translate this sentence…”.
We also saw some examples with YAML templates using prompt template libraries but I could not follow it very well.

What kind of libraries do we use and is it simple YAML file with key,value pairs such as {Request: Response}, {Example1: Content} and so on? or do we need to use specific keyword for specific models?

2

Hi @agarwalamit081

  1. You can try training on a sufficiently large and diverse dataset. Practically, achieving updates to every single weight can be hard.

  2. They rely on prompt parsing mechanisms that understands patterns. Pre-defined formats, common phrasing (like “Summarize the text”), and fine-tuned training are helpful.

  3. It depends. Some libraries use key-value pairs, but models require structured prompts depending on their training (like OpenAI’s API prompts). Libraries like LangChain simplify things but with their own conventions.

Hope it helps! Feel free to ask if you need further assistance.