Thinking aloud.. LoRA & Prompt Tuning

Course Q&A Generative AI with Large Language Models
1

I have a few queries…

  1. Will it improve performance if we apply both QLoRA/ LoRA and Prompt tuning together?
  2. Can we apply LoRA/ QLoRA for TaskA and then do Prompt tuning for TaskB so that we can reduce number of headers we need for multi task inference?
  3. Which one performs better for which task - LoRA or Prompt tuning? Is there a SuperGlue or HELM benchmark?
1 Like
2

For your questions:

  1. Most probably if you do Lora and give it some prompt tuning context will be better that without any context at all.

  2. I guess when certain data comes into the pipeline then you can ommit the Lora add on matrix, but go throught the prompt tuning path.

  3. Probably Lora because it includeds a trainable matrix with weights added on to the main model, the prompt tuning just gives a context no new weights are given in.

1 Like
3

Thank you. Do you suggest any free and open platform with the necessary computing resources where we can try these concepts (even after this course)?

1 Like
4

Free and open will be up to a certain scale, maybe Colab, AWS, IBM, they all have a free amount to be used but for large scale projects, you need to buy computing resources.

1 Like
5

@gent.spah thanks a lot. Really appreciate your prompt responses.

6

Hi, just some feedback from myside.
I think prompt tuning is not the right terminology it should be prompt engineering (in context learning) without retraining the weight or add layers to the LLM.

The prompt tuning might misleading as there is another soft prompt tuning (one of the PEFT) method to fine tune the model.

Please correct me if you have different thoughts.

Thanks,
Ming

1 Like
7

I am actually a little confused about the difference between ‘prompt engineering’ and ‘instruction fine-tuning’. It seems both are using some examples to improve the model performances on some targeted tasks. Aren’t they pretty much the same thing?

8

Below is the “completion” I got from ChatGPT 3.5 Turbo:

Prompt engineering and instruction fine-tuning share similarities in their objective of leveraging examples to improve the performance of language models on targeted tasks. However, there are distinct differences in their focus and the mechanisms through which they achieve this objective.

Prompt Engineering:

  • Focuses on shaping and designing input prompts, queries, or examples to guide the behavior of a language model.
  • The goal is to influence the model’s outputs by creating specific input patterns that are likely to elicit desired responses from the model.
  • It emphasizes the crafting of tailored prompts to effectively steer the model’s generation and influence its outputs without extensively modifying the model itself.

Instruction Fine-Tuning:

  • Involves customizing a pre-existing model by fine-tuning it with specific examples, data, or instructions related to a particular task or domain.
  • The process often includes exposing the model to task-specific data and adjusting its parameters to enhance its performance on a specific objective or set of tasks.
  • It focuses on adapting the model’s parameters through task-specific training examples, allowing for targeted optimization of the model’s behavior and performance for specific use cases or domains.

Key Differences:

  1. Approach:
  • Prompt engineering primarily focuses on shaping input prompts to guide the model’s outputs without extensive modification of the model itself.
  • Instruction fine-tuning involves modifying the model’s parameters and behavior through exposure to task-specific data and training examples.
  1. Emphasis:
  • Prompt engineering emphasizes the design and crafting of tailored prompts to steer the model’s generation.
  • Instruction fine-tuning emphasizes the adaptation of the model’s parameters and behavior through exposure to task-specific examples and data.

In essence, while both approaches utilize examples to improve model performance on targeted tasks, prompt engineering centers on customizing input patterns to guide the model’s outputs, whereas instruction fine-tuning focuses on adapting the model’s parameters and behavior through task-specific training examples and data. These distinctions underline the nuanced differences in their respective methodologies and goals. If you have further questions or need additional clarification, feel free to ask for more information.

9

My understanding is this:

  1. prompt engineering only give prompts to the model vs. instruction fine-tuning gives “training examples” (which is both prompts AND completion? not really sure about this)
  2. instruction fine-tuning modifies parameters while prompt engineering don’t? Not sure either.

Any insights?