Is QLoRA a form of QAT?

Yacine_Mazari · August 17, 2023, 1:55am

In QLoRA the base models parameters are quantized and frozen, and a small number of trainable low rank adapters are added.

So, want to check if my understanding correct.
Is QLoRA a form of Quantization Aware Training (QAT)?

Thank you

gent.spah · August 17, 2023, 7:40am

Yes it is a QAT related method.

Topic		Replies	Views
Fine-Tuning a Large Language Models with QLoRA and PEFT(LLMs) Generative AI with Large Language Models week-module-2	11	845	July 15, 2023
Week 2 Question 7 - Description of LoRA Method Generative AI with Large Language Models quiz-help , week-module-2	1	114	January 24, 2025
Clarification on LoRA Generative AI with Large Language Models week-module-2	1	56	March 23, 2025
Lesson 5: LowRank Adaptation, Quick note on X = torch.randn(...) Efficiently Serving LLMs project	1	22	February 6, 2026
Fine tuning using LoRA method Generative AI with Large Language Models week-module-2	8	1060	September 7, 2023