QLoRA
Fine-tuning 4-bit quantized base models on consumer GPUs
Training
In plain English
Fine-tuning a giant 70B AI on a single consumer graphics card by keeping the base model in 4-bit compressed memory while training high-precision adapter layers.
Technically
Quantized Low-Rank Adaptation (Dettmers et al., 2023). Uses NormalFloat4 (NF4) data type, Double Quantization (DQ), and Paged Optimizers to fine-tune a 4-bit quantized base model without performance degradation, making 70B tuning viable on a single 24GB RTX 4090/3090.