TL;DR
Parameter efficient fine tuning (PEFT) adapts a pretrained model by training a small set of new or selected weights while the original model stays frozen. The most common method, LoRA, adds two small trainable matrices next to existing weight matrices instead of updating them directly. QLoRA extends this by loading the frozen base model in 4-bit precision, which is how researchers fine-tuned a 65-billion-parameter model on a single 48GB GPU. PEFT cuts training memory, storage, and experiment time sharply, but it does not shrink the model you serve at inference time. Newer variants like DoRA and AdaLoRA close specific quality gaps that plain LoRA leaves open. Which method to pick depends on what is actually constrained, whether that is GPU memory, parameter budget, latency, or how many tasks one base model has to support at once.
Key Takeaways PEFT freezes the base model and trains a small fraction of parameters, often well under 1% to a few percent depending on the method. LoRA adds no extra inference latency once its weights are merged back into the base model; classic adapters and unmerged methods do add some. QLoRA’s 4-bit quantization made 65-billion-parameter fine-tuning possible on one 48GB GPU, according to the original paper. LoRA tends to learn less than full fine-tuning on hard, out-of-domain tasks, but it also forgets less of what the base model already knew. Rank, alpha, and target modules affect PEFT outcomes more than almost any other setting in the configuration. One base model can serve dozens of task-specific LoRA adapters at once, which changes the economics of multi-tenant LLM deployments. A 65-Billion-Parameter Model, One 48GB GPU Watch on YouTube
The Hidden Power of RAG in Modern AI Systems
Deciding between PEFT and RAG for your use case? This walks through what RAG actually does differently from fine-tuning approaches like PEFT.
In 2023, researchers fine-tuned a 65-billion-parameter language model on a single 48GB GPU while matching the performance of full 16-bit fine-tuning, according to the QLoRA paper . Before that result, adapting a model that size meant renting a multi-GPU cluster, storing optimizer states for every one of its 65 billion parameters, and waiting hours just to get a training job loaded and running. QLoRA cut that requirement down to hardware a single research lab, or a reasonably equipped enterprise team, could own outright.
That shift matters more once you move past a research demo and into an organization that wants ten specialized model behaviors instead of one general-purpose assistant. Full fine-tuning implies ten full copies of the base model, ten sets of optimizer states, and ten separate deployment artifacts to version and monitor. Training a small set of adapter weights per task, instead of every weight, turns that multiplication problem into something a single GPU and a normal MLOps pipeline can handle.
None of that removes the decisions that come after. Once training a specialist behavior costs a fraction of what it used to, the questions that matter shift to which parameters to train, how large a rank to pick, whether to merge the adapter into the base weights or keep it separate for serving, and when PEFT is not the right tool for the job at all.
What Is Parameter Efficient Fine Tuning? Parameter efficient fine tuning is a set of techniques that adapt a pretrained generative AI model to a new task or domain by training only a small subset of its parameters, or a small number of new parameters added alongside the frozen model, instead of updating every weight the model has. The base model, whether that is a Llama, Mistral, Qwen, or Gemma-class open-weight model, keeps its original weights untouched during training. Only the new or selected parameters, often well under 1% of the total parameter count, get updated by the optimizer. That single design choice is what makes PEFT so much cheaper to train than adjusting every weight in a modern LLM.
Most of the confusion around PEFT starts with dataset size. Pretraining a foundation model does require internet-scale, multi-terabyte text corpora, but fine-tuning, PEFT or otherwise, typically works with far smaller sets, from a few thousand to a few hundred thousand labeled or instruction-formatted examples for a specific task or domain. A team evaluating whether it has enough data to fine-tune is usually comparing against the wrong number if it starts from pretraining-scale requirements. It is also worth separating PEFT from the model family it gets applied to. Early PEFT papers used encoder models like BERT as test beds, but the techniques themselves are model-agnostic; today they are applied almost entirely to decoder-only, open-weight LLMs.
How PEFT Works in Four Steps Freeze the base model. Every original weight in the pretrained model is set to not require gradients, so the optimizer never touches it.Inject or select trainable parameters. Depending on the method, this means adding a new low-rank matrix pair, an adapter module, or a soft prompt, or selecting a small existing subset of weights, such as bias terms, to unfreeze.Train only those parameters. Forward and backward passes still run through the full frozen model, but gradients and optimizer states are only computed and stored for the small trainable set.Save, and then merge or swap. The trained parameters get saved as a small adapter file, typically megabytes rather than gigabytes. From there, the team decides whether to merge the adapter’s math into the base weights for a single deployable model, or keep it separate so several adapters can share one base model at serving time.PEFT vs Full Parameter Fine Tuning The table below lays out where PEFT and full fine-tuning genuinely differ, and corrects two claims that circulate often but do not hold up. PEFT does not shrink the model you serve, and it is not uniformly “better at generalizing” than full fine-tuning.
Criteria Full Fine-Tuning PEFT (LoRA Family) Trainable parameters 100% of model weights Typically under 1% to a few percent GPU memory during training Full weights, gradients, and optimizer states for every parameter Full weights for the forward pass, gradients and optimizer states only for the small trainable set Training data needed Often benefits from larger, broader task data Can work with smaller task-specific sets, though more data still helps Storage per task A full model copy per task A small adapter file per task, often a few megabytes to a few hundred megabytes Served model size One full model per task Still a full base model at inference; the adapter is small but does not replace it Merging into a single artifact Not applicable, already one model Possible with methods like LoRA; not possible with every PEFT method Inference latency Baseline model latency No added latency once merged; some added overhead if adapters stay unmerged Multi-task serving Requires swapping full models One base model can host many adapters at once Generalization and forgetting Can learn hard, out-of-domain tasks more completely, at higher risk of forgetting prior skills Tends to learn less on hard new domains, but forgets less of what the base model knew Time to a working result Longer training runs, more infrastructure to provision Faster experiments, smaller hardware footprint Governance overhead One large artifact to version per task Many small adapters to version, track, and gate against the base model they were trained on
Where SFT and Instruction Tuning Fit Supervised fine-tuning, or SFT, is a training objective. It means adapting a pretrained model on labeled input-output pairs, usually instruction-and-response examples, so it follows instructions or performs a task more reliably. SFT is not training a model from scratch, and it is not an alternative to PEFT. The two answer different questions. SFT describes what data and objective you train on; PEFT describes which parameters get updated while you train. In practice the two are usually combined, and Hugging Face’s TRL SFTTrainer accepts a peft_config argument, so a team can run standard supervised fine-tuning while only updating a LoRA adapter’s weights instead of the full model. Kanerika’s guide to LLM training covers where SFT sits alongside pretraining and reinforcement-based alignment in the broader training pipeline; this article stays focused on the parameter-efficiency layer.
Why Full Fine-Tuning Stopped Being the Default Full parameter fine-tuning updates every weight in a model, which means storing gradients and optimizer states for each one. Adam-style optimizers need roughly two extra numbers per parameter for momentum and variance terms alone, so full fine-tuning a large model can require several times the memory just to hold the base weights during training. On GPT-3’s 175-billion-parameter scale, the original LoRA paper reported training roughly 10,000 times fewer trainable parameters and about three times less GPU memory compared with full fine-tuning using Adam, while matching or exceeding full fine-tuning quality on the benchmarks the authors tested.
Two things changed at the same time to make this the practical default. First, open-weight large language models became genuinely capable, giving teams a base worth adapting instead of a closed API they could only prompt. Second, the number of specialist behaviors an organization wants from one model kept growing, covering things like a support classifier, a document extractor, a style-matched writing assistant, and a domain-specific coding helper. Full fine-tuning each of those separately means a full model copy, and a full training run, for every behavior.
Fine-tuning datasets reinforce the same point. A typical instruction-tuning or task-adaptation dataset runs from a few thousand to a few hundred thousand examples, not the terabyte-scale corpora used for pretraining. That gap between what fine-tuning actually needs and what pretraining needs is exactly why PEFT’s low parameter counts are enough. The training signal is narrow and specific, so the number of parameters that need to move to capture it is small too.
The Three Families of PEFT Methods A 2023 critical review of PEFT methods for pretrained language models, Xu et al. , groups the field into three main families, with a fourth “hybrid” category for recipes that mix ideas across them. Knowing which family a method belongs to tells you what to expect from it before you read a single benchmark number.
Additive Methods: Add New Parameters Additive methods leave every original weight alone and insert new trainable parameters into the model instead. Bottleneck adapters, prompt tuning, and prefix tuning all fall here. The frozen model still does the same computation it always did; the new parameters sit alongside it and reshape the output. This family tends to have the smallest deployment footprint per task, since only the new parameters need to be saved, but methods in it are not always mergeable into the base weights the way LoRA is.
Selective Methods: Train a Subset of Existing Weights Selective methods do not add anything new. They unfreeze a small, specific slice of the model’s existing weights, such as bias terms in BitFit or a sparse mask of individual parameters in diff pruning, and train only that slice. Because no new architecture is introduced, these methods are simple to implement and easy to merge conceptually, since the “adapter” and the base model are the same set of weights. Their main limitation is capacity, since a bias-only or sparse-mask update has less room to represent a complex new behavior than a purpose-built low-rank matrix.
Reparameterization Methods: Decompose the Update Itself Reparameterization methods, the family LoRA belongs to, represent the weight update as a lower-dimensional structure rather than a full-size matrix. Instead of learning a full update the same size as the original weight matrix, the model learns a compact factorization of it, which can then be added back into the original weights at deployment time. QLoRA, DoRA, AdaLoRA, LoRA+, VeRA, and LoftQ are all variations on this same core idea, each changing a different part of how that factorization is computed, initialized, or quantized. This is the most actively developed family in 2026, and it is where the rest of this guide spends most of its depth.
Hybrid approaches, such as Compacter, borrow mechanisms from more than one family at once, for example combining a low-rank structure with parameter sharing across layers. They are less common in production than plain LoRA or its direct variants, but worth knowing about if a standard method underperforms on an unusual architecture.
How LoRA Fine-Tuning Actually Works LoRA, short for Low-Rank Adaptation, freezes a weight matrix W in the base model and adds a parallel path made of two small matrices, A and B, so that the effective weight becomes W’ = W + BA, scaled by a factor of alpha divided by the rank r. A projects the input down to a small rank r, far smaller than the original dimensions, and B projects it back up. Only A and B are trained; W never changes during fine-tuning. Because r is small, often 8, 16, or 32 against hidden dimensions in the thousands, the number of trainable parameters is a tiny fraction of the original layer’s size, as described in the original LoRA paper .
LoRA adapters are usually attached to the attention projections inside a transformer block, the query, key, value, and output matrices, and sometimes to the feed-forward layers as well. There is no single “correct” set of target modules for every model and task; the right choice depends on where the behavior you are training actually needs to change, and it is common to test a narrower and a broader target-module set against each other. Rank and alpha both need tuning too. A higher rank gives the adapter more capacity, but more capacity is not automatically better, since past a certain point the extra parameters mostly increase training cost without a matching quality gain.
One property of LoRA’s math makes it attractive for production. Once training finishes, BA can be added directly into W to produce a single merged weight matrix, so the model that gets served has exactly the same architecture and the same latency as the original base model, as the LoRA paper states explicitly. That is different from many additive methods, which have to run the adapter at inference time alongside the frozen model and therefore add some overhead. Teams that need to switch between many task-specific behaviors at serving time, rather than deploying one fixed adapter, typically keep the adapters unmerged instead, trading a small amount of latency for the flexibility to swap behaviors per request.
LoRA Variants: QLoRA, DoRA, AdaLoRA, LoRA+, VeRA and LoftQ Once LoRA became the default starting point, most of the research effort in this family went into fixing a specific weakness rather than replacing the idea. Each variant below targets a different bottleneck.
QLoRA: When GPU Memory Is the Limit QLoRA quantizes the frozen base model to a 4-bit format called NF4, applies a second round of quantization to the quantization constants themselves to save further memory, and uses paged optimizers to avoid out-of-memory spikes during training. The trainable LoRA matrices stay in higher precision. The paper’s headline result, a 65-billion-parameter model fine-tuned on a single 48GB GPU while preserving full 16-bit fine-tuning task performance, is what made large open-weight models fine-tunable on workstation-class hardware. The paper also introduced Guanaco, a model family trained with QLoRA that reached 99.3% of ChatGPT’s performance on the Vicuna benchmark; that figure is specific to that benchmark and model comparison, not a general claim about QLoRA quality.
DoRA: Separating Magnitude from Direction DoRA , or Weight-Decomposed Low-Rank Adaptation, splits each weight update into a magnitude component and a direction component, then applies LoRA only to the directional part. The DoRA paper reports consistent gains over standard LoRA across LLaMA and vision-language benchmarks, with no added inference overhead once merged. In the Hugging Face peft library, this is a single configuration flag, use_dora=True, on top of an otherwise normal LoRA setup; note that merging a DoRA adapter with merge_and_unload alongside modules_to_save is not currently supported, which matters when planning a deployment path.
AdaLoRA: Spending the Rank Budget Where It Matters AdaLoRA allocates rank adaptively across layers using an SVD-style parameterization, instead of giving every targeted layer the same fixed rank. Layers estimated to matter more for the task get a larger effective rank, and layers that matter less get pruned down. The reported advantage over plain LoRA is strongest at low overall parameter budgets, where deciding where to spend a small rank allowance carefully makes the most difference.
LoRA+, VeRA and LoftQ LoRA+ keeps LoRA’s architecture unchanged and instead uses a different learning rate for the A and B matrices, reporting one to two percent quality gains and up to roughly twice the fine-tuning speed at the same compute budget. VeRA shares a single pair of frozen, randomly initialized low-rank matrices across every layer, and trains only small per-layer scaling vectors on top, which cuts trainable parameters far below standard LoRA while the paper reports comparable performance. LoftQ addresses a specific failure mode in quantized fine-tuning by initializing the LoRA matrices to account for the error introduced by quantization, rather than the standard random or zero initialization, which the paper shows helps most in aggressive 2-bit and mixed 2-to-4-bit setups.
Beyond these, the field keeps producing narrower initialization and structural variants, including rsLoRA, OLoRA, PiSSA, EVA, CorDA, BOFT, LoHa, LoKr, MoRA, and KronA. Most of these are worth knowing by name so a paper or a library flag does not look unfamiliar; few of them are worth adopting by default ahead of LoRA, QLoRA, or DoRA unless a benchmark on your own workload shows a real gap those three do not close.
Method Family What Gets Trained Relative Trainable Parameters Adds Latency If Unmerged? Mergeable Into Base? Best Fit Bottleneck Adapters Additive New adapter modules inserted per layer Low (around 3-4% per task in the original paper) Yes Not always Multi-task setups with strict per-task isolation Prompt Tuning Additive Soft prompt embeddings prepended to input Very low Yes No Very large models where a small trainable prompt closes most of the gap Prefix Tuning Additive Trainable prefix vectors at every layer Very low (around 0.1% in the original paper) Yes No Low-data tasks and generation-heavy workloads BitFit Selective Bias terms only Extremely low No, same weights as base Not applicable Small quick-turn adaptations where capacity needs are modest IA3 Reparameterization / Additive hybrid Learned rescaling vectors on key, value, and feed-forward activations Lower than LoRA at typical settings Minimal Yes Extremely small adapter footprint, few-shot task adaptation LoRA Reparameterization Low-rank matrices A and B beside frozen weights Low, 10,000x fewer than full fine-tuning on GPT-3 175B in the original paper No, once merged Yes General-purpose baseline for most fine-tuning workloads QLoRA Reparameterization LoRA matrices, base model held in 4-bit Same as LoRA No, once merged Yes Large models where GPU memory during training is the hard constraint DoRA Reparameterization Magnitude and directional LoRA components Slightly above LoRA No, once merged Yes, with caveats Cases where standard LoRA leaves a measurable quality gap AdaLoRA Reparameterization SVD-style adaptive rank allocation Comparable to LoRA at the chosen budget No, once merged Yes Tight parameter budgets where rank should not be fixed per layer LoRA+ Reparameterization LoRA matrices with differentiated learning rates Same as LoRA No, once merged Yes Faster convergence at the same LoRA compute budget VeRA Reparameterization Shared frozen random matrices plus small scaling vectors Far below LoRA No, once merged Yes Extremely tight storage budgets across many adapters
PEFT Methods Beyond LoRA: Adapters, Prompt Tuning and Prefix Tuning Before LoRA became the default, additive methods were the main way to adapt a large model without touching every weight, and they are still useful in specific situations today.
Bottleneck Adapters Houlsby et al.’s original adapter paper inserted small bottleneck modules, a down-projection, a nonlinearity, and an up-projection, inside each transformer layer, training only those modules while freezing the rest of the model. The paper reported reaching within about 0.8% of full fine-tuning performance on the GLUE benchmark while training roughly 3.6% of parameters per task. That was a meaningful proof that a frozen model plus a small trainable module could match near full fine-tuning, years before LoRA generalized the idea to low-rank matrices.
Prompt Tuning Prompt tuning, introduced by Lester et al. , prepends a set of trainable “soft prompt” embeddings to the input, and trains only those embeddings while the entire model stays frozen. The paper found that prompt tuning closed the gap with full fine-tuning specifically as model scale increased, becoming competitive at billion-parameter scale. It is important not to confuse this with prompt engineering. Prompt tuning trains continuous embedding vectors that never appear as readable text, while prompt engineering only edits the natural-language text you send to a model.
Prefix Tuning and P-Tuning Prefix tuning , from Li and Liang, trains a small set of continuous vectors prepended at every transformer layer, not just the input embedding layer, and the paper reports training around 0.1% of parameters while reaching comparable performance to full fine-tuning, with stronger relative results in low-data settings. P-tuning and its successor, P-Tuning v2 , train continuous prompt embeddings through a small prompt encoder while the base model stays entirely frozen. That is a meaningfully different mechanism from selecting and updating a subset of the model’s own weights, since P-tuning never touches the base model’s parameters at all and only learns what to prepend to the input at every layer.
Selective PEFT and What IA3 Actually Does BitFit is the simplest selective method, unfreezing only the bias terms throughout the network and training those while leaving every weight matrix untouched. It trains an extremely small number of parameters and needs no new architecture, which makes it fast to implement, though its limited capacity means it usually underperforms LoRA on harder tasks. Diff pruning takes a related but more flexible approach, learning a sparse mask over the full parameter set so that only a scattered subset of individual weights actually changes, rather than a structurally grouped subset like all bias terms.
What Is IA3? IA3 stands for Infused Adapter by Inhibiting and Amplifying Inner Activations, introduced in the paper behind the T-Few method and documented in the peft IA3 reference . It does not adapt the entire model’s weights, and it is not a decomposition of a weight update the way LoRA is. IA3 instead learns three small vectors per targeted layer that rescale the key, value, and feed-forward activations as they flow through the model, multiplying those activations element-wise rather than adding a new matrix product. Because it only learns rescaling vectors instead of full low-rank matrices, IA3 typically trains fewer parameters than LoRA at comparable target-module coverage, and its authors reported it as a cheaper alternative to in-context learning rather than a broader-reaching update than LoRA. This corrects a common mischaracterization. IA3 is not “Iterative Amortized Attention and Alignment,” and it does not require more memory or compute than LoRA to run.
PEFT Trade-Offs: Memory, Training Cost, Latency and Accuracy Trainable parameter count is the number most PEFT comparisons lead with, and it is also a poor stand-in for actual training cost. GPU memory during training is made up of the frozen base weights, the trainable parameters, the gradients and optimizer states for only those trainable parameters, plus activations kept for the backward pass and any quantization or temporary buffers. A method that trains 0.1% of parameters instead of 1% does shrink the gradient and optimizer memory, but the frozen base weights and the activation memory needed to run a forward and backward pass through the whole model stay largely the same either way.
Training cost follows the same logic. PEFT sharply reduces the memory spent on gradients and optimizer states, and it reduces wall-clock training time in most reported comparisons, but the forward and backward computation still has to pass through the entire frozen model. That is why QLoRA’s 4-bit quantization of the base model matters as a separate lever from LoRA’s low-rank update. One shrinks how many parameters get gradients, and the other shrinks how much memory the frozen weights themselves occupy.
Inference latency depends entirely on how the adapter gets deployed, not on which family it came from. A merged LoRA-style adapter runs at the base model’s normal speed, because the deployed weights are indistinguishable from a single fine-tuned model. An adapter kept unmerged for multi-tenant switching, or a soft-prompt method that has to be recomputed on every request, adds some runtime work. There is no PEFT method that is universally faster or slower at inference than another; it depends on the merge decision made at serving time, which the section on adapter serving below covers directly. Storage economics tell a similar story to small language models , since both approaches shrink an artifact, but a small language model shrinks the model that actually runs at inference, while a PEFT adapter shrinks only the file that gets added to a still-full-size base model.
On accuracy, a 2024 study, “LoRA Learns Less and Forgets Less,” compared LoRA against full fine-tuning on programming and mathematics tasks and found a consistent pattern. LoRA underperformed full fine-tuning on these harder, more out-of-domain tasks at standard ranks, but it also preserved more of the base model’s original capabilities than full fine-tuning did. That single finding replaces two overclaims at once. LoRA is not simply “as good as full fine-tuning,” and it is not simply “better at generalization” either. It trades some in-domain learning capacity for lower forgetting, and which side of that trade matters more depends entirely on the task.
What the PEFT Research Actually Tells Us Read in publication order, the core PEFT papers tell a coherent story rather than a pile of competing acronyms. Houlsby et al.’s 2019 adapter paper proved a frozen model plus a small trainable module could get within a fraction of a percent of full fine-tuning on GLUE. Li and Liang’s prefix tuning, and Lester et al.’s prompt tuning, both published around 2021, pushed that idea further, showing that training under 0.1% to a few percent of parameters could hold up, especially as model scale increased. Hu et al.’s LoRA paper turned a low-rank weight update into the field’s default baseline, reporting 10,000 times fewer trainable parameters and three times less GPU memory on GPT-3 175B. Dettmers et al.’s QLoRA paper then changed the memory equation entirely by quantizing the frozen base model itself, fitting a 65-billion-parameter fine-tuning job on one 48GB GPU.
Most of what followed, DoRA, AdaLoRA, LoRA+, VeRA, and LoftQ, targets a specific weakness in that baseline rather than replacing it outright. Xu et al.’s 2023 critical review is a useful check on all of it, since it groups the field into families precisely because method-to-method comparisons rarely hold up outside the exact setup a given paper used. The practical lesson from following this research is not “always use the newest method.” It is that every reported gain comes with a specific model, dataset, and hyperparameter search behind it, and a method that wins in a paper does not automatically win on your workload without your own benchmark to confirm it.
How to Fine-Tune an LLM with the Hugging Face peft Library Nearly every PEFT method described above ships as a ready configuration in Hugging Face’s open-source peft library, documented in the peft conceptual guide to LoRA and used together with Transformers and TRL. The workflow below is the current shape of that pipeline, not a conceptual outline.
Step 1: Pick an open-weight base model and clean the data. Current work targets decoder-only generative AI models such as Llama, Mistral, Qwen, or Gemma-class checkpoints, not 2019-era encoder models like BERT, RoBERTa, or XLNet. Data quality and formatting consistency in the fine-tuning set matter more than raw volume at this stage.Step 2: Load the model in 4-bit for QLoRA. A BitsAndBytesConfigprepare_model_for_kbit_training, loads the frozen base model in reduced precision; LoftQ-style initialization can be used here if the workload is expected to run at very low bit-widths.Step 3: Configure LoraConfig. This step, not knowledge distillation or pruning ratios, is the real PEFT configuration surface: rank (r), lora_alpha, target_modules (or the string "all-linear" to target every linear layer), lora_dropout, bias, and task_type. Calling print_trainable_parameters() after wrapping the model reports the exact trainable share, commonly under 1% of total parameters in the library’s own examples.Step 4: Train with TRL’s SFTTrainer. Passing peft_config=LoraConfig(...) directly into SFTTrainer runs standard supervised fine-tuning while only the adapter’s parameters actually update, which is the concrete mechanism behind the SFT-and-PEFT relationship described earlier in this guide.Step 5: Evaluate against the untuned base model. Compare the adapter’s outputs against the frozen base on the target task and, separately, on tasks the model should not have regressed on; skipping the out-of-domain check is one of the more common gaps in PEFT experiments.Step 6: Save the adapter, then decide whether to merge. The adapter saves as a small file independent of the base model. From there, merge_and_unload() folds it into the base weights for single-behavior deployment, or it stays separate for multi-adapter serving, covered next.Parameter Controls peft Default Common Starting Point Raise or Change When r (rank) Capacity of the low-rank update 8 8 to 16 for most tasks Task is complex or far from the base model’s existing skills and lower ranks underfit lora_alpha Scaling factor applied to the BA update 8 Often set equal to or double the rank Training is unstable or the adapter’s effect on outputs is too weak or too strong target_modules Which weight matrices get an adapter attached Auto-detected per model architecture, or “all-linear” Attention projections (q, k, v, o) as a first pass Standard attention-only targeting underperforms and feed-forward layers need adapting too lora_dropout Regularization on the adapter path 0.0 0.05 to 0.1 on small datasets The adapter overfits a small fine-tuning set bias Whether bias terms also get trained “none” “none” for most cases A BitFit-style combination is intentionally being tested alongside LoRA task_type Tells peft which model head and loss shape to expect No default, must be set explicitly CAUSAL_LM for most current LLM fine-tuning Working with a sequence classification or seq2seq head instead
Two library-level notes are worth flagging before running this in production. First, non-prompt-learning methods like LoRA, IA3, and AdaLoRA can now be attached and managed directly through Transformers’ own adapter integration in addition to the standalone peft API, while prompt-learning methods still need the peft library directly. Second, pin exact library versions, peft 0.21.0 on PyPI as of this writing, transformers, and trl, since adapter checkpoints are sensitive to the exact configuration and architecture mapping the training run used. Evaluating a fine-tuned model well is its own discipline; Kanerika’s LLM evaluation framework guide and its comparison of small and large language models are useful next reads once an adapter is trained, and the broader DeepSeek and MLOps guides cover the surrounding model and pipeline choices this workflow assumes.
White Paper
Navigating the Generative AI Maze
A practical guide for tech leaders on choosing between prompting, RAG, PEFT, and full fine-tuning for enterprise AI initiatives.
Download the White Paper → Loading, Merging and Serving Multiple Adapters in Production A trained adapter is not automatically ready for a multi-tenant or multi-task deployment. The peft developer guide to LoRA documents two different operations that are easy to confuse. merge_and_unload() returns a new model object with the adapter folded into the base weights and is not an in-place operation, while merge_adapter() and unmerge_adapter() toggle the merge on and off on the same model object, which is useful for testing merged versus unmerged behavior without reloading anything.
Multiple adapters can also be combined into one. The library’s add_weighted_adapter method takes several trained adapters, for example one trained with SFT and one trained with a preference-optimization method like DPO, and blends them with weights such as 0.7 and 0.3 under a combination_type="linear" setting, producing a single adapter that reflects both training signals.
For serving many adapters against one base model, vLLM’s LoRA support is the current standard pattern. It means enabling enable_lora, registering adapters through LoRARequest or the --lora-modules flag, and tuning max_loras, max_lora_rank, and max_cpu_loras to control how many adapters stay resident on GPU versus CPU at once. This is what makes the “one base model, many customers” pattern practical, since instead of hosting a separate full model per tenant, a single deployment of tools like vLLM can hold one base model and swap adapters per request. Teams comparing serving engines should also look at SGLang versus vLLM , at vLLM alternatives , and at vLLM versus Ollama before committing to a serving stack around adapter switching.
None of this removes the need to govern adapters the way any other model artifact gets governed. Every adapter should carry a record of the exact base model version, training data version, PEFT configuration, and evaluation results it was produced with, tracked in a registry such as those compared in Kanerika’s MLflow versus Hugging Face Hub versus Azure ML guide. Access control over who can deploy or swap an adapter, an evaluation gate before promotion, and a rollback path all belong in the same AI governance program that covers the rest of an LLM stack, routed and monitored the same way through an LLM gateway and LLMOps observability tooling. It bears repeating here that none of this shrinks the model that actually runs. A well-governed multi-adapter deployment still serves the full base model on every request; the adapters change its behavior, not its size.
Choosing a Method: PEFT vs Full Fine-Tuning vs RAG vs Prompting Four questions tend to settle which approach fits a given problem. Does the model need to change its behavior, tone, or output format, or does it need access to facts it was never trained on? Is the fine-tuning task close to something the base model already does reasonably well, or is it a genuinely hard new domain? How many distinct behaviors need to be supported from one deployment? And how often does the underlying knowledge change, since a model retrained every week to keep facts current is usually solving the wrong problem with the wrong tool.
PEFT and retrieval-augmented generation solve different problems and are not really competing options. PEFT changes what a model does with information it already has, adjusting its tone, its output structure, and its skill at a narrow task. RAG changes what information a model has access to at all, pulling in current or private content at query time without touching the model’s weights. A team that needs both, a model that behaves consistently in a specific style and also cites current internal documents, typically needs both techniques together, not a choice between them. Kanerika’s RAG vs fine-tuning decision guide covers that comparison in full detail and is the better read for teams still deciding between the two; this article stays focused on how PEFT itself works. For teams already leaning toward retrieval, Kanerika’s guides to advanced RAG and to context engineering versus prompt engineering cover the adjacent decisions worth making alongside it.
Main Need Prompting RAG PEFT Full Fine-Tuning Improve instructions or output format Best first option Not the right tool Useful if prompting alone is inconsistent Usually unnecessary here Add fresh or private facts Limited by context window Best fit Not the right tool alone Not the right tool alone Change behavior, style, or a narrow skill Partial, inconsistent at scale Not the right tool Strong fit, usually the starting point Consider if PEFT capacity falls short Learn a genuinely hard new domain with abundant data Weak fit Not the right tool alone Can underperform at standard ranks Best fit when data and budget allow Support many domain variants from one deployment Possible via prompt templates Possible via per-tenant indexes Strong fit via multiple adapters on one base Expensive, one model copy per variant
Common PEFT Mistakes and How to Avoid Them Starting rank too low and never testing higher. A rank of 4 or 8 chosen without any comparison can quietly underfit a task that would have worked well at 16 or 32; a short rank sweep is cheap relative to the cost of shipping an underpowered adapter.Leaving target_modules at whatever the library auto-detects. The default target modules are a reasonable starting point, not a guarantee of the best result; tasks that depend heavily on the feed-forward layers, not just attention, often need those layers targeted explicitly.Misunderstanding the alpha-to-rank relationship. Alpha scales the adapter’s effect on the output; changing rank without revisiting alpha can silently shrink or amplify the adapter’s influence in ways that look like a training bug.Overfitting to a small task-specific dataset. PEFT’s small parameter count reduces but does not eliminate overfitting risk, especially on a few hundred examples; dropout, a held-out validation set, and early stopping still matter.Skipping the untuned-base comparison. Without a baseline run of the same evaluation against the frozen base model, it is impossible to tell how much of a result actually came from the adapter.Following the “adapter should be about 10% of the model” rule. This figure has no basis in the PEFT literature and contradicts the entire point of the technique; well-performing adapters usually train under 1% to a few percent of parameters.Assuming PEFT shrinks the deployed model. The adapter file is small; the model that answers a request is still the full base model unless it has also been quantized or distilled separately.Letting evaluation or task data leak into the adapter’s training set. The same data hygiene that applies to full fine-tuning applies here: a held-out set that never touches training data is the only way to trust a reported gain.Most of these mistakes share a root cause, treating an adapter as too small to warrant the same rigor as a full model training run. Because PEFT experiments are cheap, teams sometimes skip the baseline comparisons and regression checks that would catch these issues, which is exactly backward given how easy those checks are to run once a base evaluation harness exists. The same discipline that prevents LLM hallucination in a production system also protects an adapter from quietly degrading a base model’s other skills, and the access-control and data-handling practices in Kanerika’s LLM security guide apply just as directly to a fine-tuning pipeline as to a serving one.
Where PEFT Pays Off in Enterprise Work The scenarios below are illustrative and hypothetical, meant to show where PEFT’s trade-offs line up with real constraints, not a record of a specific deployment.
Hypothetical: finance or legal document extraction. A team needs a model to pull structured fields out of contracts or filings in a specific house format. The task is narrow, the base model already reads English contracts reasonably well, and a LoRA adapter trained on a few thousand labeled documents can close the format gap without a full retraining run.Hypothetical: clinical text with PHI governance requirements. A healthcare team wants a model tuned to clinical note structure, but every training example touches protected health information. Training a small adapter on infrastructure the team already controls, rather than sending data to a third-party fine-tuning API, keeps that data inside an existing governance boundary.Hypothetical: multi-tenant SaaS with one adapter per customer. A vendor wants each customer’s model behavior tuned to that customer’s tone and terminology. Hosting one base model with a LoRA adapter per tenant, served through a router like vLLM’s LoRA support, avoids running a separate full model per customer.Hypothetical: speech or vision adaptation. LoRA-style adaptation is not limited to text models; the same low-rank update pattern is used on speech and vision-language models to adapt a large pretrained model to a narrower domain without retraining the full network.Hypothetical: private, on-premises deployments. A team running private LLMs inside its own infrastructure benefits from PEFT’s smaller compute footprint twice over: cheaper to train, and cheaper to keep multiple task-specific adapters ready without hosting multiple full models.How Kanerika Builds Custom LLM Solutions Kanerika’s AI engineering teams build and operate LLM and agent systems for enterprise clients, working across model selection, fine-tuning experiments, retrieval pipelines, and the MLOps infrastructure that keeps all of it monitored and governed after launch. That work spans LLM development , broader AI model engineering, RAG development , MLOps consulting , and AI governance , the same disciplines this guide has walked through for PEFT specifically. To be direct about what that experience does and does not cover here, none of Kanerika’s published case studies to date document a PEFT-specific deployment, and this section makes no claim otherwise. What it does reflect is delivery capability across the surrounding stack that a PEFT project depends on, model selection, evaluation, serving, and governance.
Kanerika holds ISO 27001 and ISO 27701 certifications and SOC 2 Type II attestation, is recognized as an OpenAI Select Partner, and builds AI systems on top of models including Anthropic’s Claude family. On the delivery side, LLM-based systems Kanerika has built for clients include a vendor agreement processing pipeline that cut manual review work by 82% and sped up vendor selection by 90%, and an LLM-powered ticket response system that auto-resolves 80% of tickets and reduced staffing cost by 70%. For an investment bank, Kanerika’s KlarityIQ and Karl systems delivered 43% faster information retrieval and a 35% efficiency gain in analyst workflows. These are LLM and AI engineering results, not PEFT case studies specifically, and they are presented here as evidence of delivery capability across the stack a PEFT project would run on, not as a claim that PEFT itself produced these numbers.
Teams weighing whether to run a PEFT experiment internally or bring in outside engineering support can find a broader framework for that decision in Kanerika’s guide to moving an AI pilot to production , which covers the same evaluation, governance, and operational questions this article has applied specifically to fine-tuning.
Talk to Kanerika
See PEFT and Fine-Tuning in Action
Talk to Kanerika’s AI engineering team about building and serving fine-tuned models for your enterprise use case.
Schedule a Demo → Conclusion LoRA is a sensible default starting point for most LLM fine-tuning work in 2026, not because it is the most sophisticated method available, but because it is well understood, mergeable, and cheap to experiment with. QLoRA earns its place when GPU memory during training is the actual blocking constraint, and DoRA, AdaLoRA, and the newer variants are worth testing only once a benchmark on your own workload shows plain LoRA leaving a specific gap open. Judge any PEFT experiment on quality, GPU memory, training cost, serving latency, and operational complexity together, not on trainable parameter count alone. Full fine-tuning remains the right answer when a task is hard enough that PEFT’s reduced capacity cannot reach the quality bar. And if the real problem is a model that needs access to information it was never trained on, that is a retrieval decision, not a fine-tuning one.
Frequently Asked Questions
What is parameter efficient fine tuning in simple terms? PEFT is a way to adapt a pretrained model to a new task by training only a small set of new or selected parameters while keeping the rest of the model frozen. Instead of updating every weight, methods like LoRA add a small trainable component alongside the model, which cuts training memory, storage, and time sharply compared with updating the entire model.
What is the difference between PEFT and LoRA? PEFT is the general category; LoRA is one specific method within it, in the reparameterization family. Other PEFT methods include adapters, prompt tuning, prefix tuning, and BitFit, each with a different mechanism. LoRA is simply the most widely used PEFT method today because it merges cleanly into the base model at deployment.
Is PEFT as accurate as full fine-tuning? Not always. A 2024 study found LoRA underperformed full fine-tuning on hard, out-of-domain tasks like programming and mathematics at standard ranks, though it also forgot less of the base model’s existing skills. Whether PEFT reaches full fine-tuning quality depends on the task, the rank chosen, and how far the target behavior is from what the base model already does.
Does LoRA add inference latency? No, once the adapter is merged into the base model’s weights, the LoRA paper reports the deployed model runs at the same latency as the original base model, since the architecture is identical after merging. Latency only appears if the adapter is kept unmerged to support switching between multiple tasks at serving time.
What is IA3 fine-tuning? IA3 stands for Infused Adapter by Inhibiting and Amplifying Inner Activations. It learns small vectors that rescale key, value, and feed-forward activations inside the model, rather than adding low-rank matrices like LoRA does. It typically trains fewer parameters than LoRA, not more, and does not update the entire model’s weights.
Is instruction tuning the same as PEFT? No. Instruction tuning, usually done through supervised fine-tuning (SFT), is a training objective describing what data and format you train on. PEFT describes which parameters get updated during that training. The two work together in practice: Hugging Face’s TRL SFTTrainer accepts a PEFT configuration directly, so SFT is often performed using LoRA rather than updating every weight.
Can PEFT be combined with RLHF or DPO? Yes. LoRA adapters can be trained through reinforcement learning from human feedback or direct preference optimization pipelines the same way they can through supervised fine-tuning. The peft library’s add_weighted_adapter function can even combine an SFT-trained adapter with a DPO-trained adapter into a single blended adapter.
How much GPU memory does QLoRA need? The QLoRA paper demonstrated fine-tuning a 65-billion-parameter model on a single 48GB GPU while matching full 16-bit fine-tuning performance, by quantizing the frozen base model to 4-bit precision. Exact memory needs scale with model size and sequence length, so that figure describes the paper’s specific benchmark rather than a universal number for every model.
What LoRA rank should I start with? A rank of 8 to 16 is a reasonable starting point for most tasks, matching common defaults in the Hugging Face peft library. From there, benchmark a higher rank, such as 32 or 64, against your specific task to see whether the added capacity actually improves results before committing to it in production.
Should I use PEFT or RAG? It depends on what is actually missing. PEFT changes how a model behaves, its tone, format, or narrow task skill, using data it can learn during training. RAG gives a model access to current or private facts at query time without changing its weights. Teams needing both a consistent behavior and access to changing information typically need both techniques together, not a choice between them.
