In the previous few articles, we have moved forward this main line to this:
Why Transformer can become a large model infrastructure
What does a model learn during pretraining?
Why is it necessary to align training such as SFT, RLHF, and DPO?
How the long context is made
How to adjust the inference parameters such as temperature, top-k, top-p, repeat penalty
Next, we will enter a very "engineering" topic:
When you already have a base model in your hand, how should you transform it into a model that suits your task?
When many teams do large-model training for the first time, they are most likely to fall into two extremes:
Either think "Since you want to adjust, just refer to all fine-tuning"
Either think "LoRA must be the most cost-effective, all tasks should be LoRA"
But the real situation is not a choice, but a balance of resources, goals, risks and returns.
In this article, I want to make the three most common routes clear:
What kind of scenarios is suitable for Full Fine-tuning (Full Fine-tuning)
Why can LoRA use very few parameters to change the obvious effect?
Why does QLoRA become the mainstream solution in the scenario of single-machine resource restriction
And, more critical engineering issues:
What are the parameters of rank, alpha, dropout, and target modules
4bit / 8bit, gradient checkpointing, batch size, learning rate
How to choose single card, consumer grade graphics card, multi-card server
If you are preparing to do industry Fine-Tuning, private data adaptation, vertical task alignment or multimodal follow-up training, this article will be more useful than "memorize a few nouns".
1. First, let’s talk about the conclusion: The Fine-tuning plan is not selected according to the "advanced level", but according to the goal.
Correct a misunderstanding first.
It’s tempting to see LoRA, QLoRA, and full fine-tuning as successive upgrades, as if having enough resources automatically made training every parameter the right choice.
This is actually not right.
A more accurate understanding is:
They are model adaptation problems under different constraints.
You can remember this frame of judgment first:
If you mainly want the model to learn a certain output format, style, and domain question and answer habits, first look at LoRA / QLoRA
If you want to significantly change the boundaries of the model's ability, such as large-scale continuous pre-training, deep domain migration, and the structural offset of the model, consider the full reference of fine-tuning
If your bottleneck is first of all video memory, not training time, QLoRA is usually the most realistic starting point
If your data volume is very small, it is easier to overfitting and catastrophic forgetting
So the first question in the project is not "which method is the strongest", but:
Is this training doing ability injection, style alignment, or ability reshaping?
2. Full reference fine-tuning: the upper limit of effect is high, but the resources and risks are also the highest
The meaning of full reference to fine-tuning is very direct:
The trainable parameters in the model are all participating in the update.
For a 7B model, full fine-tuning means backpropagating through all roughly 7 billion parameters, storing their gradients, and maintaining optimizer state for them.
This leads directly to three results:
1) Advantages: the most complete expression ability
Because all layers can be updated, the model can adapt to the new distribution more thoroughly.
The scenarios it fits typically include:
Continued pretraining
Mass migration, such as general corpus cut to legal, financial, biomedicine
It is necessary to significantly change the internal representation of the model, not just the output behavior
A team with sufficient data, computing power and experimental budget
2) Disadvantages: video memory and training costs are very high
It is not only the model parameters themselves that really eat resources during training, but also:
Gradient
Optimizer State (especially Adam class optimizer)
activation
A very rough but engineering useful experience is:
The video memory usage of the full participation training is often several times that of the "model weight video memory".
Therefore, it is also a 7B model, and reasoning can run, which does not mean that training can also run.
3) Risk: It is easier to forget the original ability
If the data distribution is narrow, the learning rate is unstable, or the epoch is too large, all references are easy to appear:
The original general ability has declined
The output style narrows
Performance fallback for some basic tasks
This is why many business fine-tuning scenarios are not in a hurry to fully refer to them.
Because what you really want is often not "changing the model", but "adding a layer of controllable adaptation on the premise of not destroying the original model as much as possible".
3. LoRA: Why can we call the model out with a small number of parameters?
The core idea of LoRA is to write the original huge weight update into a low rank decomposition.
Intuitively, it can be understood as:
Instead of directly modifying the entire large matrix, learn a smaller, structurally limited increment.
The benefits of doing this are very clear:
Original model weights can be frozen
The amount of trainable parameters has dropped significantly
Significantly reduced memory pressure
You can save multiple sets of Adapters for different tasks, and the switching cost is very low
This is also the reason why LoRA can quickly become the mainstream method.
For many instruction-tuning, domain-specific question-answering, and output-format tasks, a few targeted updates to key projection layers suffice. You don’t need to rebuild the model’s internal knowledge from scratch.
LoRA fits the needs just right.
Where is the most common plug of LoRA?
In Transformer, common target modules include:
q_proj
k_proj
v_proj
o_proj
gate_proj
up_proj
down_proj
In many practices, you can already get good results by doing LoRA to Q/V or Q/K/V/O in attention.
If the task is more complex, or if you want the model to adapt to the new field more deeply, the FFN-related layer will be included together.
Experience:
Only hit the attention layer, the training is more economical, suitable as a starting point
Attention + FFN is stronger, but the amount of parameters and training costs will increase
4. QLoRA: Why is it especially suitable for the scene of "the graphics card is not luxurious but want to do fine-tuning" seriously
QLoRA can be seen as:
Quantified base model + LoRA adapter training.
Its key point is not to "quantize LoRA too", but:
The base model is loaded in 4bit to reduce memory usage
What is really updated during training is still LoRA Adapter
Through the paging optimizer, quantization weight return and other skills, the single machine can also train a larger model
What does this mean?
It means that only high-end multi-card machines can do fine-tuning, but now many consumer-grade GPUs can also start trying.
For example, in practical engineering, QLoRA is often used:
Single Card 24GB Graphics Fine-Tuning 7B / 13B Weight Model
Small teams can quickly verify whether industry data is effective
Iteration Prompt + Data + Adapter combination in low budget scenarios
But QLoRA is not without a price.
Its main cost is usually reflected in:
The training throughput is slower than pure FP16 LoRA
Stability is more dependent on implementation details under some extreme configurations
If the data is large and there are many training rounds, the final upper limit may not be better than a higher resource plan
So the essence of QLoRA is not the "strongest solution", but:
Under cost constraints, a cost-effective solution is very high.
5. What are the most critical parameters, what is the control?
When many people start training, the first confusion is these parameters. The following is a direct description of the meaning of the project.
1)rank(r)
rank determines the capacity of the LoRA increment matrix.
It can be understood as "how much space you allow Adapter to have".
Common experience:
r = 4 / 8: more resource-efficient, suitable for lightweight style alignment, format learning
R = 16 / 32: more common, suitable for most domain adaptation tasks
r = 64 and above: more expressive ability, but the risk of video memory, training time, and overfitting will increase
It's not that the bigger the RANK, the better.
If the amount of data is limited, too high rank tends to just make the model remember the training set faster, rather than better generalization.
2)alpha
Alpha can be understood as the scaling factor of the LoRA update amount.
The common practice is to make alpha and rank on the same scale, or slightly larger than the rank, such as:
r = 8, alpha = 16
r = 16, alpha = 32
It affects the strength of the Adapter update to the base.
If the training is very unstable, or there is obvious overfitting, you can look back together with the learning rate, instead of just staring at the loss.
3)LoRA dropout
Its function is similar to ordinary dropout, which is to prevent Adapter Overfitting .
Common intervals:
0: Common when the amount of data is large and the task is stable
0.05
0.1
If your sample size is small and the style is very concentrated, appropriate dropout is often more effective than blindly dropping the learning rate.
4)target modules
This is one of the most underestimated parameters that affect the training effect.
Which layers you add LoRA to determine essentially which capabilities the model is allowed to adapt to.
Experience suggests:
Start with the Attention projection layer
If it is found that model learning cannot be expressed in complex fields, expand to the MLP layer
Don't come up to "full-level random fight", first do a controllable experiment
5)learning rate
LoRA / QLoRA can often use a higher learning rate than all-parameters fine-tuning, but this is not an absolute rule.
Common starting points:
LoRA / QLoRA: 1E-4 to 2E-4 common
All-in-Tuning: 1e-5 to 5e-5 is more common
If the task is conservative, the data quality is average, and the sample size is not large, I would recommend starting with a smaller learning rate.
Because in the large model fine-tuning, the most expensive thing is not "more training for one more day", but the direction is found in the wrong direction after training.
6)batch size / gradient accumulation
What really matters is not the Micro Batch written in your configuration, but:
effective batch size.
That is:
When there is not enough video memory, the most common practice is not hard descending sequence length, but:
Reduce Micro Batch first
Then use the gradient accumulation to make up the valid batch back
This is one of the most practical tuning methods in stand-alone training.
6. How to choose resources: what are the suitable solutions for single card, multi-card and server
Here is a very practical engineering judgment.
1) Consumer-grade single card (such as about 24GB)
The more realistic starting point is usually:
7b magnitude model
QLoRA or LoRA
Turn on Gradient Checkpointing
Reasonable control Sequence length
This type of configuration is best for:
Industry Q&A adaptation
The internal knowledge-style migration of enterprises
Structured output optimization
2) Stand-alone multi-card / medium training resources
At this point you can start thinking:
LoRA with a higher rank
longer context training
Adapter Fine-Tuning of 13b or even larger models
If the target is business adaptation, the LoRA system is usually still more cost-effective.
3) High budget server cluster
Only when you meet the following things at the same time, all references to fine-tuning are more worthy of serious progress:
The amount of data is large enough
Clearly know that it is not just the output style to change
There is a complete evaluation system to monitor forgetting and degradation
Can bear multiple rounds of training and rollback costs
Otherwise, many so-called "more advanced" decisions will only make mistakes more expensive in the end.
7. Deployment perspective: the training method will directly affect the online method
This matter is often overlooked.
Many teams only look at the loss during training, not how to go online at the end, but only when they reach the deployment stage are they found that the plan is not smooth.
Deployment Benefits of LoRA / QLoRA
Adapter files are small for easy version management
The same base model can hang multiple sets of tasks Adapter
Suitable for grayscale, A/B and fast rollback
Friendly to multi-tenant or multi-business line scenarios
Deployment cost for full reference to fine-tuning
Each version is a full copy of the model
Higher storage and distribution costs
Rollback is heavier
Model governance and version auditing are more complex
If you are doing production services, not academic Benchmark, Deployment costs themselves are part of the training scheme choice.
8. The most common pit: it is not that the training cannot run, but the goal is not clearly defined
Finally, a few of the most common questions.
1) The amount of data is very small, but blindly pursue a big Rank
The result is usually the training set works well, and the Validation Set and the real online scene are a mess.
2) Just look at the loss, not the task indicators
In many SFT/LoRA tasks, the loss of loss does not mean that the structured output is more stable, and it does not mean that the fact is more consistent.
Be sure to check downstream indicators, such as:
Format Accuracy
fact consistency
command compliance
Whether the behavior of abstention is abnormal
3) Misjudging "insufficient domain knowledge" as "fine-tuning is not enough"
Some problems are actually more suitable for RAG supplement knowledge, rather than forcibly put new information into the model parameters by force-tuning.
If the knowledge update is frequent and the version changes rapidly, the search enhancement is preferred, and do not use fine-tuning as a database.
4) Regardless of the context length during training, long input is required during deployment
The distribution of training and deployment is too large, and the final effect is often unstable.
If the long document, long question and answer, and multiple rounds of dialogue will be processed when it is online, the corresponding distribution should be covered as much as possible during the training stage.
9. How to choose: give you a decision table that can be directly implemented
If you just want to get a practical conclusion first, you can directly judge this:
Prefer the case of LoRA
The goal is field adaptation, format learning, style adjustment
I hope that the training is fast, the deployment is light, and the version switching is convenient
Small data scale
Do not want to obviously destroy the general abilities of the base model
Prefer the case of QLoRA
Video memory tension
Only single card or limited GPU
Need to verify task feasibility under low budget
The training speed can be slightly slower, but the threshold must be lowered
Consider the case of full reference to fine-tuning
There is a large number of high-quality field data
The goal is deep capability migration, not just output alignment
Sufficient training resources and evaluation system
Can withstand higher deployment and rollback costs
If I had a universal advice, it would be:
90% of the business fine-tuning projects should first strengthen the baseline from LoRA / QLoRA, rather than do full fine-tuning.
Because what really determines the success or failure of a project is usually not "you use the heaviest training method", but:
Is the data clean
Is the task definition accurate?
Evaluate whether the loop is closed
Is it controllable online?
10. Conclusion: fine-tuning is not "more ruthless" than who, but "modified more accurately" than who
The most exciting thing in large-model training is the amount of parameters, the number of graphics cards and the length of training.
But what can really make a project is often not these surface indicators, but whether you figure it out:
Which abilities should be obtained by fine-tuning
What knowledge should be provided by RAG
What problems are actually the inference parameters and system design problems
LoRA, QLoRA, and full reference are not a struggle for faith.
They are just engineering tools under three different resource constraints.
Choosing the right tool is much more important than a heaper tool.
Next, I will continue to write down:
Large language model Deployment actual combat: how to choose FP16, INT8, 4bit quantization? Throughput, precision and real trade-off of video memory.
This article will include another practical problem after training:
The model can be transferred out, which does not mean that it can run cheap, stable and large-scale.
If you are preparing to make your own model adaptation, I hope this article can help you take the detour of "insufficient video memory, unstable effect, and troublesome online".