Waiting for the budget fine-tuning experiment cover

  • Date: 2026-10-03
  • For graduate students, early-career researchers and research engineers
  • Experiment range: real SmolLM2-135M-Instruct, SST-2; single seed, CPU short budget fine-tuning, no reservation confirmation set is used.

Abstract

Expanding the Training Set from eight to 32, the score remains unchanged, can you say "more data is useless"? First check whether the two groups have seen the same number of supervision signals. In this article, the fixed answer is supervised token and the number of update steps, and the eighty and thirty-two repetitions are repeated four rounds. The correct number of JSON joints in the two groups is 58/64, and the sample answers are the same; ordinary word answers have dropped from 57/64 before training to 33/64. This result does not support the benefits of unique sample additions, but reveals the trade-off between output conventions. What is really worth reproducing is the chain of evidence between budgets, samples and actions.

Contents

  1. Reproducing value and paper boundaries
  2. Core Idea: Counting Three Budgets
  3. A guide to the official implementation
  4. A minimal experiment and evaluation protocol
  5. Actual results and failure checks
  6. Code and instructions for reproducing the results
  7. Testable research questions
  8. Summary and References

Reproducing value and paper boundaries

Experiment 089 verified that the answer label mask was correct, but four updates per group did not teach the model the JSON format. Experiment 090 asks how to spend more training: add distinct examples or repeat existing ones? We held the model, template, optimizer, and development split fixed, varied the training pool size and number of repetitions, and recorded both the budget at each step and the output for each example.

Paper report: Muennighoff et al. in "Scaling Data-Constrained Language Models" study the duplicate data and calculation budget in the pre-training research, pointing out that the benefits of duplicate data cannot be extrapolated indefinitely. [1] This article cited its problem awareness, did not use its scale law coefficient, and did not reproduce the original experiment of the small emotion fine-tuning. The pre-trained full-text objective is significantly different from the task of supervising only eight answers to the token.

The actual verification this time: Continue to use the SmolLM2-135M-Instruct that has been trained in instructions, rather than the language model trained from zero. [2] SST-2 is the emotional data of English film reviews; the training line may also be a phrase fragment, and the thirty-two indexes are not equal to 32 independent film reviews, and they do not directly represent the Chinese ability. [3][4]

Core Idea: Counting Three Budgets

A token is the basic unit of the tokenizer to handle the text. SFT, that is, supervising fine-tuning, lets the model improve the probability of correct answers when given context. set in the batch ii Line tt The mask of the target is mitm_{it}, take one of the answer position, prompt and fill the position to take zero:

L=−∑i,tmitlog⁡pθ(xit∣xi,<t)∑i,tmit,Bsup=∑u,i,tmuit.L=-\frac{\sum_{i,t}m_{it}\log p_\theta(x_{it}\mid x_{i,<t})}{\sum_{i,t}m_{it}},\qquad B_{\mathrm{sup}}=\sum_{u,i,t}m_{uit}.

θ\theta are model parameters, xx is token, uu Iterate through update batches; BsupB_{\mathrm{sup}} Counting is the goal that really counts for the loss in all updates, including the ones that are repeatedly seen. When the label is handed to the model, it is not shifted in advance: the output of the current position predicts the next position, and the official loss function is responsible for the shift. [5]

Also remember BinB_{\mathrm{in}} Enter the total number of tokens for non-Padding, BpadB_{\mathrm{pad}} is the total number of actual batch tensor locations. padding is a placeholder that fills the sequence of different lengths of the same batch. The loss of only masking prompts will not eliminate its forward pass, so "waiting to supervise token" does not mean "waiting for all calculations". The so-called fairness in this article is limited to the same amount of supervision, the number of updates and the category of each batch; the difference in the input length must still be disclosed.

Nested training pool with the same supervisory budget

The two models in the figure are initialized from the same checkpoint, and the updated parameters are not shared during training. Budget brackets cover two groups, dotted lines enter the audit; the confirmation set is isolated. The figure is only shown in the main comparison, and the short training auxiliary group is shown in the table below.

A guide to the official implementation

Read the pinned version of the model first tokenizer_config.json, check the message template, the starting point of the answer, and the EOS, that is, the end tag; read the Transformers apply_chat_template, to confirm that there is no duplicate adding special token. This text common.py/build Check that the complete sequence is consistent with the generated prefix by ID, keep the real EOS at the end of the answer, and remove the additional newline of the template.

The second stop is LlamaForCausalLM.forward and ForCausalLMLoss. This batch input is [2,L] and output logits as [2,L,49152], the last dimension is the size of the vocabulary. Prompt and complements are labeled as -100, the model skips these targets; PAD and EOS are both ID 2 and must be distinguished by the true length, and all tags with values 2 cannot be deleted. The maximum difference between independent shift cross entropy and official loss is 1.19×10−71.19\times10^{-7}.

Finally read PyTorch SGD.step and generator argmax Branch: The former performs gradient updates, and the latter selects the token with the greatest probability at each step. The source map fixed the real submission, specific lines and licenses of Transformers 4.49.0 and PyTorch 2.6.0; the five official source codes have been re-downloaded, which is consistent with the installation file byte. Self-writing scheduling and auditing belong to the teaching implementation, and the official code is directly called by the model, loss, optimizer and decoding.

A minimal experiment and evaluation protocol

Both the model and tokenizer revision are 12fd25f77366fa6b3b4b768ec3050bf629380bac, the data revision is 8d51e7e4887a4caaa95b3fbebbf53c0490b58bbb. SHA-256 of seventeen resource files are written to the lock file; the cache identity also includes templates, prompts, decoding, splitting and executing source code hashes.

From the official train press SHA256('89:'+idx) Sixteen lines are selected in each of the positive and negative categories, and then the negative and positive alternating. The first four lines of each category form a small training pool, and the eight samples of 089 are just used, and the order is changed to balance each batch. Thirty-two lines are completely repeated without normalization, and do not completely coincide with all the development and confirmation samples; this inspection cannot rule out phrase correlation, near duplication or upstream pre-training pollution.

The development set follows the 64 articles of 088; the subsequent 128 articles confirm that the sample is not reasoned. The output target is unified as emotional JSON, and after the actual tokenization, each item is exactly eight supervised tokens, containing EOS. The maximum length is 256, and the official operation is not truncated, and there is no stitching and packaging of multiple samples.

Condition Unique training line number of rounds Update steps supervise token enter token
short training aids 8 4 16 256 2,736
Repeat the eight, the main control 8 16 64 1,024 10,944
Thirty-two expansions, the main control 32 4 64 1,024 10,556

The last three columns answer "update several times", "how many goals to learn" and "how many contexts to deal with". The main group has exposed 128 lines, two rows, and sixteen effective targets in each batch; but the positions with padding are 11,776 and 11,344, and the cost of attention shape is also different. This table cannot claim to be completely FLOPs.

It is also necessary to distinguish the unique number of rows and the amount of effective information. Although the two emotional tags are from different texts, most of the target answers are the same structure; repeated punctuation and key names are also included in the supervision volume. Therefore, this paper does not regard all 1024 goals as new emotional knowledge. If the answer is changed to a long explanation, the target length must be recalculated, and the budget cannot be continued by multiplying the number of samples.

All three groups were reloaded from the original weight, and all 134,515,008 parameters participated in SGD; the learning rate was 0.001, no momentum, no weight attenuation, and a gradient clipping threshold. Fixed SEED 90, CPU float32, four threads, Eager attention; eval The mode is used to turn off random modules, and automatic derivation is still reserved. The cyclic sequence does not reshuffle, do not stop early, and do not pick check points according to the development score. The short training group deliberately reduces the amount of supervision and is only used to observe the changes of the repeated training budget.

The primary indicator is JSON combined with Accuracy JJ: correctly formatted FF and the content agent is correct CC. The format requires only one Sentiment key and legal lowercase label; the content agent requires the only explicit emotional word matching data label in the answer, which is not an artificial semantic judgment. The common word convention is reported together as a diagnosis; all states are greedy to generate up to thirty-two new tokens.

Actual result: the total score is the same, but the cost cannot be omitted

Status The correct number of ordinary answers JSON format passes JSON content proxy is correct JSON joint is correct
Before training 57/64 0/64 50/64 0/64
short training aids 59/64 0/64 53/64 0/64
repeat eight 33/64 64/64 58/64 58/64
Thirty-two expansions 33/64 64/64 58/64 58/64

Not only are the total scores equal to the main group, but the 128 ordinary/JSON answer texts are also sampled one by one. The pairing of the main indicators wins, draws, and loses 0, 64, and 0. This only shows that there is no behavior difference observed at the current development point, and it is not proved that the parameters are the same, the overall equivalent or unique data is never used; single seeds and tiny training pools limit the conclusion.

Use the correct prefix to calculate the answer to the original small training pool, that is, NLL: 1.0283 before training, 0.6661 after short training, and the two main groups are 0.1613, 0.1720. It measures the probability that the correct answer is given by the model at a given history, not the success rate of free generation. Similar measurements on the 32 training pools also declined, but it contains training data and cannot be packaged as no generalization results.

Failure is more worth saving. The two main groups generate 62 negatives and two positives under the ordinary convention; from short training to repetition, the content is paired with five wins, twenty-eight draws, and 31 losses. The format is still completely legal, and the errors mainly fall on the label selection. Showing only the JSON success rate masks this convention sensitivity.

Troubleshooting and limits of the evidence

The step-by-step records confirm that each main condition used 64 steps and 1024 effective target tokens, with one positive and one negative example per batch. We also checked for leakage. The validation labels came from the official validation split, and no confirmation indices entered generation. We retained all 512 raw generations. The auditor decoded them again and independently parsed their formats, labels, and pairings. The scoring rule remained fixed even when the results were negative.

When running again, you need to check the model status, not just the score. All three final checkpoints are saved, the first layer of attention projection weight changes are non-zero, and the total number of parameters remains unchanged. Independent audit reconstructs duplicate schedules from the original training index to prevent two groups from misuse of the same sample. The model gives the same text in two groups, which only shows that the greedy decisions under these prompts are consistent, and it does not mean that the two weights are the same.

The author's inference: The same JSON skeleton may make a small amount of data enough to learn the output method, and repeated updates under one convention may change the label preference of another convention. This is one of the competition explanations; category word probabilities, hint sensitivity, and optimization paths are all likely to be involved. This time, no independent causal contribution was detected, and it is not directly called catastrophic forgetting.

The resource test run first executes two steps and two development samples in each group, and the peak RSS is 2.62 GB, and then the official operation is carried out after the actual estimate. Officially end-to-end 118.54 seconds, peak RSS 2.76 GB; three groups of training are about 5.45, 20.42, and 20.71 seconds respectively. Time is imported from a third party, including checking, loading, training, reasoning, NLL and saving; GPU memory is not measured. Loss of different batches cannot pretend to be a fixed Validation Set learning curve.

Code and instructions for reproducing the results

The source code has been uploaded to the public repository: Source tree at the pinned commit、README and run instructions、SOURCE_MAP official source code correspondence. The fixed version is 11c29a7deefe, please use the commit to reproduce to avoid subsequent master branch changes.

After installing fixed dependencies in README and running resource preparation, the minimum command is python run.py --cache ./cache --out smoke-new --private ./private-new --smoke. It is expected to get a step-by-step budget, original answers, indicators, environment and stage records; private catalogs save checkpoints and input tokens. Remove the test run parameters to perform all the experiments in this article, and then use audit.py independent verification. The downloader multiplexes the caching of the caching and does not require a large file to be re-downloaded.

SST-2 Data card license labeling UNKNOWN, publicly includes no original sentence, reversible input token or original data; model weights, environment and cache are not distributed. The source code, lock file, small derivation verification record and license are completely preserved, and the syntax and the decompression entrance are separately accepted; the cross-platform level is consistent with manual verification.

Testable research questions

The first hypothesis is "the current budget is the main format, and more unique texts have not changed the visible behavior". The counter-evidence can come from after the same supervision amount is frozen, and there is a stable one-case difference between the nested training and sampling seeds; the data cannot be continued until one wins.

The second hypothesis is "normal answer bias comes from training JSON conventions only". The next article can pre-fix the director volume, change the ratio of ordinary and JSON data, and measure the target format and reserve convention at the same time. If it is still biased after mixing, the simple format competition interpretation will be weakened. Multi-proportion is only selected in the development set, and the final confirmation that the data continues to be sealed.

Summary

This article completes a real, auditable but limited scope and other supervisory budget comparison. The unique sample is enlarged by quadruplets without changing the current output; increasing the repetitive training volume learns JSON, but damages the ordinary answer. The research progress does not have to be manifested as a method to win: to determine which budget is equal, which behavior is improved, and which ability is damaged, and only leave a starting point for subsequent rebuttal experiments.

References

Retrieve and direct access date: 2026-10-03. The paper report, official realization and this actual measurement are respectively marked; no dynamic list is cited.

  1. Niklas Muennighoff et al., 2023, Scaling Data-Constrained Language Models,arXiv v1; official project. Only as a source of problem for duplicate data studies.
  2. HuggingFaceTB, SmolLM2-135M-Instruct Fixed Model Card. Confirm model identity, instruction training and license.
  3. Richard Socher et al., EMNLP 2013, Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank. Data research background.
  4. StanfordNLP, SST-2 Fixed Data Card. Check division, label, language and license.
  5. Hugging Face, Transformers fixed causal loss;PyTorch, Fixed SGD implementation. Check the target shift, denominator and update.