
- Date: 2026-10-02
- For graduate students, early-career researchers and research engineers
- Scope: SmolLM2-135M-Instruct, SST-2 English emotional classification; small budget training audit, not a complete benchmark score.
Summary: First prove that the loss does supervise the part you want to teach
Seeing the loss in the training log does not mean knowing what the model is learning. SFT is supervised fine-tuning, that is, update the existing model with the given input and target. This article narrows the question: After the chat template becomes a token sequence, is the loss covered by prompts, answers, end marks, or unintentional filling? Token refers to the discrete text unit generated by tokenizer, which does not necessarily correspond to a Chinese character or a complete word.
This time, two groups of real and full parameter updates were actually run, and each group of eight training samples and four steps. The mask and gradient check pass, but the number of JSON format success is still zero. This is not a failure that needs to be hidden: it separates the "training pipeline effective" from "behavior has improved", and also reminds us not to take a smaller loss as a better way.
1. Reproducing value: undertaking the failure of the format, rather than rebuilding the baseline
088 It has been found that the same model performs differently under the pure tag and JSON output convention. This article uses its models, tokenizers, prompts and scorers, and only adds a new supervision position comparison; the development and evaluation adopts the fixed first eight of the original 64 articles, and the remaining 128 confirmation samples continue not to reason. The scores of these eight items cannot be directly subtracted from the total score of the 64th article in the previous article.
The model is still an Instruct checkpoint that has been trained by official instructions. The model card indicates that its instructions are trained to use open and organized data, and are optimized by preferences; this article does not reproduce that complete process, nor does it claim to start with the basic model of fine-tuning. [1] The new training sample is from the official train division of the fixed version SST-2, and the original verification division is only used for behavioral observation. [2][3]
The real new product is not a beautiful curve, but supervises the ledger, step-by-step gradient, two final parameter status and original output before and after the update. Readers can ask: Has the first answer token been taught? Has the end mark been deleted by mistake? Are samples that are not answered still being sent to training?
2. Core thought: input is visible, does not mean directly included in the loss
Let the batch size be B, the sequence length after filling is t, and the shape of the model output fractional tensor Z is [B, T, V], and V is the size of the vocabulary. For the input sequence x and supervised mark m, the average cross entropy used is:
Here the position is numbered according to the zero start point, m is a indicating that the target participates in the supervision, and the zero means ignore; the target of the position t is predicted by the output of the previous position. The denominator is the number of tokens that are truly supervised, not the number of samples, nor is it the area of the rectangle after filling. When the denominator is zero, an error should be reported before entering the training.
The full-sequence group oversees all real positions after the first token; only the answer group only oversees the assistant's answer and its true end tag EOS. Both groups are kept as input as inputs, and the fill position is not supervised. There is a newline after the last EOS generated by the template. This article is clearly deleted in the two groups together, and this decision is not hidden in the data processing.
The label mask specifies which positions count toward the loss. The attention mask controls how positions such as padding participate in attention. Assigning the ignore value to prompt labels leaves the prompt in context, and gradients can still flow through its internal representations. The answer still depends on the preceding task instructions and film review. That distinction is what the later experiment tests.

3. Guide to the official implementation: follow the real execution link
First look at the tokenizer_config of the model. Its chat template is written to the start and end tags of the role, and the PAD filler shares ID 2 with the EOS terminator. [1] Therefore, the shortcut "whatever is equal to pad_token_id is set to -100": that will delete the real answer at the same time. We constructed the labels according to the true length and answer boundaries, and checked the counterexamples with the wrong method. Eight samples are missing one EOS that should be supervised.
Look at Transformers' apply_chat_template. The program renders the prompts with the starting point of the assistant, as well as the dialogue containing the complete answer, asserting that the former is indeed the prefix of the latter. [4] Use this boundary mask after passing, not guess the token number by the length of the string. This time, it is only proved that this single-wheel template is available; multi-round dialogue, boundary tokenization merge or tool messages need to be re-audited, and a fixed offset cannot be copied.
Then read the vocabulary projection and loss_function call of LlamaForCausalLM, and then enter the ForCausalLMLoss of the pinned commit. The official function has completed the label shift and sets the ignore value to -100. [4] The local direct transmission of unshifted tags, and at the same time independently calculate the cross entropy of "front t−1 logits to the back t−1 labels". If the caller is shifted in advance again, the model will be trained to predict the subsequent position of the wrong.
Finally, catch up with the parameter update of PyTorch SGD. [5] There is no local rewrite of the model mathematics; the four Transformers installation source code and SGD files are all the same bytes as fixed official submissions. Template, mask construction and auditing are independent teaching implementations, and the source map clearly distinguishes between direct calls, multiplexing author codes and custom rules.
4. Minimum experiment: first freeze the sample and budget, and then update the parameters
The training set is sorted according to SHA256('89:'+idx) to take the first eight, plus or minus four; this is a pre-registered selection rule, and the samples are not screened according to the training effect. The verification follows the order of the previous SHA256('88:'+idx) . The normalized text checks that the selected training samples were not found completely coincide with the development and reserved samples, but this does not rule out near-repetitive or upstream pre-training contamination. The SST-2 training data also includes movie review clips, and IDX cannot be interpreted as an independent and complete film review. [3]
Both runs started separately from the same pretrained weights, using identical full inputs and sample order, a batch size of two, four steps and a learning rate of 0.001. Both used SGD with no momentum or weight decay and a global gradient-norm clipping threshold of one. All 134515008 parameters were optimized. The model stayed in eval mode to disable stochastic layers, with automatic differentiation enabled. Calling eval does not disable gradients, and training was not wrapped in inference mode.
The device is CPU, Float32, and four threads, explicitly using Eager attention. First run two test runs in each group, one step, and then continue the four-step formal experiment according to the measured resources. The total test run time is 5.518 seconds, and the peak value is about 2.33 GB; the official total time is 9.943 seconds, and the peak is about 2.66 GB, both within the budget of 900 seconds and six GiB. The total time is calculated from the introduction of the third-party library, including resource verification, model loading, training, saving and evaluation; it is not a pure training speed. GPU memory is not measured.
5. This actual verification: the mask is correct, and the behavior can still not improve
The first batch of input shapes is [2,92], and logits are [2,92,49152]; the three axes are batch, position and vocabulary, respectively. The maximum absolute difference between independent cross entropy and official loss is 4.7684×10⁻⁷. The first answer target, real EOS and fill labels pass item by item; all logits corresponding to ignore the target are zero, but prompt the input of embedding is non-zero. The latter said that the prompt is still involved in the calculation of the answer, and the prompt cannot be announced according to the previous one "not been learned".
The projection matrix of the first layer of the two groups has undergone measurable changes, and the maximum absolute change amount is about 2.11×10⁻⁶ and 2.50×10⁻⁶ respectively. The final parameter status and hash has been saved locally. The small change proves that the optimizer has been executed, but cannot prove that the model has learned the task alone.
| The same group of eight development samples | Before update | Four steps of the full sequence | only answer four steps |
|---|---|---|---|
| Pure Label: Format Compliance / Tag Correct | 8 / 6 | 8 / 7 | 8 / 7 |
| JSON: Format compliance/tag is correct | 0 / 6 | 0 / 6 | 0 / 6 |
| JSON: Format and tag are correct at the same time | 0 | 0 | 0 |
The score is followed by F-format compliance, C uniquely explicit emotional word matching source tag, and J are satisfied at the same time. C is an automatic proxy, and there is no human semantic annotation. Forty-eight generations are reserved, all samples enter the denominator, and JSON is not automatically repaired. Both groups only changed the pure tag of IDX481 from Negative to Positive; the original answer of JSON remained unchanged. A single-sample change is not enough to claim the overall accuracy increase, and it cannot prove that the two supervisory equivalences.
6. Failure investigation: first check the denominator, and then explain the training phenomenon
Each group enters a total of 684 real tokens, but the whole sequence effectively supervises 676, and only 64 answers are answered. The first loss is about 3.1793 and 0.8639 respectively, the latter is smaller does not mean better training: the predicted target set and normalized denominator of the two are different. This is a supervised policy comparison of the synchronous number and input, not waiting for the performance race to supervise the token budget; all updates also trigger gradient clipping.
The four batches of samples are different, so the first and last differences of Loss cannot be regarded as the learning curve on the same batch. To measure whether the fixed target is fitted, a fixed probe should be set up and the evaluation method should be maintained; there is no by-select a subset that is easy to fall. The forward context is still the same, and the supervision token cannot be written as ten times less than the calculation cost.
The truncated audit gives a more direct failure: when the upper limit is 64, the answer targets of the eight samples disappear; at 96 o'clock, only one EOS is lost; 128 and 256 are lost. The formal training uses 256 and has no truncation, and it fails when it encounters a long answer. These results come from the real sequence location and are not artificially elongated presentation inputs.
The investigation sequence should therefore be the boundary of the character, the first answer alignment, the real EOS, padding, the effective denominator, and the finite gradient, and finally discuss the ability. The first public download is limited by the sandbox proxy connection, and then the normal download is passed and the fixed hash verification is passed; the data is not replaced or the failure is used as an experimental success. Code execution, data problems and model negative results need to be recorded separately.
7. Testable research problems: split the "unchanged" into a competitive explanation
Two explanations are possible. Four updates may have been too few to change the greedy output. Alternatively, target probabilities may have shifted under teacher forcing without crossing the decision boundary for the first freely generated token. Under teacher forcing, the model receives the correct prefix during training. In free generation, it produces its own prefix. We did not save the full distributions before and after training, so the experiment cannot distinguish these explanations or identify the mechanism.
In the next article, the data and repetitions of the same valid supervised token can be compared under the frozen development agreement, and the fixed target likelihood, the first generation token and f/c/j is registered. If the loss decreases but the format continues to fail, it is necessary to check the difference between training and generation conditions; if only repeating the same book can improve, it is necessary to leave a task instance verification memory interpretation. These are the follow-up experiments that are subject to manual verification, and do not promise to improve in advance. Confirmation sets do not participate in these options.
Code and instructions for reproducing the results
This article has been released in a fixed version: Full source code directory、README and run instructions、SOURCE_MAP official source map. Forty distribution files have been anonymously pulled and checked one by one SHA-256, and directories, descriptions, maps and entrances can be publicly accessible; the independent audit and real training test run of the open copy have passed, and the loss, gradient and output are consistent with the local. Links are pinned to the same commit and do not change with the master branch.
After a fixed dependency is installed and runs the downloader, the minimum command is python run.py --cache ./cache --out smoke-new --private ./private-new --smoke. It executes two real training samples, one step and two development samples of each set of two development samples to generate, produce loss, gradient, mask, output and checkpoint; delete the test run parameter can redo the formal experiment. Independent review use audit.py, Role and Error EOS mask counter-example use role_audit.py.
The actual download, preprocessing, two-group training, development generation and independent auditing are actually completed. Cache keys include models, tokenizers, data, templates, decoding, partitioning and executing code. The SST-2 card license is unknown, and the public package does not contain original movie reviews or reversible input tokens; these local audit materials, model weights and training checkpoints are not uploaded, and the reader re-acquires and runs from a fixed source.
Summary
The supervising location is part of the experimental hypothesis. The scoring responsibility, the meaning of each gradient and the original output of each token can be checked first, and then the training effect can be discussed. This minimal experiment leaves evidence that runs correctly but the format is not improved, and also provides a clear starting point for the next step to control the training budget.
References
Retrieval and actual access date: 2026-10-02. When the web page fails to obtain the official original text of the same fixed version, the access record is saved with the acceptance.
- HuggingFaceTB: SmolLM2-135M-Instruct Fixed Model Card; Fixed chat template with special symbols.
- Richard Socher、Alex Perelygin、Jean Wu、Jason Chuang、Christopher D. Manning、Andrew Ng、Christopher Potts,2013: Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank, EMNLP. This paper uses data and does not reproduce the paper tree model.
- Stanford NLP: SST-2 Fixed Data Card.
- Hugging Face,Transformers 4.49.0: chat template entry、Model loss call、Shift and ignore the original text of the source code.
- PyTorch 2.6.0: SGD Single Tensor Update.