Cover illustration

  • Date: 2026-10-27
  • For graduate students, early-career researchers and research engineers
  • Data deadline: 2026-10-27; search for nearly 12 months of work and retain the necessary classic research. This article is a deep reading of the literature, no model error correction experiment is run, and it does not claim to be fully covered.

Summary: The second answer, why is it more credible?

Let the model "check it again", it will often write a reflection and change the answer. The reader can easily understand this process as: the model has found a bug, so the error is corrected. But the same appearance may also come from resampling, hints in hints, or new information provided by the actuator. The answer has changed, and it can only prove that the modification has been made.

**The judgment of this article is: self-correction needs to simultaneously examine the ability of error identification, modification ability and retaining correct answers; the results of different feedback and training conditions cannot be combined into a sentence "reflection is effective" or "reflection is invalid". The main difference between CRITIC and SCoRe is the main difference between Self-Refine, Huang, etc., and firstly, it lies in the task and information conditions, not which camp they belong to. Related Definitions and Settings: Huang et al. §2–3.

The question in this article is how to decide whether to change it or not after one answer, without discussing the complete closed loop of reusing the self-generated materials for training, and not repeating the budget comparison of multi-agent division of labor.

"Change yourself" contains at least three things

inner self-correction It means that AT Inference Time has not received new search results, tool feedback or correctness labels, and the model will make a judgment based on the title and existing output. It does not mean that the model has never been trained in artificial data or rewards. external feedback This is the modification of the new observation, such as the execution error, web page evidence, or the label of "this answer is wrong".

Also distinguish Generate new candidates and Choose the final answer. Even if the second answer is more likely to be correct, the system still has to decide whether to replace the first one. Conversely, even if there are correct answers in the candidates, the system may not be able to recognize them. An evaluator who knows the standard answer and only allows the wrong question to enter the modification has already completed some judgments for the model; this setting of knowing the truth value is usually called Oracle.

Three competing explanations can help disentangle the mechanisms. First, computing the answer again or trying another approach may suffice, even without a written self-assessment. Second, the model may use its own critique to locate an error, in which case the content of the critique matters. Third, a new tool observation or correctness signal may change what information is available. All three can operate together. Separating their effects takes distinct interventions, and one successful conversation cannot tell us which dominates.

Intuitively, it is three different actions to re-transcription the problem-solving draft once, point out which step is not true, and use a calculator to check the intermediate results. In the end, you may get the correct answer, but the scientific research question is: Which of these actions is removed, and the improvement will disappear?

What seems to be an opposing paper?

Self-Refine: Improving the output is not equal to universally identifying inference errors

Self-Refine allows the same model to generate, feedback and modify in turn, without additional weights; feedback prompts include task demonstrations. Its tasks cover dialogue, code and constraint generation, and the evaluation has both task scores and preference judgments. In Table 1 of the official paper, the GPT-4 mathematical problem solving rate has changed from 92.9 to 93.1, which is not the same evidence as the substantial improvement of several generation tasks. Madaan et al., §2–3, Table 1.

Therefore, the average improvement of cross-task cannot be interpreted as "the model can reliably discover its own logical errors". On the other hand, the mathematical benefits are small and do not erase the value of improving expression or satisfying constraints. Appendix K.1 of the paper also lists the mathematical results using correct information separately, and cannot be mixed with ordinary self-feedback. What really needs to be unified here is to measure objects, not slogans.

Huang et al.: Stop the rules will secretly bring in the answer information

Huang et al. compared modifications with Oracle tags and no external feedback. The former is stopped, and the latter must be decided by itself. On its GSM8K subset of GPT-4, the standard prompt is 95.5%, and Oracle is modified to 97.5%; the first round of internal modification is 91.5%. These are the fixed settings of the paper at the time. GPT-4 uses 200 samples of sampling, which cannot be regarded as the general upper limit of today's model. §3.1–3.2, Table 2–3.

This control reveals that the model is not given a complete answer, and information that is sufficient to protect the correct solution may also be leaked. It does not prove that a re-examination must be harmful on all tasks. Whether the initial prompt is sufficient, and whether the modification prompt is implied "must be wrong", it should also be an experimental condition, rather than hidden behind the method name.

CRITIC: The tool provides new observations, but does not take over all judgments

CRITIC connects search, code interpreter, etc. to the criticism process. In the question and answer experiment, the ablation removes the search interface and allows the model to generate evidence by itself; the complete method performs better in the corresponding settings. It should be noted that ordinary CRITIC is separate from the Oracle version with an asterisk and only modifying the wrong sample. Gou et al., §3–4.1, Table 1.

These results support the value of external observations while leaving open how much the model gains from introspection. Search results still have to be checked for relevance, and a program can run successfully while modeling the wrong problem. Removing a tool changes both the information the model sees and the cost of interacting. It cannot isolate a single “capacity for reflection” under otherwise identical resources. Appendix A of the paper also documents limitations involving prompts, latency, and generalization across tasks.

SCoRe: Modified independently during the test, which can be the ability after training

SCoRe adopts two-stage reinforcement learning: first let the follow-up attempts to learn to improve, and then use rewards to shape and encourage effective error correction, and suppress the correct answer to correct mistakes. Training rewards come from math answer matching or code tests; test modification prompts do not reveal whether the first answer is correct. Kumar et al., §3, §5–6.

In the official paper table 2, the Gemini 1.5 Flash starting point model answered 52.6% and 41.4% twice, and the SCoRe after training was 60.0% and 64.4%. This is the evaluation left by MATH500; training to join the other 4500 questions in the original MATH Test Set. You can't take these numbers directly with the complete original test set elsewhere, and you can't mix the first answer before and after the training and the modification income. §6, Table 2.

The limited inference made in this paper is that "normal prompts are not stable to induce" does not mean "cannot be obtained". However, there is no need to feedback from the question after training, and there is no external supervision during the training process, which is a different proposition.

Route Conditions that need to be fixed first Observe what is mainly supported Still can't prove anything
Self-Refine Tasks, tips, preferences or correctness indicators Some outputs can be improved with feedback iteration All kinds of reasoning errors can be identified
Huang et al. Whether to use the true value to decide whether to stop Judgment and stop will change net income All models can never correct errors
CRITIC Tools, Demonstration, Normal or Oracle Mode External observations can help modify All gains come from internal reflection
SCoRe Training rewards, starting points and reserved divisions After special training, you can get modification benefits Unsupervised introspection or generic migration

These papers do not share the starting point, training data and total budget of the model. The table is used to compare the conditions of evidence and is not used to arrange who has the highest accuracy.

In addition to the total score, it depends on which answers have been changed.

In September 2026, Tianzhu Zhang's preprint further distinguishes between "wrong correction pairs" and "correction corrections", and studies when to call the modification process. This paper adopts its transfer statistical perspective, but does not regard the selector benefit as an established conclusion: the author admits that some historical results have not corresponded to the sample ID of the freeze configuration and non-overlapping samples, and the independent evaluation evidence is incomplete. §2.3、§4.2、§6.

For the same batch of questions, fixed parser and binary correctness judgment, the accuracy change satisfies:

ΔA=n错→对−n对→错N.\Delta A=\frac{n_{\mathrm{错\to对}}-n_{\mathrm{对\to错}}}{N}.

Among NN is the whole number of questions, two nn They are the number of questions for two types of transfers; this is the counting identities, not this model experiment. It shows that the same net gain can come from conservatively fix a small number of errors, or from a large amount of repair and a large amount of damage offset.

It should also be reported separately: how many of the initial error questions have been fixed, and how much is destroyed in the initial pair. If the denominator of the two is different, the percentage cannot be directly subtracted; when the initial sample of a certain type is empty, the corresponding condition rate should be recorded as undefined. The more reliable the first answer, the fewer the potentially repairable questions, and the slight disturbance to the correct answer is worth paying attention to; this is the risk brought by the change of the base, and the non-high-capacity model must not be suitable for modification.

This preprint also suggests another layer of boundaries: how to score missing answers, how to implement KEEP/CHANGE, and whether the reasoning is truncated by the budget may CHANGE the transfer statistics. §2.1–2.2. Therefore, "the final answer is better" still needs to be based on the same problem and analysis rules; we cannot leave only samples that can be parsed and changed to success.

A follow-up experiment to distinguish competing explanations

Self-correction control route and independent evaluation

Figure: This is an experimental framework proposed in this paper and has not yet been implemented. The three routes run separately, and the candidates are not combined into a stronger system; the resampling does not look at the old draft, and the self-assessment and the tool route receive the same old draft. The orange arrow indicates that an external observation has been added, and the right side is only responsible for leaving the evaluation and no feedback on the true value; the green and orange symbols represent the two types of results of repair and destruction, and do not indicate the measurement trend. The baseline that retains the original answer is not drawn separately.

Graduate students can first choose a small task with a clear grading rule, a fixed model version and a collection of questions, save the top and answer of each question, and then make three comparisons: re-answer without reading the old draft. The baseline that is not modified is also reserved. All routes use the same complete task description, avoiding the second round of replenishment requirements.

The first round of comparison should be fixed and the output rules should be fixed, and the new answers of each route should be accepted by default, and the changes in the generation stage should be measured first. In the second round, the selection rules of "reservation or replacement" are introduced, which are compared with always reserved and always replaced. If you use the truth value to pick a winner from the beginning, you can't judge what the model has learned.

Budgets should also be recorded in layers. The same number of calls does not represent the same number of tokens: criticism, historical context and tool return will consume resources. The results and actual delays of the fixed total token can be reported at the same time, and the external call fee can be listed separately; the samples that fail due to budget truncation must be retained. Mechanism comparison and efficiency comparison can be two experiments, and there is no need to pretend that a table has already controlled both.

To further test whether the criticism has a causal effect, you can design the correct criticism, criticism from other topics, and equal-length text comparison without diagnostic content at the development stage, and then freeze the design. If there is still a similar improvement in criticism, it will at least weaken the explanation of "the specific diagnostic content will lead to repair"; if only the feedback of the real tool is valid, you should continue to check whether the new information is sufficient and whether the wrong feedback will be misled in reverse. Any result will also require cross-question verification.

Finally, use the data development selection rules and evaluation rules that do not overlap each other, and save the reasons for transferring each question, original output, and parsing failure and stop. Hidden answers or tests required for scoring can only be used for final evaluations. When it is necessary to manually judge the correctness of the process, it should be completed by the real annotation, and the model's own approval cannot be used as a semantic gold label. These experiments are not carried out, and the above described are all verifiable designs.

Summary: Split "reflection" into measurable actions

The more secure conclusion is that rewriting may be useful, tool feedback may be useful, and special training may also make modifications more effective; they do not automatically prove the same intrinsic mechanism. When reading the paper, first confirm who provides the information, who decides to stop, who chooses the final answer, and then check whether the repair and damage are calculated.

What's next worth verifying is not whether the model will say "I'm wrong", but whether it can find an answer worth modifying when the budget and information are controlled, and keep the correct answer. This will turn a beautiful reflection into research propositions that can be supported or overturned.

References

  1. Aman Madaan et al., 2023, Self-Refine: Iterative Refinement with Self-Feedback. The official paper of NeurIPS 2023.
  2. Jie Huang et al., 2024, Large Language Models Cannot Self-Correct Reasoning Yet. ICLR 2024 official paper; the first preprint was published in 2023.
  3. Zhibin Gou et al., 2024, CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing. ICLR 2024 official paper.
  4. Aviral Kumar et al., 2025, Training Language Models to Self-Correct via Reinforcement Learning. ICLR 2025 official paper; first preprint was published in 2024.
  5. Tianzhu Zhang,2026, When Should LLMs Trust Their Own Revisions? A Risk-Aware Study of Intrinsic Self-Correction. arXiv: 2609.35832v1, 2026-09-23, preprint, no official employment is confirmed.

Apps by the author

The following is the author's own product promotion. In addition to writing a technical blog, I have also independently developed some daily gadgets, welcome to learn:

  • Daily Expense Journal: Record the daily income and expenditure, arrange the budget and savings goals, so that the whereabouts of each money is more clear. View on the App Store.
  • One Small Action: Select small actions according to the current time and energy, record the feelings after completion, and leave a little clue for daily actions and reviews. View on the App Store.

Find more articles and projects on my website: zffjzz.top. These works have no relationship with the papers cited in this article.