
The cover is a concept generated by AI, not the original diagram of the paper, the real device or the experimental results.
- Interpretation date: 2026-10-25
- Data deadline: 2026-10-25 20:29:31 (Beijing time)
Early on October 23, Beijing time, Mahdi Salmani and colleagues at the University of Southern California submitted their PKTP preprint on pruning reasoning paths. The model starts several attempts at a problem, then decides which to pursue. It chooses paths for the chance that at least one in the group will succeed. A path’s individual promise is only part of that decision. Given a limited compute budget, which lines of reasoning are worth keeping together? Paper and revision history
From picking top students to choosing complementary problem-solving groups
Let's take a look at the two existing research routes. A "probe" is used to read the internal state of the model to predict whether the intermediate answer is correct; the probe can be understood as an additional small discriminator, which is not equivalent to the confidence that the model expresses in natural language. Another one that encourages the exploration of a wider variety of paths, based on differences in internal representations. The two provide quality signals and diversity signals respectively. Probe Research Summary、RepExp Author Institution Introduction
You can understand the difference with a hypothetical example. Although the four answers are different, they have misused the same theorem; the fifth text is not so smooth, but it bypasses this mistake. Only sorted by the quality of the order, you may leave all the first four copies. Conversely, only the pursuit of differences, and the answer to the digression may be selected. This example is not the paper data, but only shows that the value of a set of answers cannot be simply added.
PKTP attempts to jointly estimate the success or failure of the path, and uses the internal representation to describe the association, and then select candidates one by one. The actual optimization is the approximation goal that is easy to calculate, so the title of "direct optimization" cannot be understood as the accurate probability of knowing the true success, or can always find the best combination in the world. methods part
How much question does this experiment answer?
The paper is evaluated mathematically on eight open weight inference models. The main setting is to continue from 128 paths, according to the first 4000 tokens, select 1, 2, 4, 8 or 16 to continue. The authors report that the advantages are relatively obvious when the baseline is only selected by quality or only by diversity. experimental part
Check the denominator. For each model, the researchers excluded questions answered correctly every time or incorrectly every time across 128 attempts. They used eight-fold cross-validation and Qwen3-32B to judge answer equivalence. The reported results apply to this filtered question set. They do not estimate accuracy on the full original bank. Data and verification caliber
My judgment is that this kind of screening is suitable for observing "how to choose when there is a choice", but it leaves another question: how many of the actual tasks cannot generate the correct candidate at all? If the failure mainly occurs at this step, no matter how detailed the selector is, it cannot make up for the answer out of thin air. In the future report, if the full number of questions and screening subsets are displayed at the same time, it is easier for researchers to judge which link the income comes from.
In the same way, cross-validation allows the selector to be checked on topics that are not used for its training, but the possibility that the relevant questions have been seen in the pre-training stage of the basic model is not excluded accordingly. The two training processes correspond to two different boundaries of evidence.
There are correct answers in the collection, and there is still a distance from delivering the correct answers
Pass@k measures here at least one of the final correct chances. For the researchers, it reflects the coverage of the candidate set; for those who really want to get an answer, there is still a lack of the final selection step.
The earlier Self-Consistency study connects "sampling multiple ideas" with "summary as answers", and selects the results through consistency between paths. However, consistency cannot be logically guaranteed to be true and correct: if multiple paths share the same error, most opinions may still be wrong. original paper abstract
Therefore, the subsequent inspection can report two results at the same time: whether there is a correct solution in the set, and whether the solution finally handed over is correct. If the former improves and the latter does not improve, it should continue to study and verify or summarize the links, and cannot directly declare that the end-to-end capability has improved. For scientific research, this is more critical, because many tasks do not have a ready-made standard answer for the system to check at any time.
Continue a few less, not automatically means lower total cost
Cutting the path can indeed reduce subsequent generation, but you have to pay for the previous work. Generate all prefixes, read internal states, train and run selectors, all part of the complete calculation process. Fair comparison requires clear opponents: do you count all the candidates, or only generate a few complete answers from the beginning? The cost baseline of the two is different.
This article also wants to see a complete cost-success rate curve: fixed total generated amount or actual calculation budget, and then compares random sampling, quality ordering, diversity selection and joint selection. If you increase the prefix length to make the choice more accurate, you should also show how much the overhead is increased, rather than just keep the side of the success rate.
This is also a detail worthy of advance agreement in the reproduction of scientific research. The GPU time, peak video memory and waiting time may not necessarily decrease simultaneously; whether the hidden state can be obtained efficiently will also affect the applicability of the method in different inference systems. What is proposed here is the follow-up evaluation requirements, and there is no measured saving ratio in this paper.
What evidence should we wait for next?
PKTP shifts the selection problem from picking the single most promising path to keeping several paths whose chances of success complement one another. The evidence so far comes from the math experiments in the authors’ preprint. This search found neither an independent replication of PKTP nor a clear link to the authors’ code, so broad validation remains an open question. Current paper entrance
Signals worthy of tracking include: public realization and problem-by-question results, migration of different fields, manual random checks on answer judgments, and comparisons after including all expenses. If this evidence can continue the benefits, the joint choice is more reasonable to become a reliable method of inference budget allocation. The more secure conclusion is that we have obtained a specific and verifiable selection principle, rather than the general trick of "making the model think less and must be smarter".
References and reading range
- Salmani et al.: Direct Pass@K Optimization for Thought Pruning, V1 was submitted at 2026-10-22 17:59:48 UTC, that is, Beijing time 10-23 01:59:48. Read summary and submission records; according to the introduction of the preprint, peer review and reception are not verified, and the date of occurrence of the experiment is unknown.
- PKTP v1 HTML body, read the introduction, method, experiment and conclusion, and check the definition of Appendix F to indicate the definition; did not review the appendix certificate or run the experiment item by item. It belongs to the same research as the previous entrance and cannot be regarded as an independent verification.
- Zhang et al.: Reasoning Models Know When They're Right, 2025-04-07 starter. Read summaries and submission records are only used to interpret the probe routes, and do not refer to their savings proportions.
- Microsoft Research:Representation-Based Exploration for Language Models, the institution page is marked with October 2025, ICLR 2026, the specific release date is unknown. The read author agency summary, as a background of diversity routes, is not an external reproduction of PKTP.
- Wang et al.: Self-Consistency improvements, 2022-03-21 first, the read page is V4 revised 2023-03-07, labeled ICLR 2023. Read summaries and version records are only used to differentiate between sample and answer summary.
Apps by the author
The following is the author's own product promotion. In addition to writing a technical blog, I have also independently developed two apps to help organize daily life. Welcome to understand:
- Daily Expense Journal: Record daily income and expenditure, arrange budget and savings goals, and make each account and plan more clear. View on the App Store.
- One Small Action: Choose small actions according to the current time and energy, record the feelings after completion, and leave a clue to the daily review. View on the App Store.
Find more articles and projects on my website: zffjzz.top. Your feedback helps me improve these tools.