Cover illustration

  • Date: 2026-10-28
  • For graduate students, early-career researchers and research engineers
  • Data deadline: 2026-10-28; search for work in the past 12 months, and retain the relevant classic research. This article is deeply read as evidence, not running the model experiment, and does not claim to be fully covered.

Summary: A path that leads to may have different origins

The agent continuously completes the search, selection, filling and submission, and it seems that the whole process is thought out first. However, the same success trajectory may also come from the best branch left after repeated attempts, or the environment prompts the next step every time you take it. The results are true, and the explanations on how the results are produced have not yet been determined.

This paper compares formal planning, language agent tree search and planning diagnosis research. The current evidence supports some models that can generate effective action sequences, and also supports external search to improve system performance; but only show the success trajectory, and it is impossible to determine that the benefits come from the model foresight, familiarity strategy, candidate screening or environmental feedback."Planning or search" is either: search can be the implementation method of planning, and what is really asked to ask is who is expressing the status, who is providing the future, and who bears the cost of failure. formal evaluation、Web tree search

Distinguish goals, means and evidence first

Planning is to find an action arrangement that can reach the target from the initial state; search is to expand, compare and discard between multiple candidate states or action sequences. The former describes the problem to be solved, and the latter describes a solution. Seeing the tree branch, you cannot deny the plan accordingly; when you see the natural language plan, you cannot confirm that the model has a reliable future deduction. concept background

Status means "what are there now and which conditions are true"; the transfer rule answers "what will change after action". For example, you must empty your hands before moving the box, and the original position will be vacated after moving the box. Here is a concept example, not the experimental results in the paper. If the model only repeats the action name, but forgets the conditions of the previous action change, it may still generate a plan that is smooth but cannot be executed.

There are also three types of "futures". model prediction It is to speculate on the successor status before the action; environmental testing It is to actually execute a candidate, and then read the return result; Symbol Simulation It is to calculate the successor state according to the given rule. They can all assist in selecting actions, but have different sources of information. External programs provide precise transfers, which does not mean that these transfers have also been independently grasped by the language model.

Therefore, there are at least three distinct interpretations of success: the model organizes actions in advance according to the status and constraints; the model calls the familiar policy template and adapts the current object; the system filters out errors through more branches and feedback. The three can coexist. The research task does not choose an answer based on the output style, but designs a test with different predictions after the design is changed.

What kind of evidence is there to generate an effective plan without tools?

Research on Corrêa, Pereira, and Seipp In the 2026 revision, the new PDDL task was used to evaluate the cutting-edge model. PDDL is a formal language describing action conditions, effects and goals. The model is tried once every question, and the tool is not enabled; the generated result is checked by the VAL validator for verification. Inputs also include cross-domain examples and prompt lists. In the standard task, Gemini 3.1 Pro solves 245/360 questions, and Scorpion Maidu solves 234 questions; after the symbol name is confused, the former solves 231 questions. §2, Table 1–2, Appendix B–C

This is more binding than showing a few beautiful conversations: the scheme must meet clear rules and test instances are newly generated. It supports the model with observable plan generation capabilities on these tasks, and can no longer generalize all subsequent models with the failure of the earlier model.

However, the paper measures the coverage of "find any effective plan", which is not the shortest plan guarantee; the model call and the resources of the classic planner are not allocated and other costs. The main text is referred to as the default API settings, and the appendices are additionally listed with sampling parameters. Therefore, this article uses the appendix to limit the protocol, and does not write the coverage lead as the efficiency leading.

Name confusion weakens the help of familiar words, but does not erase the task structure. The author also observed that the model can recognize some fields from the structure. This article inference is that the new example plus symbol renaming can reduce the credibility of some direct memory explanations, but it is not enough to exclude the structural solutions that have been learned. The domain name is not stated in the visible reasons, nor can it be proved that there is no use of the relevant priors internally. §2 "Information Leakage", Appendix B

This is not to belittle the achievement. Using learned algorithms to solve new instances is inherently valuable generalization; just "adapting policies on known structures" and "establishing available state models under new rules" are still two problems that need to be measured separately.

After joining the tree search, which system does the result belong to?

LATS combines language model generation actions, candidate evaluations, tree searches and feedback. Its WebShop sets 50 instructions to evaluate, and the upper limit of iteration of LATS, ReAct and Reflexion is 30; LATS extends multiple subnodes each time. The authors report that the search improves performance under this setting, and at the same time, it is clear that the decision task depends on the environment and can fall back. §4, §5.3, Appendix B

What needs to be restrained here is attribution. "Same 30 rounds" only aligns one unit of counting, and cannot automatically become the same number of calls, tokens, or observations. An iteration in the tree can generate and evaluate multiple candidates. All the upgrades of the system are called "the basic model is better planned", and the help provided by the control program and the environment will be missed; on the other hand, it is said that "it's just a search, so it's not a power", and it also ignores the role of candidate quality and evaluates the effect of quality on search.

Koh and others further search the actual reach status in the web environment, and use the model value function to select branches. The value function is a scoring rule for whether the candidate state is worth continuing. The corresponding baseline success rate of GPT-4o in VisualWerena is 18.9%, and after adding search, it is 26.4%; Backtracking is achieved by resetting the replay action sequence. §3–4, Table 2, Appendix A.4.1

The execution path is short, which does not mean that you only interact with the environment a few times. The final five steps may already have many probes, scores and replays. This result can prove that the search system is useful in this protocol, and it cannot prove that the follow-up status of these web pages is not required for the model to be probed.

The paper also lists irreversible actions as limitations. §5.4 For the researchers, this is not an accompanying deployment reminder, but a part of the task definition: the world that allows to revoke, the branch of failure is turned into information; in the world that cannot be revoked, the branch of failure may directly change whether the goal is still achievable. The "Search Budget" of the same name in the two environments may not buy the same ability.

If you take out the plan and score it, do you see the plan?

APB distinguishes generating a complete plan from predicting subsequent steps given feedback already available. It also tests redundant tools, faulty tools, and unsolvable tasks. A model judge uses reference answers to assess the results, without a PDDL-style formal proof. The published paper reports that accuracy fell when the models had to predict three steps ahead instead of one. The authors calibrated the judge against their own manual annotations, but some judging errors remain. §3, §5.2, Appendix D

This dismantling can help locate the behavioral difference that "will continue to take one step, but cannot stably arrange subsequent multi-steps". However, the multistep output also increases the need to be correct at the same time, and the decline cannot be attributed to a specific internal mechanism. The program is rated as reasonable and is diagnostic evidence; it is still not the same end point as the success of execution in the environment.

The APB also validates the purpose of the diagnostic revision in two controlled environments: the revision team starts from the same first draft and obtains a review feedback before execution. The reviewer can access hidden environmental supervision, and the feedback has anti-leakage constraints; therefore, this is a system control with additional diagnostic information, and cannot be regarded as a model to correct itself out of thin air. The paper also makes it clear that its framework does not model random transitions and updates of belief states. §5.1, Appendix C.10, Limitations

Several studies do not form two camps, "Word Planning" and "Unable to Plan". The objects they observe are different:

Entrance to Evidence critical condition what to observe Can't prove anything yet
formal plan Given a rule, one generation, after-fact verification Some models generate valid sequences Optimization, etc. Cost advantages, internal algorithms
LATS Multi-branch, feedback, retractable Improved performance of the measured system The proceeds all come from the base model
Web tree search Practical testing, scoring, and resetting replays gain after searching at the same point Equivalent benefits in irreversible environments
APB diagnostics Fixed task material, model review and controlled execution verification Program Defects and Revised Values Incorrect mechanism measurement, open world guarantee

How to make the two explanations give different predictions?

Evidence Framework for Planning and Searching

Figure: The original analysis framework, not the measurement results of the paper. The upper lane indicates the predicted consequences of the prediction, and the bottom lane represents the environmental detection, and both can enter the selection, execution and verification. The purple dotted line means only the proposed intervention; where the lines intersect are not additional connections. The internal deduction is not reversed through verification, and the check in the figure only means that the candidate is selected.

Below is a subsequent experimental design that has not yet been performed. Rather than collecting a batch of successful screenshots, it is more worth keeping the entire search process and making the source of information a controllable variable.

The first group: first promise prediction, then allow action. Fixed model, task and observation history, let it first write the prerequisites, subsequent status and subsequent dependencies of the candidate actions, and then query the environment after sealing. The control group was directly detected and then selected. Records which state the prediction errors are in, not just to see if it was successful at the end. If both groups are successful, but no detection groups have repeatedly predicted errors, then the system success cannot be attributed to the accurate prediction.

This design also has boundaries: requiring explicit write states may consume extra budget and change the way you answer. Therefore, the performance baseline that does not require status reports should be retained at the same time, to avoid directly interpreting "reporting in the specified format" as "no status representation".

Second group: Change the name separately and change the rules. First do the renaming of the symbol that keeps the transfer structure unchanged, and then only change the premise or effect of an action, and explicitly give the new rule to the model. The former tests the dependence on surface expression, and the latter tests whether to update the existing strategy. The task needs to be checked by the independent solver, and the development and evaluation sets are divided according to the structure, so that the neighbors of the same template cannot enter the two sides at the same time.

If the model can handle the new name, but sticks to the old steps under the new rules, it is more in line with the insufficient explanation of the policy migration; if it can correctly predict the consequences and adjust the action on a variety of unsister rules, it will add evidence of the derivation of transferable status. This is still behavioral support, not the only proof of internal implementation.

Group 3: Remove the right of backwards separately. Compare a plan, independent candidate screening, and step-by-step tree search in a fully reset simulated environment and a simulated environment with irreversible state changes. Irreversible conditions can be implemented with rules such as consuming unique props, and do not need to take risks on the real website. Each group records all environment queries, model tokens, evaluation calls, replay times and times.

The budget should be drawn as a relationship of performance changes with resources, rather than just reporting a "number of the same round" result; if all resources cannot be aligned at the same time, they will report their respective constraints separately. In addition to the final success rate, invalid actions, error stops and failure branches should also be reserved. In this way, we can see whether a method is to eliminate the bad plan earlier, or rely on more reversible trial and error remedies.

Summary: Write a competency statement where evidence can reach

The success trajectory indicates that the system has taken a road; the independently verified scheme further shows that this path meets the given rule. To judge whether the model has grasped the consequences of the action in advance, it is also necessary to predict the action, rule intervention and information budget control.

The evidence leaves room for genuine progress in planning, and the use of search does not by itself count against it. **Which part of the system becomes more reliable with particular state information, access to feedback, and opportunities to backtrack? How much of that ability survives when those conditions are removed?** Answering those questions lets evidence accumulate across demonstrations. Labeling a single success “real understanding” or “mere trial and error” tells us much less.

References

  1. Augusto B. Corrêa、André G. Pereira、Jendrik Seipp, Frontier Large Language Models Rival State-of-the-Art Planners, 2025 first draft, 2026 revision. arXiv V2 full text. References in preprints; official publication has not been verified. V1 titled The 2025 Planning Performance of Frontier Large Language Models, this article does not mix the results of the two versions.
  2. Andy Zhou、Kai Yan、Michal Shlapentokh-Rothman、Haohan Wang、Yu-Xiong Wang, Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models,ICML 2024. formal page; method details arXiv V3 full text, the official PDF was not successfully read this time.
  3. Jing Yu Koh、Stephen McAleer、Daniel Fried、Ruslan Salakhutdinov, Tree Search for Language Model Agents, TMLR 2025 (first draft of 2024). Journal of Journals Confirmation of publication; the experimental and restrictive basis of this paper 2026 arXiv v4, OpenReview official PDF access is blocked by browser authentication.
  4. Haoyu Sun、Wenxuan Wang、Mingyang Song、Jujie He、Weinan Zhang、Yang Liu、Yang Yang、Yu Cheng, Agent Planning Benchmark: A Diagnostic Framework for Planning Capabilities in LLM Agents,Findings of EMNLP 2026. formal page and Published paper. arXiv's latest V4 has not been successfully read, and the key conclusion of this article is returned to the official full-text verification.

Apps by the author

The following is the author's own product promotion. In addition to writing a technical blog, I also independently develop some small tools that make daily life more organized, welcome to understand:

  • Daily Expense Journal: Record the daily income and expenditure, arrange the budget and savings goals, and arrange a good place for each money. View on the App Store.
  • One Small Action: Choose small actions according to the current time and energy, record the feelings and discoveries after completion, and let the daily review start with a small thing. View on the App Store.

Find more articles and projects on my website: zffjzz.top, you are also welcome to share your experience.