
- Date: 2026-10-25
- For graduate students, early-career researchers and research engineers
- Data deadline: 2026-10-25; search for nearly 12 months of work, and retain the early research required for disputes. This article is deeply read as evidence, without executing model experiments, and does not claim to be fully covered.
Summary: The team won, who contributed the gain?
Let several models answer, discuss, and vote at the end, and the score exceeds one answer, which sounds a lot like collective wisdom. But this comparison also increases the chances of attempts, readable text and selected answers. Even if members never communicated, they may get benefits only by summarizing independent answers.
Comparing the four studies, this paper asks three distinguishable questions: Is it enough to use several more times, whether the exchange reasons are further effective, and whether the division of labor is reduced to reduce duplication of work? **Existing evidence supports the possibility of independent sampling as a strong baseline, and also retains the possibility that some communication and summary configurations are more effective; it is not yet possible to interpret the benefits of all teams as division of labor, let alone promote the use of non-profits on a certain benchmark as useless. See Choi et al.、Efficiency control see Wunderlich, etc..
Split "multi-agents" into three actions
The agents here can be different sessions of the same model, and do not necessarily have different weights or knowledge. self-consistency It is to independently generate multiple answers, and then vote according to the final answer; multi-agent debate Let members read peer output and revise; Division of work The problem is divided into complementary sub-tasks, and finally the results are synthesized. The three may be combined, but not the same variable.
You can use a hypothetical scenario to understand the difference: several assistants check the same paragraph, and then vote, which is a repeated attempt; exchange of the counter-examples found in each of them is the exchange of information; This example is not this experiment, and it is not guaranteed that the last one is the best. If the definition of the core is wrong, the subsequent derivation and conditional verification may also be jointly established on the wrong premise.
"Professional status" is not the same as professional ability. Label the same model as a mathematician, reviewer, and engineer, and only prove that the prompts are different; if you want to claim to form complementary expertise, you also need to observe their stable advantages in different sub-tasks. Conversely, models without role names may also obtain complementary information through tools or different evidence.
A successful trajectory gives us something to explain. Even when the final answer is correct, we still need to determine whether the agents corrected one another, followed the majority, or simply supplied a correct candidate that the aggregator happened to select.
What did the four studies compare?
How much has the vote taken?
The NeurIPS 2025 paper by Choi et al. separates the initial answer vote from the subsequent debate. The main comparison of five members was evaluated, and Qwen2.5-7B-Instruct and Llama3.1-8B-Instruct were evaluated on the same batch; the debate did not stabilize beyond the initial voting. The number of members is fixed here, and the saved debate budget is not redistributed to the voting group. It can show the optimal proof of the initial integration strong, but not all cost configuration. §2–3, Table 1
The paper also establishes an idealized belief update model. Its theorem requires that the neighbor's average correct answer belief is equal to the current member's belief, and the next round of expectations remains unchanged. The result of this premise cannot be rewritten as "any real language model discussion cannot be gained". Whether real communication can introduce verifiable new reasons is another empirical question. §4 Theorem 2
The configuration is appropriate, the communication may still be valuable
Wunderlich et al., in the ACL 2026 Student Research Workshop paper, compares Accuracy and cost for sampling, reflection, debate and layered summary. Layered summary is called MoA: multiple proposals are synthesized layer by layer, and finally the answer is output; it is not automatically equivalent to professional division of labor. In the main experiment, Llama 3.1 70b and a thousand sampling questions were used. In the tested MMLU-Pro configuration, the highest accuracy of sampling, debate and MoA was 68.7%, 70.0%, and 71.4% respectively. §3–4.1, Figure 2
The key limitation is: these points are Comparable Estimated Budget, it is not the exact same time-consuming time. The cost model is counted in floating-point operations and video memory handling, using specific quantization, batch processing and hardware assumptions; it is not an end-to-end delay in measured measurements. The paper also acknowledges the uncertainty of sampling. Therefore, the result provides support for some configurations, and it cannot be guaranteed that each difference is significant, or the sorting after the model change and hardware is unchanged. Appendix B, Limitations
The same number of generations does not mean the same bill
The preprints of Ferreira et al. in September 2026 allowed the three members to run three rounds each, and used nine independent samplings for control. The debate did not show additional stability in its small open weight model pool and measured discrete tasks. The matching mentioned in the text is first of all Generation times match; Reading companion text still brings additional input costs. The manuscript is annotated with ICLR 2027, which has not been verified and accepted. §3.2, §5.1, Table 1
A correction worth learning is that the discussion record of some long inference models is full of service context, resulting in the last round of empty output. After expanding the window and re-run the affected configuration, some of the original disadvantages are close to disappearing. This means that the failure may come from the implementation capacity. Its extreme character prompt is not equal to task expertise training, and it cannot use this result to deny all divisions of labor. §5.5、§7
Being able to answer questions does not mean being able to execute together
Kim et al. officially published in Nature Machine Intelligence's research, expanding the comparison to the task of requiring tool feedback. In the configuration of the six benchmarks and five types of architectures, financial analysis and sequential planning present collaborative effects in different directions. The official version of the description matches the system-level inference token budget limit and tool access, and the number of iterations is mainly the termination limit; the actual consumption may still be different. Main results、Methods
The authors attribute some of the differences to how readily a task can be split up. But switching benchmarks also changes the environment and the scoring method, so the comparison cannot isolate the effect of dividing the work. The paper explicitly limits what it can predict about absolute scores across domains, and some validation subsets contain only twenty questions. The evidence suggests that the benefit depends on the task. It gives no universal threshold for when collaboration will help. Robustness and sensitivity analysis、Discussion
| Research angles | control or cost caliber | main observation | cannot be launched directly |
|---|---|---|---|
| Initial Voting and Debating | fixed number of members | Initial integration is strong baseline | Discussion is never valid |
| Configuration efficiency comparison | Estimate calculation and transfer time | Some summary configurations dominate | Professional division of labor has been identified |
| Debate and Repeat Sampling | Fixed generation times | Unstable communication advantages | Total Token, the same delay |
| tool interaction architecture | system budget cap | The effect of the task is different | Unification of the new field |
Each row in the table corresponds to the above papers; the task, model starting point, sampling and scoring are different, and the score ranking of cross-paper is not made. The first item is the official paper of NeurIPS, the second item is the paper of the ACL Student Research Workshop, the third item is still a preprint, and the fourth item is a formal journal version.
Disputes really fall on "budget" and "information"
These studies do not constitute a simple front and back camp. The number of fixed members, the question is what happens after the existing team discusses; the fixed number of generations, the question is how to allocate the call opportunity; the fixed system budget limit, the question is what architecture is effective when the resource is limited; compromise with cost. Different problems, the conclusions can be established at the same time.
The cutting-edge efficiency can be intuitively understood as: in the tested scheme, there is no other solution that is cheaper and more accurate at the same time. But it relies on which configurations have been searched. If only the team is carefully adjusted, but only one attempt is given to the single unit, the frontier will be biased towards the team; if it is only tested for a cumbersome discussion protocol, it cannot represent the entire communication mechanism. The configuration selection itself should use the development set, then freeze the scheme and compare on the standalone test set.
Costs should at least be differentiated and read. Multiple members re-read the full history each time, even if the final output is equally short, it may consume more resources. Parallel execution can shorten the wait time, but does not necessarily reduce the total calculation; the same number of tokens may not correspond to the same cost across different models. Researchers should first explain that they limit the number of calls, total computing power, delay or spend, and avoid the conclusion of the sliding between these meanings.
Information also needs to be accounted. If the team takes more databases, additional search results, or stronger referees, it wins a system that contains these resources. To ask "whether the communication itself is valid", the control must have the opportunity to obtain the same evidence. To ask "whether the division of labor is effective", it is necessary to further control the total amount of evidence, and only change who sees what first, how to assign tasks and synthesis answers.
A follow-up experiment to distinguish competing explanations

The picture shows the comparison framework proposed in this article, which has not been implemented. The solid line represents the preset information flow, and the dotted line connects the common observation indicators; the three routes are not the causal mechanisms that have been proved, and do not imply which one wins. The top resource matching must be implemented as a clear budget rule, and additional baseline for the monomer is reserved.
You can start with a type of task that can be verified and split into sub-questions, and first fix the model checkpoint, problem division, evidence pool, tool permissions, decoding and scoring. Save a set of initial candidates, and then construct three routes: vote after independent sampling; revise each other after reading the same candidates; process and synthesize after processing according to the pre-defined complementary sub-tasks. In addition, give the same total resources to the monomer, and let it plan, inspect and revise it on its own.
It is best to do two experiments separately here. Mechanism Reuse the same batch of initial candidates to study what the exchange has changed; Efficiency experiment Allows each program to allocate sampling and exchange independently within the same total budget. The former is more convenient for attribution, but does not represent the optimal configuration; the latter is closer to the resource trade-off, but cannot rely solely on the total score positioning mechanism. Mixing the two into a list will re-introduce the original problem.
Then do targeted interventions. For the debate group, compare the real reasons, only show the answers, and the reasons that are similar in length but are not related to the current question. If only the real reason produces additional error correction, can the content exchange be more supported and contributed; if only the majority of the answers are also effective, the explanation of the answer should be considered aggregation or herd. Irrelevant reasons may cause additional interference, so it is still necessary to keep a completely communication-free control, and not only "real" and "out of order".
For the division of labor, keep the evidence unchanged, compare the assignments, random assignments, and each person to see all the evidence. The same decomposition scheme should also be allowed to perform the same decomposition plan: otherwise the team advantage may come from better task descriptions, not multisubjective collaboration. Deleting the name of the character but keeping the actual division of labor can also check the impact of the name and the work content. The allocation rules here should be determined before looking at the test answers.
In addition to the final Accuracy, it is also necessary to keep the track-by-question track: whether the initial candidate has the correct answer, which errors have been corrected, which correct answers have been corrected, whether the members repeatedly check the same evidence, and whether the key constraints are lost when synthesizing. The consistency rate of answers can only measure consensus and cannot replace correctness. The correction and destruction are counted separately, so that the average increase will not make the average increase mask the specific type of damage.
Finally, the processing methods of resource depletion, tool failure, empty answer and context overflow are prescribed in advance, and the stability is checked across random seeds. If you have already seen a batch of questions when you choose a configuration, you no longer regard it as an independent validation set . These are future experimental designs; this article does not run these routes, nor does it produce new Accuracy .
Summary: From "many people" to a rebuttal explanation
To judge the research value of multiple agents, there is no need to believe in the superiority of the team, nor to deny the cooperation first. The more useful sequence is: first prove that the income exceeds independent sampling, then check whether the real information exchange contributes error correction, and finally verify whether the task allocation reduces duplication and reserve constraints under the same evidence and resources.
A more secure conclusion is that, Sampling, communication, summarization and division of labor need to provide evidence separately. A system is indeed more accurate and can be a valuable result; only by allowing the competitive interpretation to accept the same strict control can we further answer why it is more accurate and when the benefits will disappear.
References
- Hyeong Kyu Choi、Xiaojin Zhu、Sharon Li(2025), Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?. NeurIPS 2025 official version.
- Florian Valentin Wunderlich、Lars Benedikt Kaesberg、Jan Philip Wahle、Terry Ruas、Bela Gipp(2026), Multi-Agent Reasoning Improves Compute Efficiency: Pareto-Optimal Test-Time Scaling.ACL 2026 Student Research Workshop; Published paper.
- Leonardo Ferreira、Gardenia Liu、Kaden Zheng(2026), Beyond Symmetric Agents: Cognitive Diversity and Multi-Agent Debate in Small Language Models. arXiv V1, 2026-09-26; submission status is not equal to employment.
- Yubin Kim et al. (2026), Capable language models can outgrow the benefits of collaboration. Nature Machine Intelligence 8, 1157–1172; early preprint title is Towards a Science of Scaling Agent Systems. Author Code maintenance record Indicates that the analysis script has corrected the default configuration set; this article does not run its code, and does not assume the published value is called this recurrence.
Apps by the author
The following is the author's own product promotion. In addition to writing a technical blog, I have also independently developed two small tools that make everyday life more organized. Welcome to understand:
- Daily Expense Journal: Record the daily income and expenditure, arrange the budget and savings goals, so that the whereabouts of each money is more clear. View on the App Store.
- One Small Action: Select small actions according to the current time and energy, record the feelings and discoveries after completion, and leave a clue for the daily review. View on the App Store.
Find more articles and projects on my website: zffjzz.top. Welcome to share the experience of use, help these gadgets to improve slowly.