Cover illustration

Date: 2026-06-23
Suitable for readers: graduate students, LLM/Agent researcher, engineering reader who is calling the system of construction tools

Abstract

In the past year, the main line of Tool Learning has advanced from "Let the model call functions" to "let the model complete tasks stably in multiple rounds". Early Work Concerned about API selection, parameter padding, and formatting correctness; the focus of the past year has shifted to interactive status, user collaboration, description of long context tools, dynamic Tool collection, RL training, and security boundaries. In this paper, the representative papers and benchmarks are sorted according to the research route: the classical foundation includes ReAct, Toolformer, API-Bank, Gorilla, ToolLLM; the focus of the past year includes BFCL V4, ToolSandbox, tau^2-bench, ASTRA-bench, ComplexFuncBench, ACEBench, RAGEN, EGPO/RC-GRPO, EigenData and MCP Safety related research. The engineering conclusion is very direct: the Tool call is not a "prompt skill", but a set of distributed system interfaces that can be evaluated, auditable and rollable.

Contents

  • Research background: Why the tool call becomes the core problem again
  • Nearly One Year Roadmap: From Schema Matching to Interactive Agent
  • Key papers by research question
  • Comparing the methods
  • Technical trends
  • Implications for deployment
  • Limitations and open questions
  • Questions to watch
  • References

Research background: Why the tool call becomes the core problem again

LLM itself is good at language modeling, but real applications often require external capabilities: checking databases, calling business APIs, writing tickets, running code, retrieving files, browsing web pages, and operating UI. The tool call rewrites "answer" into a closed loop of "select tools, generate parameters, perform actions, read results, and continue to make decisions". OpenAI's Function Calling / Tools document also puts functions, built-in tools, remote MCP, and tool search in the semantics of the same set of tools: the model generates Tool Call, the application executes and returns the tool output, and then the model continues to generate a reply or the next call.

This brings two research difficulties. First, the output of the language model is no longer just a natural language, but a structured action with side effects; any parameter error may lead to changes in the state of the real system. Second, the success of the Agent depends not only on whether a single call is correct, but also on multiple rounds of status tracking, error recovery, user clarification, permission control, and execution environment. That said, the problem boundaries of Tool Learning are expanding from NLP Benchmark to software engineering systems.

Tool Learning Roadmap

roadmap for the past year

Stage 1: Use the Prompt / ReAct tool. ReAct alternates the Reasoning trace and action, allowing the model to invoke the external environment while inferring. Its importance is not in a fixed Prompt template, but in the arrangement of "thinking" and "action" into a trajectory. For today's Agent system, ReAct is still the most common execution loop prototype: planning, invoking, observing, and correcting.

Stage 2: Supervised tool learning. Toolformer uses a small number of examples and self-supervised screening training models to decide when to call the API, what parameters to pass, and how to use the results. API-Bank, Gorilla, and ToolLLM extend the research focus to the real API collection, tool retrieval, call format, and training data structure. The core problem at this stage is whether the model can map the natural language intent to the correct tools and parameters.

Stage 3: Interactive and state-based evaluation. The Benchmark of 2024 to 2026 is obviously no longer satisfied with whether a single round of JSON matches. ToolSandbox introduces stateful tool execution, implicit state dependencies and user simulator; tau^2-bench model users as participants who can use tools; ASTRA-bench Emphasizing personal context, time evolution events and complex intentions; the official page of BFCL v4 also clearly extends the direction to Holistic Agentic Evaluation, and the last update of the label list is 2026-04-12.

Stage 4: RL and process rewards. In the past year, I have regarded Tool Calling as a strategy problem that can be reinforced learning. RAGEN focuses on track-level training, exploration and reward instability in multiple rounds of Agent RL; EGPO attempts to train a more robust function calling with exploratory enhanced GRPO; for RC-GRPO Reward-Conditioned Trajectory Policy improves the diversity of exploration within the group of Tool Calling; ToolPRM combines the Inference Scaling called by the structured tool with the process reward model. The common judgment here is that SFT alone is not enough to cover failed recovery in interactive environments.

Key papers by research question

1. Basic paradigm: from language generation to action trajectory

ReAct provides the minimum available Agent execution loop: the model explicitly generates inference trajectories and interspersed actions. It is suitable for tasks with few tools and clear environmental feedback; the disadvantage is that the intermediate inference exposed by Prompt is not necessarily suitable for production systems, and it is also easy to mix "interpretative text" with "executable actions".

The contribution of Toolformer is to turn tool calls into learnable token-level behaviors: the model not only needs to know the answer, but also determines when to use calculators, search, translation, calendar and other tools. It proves that the tool use can be internalized into the model with data filtering and training. But today, its environment is still relatively ideal: the tool collection is fixed, the call risk is low, and the interaction depth is limited.

2. API scale: retrieval, routing and parameter generation

API-Bank constructs a runtime API environment and tool use dialogue, and proposes the ability to divide and divide, such as planning, retrieval, and invocation. Gorilla emphasizes large-scale API connections and reduces API Hallucination; ToolLLM/ToolBench advances to 16000+ real API Instruction tuning and evaluation. They have established the three-piece set of today's Function Calling model training: tool description corpus, tool retriever, and call trajectory data.

In engineering, this route reminds us not to put all tools into the context at once. The official tool documentation also emphasizes that too many initial exposure functions will increase the difficulty of token and selection. It is recommended to use a smaller initial tool set, clear schema, strict mode, and tool search / allowed tools and other mechanisms to control the tool control tool.

3. Benchmark migration: from AST to task completion

Early versions of BFCL assessed function calls using abstract syntax trees. The official V4 page describes a shift from tool use to agent evaluation that incorporates real-world data, periodic updates, cost, and latency. ToolSandbox emphasizes state dependencies and on-policy evaluation through dialogue. ACEBench separates ordinary, special, and agent scenarios to analyze distinct error types. ComplexFuncBench focuses on long contexts, multi-step constraints, and argument filling. IFEval-FC evaluates formatting instructions in parameter descriptions. In production, even a correct argument value can cause a failure if it is in the wrong format.

This transformation is critical: a single step Schema Accuracy can only prove that the model will "write out a function call", not to prove that it can complete the task when the business state changes, user supplementary information, permission restrictions and tool failure.

4. Interactive Agent: The user is no longer a passive input box

The Dual-Control design of tau^2-bench is very representative: in real customer service, IT support, and operation systems, users may not only answer questions, but may also modify the system status by themselves. Agents must coordinate user actions with tool actions, explain the next step, and avoid failure due to state competition. ASTRA-bench puts personal contexts, long-term events and complex references into the evaluation, and lets the tool call into the "personal assistant" scenario.

The inspiration of this type of Benchmark to the engineering architecture is: Agent Memory, user confirmation, status lock, transaction boundary, and observation refresh, which is equally more important than the function-call accuracy of a single model.

5. RL tool strategy: let the model learn to explore and restore

Supervised fine-tuning (SFT) tends to imitate training trajectories, which suits stable APIs and single-turn tasks. Multi-turn tool environments introduce sparse rewards, error propagation, and costly exploration. RAGEN studies instability in multi-turn agent RL and finds that without fine-grained rewards that account for reasoning, models may learn superficial strategies. EGPO, RC-GRPO, and R2IF explore ways to adapt GRPO-style training to function calling. Some modify advantage estimates, some control trajectory quality, and others align the reasoning process with decisions about when to call a function.

These results still need to be viewed with caution: many of the papermarks reported by the papers rely on specific model, data and evaluation implementations; whether it can be generalized to real business APIs still needs to reproduce experiments and online A/B verification. This article does not regard the SOTA ranking claimed by the paper as a stable fact, and the relevant figures are recommended to be re-checked with the current page of the paper version with the current page of the paper.

Comparing the methods

Route Representative work What is the main solution Engineering heuristics
Prompt / ReAct ReAct Let reasoning and action alternate Suitable for prototypes; separation of executable actions and interpreted texts to be separated in production
Self-Supervision Tools Learning Toolformer Learn when to call tools and how to use results Extend training data with a small number of seed demonstrations
API retrieval with SFT API-Bank, Gorilla, ToolLLM Large-scale tool selection, parameter generation, API illusion Tool description quality and search recall decision upper limit
Function calling benchmark BFCL, ACEBench, ComplexFuncBench, IFEval-FC Structured calls, formats, long contexts, multistep constraints Not only the Exact Match is evaluated, but also the post-execution status
Stateful / conversational eval ToolSandbox, tau^2-Bench, ASTRA-bench Status dependencies, user collaboration, personal context Transactions, rollbacks, user acknowledgments and audit logs are required
RL tool policy RAGEN, EGPO, RC-GRPO, ToolPRM Multiple rounds of exploration, sparse rewards, process supervision Reward design and environment simulation are more important than algorithm names

Technical trends

First, the tool schema is from "document" to "control plane". Function name, parameter enumeration, required field, additionalProperties, strict pattern, and format requirements in the parameter description will directly affect the model behavior. A good schema should make the illegal state as inexpressible as possible, rather than expecting the model to remedy it with common sense.

Second, the number of tools has become a context engineering problem. A large number of APIs will increase selection confusion, token cost and delay. Tool Search, namespace, layered routing, task-based Allowed Tools, and cache tools are becoming the basic capabilities of Agent Runtime.

Third, the evaluation shifts from offline answers to executable environments. In the future, the more valuable BenchMark will record the database status, environment changes, user simulation trajectory, and failure recovery process, instead of just looking at whether the model has generated a reference JSON.

Fourth, security research began to catch up with the protocol ecology. MCP makes tool access more standardized, but also expands the risk of tool permissions, prompt injection, cross-tool data leaks, and malicious server descriptions. Papers such as MCP Safety Audit remind us that the tool protocol is not a security boundary, but permission, isolation, approval and observability.

Implications for deployment

  1. Use tool calls as RPC system design. Each tool requires schema, idempotent description, error code, permission model, timeout, retry and audit fields.
  2. Do the gadgets first, and then do dynamic discovery. A small number of high-confidence tools are exposed by default; the long tail tool is delayed by retrieval or tool search.
  3. Include "Whether Call Tools" into the review. Many failures are not parameter errors, but excessive calls, miss calls, and forced calls when the information is insufficient.
  4. Record the full trajectory, but do not show the complete reasoning unconditionally. The production system saves tool calls, observations, state differences, and user confirmation without having to use the internal chain-of-thought as part of the user interface.
  5. Use transaction boundaries for side effects tools. Permission verification and user confirmation before writing operations; support compensation or rollback when failure; high-risk actions enter manual approval.
  6. Build a local Benchmark. The open list can only provide directions, and you must use your own API, data distribution, permission constraints and exception paths to construct task sets before real launch.

Limitations and open questions

The biggest problem with Tool Learning at present is the fragmentation of the evaluation. Different Benchmark's Tool collections, actuators, user simulators and scoring functions are very different, and it is difficult to directly migrate to business scenarios. The second problem is the quality of synthetic data: large-scale Tool tracks are easy to override formats, but do not necessarily cover the vague expressions, error messages and state competition of real users. The third problem is the security boundary: the model can generate the correct Tool call, which does not mean that it should be authorized to execute the call.

There is also a controversy that is easily overlooked: whether RL must be better than SFT. For fixed, low-risk, and strong Schema tools, SFT may be sufficient to increase strict verification; for long-range, multi-round, and state-changing environments, RL or process rewards are more likely to reflect value. In engineering, the training complexity should not be increased for "Agent RL", and it is more secure to use the executable evaluation to locate the bottleneck first.

Questions to watch

First, whether the benchmark called by the tool will shift from "function accuracy" to "business status accuracy". EigenData's audit and repair of BFCL-V3 has pointed out that there may be systemic errors in Schema, implementation and reference trajectories, and such Benchmark Hygiene will become more and more important.

Second, how to make safe isolation of dynamic tool ecology. MCP, plug-in, and remote tool servers make the ability to discover more flexible, and also allow the supply chain risk to enter the Agent system. How to define minimum permissions, tool reputation, sandbox, and cross-tool data flow is a key issue in the next stage.

Third, whether the small model and end-side Function Calling will mature. Hammer, Granite Function Calling and other job description small models specially trained have the opportunity to have practical value in local tasks; but they still need to face the tool set change and complex dialogue state.

Fourth, how the RL tool strategy is combined with interpretability. Process rewards, trajectory filtering, and reward-conditioned exploration are all resolving the same contradiction: the model needs to be explored, but the production system needs to be controlled. Whoever puts the training benefits, interpretable calls and execution security within the same framework is closer to the deployable agent.

Summary

Tool Learning's research focus has been upgraded from "Will the model output function calls" to "Whether the agent can stably complete tasks in a real Tool environment". The most noteworthy thing in the past year is not a single point of SOTA, but a paradigm change: the Tool surface must be retrieved and governed, the call trajectory must be executed and audited, the evaluation should be covered with the status and user cooperation, and the training should be processed, exploration and failure recovery. For engineering teams, the most practical strategy is to include LLM Tool calls into regular software engineering discipline: gadgets, strong schemas, executable reviews, sandbox permissions, log audits, and step-by-step rights.

References

Search date: 2026-06-23.

  1. Hu et al., “Agentic Tool Use in Large Language Models”, arXiv, 2026-04-01. https://arxiv.org/abs/2604.00835
  2. Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models”, arXiv / ICLR camera ready, 2022-2023. https://arxiv.org/abs/2210.03629
  3. Schick et al., “Toolformer: Language Models Can Teach Themselves to Use Tools”, arXiv, 2023-02-09. https://arxiv.org/abs/2302.04761
  4. Li et al., “API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs”, arXiv / EMNLP 2023. https://arxiv.org/abs/2304.08244
  5. Patil et al., “Gorilla: Large Language Model Connected with Massive APIs”, arXiv, 2023-05-24. https://arxiv.org/abs/2305.15334
  6. Qin et al., “ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs”, arXiv, 2023-07-31. https://arxiv.org/abs/2307.16789
  7. Berkeley Function Calling Leaderboard (BFCL) V4, official leaderboard, last updated 2026-04-12 on page. https://gorilla.cs.berkeley.edu/leaderboard.html
  8. Lu et al., “ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities”, arXiv, revised 2025-04-16. https://arxiv.org/abs/2408.04682
  9. Barres et al., “tau^2-Bench: Evaluating Conversational Agents in a Dual-Control Environment”, arXiv, 2025-06-09. https://arxiv.org/abs/2506.07982
  10. Xiu et al., “ASTRA-bench: Evaluating Tool-Use Agent Reasoning and Action Planning with Personal User Context”, arXiv, 2026-03-02. https://arxiv.org/abs/2603.01357
  11. Chen et al., “ACEBench: Who Wins the Match Point in Tool Usage?”, arXiv, 2025-01-22, revised 2025-11-20. https://arxiv.org/abs/2501.12851
  12. Zhong et al., “ComplexFuncBench: Exploring Multi-Step and Constrained Function Calling under Long-Context Scenario”, arXiv, 2025-01-17. https://arxiv.org/abs/2501.10132
  13. Wang et al., “RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning”, arXiv, 2025-04-24. https://arxiv.org/abs/2504.20073
  14. Hao et al., “Reasoning through Exploration: A Reinforcement Learning Framework for Robust Function Calling”, arXiv, submitted 2025-08-07, revised 2025-10-10. https://arxiv.org/abs/2508.05118
  15. Zhong et al., “RC-GRPO: Reward-Conditioned Group Relative Policy Optimization for Multi-Turn Tool Calling Agents”, arXiv, 2026-02-03. https://arxiv.org/abs/2602.03025
  16. Chen et al., “EigenData: A Self-Evolving Multi-Agent Platform for Function-Calling Data Synthesis, Auditing, and Repair”, arXiv, 2026-03-05. https://arxiv.org/abs/2603.05553
  17. Rabinovich and Anaby-Tavor, “On the Robustness of Agentic Function Calling”, arXiv, 2025-04-01. https://arxiv.org/abs/2504.00914
  18. “Function calling”, OpenAI API documentation, accessed 2026-06-23. https://developers.openai.com/api/docs/guides/function-calling
  19. “Using tools”, OpenAI API documentation, accessed 2026-06-23. https://developers.openai.com/api/docs/guides/tools
  20. “MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploits”, arXiv, 2025-04-04. https://arxiv.org/abs/2504.03767