
- Date: 2026-10-29
- For graduate students, early-career researchers and research engineers
- Scope: Evidence control of three first-hand studies; model experiments that are not run, and subsequent experiments are all proposals.
Summary: Change a tool, what exactly?
A model learns to query the weather during training, then successfully calls an unfamiliar flight API at test time. That looks like transfer, but what exactly was unfamiliar? Perhaps only the API name was absent from fine-tuning. Or the parameter units, response structure, and execution conditions may all have changed. Success in the first case tells us little directly about the second.
The judgment of this article is: Tool learning can indeed be migrated under a limited protocol, but the calling code, tool selection and execution effect are different evidence objects. API Pack shows code generation on the new interface; ToolSense reveals the separation between the search performance and the tool knowledge probe; PA-Tool shows that the fixed model will also behave differently due to the change of the interface name. The three papers do not constitute a simple support and opposition camp. We need to ask: what is fixed, what is changed, and what is rated?
From the name to the effect, there are several layers of abilities in the middle
The API is the interface for the program to call another program. Tool mode, also known as Schema, is the description of the interface name, parameters, type and description. The semantics of the tool is related to the meaning of the call: what prerequisites are needed, what information is returned, and how the environment will be changed. This article transforms the "understanding of semantics" operations into a set of testable behaviors, without referring to the internal experience that cannot be directly observed.
Take a hypothetical temperature tool as an example: the model may fill legal structured parameters, but use Fahrenheit as a degree of Celsius; it may correctly select the sensor, but use "reading the old value" as "measure immediately". These are not the results of this article, but the description of the format, selection, parameter meaning and state effect can be wrong.
There are at least three competitive explanations. First, the model has learned the common call template, and when you see a similar description, you fill in the name and parameters. Second, it will use the current document to establish a local correspondence, so it can handle new interfaces, but it may not be able to infer the side effects that are not specified. Third, it forms a combinable operation rule, which can adjust the call order according to the premise and effect. The three may coexist; the success of the task itself usually cannot identify each contribution.
Therefore, "no tool" must be indicated relative to which training stage. Fine-tuning data isolation does not guarantee that pre-training has never been exposed to public documents. Changing the name does not mean changing the rules; in turn, the replacement of the uncommon identifier causes failure, which may be a burden of reading and writing and tokenization, not just the disappearance of semantic ability.
Code generation on the new API, how far is the migration proven?
The API Pack divides the test into new instructions, the new endpoint of the known API, and no API at all three levels; the endpoint is the specific operation in the interface. The study uses the starting point model such as Codeellama for fine-tuning, and compares prompt settings such as examples, random examples, and retrieval examples. This layering makes the "switching method" and "changing interface" no longer confuse. API Pack,§4.
However, its correctness is determined from the similarity of the code sequence, and the heuristic threshold value of 0.9 is used, which is not to check the environment status after running the interface. The paper also states that data processing makes it difficult to perform evaluation, the task is not designed as a multi-API combination, and the instructions need to be clear about the API name. Therefore, it supports call generation migration under limited conditions, and cannot directly prove the autonomous discovery tool or master side effects. Tongwen §4.2, §6.
This does not make the result lose its value. Generating usable interface code is inherently independent ability; the problem is what limits the conclusion. Retrieving examples brings external information, oracle settings also provide special correct example conditions, and it is impossible to describe such results as a model to independently master unfamiliar tools.
This article further infers that if the two calls are almost the same, only the parameter unit is different, the close of the string does not necessarily mean the task is correct; if the two programs are written differently and the effect is the same, the difference in the string does not necessarily mean failure. This logic explains why it is necessary to perform layer supplements, and does not claim that the above benchmark has appeared in a specific misjudgment of an unverified.
Finding a tool does not mean being able to explain it
ToolSense is a parametric tool retrieval: associate the tool to a special identifier, and then train the model to output the identifier from the query. It tests query changes, free generation and constraint generation, and tool attribute question and answer respectively. Constraint decoding will limit the output to the legal identification path, helping the system to avoid tools that do not exist, but the legality does not guarantee that the semantic judgment is correct. ToolSense,§1—4.
For Gemma3-4B in the TG configuration after stage two, the paper reports 95.7% on standard G1 constrained retrieval and 31.4% on a four-choice tool probe. The first measures whether at least one annotated tool appears in the top 50 results. The second measures accuracy on questions about tool attributes. These are different tasks, and neither measures successful execution. The probe supplies only the tool identifier and the question, without the full description. The same text §3.2—3.3, Table 2 and Table 4.
The most worthy of reservations here is the separation of measurements, not the slogan of "the model does not understand tools at all". The scope of the paper is limited to a specific two-stage paradigm and tool catalog; the diagnostic set is generated by the model and verified manually by sampling, and the end-to-end execution is reserved for subsequent research. Tongwen Limitations.
There is also a comparability trap: the new query set changes not only the wording, but also the number of ambiguity and target tools. The decline after the distribution change is worthy of attention, but all the losses cannot be attributed to "less keywords". The failure of the knowledge probe does not mean that all knowledge is erased: it first states that in the form of this problem and the reading mode, the relevant knowledge is not reliably expressed. This is the limitation of the intensity of evidence in this paper.
What does changing the name mean?
PA-Tool provides another intervention: a fixed model, generate candidate names according to the tool description, and then select the name according to the degree of similar aggregation between the candidates. This concentration is called Peakedness, and the author uses it as a proxy for the model to be familiar with naming habits. The method modifies the tool name and parameter name, keeps the description, and connects the original interface in mapping. PA-Tool, §3 and Limitations.
The result support naming is a factor that affects the performance of the tool, but not benefits everywhere. For example, in Table 4, the Call+Retrieve of Llama3.2-3B in API-Bank dropped from 10.9% to 9.2%; the retail multi-round task in the same table has increased from 4.5% to 5.6%. The former checks that the tool matches the parameters, and the latter checks that the task is completed, and the two columns cannot be regarded as the same ruler. Tongwen §6.6, Table 4, Appendix B.
This result constrains two overly strong explanations. You can't assert that the model has no relevant ability just because the original interface fails; and you can't assert that the model has obtained new operation knowledge just because the renaming is successful. The author's interpretation of pre-training familiarity is supported by proxy indicators, but it does not mean that the pre-training source of each name is directly tracked. The study also limits the English model, and the cross-language effect still needs to be tested separately.
This article is more inclined to see this as a question of "whether the ability can be expressed through the interface". Improving accessibility has practical value, but if the research problem is a semantic migration, it is necessary to continue to change the execution rules, rather than stopping on a better-sounding, more common name.
Put the three grades back in their respective experiments
| Research | Change and Condition | what can you observe | can not be asserted |
|---|---|---|---|
| API Pack | Test a new interface after fine-tuning, and the prompt can include examples | The matching performance of the call code | Multiple tools have been executed |
| ToolSense | Query and decoding conditions changes on the tool directory | Retrieve, identify generation and attribute probes | All agents do not understand semantics |
| PA-Tool | Fixed model, change tool and parameter naming | Effects of Interface Expression on Performance | New operating rules have been mastered |
Tables are the induction of measurement objects in this paper. The starting points, training materials, reasoning process and indicators of the three models are different, and there is no unified score suitable for direct sorting. Even if they are all called "generalization", the new API, query distribution changes, and interface renaming are not the same intervention.
Budgets are also recorded with conclusions. fine-tuning consumes training resources, sample retrieval provides information, and candidate name generation requires additional calls. Fixed the final number of answers does not erase these differences. If the comparison is a complete system, the cost can be included as a whole; if what is compared is what has been learned, the starting point and visible information should be fixed, and the isolation mechanism should be isolated item by item. Differences between papers should help locate gaps, rather than create wins or losses.
Next Step: Make Names, Rules, and Combinations Change respectively

Figure: Fixed model, test the original interface, name change, rule change and new combination, and then execute and check the status. The arrow indicates the proposed test process, not the confirmed internal causal mechanism; this figure has no measurement data, and the experiment has not been carried out.
A feasible small study can use a self-built offline tool environment, such as a limited list of items to read and write. It does not require a real account, nor does it need to expand the system complexity to a complete agent platform. The key is to write the tool premise, effect and judgment procedures in advance, and divide the training and test rules before the task is generated.
The first group is only renamed. The function, description content and parameter type are consistent, and the familiar name is replaced with a neutral identity, and at the same time, it provides a complete mapping; a new name comparison of the same length and normally readable is set. Statistical legal format, correct tool and parameter errors respectively. In this way, it is possible to distinguish between name prior, read and write burden and document usage, instead of calling any decline "memory dependence".
The second group reserves the name, and changes a rule with a clear document, such as changing the "Addition entry" to "Replacement item", or changing the numerical unit. Documents must be updated synchronously, and cannot be used to create unsolved tasks with error descriptions. If the model still acts according to the old rules, it constitutes evidence of behavior that cannot be adjusted according to the new semantics. A comparison of significant hints of changes should also be set, and it is excluded that only small prints are missed.
The third group used the tools that they had seen separately to form a dependency that has not been seen before. It is required to obtain a valid identification first and then perform the modification, and check the final status and the state that is prohibited. Just looking at the final answer text will miss "proclaimed success but not execution"; just looking at one target field will also miss other content by the way. Judgment rules should be frozen before the model is output.
Finally, the execution feedback is taken as a separate factor: a group is only read after reading the instructions, and the other group allows limited trial calls and reads the returned information. Record each attempt, error message, and retry cost, and distinguish the first migration and interaction adaptation. If it is allowed to succeed after repeated trial and error, the conclusion should be written "Trusted with feedback", and cannot be written as the first understanding.
The same question is used for analysis, and the fluctuation of the failure type and the repeated operation is retained. If the name change is valid and the rule change is invalid, the interface familiarity is supported to explain; if it can be adjusted steadily with the new rule and maintains correct in the new combination, the evidence for semantic use is added. Even so, the behavioral results are still not enough to uniquely determine the internal algorithm. The value of the research comes from excluding specific alternative interpretations, rather than issuing a general certificate to "real understanding".
Summary: Migration requires a clear object
Instead of asking whether the tool learning can be migrated, it is better to ask: is the migration format, document correspondence, attribute knowledge, or executable rules? Existing studies have allowed us to see partial migrations, as well as the strong impact of interfaces and measurement conditions. The next time you read the new tool list results, first look for data division, visible documents, feedback budget and success criterion. Only when these boundaries are clear, the success trajectory can gradually become evidence of capacity.
References
Retrieval and data deadline: 2026-10-29. Retrieve the related work for nearly 12 months and keep the earlier API Pack; do not claim to cover the full coverage. The following three are the core sources. The published identity of the two ACL-based papers has been checked on the official page. Due to the failure of the official PDF access, the experimental details are limited to the clearly marked arXiv version.
- Zhen Guo、Adriana Meza Soria、Wei Sun、Yikang Shen、Rameswar Panda. API Pack: A Massive Multi-Programming Language Dataset for API Call Generation. ICLR 2025. Published paper.
- Ashutosh Hathidara、Sai Shruthi Sistla、Sebastian Schreiber、Sahil Bansal. ToolSense: A Diagnostic Framework for Auditing Parametric Tool Knowledge in LLMs. Findings of EMNLP 2026. formal page; Deep reading version: arXiv v2, 2026-08-28.
- Jonggeun Lee、Woojung Song、Jongwook Han、Haesung Pyun、Yohan Jo. Don’t Adapt Small Language Models for Tools; Adapt Tool Schemas to the Models. ACL 2026. formal page; Deep reading version: arXiv v3, 2026-04-19.
Apps by the author
The following is the author's own product promotion. In addition to blogging technology blogs, I have also independently developed some small tools to make everyday life more organized. Welcome to understand:
- Daily Expense Journal: Record daily income and expenditure, arrange budget and savings goals, so that each money has a clear place to go. View on the App Store.
- One Small Action: Select small actions according to the current time and energy, record the feelings after completion, and leave a little clue for daily actions and reviews. View on the App Store.
Find more articles and projects on my website: zffjzz.top.