In the previous article, we split the large-model production service into several aspects that are more engineering:

  • How to hold high concurrency
  • What should monitoring indicators focus on?
  • When does the flow limit have to be done?
  • How to prevent the entire service from being dragged to death in failure recovery

But when the underlying reasoning service is gradually stabilized, the upper-level application will soon enter another stage:

The simply "chat" model is not enough, the system begins to require the model to call tools, access external information, decompose tasks, and execute multistep processes.

At this time, many teams will encounter new problems at the same time:

  • Are Function Calling and Tool Use the same thing?
  • Why does the model clearly "know" to check the weather, but it may not be able to generate tool parameters correctly?
  • Why is Agent Demo so smart that as soon as you enter production, you start infinite loops, messing with tools, and costs soaring?
  • tool_choice、temperature、max_steps, how to match the parameters of timeout, retry, and observation window?
  • When should the model be freely planned and when should the process be written to death?

Behind these questions, it is essentially not "whether the model will call the function", but:

Are you using the language model from a text generator to an execution system that will perceive the environment, invoke external capabilities, and continue to make decisions based on feedback.

This article focuses on this issue.

I want to take the four layers to dismantle:

  1. What is Function Calling and which layer of problems it solves
  2. How does the complete closed loop of Tool Use run?
  3. Why isn't the "adjusting a few more tools" as simple as the Agent?
  4. How to match the most critical parameters, constraints and anti-loss control mechanism in engineering

If you have gone from "Give the model a prompt" to "Let the model receive search, database, browser, code execution, business API", this article is basically the starting point of understanding the Agent system.


1. Let’s talk about the conclusion first: Function Calling, Tool Use, and Agent are three layers of abilities, don’t mix one word

In many discussions, these three words will be mixed.

But if it is not disassembled, the system design is easy to mess up.

1) Function Calling: Let the model make a "call request" according to the convention format

The core problem that Function Calling solves is:

Don't let the model describe "I want to adjust an interface" in natural language, but let it output structured, machine-resolvable call intent.

For example you define a function:

  • get_weather(city, date)

Ideally, the model should not output:

  • "I suggest to check the weather in Shanghai tomorrow."

Instead, it should output something like:

  • Tool name: get_weather
  • Parameters: {"city":"上海","date":"tomorrow"}

So the focus of Function Calling is not "really executing the Function", but:

  • Tools describe how to expose to a model
  • Is the parameter schema clear enough
  • Whether the model can stably produce legal parameters
  • Can the upper-level program safely parse the results?

It's essentially a Structured output interface layer.

2) Tool Use: Let the model be called, and then the system really executes it, and recharges the result

Tool Use is one step more than Function Calling.

It requires the system to form a complete closed loop:

  • Model judges whether tools are needed
  • Model output tool name and parameters
  • External actuator actual call tool
  • Tools return results
  • The result is fed back to the model
  • Models continue to answer or continue decisions based on new observations

So the key to the Tool Use is not just schema, but Execution and Feedback.

This means that the system begins to face the uncertainty of the real world:

  • Tools will time out
  • Parameters may be illegal
  • return possibly empty
  • The external API will fail
  • Multiple tool results may contradict each other

From this level, the problem is no longer just "whether the output format is correct", but "the system behavior is stable and unstable".

3) Agent: Let the model continue to do "thinking-action-observation" in the multistep loop

Agent goes further.

Agent is not a call, but a loop:

  • Decide what to do next according to the goal
  • Select a tool or subtask
  • perform action
  • observation
  • Determine whether to continue, modify the plan or end

So what the Agent system really adds is not some API field, but:

The decision-making link is longer.

Once the link becomes longer, the problem is amplified synchronously:

  • Will one wrong step get farther and farther?
  • Will tool calls continue to accumulate costs?
  • Will it enter the loop
  • Is it possible to misuse dangerous tools?
  • How the observations are compressed into the context
  • When to stop and when to give it back

Therefore, the most practical way of understanding is:

  • Function Calling is Structured call interface
  • Tool Use is call closed loop
  • Agent is multistep decision system

Don't take them as one thing.


2. Why the large model needs tools: not because the model is not smart, but because it is naturally restricted by three types

When many people come into contact with Agent for the first time, they will mistakenly think that "tools are the ability to supplement weak models".

Actually not accurate.

The more essential reason is:

1) Model parameters are not real-time worlds

No matter how big the model is, it just presses the statistical laws that you have seen during training into the parameters.

So it is naturally not good at directly solving this kind of problem:

  • It won't rain in Shanghai today
  • How many orders are in the current database
  • What does a web page show now
  • What is the latest CI status of a code repository

These are not "language knowledge" problems, but Real-time external state issues.

The meaning of the tool call is to connect the model and the real-time environment.

2) Model parameters are not precise executors

Even if the model knows that "should be counted", it does not mean that it is suitable for its own calculation.

For example:

  • Complex Mathematical Computing
  • SQL query
  • Tune API
  • file system operation
  • Browser click and page read

Many tasks are not solved by speaking more, but relying on Determine the deterministic execution to a specialized system.

3) Model parameters are not long-term workflow managers

In multistep tasks, models often encounter:

  • Intermediate state needs to be saved
  • Failure requires retry
  • The result of a step is passed to the next step
  • Timeout and permissions to check
  • Some movements require manual confirmation

These are more like system engineering than single rounds of Q&A.

So the existence of tools and agents is not an alternative model, but acknowledging the boundaries of the model.

Summarize in one sentence:

The model is responsible for understanding, planning and generating; tools are responsible for visiting the world, performing actions and returning deterministic results.


3. How does Function Calling work: the essence is "give the model a restricted action space"

Many platforms talk about Function Calling as an API capability, but from a mechanism point of view, it is more like:

You tell the model explicitly in the form of a set of allowed actions.

1) Tool definitions usually contain at least three types of information

An available tool description, at least make it clear:

  • tool name
  • Instructions for the use of the tool
  • Parameter Structure and Field Constraints

For example, a search tool, you can't just write:

  • search(query)

Better descriptions usually fill up:

  • This tool is suitable for real-time information
  • query It should be a specific problem rather than an empty theme
  • top_k What is the value range of
  • under what circumstances should it not be called

Because the model is not a compiler.

Whether it will choose the right tool and fill in the right parameters depends to a large extent on whether the description you give is enough to judge.

2) The more blurry the Schema design is, the easier it is for the model to "look and understand, but it is actually messed up"

The most common questions include:

  • The field name is too abstract, the model does not know what to fill in
  • Multiple tools overlap, the model is difficult to distinguish
  • There are too many options for parameters, and the model is easy to fabricate values
  • There is no counterexample in the description, and the model does not know when it should not be adjusted

For example:

  • action_type
  • data
  • options

This field name is considered universal from the engineer's point of view, but it is difficult to judge the semantic boundary from the model perspective.

The generally better schema principle is:

  • The field name should be as clear as possible
  • Enumeration values are as small as possible and the boundaries are clear
  • The required parameters should be few and the key
  • Write down applicable and disabled scenarios in the description

3) Function Calling is not "guaranteed correct call", just improves call controllability

Avoid a misunderstanding here.

Many people think that as long as the Function Calling is opened, the model will be like a program that strictly follows the type system.

Facts are not.

The model is still possible:

  • Wrong tool
  • Missing parameters
  • Generate illegal enumeration
  • Hard adjustment tool on this direct answer
  • Pretend you know the answer while the tuner tool

So Function Calling is more like:

Compress the original completely free text into a space that is easier to verify, easier to refuse, and easier to retry.

This is already important, but it does not mean that "the problem is solved".


4. The complete link of the Tool Use: it is not the end of the model output, but enters a failed execution system

Once the real adjustment tool is started, the system goes from "Generate Problem" to "Execution Problem".

A typical link is usually long like this:

  1. User puts goals
  2. Model judges whether tools are needed
  3. Model output tool name and parameters
  4. System check parameters legitimacy
  5. The executor initiates the actual call
  6. Tools return results or errors
  7. The system compresses the result into observation back to the model
  8. The model decides to end, asks, changes parameters, or continues to call other tools

In this link, almost every layer may make an error.

1) Parameter verification is the first gate

Do not directly hand over the parameters generated by the model to the actuator.

At least do:

  • type check
  • Required field check
  • Enumeration value check
  • Length and range limits
  • Dangerous character filtering
  • Permission related verification

Because the model generates "high probability text", it is not a natural trusted input.

2) The actuator and the model should be decoupled

An important engineering principle is:

The model is responsible for proposing the action, and the actuator is responsible for the real landing action.

Don't let the model have unrestricted execution rights on its own.

A more secure architecture is usually:

  • Models can only request tools
  • Tool whitelist is controlled by the system
  • Executors do permission checking, timeout, auditing, retry
  • Some high-risk actions must be manually confirmed

The advantage of this is that even if the model is wrongly judged, it will not directly cross the boundary of the system.

3) The return result must be "observed compression"

The result of the tool is not the better the original.

If you put a lot of HTML, JSON, full log text, and irrelevant fields back to the context, it is easy to have problems:

  • Context dilation
  • interference model judgment
  • Token cost soars
  • Important information is submerged

A better practice is usually:

  • Keep only the necessary fields
  • Clearly marked status: success, failure, empty results
  • Cut and summarize long results
  • Preserve key evidence instead of all noise

The quality of Observation will directly affect the Agent's next step to judge the quality.


5. Why is it easy to "look very strong, but actually very brittle"

Many problems with the Agent Demo are not the first step, but the second step starts to get out of control.

This is the typical risk of a multistep system.

1) One step is wrong, may be enlarged in the next steps

For example, the first step of the model misunderstood the user needs.

In a single round question and answer, the consequences may only be a little bit biased.

But in the Agent, errors continue to spread:

  • Wrong tool
  • Query wrong object
  • get the error Observation
  • Plan the next step based on the error Observation
  • Finally, the result is a serious but completely wrong direction

So the most difficult thing about the Agent is not "can it start", but Will the error spread in chains?.

2) The stronger the free planning, the more difficult to predict the behavior

Let the model freely choose tools, freely dismantle tasks, and freely decide when to end, and look more intelligent.

But the price is:

  • decreased behavioral stability
  • Cost is harder to control
  • Online problems are more difficult to reproduce
  • Security borders are harder to verify

Therefore, production is rarely a "pure free Agent".

The more common practice is:

  • Limited Toolset
  • limited number of steps
  • clear termination conditions
  • Approval for high-risk actions
  • Semi-structured process arrangement for key links

3) The bottleneck of Agent is often not model ability, but context management

After a multistep cycle, the context will expand rapidly.

Questions include:

  • History Observation is getting longer and longer
  • Early key constraints are diluted by subsequent content
  • The model begins to forget the initial goal
  • Token costs continue to accumulate

Therefore, mature systems usually do not put the whole process in the context of the whole process without brains.

Rather will do:

  • Intermediate state structured storage
  • Summary of historical steps
  • Critical constraints are left alone outside ScratchPad
  • Crop the Observation

The stability of the Agent depends largely on how you manage the context, not just how strong the base model is.


Six, the most critical parameters, how to understand

The following is the most common and most easily misused batch of parameters in the project.

1)tool_choice: Let the model choose freely, or force a tool

It is essentially controlled by:

  • Whether to allow the model to decide whether to call the tool for itself
  • Whether it is required that the model must call the specified tool
  • Whether tool calls are prohibited, answer directly in natural language

Applicable suggestions:

  • Open Q&A scenarios: allow the model to choose freely, but have the rules of the bottom line
  • Determine process scenarios: Force the specified tool to be more stable
  • Test/Assessment Scenario: Fixed tool selection is easier to compare

If a step is theoretically necessary to check the database, then don't let the model "think whether to check".

Direct forced calls are usually more stable.

2)temperature: Usually more conservative than chatting in agents

In general chat, slightly higher randomness may be more natural.

But in tool calls and Agent tasks, too high temperature usually means:

  • Tool selection is more floating
  • Parameter filling is more unstable
  • Multistep links are more difficult to reproduce
  • Repeated execution results of the same task are more different

Experience:

  • Structured output, retrieval, database query, and code execution, the temperature should usually be low
  • Only when creative generation and candidate schemes diverge, is it suitable for moderate improvement?

Do not use the Agent as a creative model to match the sampling parameters.

3)max_steps: It is not a random fuse, but the cost and risk valve

This is one of the most important protection parameters in the Agent system.

It controls:

  • How many rounds of "thinking-action-observation" cycles are allowed in a task

The value is too small:

  • Legal and complex tasks can't be completed

Value is too large:

  • easy loop
  • cost out of control
  • Failure has been delayed for a long time to reveal

The more practical ideas are:

  • Simple task to small steps
  • Complex tasks give different upper limits according to the task template
  • Trigger the summary or hand over the labor when the upper limit is close to the upper limit

max_steps essentially Agent's budget parameters, which is not a dispensable default value.

4) Timeout parameter: the model timeout, tool timeout, and total task timeout must be distinguished

Many systems have only one unified timeout, which is usually not enough.

A more reasonable way to dismantle is:

  • Timeout for single model generation
  • Timeout for single tool execution
  • The total task timeout
  • Queuing for timeout

Because the reasons behind these four timeouts are different:

  • The model is slow, it may be that the context is too long or the service is congested
  • Tools are slow, possibly an external API or network problem
  • Total task is slow, maybe too many loops
  • Queuing timeout may be a system capacity problem

Only when you take it apart will you know which layer to optimize.

5) Retry parameter: not all failures should be retryed

Retrying seems reasonable, but especially dangerous in the Agent.

Because a failed retry can trigger:

  • More token consumption
  • More external API calls
  • longer task link
  • repeat side effects

should distinguish:

  • retryable error: Temporary timeout, short-term network error, current limit
  • Errors that cannot be retryed: Illegal parameters, insufficient permissions, and inconsistency in business constraints

and to match:

  • Maximum number of retries
  • exponential withdrawal
  • idempotent design
  • Add deduplication keys to side effects

Otherwise, it is easy for the Agent to magnify an instantaneous abnormality into a serial call accident.

6) Observation window and context crop parameters: decide whether the Agent will "get more and more confused"

Many frameworks like to plug the complete process back into the context.

Convenient in the short term, and high risk in the long run.

Requires clear control:

  • Keep the last few steps Observation
  • When to Summary of Historical Steps
  • Which system constraints need to live in context
  • Which intermediate results are only placed in the external state, not refilling the model

This parameter group often affects the stability of the system more than the "replacement model".


7. What really matters in production is not to make the Agent more free, but to make it more controllable

When many teams are just making agents, the goal is to be "as autonomous as possible".

But when it comes to production, the priority becomes four other things:

  • predictable
  • Can be audited
  • can be played back
  • Stop loss

1) Tool whitelist and permission grading must be done

Not all tools should be exposed to the same Agent.

Safer designs are usually:

  • Only expose the minimum toolset required
  • Separate read and write operations
  • External side effect tools are individually approved
  • Divide the permissions by user identity, tenant, and environment

An Agent that can call ten tools is not necessarily better than an Agent that only calls two tools.

It may just be more dangerous.

2) High-risk actions must have a human confirmation node

For example:

  • Send email
  • Place an order
  • delete data
  • Modify the database
  • Execute production commands
  • send out messages

These actions should not be executed just because the model is "very sure".

A more secure model is:

  • Suggestions for the Sons of Action
  • Show the tools and parameters that will be called
  • Waiting for manual confirmation
  • re-execute

This will sacrifice a little fluency, but can be exchanged for controllability.

3) Logs and tracks must be replayable

If there is a problem with the Agent system, if you can only see "the final answer is wrong", it is basically inexhaustible.

At least playback:

  • Each step Prompt or system state
  • Tool selection at every step
  • parameter content
  • Observation content
  • error type
  • Total steps, total token, total time-consuming

Otherwise, you don't even know the problem is:

  • The model is wrong
  • schema is not clear
  • Tools return dirty data
  • Crop strategy cuts out key information
  • Timeouts and retry strategies fight each other

4) There must be a clear termination condition, not "until the model says the end"

The termination conditions are preferably multiple:

  • Achieve goals
  • Maximum number of steps reached
  • total timeout
  • reach the cost budget
  • No effective progress in several consecutive steps
  • Repeat the same tool call continuously

If the termination condition is only "the model decides to end by itself", then the system will encounter an infinite loop or pseudo-complete sooner or later.


8. When should I go to the Agent, and when I don't need the Agent

This is a very practical question.

Because many tasks are packaged as agents, in fact, the process is not designed well.

Scenarios for Agents

There are usually several features:

  • The task path is not fixed
  • Need to make dynamic decisions based on intermediate results
  • Multiple tool collaborations may be involved
  • Requires multistep search, screening, verification
  • It is difficult to write all the processes before labor

For example:

  • Multi-source information retrieval and cross-validation
  • Complex report analysis
  • Browser Task Automation
  • Codebase targeting problem and try to fix
  • Open tasks that need to be explored

Scenarios that are not suitable for Agent

If the process is inherently certain, it is actually more suitable for workflows rather than agents.

For example:

  • Check the database → fill in the template → return result
  • Fixed field extraction → JSON output
  • Fixed API query → result shaping
  • After approval, perform a determined action

This type of task is often more stable, cheaper and easier to debug with orchestration.

A very practical criterion for judgment is:

If you can spell out dependable rules for a workflow, implement the workflow directly. Introduce an agent only when you genuinely cannot specify the intermediate decisions in advance.


9. What does an Agent system that is closer to reality usually look like

If you abstract a relatively mature Agent system, the common structure will be:

  • task entry layer: Receive user goals and do identity and permission verification
  • Planning/Decision Level: Decide whether the next step is to answer, call the tool or continue searching
  • tool execution layer: Whitelist tool, parameter verification, permission control, timeout and retry
  • state management: Save task status, intermediate results, historical summary, budget consumption
  • Observation and Audit Layer: Logs, tracks, indicators, alarms, playback
  • artificial takeover layer: Confirmation of high-risk action, failure to complete the bottom line, manual correction

In this structure, the base model is only a middle part.

What really decides whether the Agent can enter production is not just whether the model is smart enough, but:

  • Is the tool boundary closed?
  • Is the status managed?
  • Is there any isolation error?
  • Is the cost a good budget?
  • Is there a way for people to take over in time?

10. The three most practical landing suggestions

If you're going to get Function Calling or Agent to real business, I'd recommend doing the following three things first.

Suggestion 1: Start with the "single tool closed loop", don't pursue a general Agent at the beginning

Start with a clear task:

  • When to call
  • how parameters are generated
  • How to compress the returned result
  • How to fail

After the single tool is closed and stabilized, it will expand to multi-tool collaboration.

This is much more reliable than stacking a dozen tools at the beginning and pursuing "all-round agents".

Recommendation 2: Schema design as a product design, rather than handwriting a JSON

Tool names, parameter names, field descriptions, enumeration values, and applicable boundaries will directly affect the model behavior.

Many so-called "models don't tune tools" are essentially schemas that are too poorly written.

Suggestion 3: Prioritize the budget and termination conditions, and then pursue a sense of intelligence

Control first:

  • maximum number of steps
  • Total timeout
  • Tool timeout
  • Retry upper limit
  • Available tools range
  • Approval of high-risk action

Make the system stable first, and then consider making the Agent more free.

In the production system, "controlled 80 points" is usually more valuable than "occasionally stunning, occasional 95 points for rolling over".


11. Final summary: Agent is not an API function, but a system design that connects the model to the environment

When many people come into contact with Function Calling and Agent for the first time, they will think that this is just a few new fields added by the model manufacturer.

But from an engineering point of view, it really changes a lot.

Because from this moment, the model is no longer just answering the question, but in:

  • Understand the goal
  • select action
  • Caller
  • read feedback
  • Continuous decisions in a multistep loop

So the core conclusion can be condensed into one sentence:

Function calling gives the model a structured way to request a call. Tool use closes the loop by executing the request and returning the result. An agent keeps deciding what to do as a task unfolds over multiple steps.

The key to production is not to make the Agent much like a person, but to do it:

  • Boundary
  • Have a budget
  • Audited
  • a termination condition
  • Stop loss after error

If none of these, the Agent is often just a more expensive, more difficult-to-treat Prompt casing system.

If these are all done, the model will really have the opportunity to move from "talking" to "can do things".

Next, follow this main line and continue to advance, a very natural problem is:

Why do many businesses end up going to RAG instead of relying solely on agents or larger parametric models to make up the blind spots of knowledge?

That will further advance the topic to the engineering level of "knowledge access" and "retrieval enhancement".