
Who this is for
- Developers who have been able to call the API but the output is unstable
- Product engineers who need tuning to generate quality
What problem does this article solve?
Parameters are messed up, which leads to the shortness of the answer, the erratic style, and the uncontrollable cost.
Key concepts
- temperature
- top_p
- token upper limit
- stop condition
- reproducible experiments
Walkthrough
- Prepare a fixed set of test questions
- Compare different parameter combinations
- Record output quality and token consumption
- Set recommended parameters for different scenarios
Python code example
The companion demo is generated in:
cd examples/llm-guides/002_llm_api_parameters
python demo.py
The example follows two principles:
- From
OPENAI_API_KEY, load the key at runtime to avoid hard-coding it. - Without an API key, the example runs in mock mode so you can follow the workflow.
The essential call looks like this:
import os
api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
print("未配置 OPENAI_API_KEY,进入 mock 模式。")
else:
# 真实项目中在这里初始化 SDK 客户端并发起请求。
print("已读取 OPENAI_API_KEY,可以调用模型。")
Common pitfalls
- Publishing after a single successful demo run leaves out error handling, logging and human review.
- Including API keys, real file paths, account details or business data in published articles or code.
- Readers can struggle to reproduce an example if it is unclear which parts are mocked and which run in production.
- Citing an API or framework feature without checking its documentation can mislead readers when versions change.
Summary
- LLM Guide 02 explains how to tune temperature, max_tokens, and top_p. Changing settings haphazardly can produce erratic response lengths and styles and make costs hard to control.
- Make the workflow reproducible, reversible and easy to review. One good generation tells you little about how it will hold up in use.
- Before publishing, check current documentation for model capabilities, API parameters and third-party frameworks.
CSDN tags
Large model, Prompt, Python, parameter tuning, AIGC
WeChat article summary
Controlled comparisons show how generation settings affect the output and how to make results more consistent, less costly, and easier to reproduce.
Alternative title for Zhihu
- Is it better to turn up Temperature?
- How to set the model generation parameters?
- Why is the same Prompt different every time?
References and further reading
- Check the OpenAI documentation for your model, SDK and account permissions.
- Python documentation on environment variables, virtual environments and exception handling.
- Consult the documentation for the tools used in the example, such as LangChain, LlamaIndex, Haystack or the vector database.