
Who this is for
- chatbot developer
- AI writing tool developer
- Python beginner
What problem does this article solve?
The non-streaming interface has a long wait time, and the user mistakenly thinks that the program is stuck.
Key concepts
- flow response
- Incremental Token
- Terminal output refresh
- Timeout processing
Walkthrough
- Prepare the CLI input loop
- Enable streaming requests
- Print response content block by block
- Increase Ctrl+C and error message
Python code example
The companion demo is generated in:
cd examples/llm-guides/003_llm_api_streaming_cli
python demo.py
The example follows two principles:
- From
OPENAI_API_KEY, load the key at runtime to avoid hard-coding it. - Without an API key, the example runs in mock mode so you can follow the workflow.
The essential call looks like this:
import os
api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
print("未配置 OPENAI_API_KEY,进入 mock 模式。")
else:
# 真实项目中在这里初始化 SDK 客户端并发起请求。
print("已读取 OPENAI_API_KEY,可以调用模型。")
Common pitfalls
- Publishing after a single successful demo run leaves out error handling, logging and human review.
- Including API keys, real file paths, account details or business data in published articles or code.
- Readers can struggle to reproduce an example if it is unclear which parts are mocked and which run in production.
- Citing an API or framework feature without checking its documentation can mislead readers when versions change.
Summary
- LLM Guide 03 builds a command-line chat interface with streaming output. When a non-streaming interface makes users wait for the entire response, they may think the program has frozen.
- Make the workflow reproducible, reversible and easy to review. One good generation tells you little about how it will hold up in use.
- Before publishing, check current documentation for model capabilities, API parameters and third-party frameworks.
CSDN tags
Large model, Python, streaming output, ChatGPT, command line
WeChat article summary
A command-line chat tool illustrates why streaming responses can make an LLM application feel much more responsive.
Alternative title for Zhihu
- Why do mockup applications like streaming output?
- How to implement verbatim AI answers in Python?
- What user experience problems can flow response solve?
References and further reading
- Check the OpenAI documentation for your model, SDK and account permissions.
- Python documentation on environment variables, virtual environments and exception handling.
- Consult the documentation for the tools used in the example, such as LangChain, LlamaIndex, Haystack or the vector database.