Cover illustration

Who this is for

  • chatbot developer
  • AI writing tool developer
  • Python beginner

What problem does this article solve?

The non-streaming interface has a long wait time, and the user mistakenly thinks that the program is stuck.

Key concepts

  • flow response
  • Incremental Token
  • Terminal output refresh
  • Timeout processing

Walkthrough

  1. Prepare the CLI input loop
  2. Enable streaming requests
  3. Print response content block by block
  4. Increase Ctrl+C and error message

Python code example

The companion demo is generated in:

cd examples/llm-guides/003_llm_api_streaming_cli
python demo.py

The example follows two principles:

  • From OPENAI_API_KEY , load the key at runtime to avoid hard-coding it.
  • Without an API key, the example runs in mock mode so you can follow the workflow.

The essential call looks like this:

import os

api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
    print("未配置 OPENAI_API_KEY,进入 mock 模式。")
else:
    # 真实项目中在这里初始化 SDK 客户端并发起请求。
    print("已读取 OPENAI_API_KEY,可以调用模型。")

Common pitfalls

  • Publishing after a single successful demo run leaves out error handling, logging and human review.
  • Including API keys, real file paths, account details or business data in published articles or code.
  • Readers can struggle to reproduce an example if it is unclear which parts are mocked and which run in production.
  • Citing an API or framework feature without checking its documentation can mislead readers when versions change.

Summary

  • LLM Guide 03 builds a command-line chat interface with streaming output. When a non-streaming interface makes users wait for the entire response, they may think the program has frozen.
  • Make the workflow reproducible, reversible and easy to review. One good generation tells you little about how it will hold up in use.
  • Before publishing, check current documentation for model capabilities, API parameters and third-party frameworks.

CSDN tags

Large model, Python, streaming output, ChatGPT, command line

WeChat article summary

A command-line chat tool illustrates why streaming responses can make an LLM application feel much more responsive.

Alternative title for Zhihu

  • Why do mockup applications like streaming output?
  • How to implement verbatim AI answers in Python?
  • What user experience problems can flow response solve?

References and further reading

  • Check the OpenAI documentation for your model, SDK and account permissions.
  • Python documentation on environment variables, virtual environments and exception handling.
  • Consult the documentation for the tools used in the example, such as LangChain, LlamaIndex, Haystack or the vector database.