streaming-responses
Use when streaming tokens incrementally from an LLM via liter-llm over SSE or async iterators. Covers chat_stream, delta handling, and null-content chunks.
Informations de source
- Dépôt
- xberg-io/liter-llm
- Dernière activité de la source
- 30 septembre 2026 à 16:23
- Langue détectée de SKILL.md
- anglais
- Étoiles
- 260
- Forks
- 23
Options d'installation
Le prompt qui vérifie d'abord la source est sélectionné par défaut. Vous pouvez passer à une commande directe ou télécharger une copie locale.
Vérifiez les fichiers source
Lisez SKILL.md et les fichiers associés affichés par SkillsMP avant de décider de l'installer.
Affichage de SKILL.md
SKILL.md
Instructions source · Aperçu en lecture seule- name
- streaming-responses
- description
- Use when streaming tokens incrementally from an LLM via liter-llm over SSE or async iterators. Covers chat_stream, delta handling, and null-content chunks.
<!--
AI-RULEZ :: GENERATED FILE — DO NOT EDIT
Content-Hash: blake3:13410510899be94ef40712ffcb27a284ef4ac4977b6e45b38b014d60e913f995
Source-Hash: blake3:651c31e789b5b3a98bbebecbda0ea4f0b6222b63c8264db7c677af355c5fe27b
Schema-Version: v1
-->
# Streaming Responses
Use `chat_stream(...)` to receive tokens as they are produced instead of waiting
for the full completion. The proxy streams over SSE; bindings expose async
iterators.
## Python
```python
import asyncio, os
from liter_llm import create_client
from liter_llm._internal_bindings import ChatCompletionRequest
async def main() -> None:
client = create_client(api_key=os.environ["OPENAI_API_KEY"])
request = ChatCompletionRequest.from_json(
'{"model":"openai/gpt-4o","messages":[{"role":"user","content":"Tell me a story"}],"stream":true}'
)
async for chunk in client.chat_stream(request):
if chunk.choices and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
print()
asyncio.run(main())
```
## TypeScript
```typescript
import { createClient } from "@xberg-io/liter-llm";
const client = createClient(process.env.OPENAI_API_KEY!);
const chunks = await client.chatStream({
model: "openai/gpt-4o",
messages: [{ role: "user", content: "Tell me a story" }],
});
for await (const chunk of chunks) {
process.stdout.write(chunk.choices?.[0]?.delta?.content ?? "");
}
```
## Notes
- The first and last chunks often carry null content. Always null-check
`chunk.choices[0].delta.content` (Python) or
`chunk.choices[0]?.delta?.content` (TypeScript) before using it.
- Tool-call deltas arrive in `delta.tool_calls` (Python) /
`delta.toolCalls` (TypeScript); accumulate `function.arguments` fragments
across chunks before parsing.
- Through the proxy, request streaming with `"stream": true` on
`/v1/chat/completions`.
Voir sur GitHub