菜单
HuggingFace Text Generation Inference (TGI). High-performance LLM serving with continuous batching, tensor parallelism, watermarking, and OpenAI-compatible API. Native HF model hub integration.
默认使用会先检查来源的 Prompt；你也可以切换为直接命令，或下载本地副本。
决定是否安装前，请先阅读 SKILL.md，以及 SkillsMP 当前展示的配套文件。
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
直接命令不会经过审查 Prompt；运行前请先检查来源。
npx skills add https://github.com/mkurman/zorai --skill huggingface-tgi命令会保持在同一行。复制前请横向滚动并检查完整内容。
想先保存到本地？可下载 SkillsMP 当前能够提供的文件。
Use when provisioning Prime Intellect GPU compute, managing pods/disks/sandboxes, running hosted RL training via prime lab, installing or publishing RL environments, or exposing local services via Prime Tunnel. Covers the `prime` CLI (PyPI: prime) for all Prime Intellect platform operations.
Shared Python package for Science Skills, currently containing http_client -- a unified HTTP client with rate limiting, retries, and exponential backoff. Not a standalone agent skill. Do not invoke directly.
Retrieve and analyze AlphaFold predicted structures for a protein. Use when the user provides a specific UniProt Accession ID and wants structural confidence metrics (pLDDT), domain boundary analysis, or disorder assessment. Do not use if the user only has a protein name, gene name, or amino acid sequence — ask for a UniProt ID first.
基于 SOC 职业分类
正在显示 SKILL.md
| name | huggingface-tgi |
| description | HuggingFace Text Generation Inference (TGI). High-performance LLM serving with continuous batching, tensor parallelism, watermarking, and OpenAI-compatible API. Native HF model hub integration. |
| tags | ["tgi","llm-inference","huggingface","serving","text-generation","api","zorai"] |
Text Generation Inference (TGI) is a production-ready LLM serving solution from Hugging Face. It provides optimized inference with continuous batching, quantization (GPTQ, AWQ), tensor parallelism, flash attention, and an OpenAI-compatible API.
# Docker deployment (recommended)
docker run --gpus all -p 8080:80 -v $HOME/models:/data ghcr.io/huggingface/text-generation-inference:latest --model-id Qwen/Qwen2.5-1.5B-Instruct
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="none")
response = client.chat.completions.create(
model="tgi",
messages=[{"role": "user", "content": "Hello!"}],
)
print(response.choices[0].message.content)
stream = client.chat.completions.create(
model="tgi",
messages=[{"role": "user", "content": "Write a poem"}],
stream=True,
)
for chunk in stream:
print(chunk.choices[0].delta.content or "", end="")