groq
[WHAT] Fast LLM inference via Groq API (chat) + Ollama (embeddings) [HOW] Groq for chat completions (llama-3.3-70b-versatile), Ollama nomic-embed-text for embeddings [WHEN] Need fast inference, embedding text for RAG, chat completions [WHY] Groq provides fastest LLM inference; Ollama handles local embeddings (Groq has no embedding API) Triggers: "groq embed", "groq chat", "groq complete", "embed with groq", "fast llm"
来源信息
- 仓库
- lev-os/agents
- 最近来源活动
- 2026年3月7日 00:14
- 检测到的 SKILL.md 语言
- 英语
- 星标
- 21
- 分支
- 2
安装方式
默认使用会先检查来源的 Prompt；你也可以切换为直接命令，或下载本地副本。
检查来源文件
决定是否安装前，请先阅读 SKILL.md，以及 SkillsMP 当前展示的配套文件。
文件资源管理器
4 个文件正在显示 SKILL.md
SKILL.md
来源说明 · 只读预览- name
- groq
- description
- [WHAT] Fast LLM inference via Groq API (chat) + Ollama (embeddings) [HOW] Groq for chat completions (llama-3.3-70b-versatile), Ollama nomic-embed-text for embeddings [WHEN] Need fast inference, embedding text for RAG, chat completions [WHY] Groq provides fastest LLM inference; Ollama handles local embeddings (Groq has no embedding API) Triggers: "groq embed", "groq chat", "groq complete", "embed with groq", "fast llm"
# groq
Fast LLM inference via Groq API for chat, Ollama for embeddings.
## Setup
**Environment:**
- `GROQ_API_KEY` - Required for chat completions
- Ollama running locally for embeddings (`ollama serve`)
**Install dependencies:**
```bash
cd ~/.claude/skills/groq
pip install groq requests
```
**Pull embedding model (first time):**
```bash
ollama pull nomic-embed-text
```
## Usage
### Chat Completion
```bash
# Simple chat
./scripts/chat.py "Explain quantum computing in 2 sentences"
# With system prompt
./scripts/chat.py "Write a haiku" --system "You are a poet"
# Different model
./scripts/chat.py "Hello" --model llama-3.1-8b-instant
# JSON output
./scripts/chat.py "List 3 colors as JSON array" --json
```
### Embeddings
```bash
# Embed text (returns JSON array of floats)
./scripts/embed.sh "Hello world"
# Embed from stdin
echo "Some text to embed" | ./scripts/embed.sh
# Python direct
./scripts/embed.py "Hello world"
```
## Models
### Chat Models (Groq)
| Model | Context | Speed | Use Case |
|-------|---------|-------|----------|
| `llama-3.3-70b-versatile` | 128k | Fast | Default, general purpose |
| `llama-3.1-8b-instant` | 128k | Fastest | Simple tasks |
| `llama3-70b-8192` | 8k | Fast | Legacy |
| `gemma2-9b-it` | 8k | Fast | Instruction following |
### Embedding Model (Ollama)
| Model | Dimensions | Notes |
|-------|------------|-------|
| `nomic-embed-text` | 768 | Local, fast, good quality |
## Output Format
### Chat
Plain text response to stdout. Errors to stderr.
### Embed
JSON array of floats:
```json
[0.123, -0.456, 0.789, ...]
```
## When to Use
| Scenario | Command |
|----------|---------|
| Quick question | `./scripts/chat.py "What is X?"` |
| Code generation | `./scripts/chat.py "Write Python for Y"` |
| Embed for RAG | `./scripts/embed.sh "document text"` |
| Batch embed | `cat docs.txt \| while read line; do ./scripts/embed.sh "$line"; done` |
## Error Handling
- Missing `GROQ_API_KEY`: Chat fails with clear error
- Ollama not running: Embed falls back to error message
- Rate limits: Groq has generous limits but will return 429 if exceeded
## Related Skills
| Skill | Use When |
|-------|----------|
| **oracle** | Need GPT-5, Claude, multi-model comparison |
| **lev-find** | Unified search with embeddings already indexed |
| **brave-search** | Web search, not embeddings |
在 GitHub 查看