一键在 Manus 中运行任何 Skill

stt-integration

星标701

分支110

更新时间2026年3月23日 05:06

Add or modify speech-to-text providers in assistant-api with transport-aware ingestion (WS/SDK/HTTP), transcript packet correctness, and UI/provider wiring.

安装

用 Codex 或 Claude 帮你安装复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。

在 Manus 中运行

来源

rapidaai

rapidaai/voice-ai

打开 GitHub 仓库查看创作者相关仓库

下载

在 Manus 中运行

STT Integration Skill

Mission

Integrate STT providers that emit reliable interim/final transcripts, interruption signaling, and stt_latency_ms metrics.

Inputs expected from user

Ingestion model: streaming WebSocket, SDK callback stream, or HTTP chunk/segment submit.
Interim transcript support required or final-only acceptable.
Required language/model controls.

If user does not answer:

Follow nearest streaming baseline and preserve current transcript packet semantics.

Hard boundaries

In scope:

api/assistant-api/internal/transformer/<provider>/stt.go (+ provider option/callback helpers)
api/assistant-api/internal/transformer/transformer.go
optional contract updates: api/assistant-api/internal/type/stt_transformer.go, packet.go
provider STT JSON + UI config files/components

Out of scope:

TTS-only transport changes
telephony channel implementation
EOS/VAD algorithm changes

Transport mapping

WS callback baseline: deepgram, assembly-ai, sarvam
SDK baseline: azure, google, aws
HTTP/other baseline: provider-specific fallback keeping packet contract

Packet contract

Input:

UserAudioPacket

Required outputs:

SpeechToTextPacket (interim/final)
InterruptionPacket{Source:"word"} when provider exposes speech-onset word signal
ConversationEventPacket{Name:"stt", ...}
MessageMetricPacket{Name:"stt_latency_ms"} per utterance

Implementation workflow

Select baseline provider by transport.
Implement STT in provider folder and callback/parser helpers.
Register provider in STT factory switch.
Wire UI model/language metadata and provider form.
Validate partial/final ordering and context IDs.
Add tests for init, interim/final, reconnect/retry, latency metric.

Validation commands

go test ./api/assistant-api/internal/transformer/... -run TestSTT
go test ./api/assistant-api/internal/transformer/<provider>/...
cd ui && yarn test providers
./.claude/skills/stt-integration/scripts/validate.sh --check-diff --provider <provider>

同仓库更多 Skills

同仓库

noise-reduction-integration

rapidaai/voice-ai

Add or modify noise-reduction providers in assistant-api with denoiser factory wiring, packet safety, and UI option compatibility.

2026-03-23701

noise-reduction-integration

rapidaai/voice-ai

Add or modify microphone noise-reduction providers in assistant-api with denoiser factory wiring, packet safety, and option/UI compatibility.

2026-03-23701

end-of-speech-integration

rapidaai/voice-ai

Add or modify end-of-speech integrations in assistant-api with strict separation from VAD internals. Use for transcript/audio/history-aware turn-finalization logic, provider wiring, and EOS UI config.

2026-03-23701

llm-integration

rapidaai/voice-ai

Add or modify LLM/integration providers in integration-api with caller factory wiring, unified provider routing, streaming behavior, and metric/audit compatibility.

2026-03-23701

local-setup-and-run

rapidaai/voice-ai

Explain and validate local setup paths for this repo with Docker and without Docker. Use when developers need exact prerequisites, startup commands, health checks, and troubleshooting steps.

2026-03-23701

system-understanding

rapidaai/voice-ai

Build a code-grounded implementation plan before coding. Use to trace packet flow, factory boundaries, provider config loading, and exact file-level change scope for a requested integration.

2026-03-23701

name	stt-integration
description	Add or modify speech-to-text providers in assistant-api with transport-aware ingestion (WS/SDK/HTTP), transcript packet correctness, and UI/provider wiring.

STT Integration Skill

Mission

Integrate STT providers that emit reliable interim/final transcripts, interruption signaling, and stt_latency_ms metrics.

Inputs expected from user

Ingestion model: streaming WebSocket, SDK callback stream, or HTTP chunk/segment submit.
Interim transcript support required or final-only acceptable.
Required language/model controls.

If user does not answer:

Follow nearest streaming baseline and preserve current transcript packet semantics.

Hard boundaries

In scope:

api/assistant-api/internal/transformer/<provider>/stt.go (+ provider option/callback helpers)
api/assistant-api/internal/transformer/transformer.go
optional contract updates: api/assistant-api/internal/type/stt_transformer.go, packet.go
provider STT JSON + UI config files/components

Out of scope:

TTS-only transport changes
telephony channel implementation
EOS/VAD algorithm changes

Transport mapping

WS callback baseline: deepgram, assembly-ai, sarvam
SDK baseline: azure, google, aws
HTTP/other baseline: provider-specific fallback keeping packet contract

Packet contract

Input:

UserAudioPacket

Required outputs:

SpeechToTextPacket (interim/final)
InterruptionPacket{Source:"word"} when provider exposes speech-onset word signal
ConversationEventPacket{Name:"stt", ...}
MessageMetricPacket{Name:"stt_latency_ms"} per utterance

Implementation workflow

Select baseline provider by transport.
Implement STT in provider folder and callback/parser helpers.
Register provider in STT factory switch.
Wire UI model/language metadata and provider form.
Validate partial/final ordering and context IDs.
Add tests for init, interim/final, reconnect/retry, latency metric.

Validation commands

go test ./api/assistant-api/internal/transformer/... -run TestSTT
go test ./api/assistant-api/internal/transformer/<provider>/...
cd ui && yarn test providers
./.claude/skills/stt-integration/scripts/validate.sh --check-diff --provider <provider>