| name | openai-transcribe |
| description | Transcribe audio with OpenAI speech-to-text models (gpt-4o-transcribe, gpt-4o-mini-transcribe, whisper-1) using prompt-guided terminology, segment-level timestamps, optional diarization via post-processing, and reliable artifact delivery. Use when converting interviews, calls, podcasts, or meeting recordings into reviewable text. Requires OPENAI_API_KEY. |
OpenAI Transcribe
Produce trustworthy, reviewable transcripts from audio using OpenAI's speech-to-text APIs with deliberate model selection, terminology priming, chunking for long media, and explicit confidence reporting.
Stack Baseline (2026)
| Concern | Choice | Notes |
|---|
| Default model | gpt-4o-transcribe | Highest accuracy; multilingual; lower WER than whisper-1 |
| Cost-tier model | gpt-4o-mini-transcribe | Use for bulk/low-stakes work |
| Legacy fallback | whisper-1 | Only when needing verbose_json segment timestamps natively |
| Endpoint | POST /v1/audio/transcriptions | Multipart upload |
| Streaming | stream=true (Realtime/SSE) | Live captions, voice agents |
| Max file size | 25 MB per request | Chunk longer files (≤10 min slices recommended) |
| Supported formats | mp3, mp4, m4a, wav, webm, flac, ogg, mpga, mpeg | Re-encode unusual codecs to wav 16kHz mono first |
| Diarization | Not native | Use pyannote.audio 3.x or AssemblyAI for speaker labels, then align |
| Timestamps | timestamp_granularities=["segment","word"] (whisper-1) or response includes segments (gpt-4o-transcribe) | |
| SDK | openai Python ≥1.50, Node openai ≥4.70 | |
When to Use
- Interview, podcast, lecture, or meeting transcription.
- Compliance recordings requiring searchable text.
- Generating subtitle/caption tracks (SRT/VTT).
- Building a voice-input feature where latency tolerates batch.
Prerequisites
OPENAI_API_KEY exported in environment (never hard-coded).- Audio file path and duration known; file readable and within format support.
- Decision on output: plain text, timestamped, speaker-labeled, SRT/VTT, or summary derivatives.
- For long audio:
ffmpeg available for re-encoding/chunking.
Instructions
- Inspect input. Check duration, container, sample rate, channels, and noise floor:
ffprobe -v error -show_entries format=duration,bit_rate,format_name \
-show_entries stream=codec_name,sample_rate,channels file.m4a
- Normalize if needed. Convert to 16 kHz mono WAV for best accuracy and smallest size:
ffmpeg -i input.m4a -ac 1 -ar 16000 -c:a pcm_s16le normalized.wav
- Chunk if >25 MB or >25 min. Split on silence to avoid mid-word cuts:
ffmpeg -i normalized.wav -f segment -segment_time 600 \
-c copy chunks/part_%03d.wav
- Prime terminology with the
prompt parameter (≤244 tokens) — list proper nouns, product names, jargon. The model treats it as prior context, not literal text:
from openai import OpenAI
client = OpenAI()
prompt_terms = (
"Speakers: Dr. Acharya, Mei-Lin Park. "
"Project: Helios. Acronyms: SLO, RTO, FAPI."
)
with open("normalized.wav", "rb") as f:
result = client.audio.transcriptions.create(
model="gpt-4o-transcribe",
file=f,
language="en",
prompt=prompt_terms,
response_format="json",
temperature=0,
)
transcript = result.text
- For per-segment timestamps, use
whisper-1 with verbose_json:
v = client.audio.transcriptions.create(
model="whisper-1",
file=open("normalized.wav", "rb"),
response_format="verbose_json",
timestamp_granularities=["segment", "word"],
)
for seg in v.segments:
print(f"[{seg.start:.2f}-{seg.end:.2f}] {seg.text}")
- Reassemble chunks by adding cumulative offsets to each chunk's segment timestamps before merging.
- Diarize (optional). Run
pyannote/speaker-diarization-3.1 separately, then assign each transcript segment to the speaker whose RTTM span overlaps most. Never fabricate speaker labels without a diarization step.
- Clean for human review. Fix punctuation/casing only; preserve numbers, names, product terms verbatim. Mark unclear regions with
[unclear] rather than guessing.
- Emit artifacts: raw response JSON, cleaned
.txt, optional .srt/.vtt, and a confidence_notes.md listing low-quality regions (high WER risk: overlapping speech, music bleed, clipping).
- Cost & latency check. gpt-4o-transcribe is billed per minute of audio; log duration × rate before bulk runs.
Common Pitfalls
| Pitfall | Impact | Fix |
|---|
| Sending raw 48 kHz stereo m4a | Higher cost, no accuracy gain | Downmix to 16 kHz mono WAV first |
Treating prompt as a transcript prefix | Model hallucinates the prompt into output | Use only as terminology hint; verify output start |
| Splitting on fixed time without silence detection | Words cut mid-syllable, segment loss | Use silencedetect filter or pydub.split_on_silence |
Hardcoding OPENAI_API_KEY in scripts | Credential leak in repo/CI logs | Read from env / secret manager only |
| Trusting transcripts of low-SNR audio | Confident-sounding hallucinations | Compute audio RMS; flag segments below threshold |
| Assuming diarization from speaker-like punctuation | Wrong speaker attribution | Run a real diarizer; align by overlap |
| Retrying on 429 without backoff | Rate-limit spiral | Exponential backoff + Retry-After header |
| Logging full transcript to stdout in CI | PII leak in build logs | Write to file only; redact in summaries |
Output Format
Deliver to the caller:
transcript.raw.json — provider response, untouched.transcript.clean.txt — punctuation/casing normalized; verbatim content.transcript.srt or .vtt — when timestamps requested.confidence_notes.md — list of [start–end] ranges flagged low-confidence with reason (noise, overlap, accent).- Optional derivatives (only if requested):
summary.md, action_items.md, decisions.md.
Authoritative References
