deepgram-python-audio-intelligence

# Using Deepgram Audio Intelligence (Python SDK) Analytics overlays applied to `/v1/listen` transcription: summarize, topics, intents, sentiment, language detection, diarization, redaction, entities. Same endpoint / same client methods as STT — enable features via params. ## When to use this product - You have **audio** (file, URL, or live stream) and want analytics alongside the transcript. - REST is the primary path — most analytics are REST-only. **Use a different skill when:** - You want a pure transcript with no analytics → `deepgram-python-speech-to-text`. - Your input is already transcribed text → `deepgram-python-text-intelligence` (`/v1/read`). - You need conversational turn-taking → `deepgram-python-conversational-stt`. - You need a full interactive agent → `deepgram-python-voice-agent`. ## Feature availability: REST vs WSS | Feature | REST | WSS | |---|---|---| | `diarize` | yes | yes | | `redact` | yes | yes | | `punctuate`, `smart_format` | yes | yes | | Entity detection | yes | yes | | `summarize` | yes | **no** | | `topics` | yes | **no** | | `intents` | yes | **no** | | `sentiment` | yes | **no** | | `detect_language` | yes | **no** | | `custom_topic` / `custom_intent` | yes | **no** | For the WSS-only subset, same code path as `deepgram-python-speech-to-text`. ## Authentication ```python from dotenv import load_dotenv load_dotenv() from deepgram import DeepgramClient client = DeepgramClient() ``` Header: `Authorization: Token <api_key>`. ## Quick start — REST with full analytics ```python response = client.listen.v1.media.transcribe_url( url="https://dpgr.am/spacewalk.wav", model="nova-3", smart_format=True, punctuate=True, diarize=True, # speaker separation summarize="v2", # "v2" for the current model; True also accepted on /v1/listen topics=True, intents=True, sentiment=True, detect_language=True, redact=["pci", "pii"], # or Sequence[str] language="en-US", ) r = response.results print("transcript:", r.channels[0].alternatives[0].transcript) print("summary:", r.summary) print("topics:", r.topics) print("intents:", r.intents) print("sentiments:", r.sentiments) print("detected_language:", r.channels[0].detected_language) # Speaker diarization for word in r.channels[0].alternatives[0].words or []: speaker = getattr(word, "speaker", None) if speaker is not None: print(f"Speaker {speaker}: {word.word}") ``` ## Quick start — REST file ```python with open("call.wav", "rb") as f: audio = f.read() response = client.listen.v1.media.transcribe_file( request=audio, model="nova-3", diarize=True, redact=["pii"], summarize="v2", topics=True, ) ``` ## Quick start — diarization with word-level timings Enable speaker separation and word-level timestamps in a single request, then iterate the per-word objects to build a speaker-labelled transcript with timing. ```python response = client.listen.v1.media.transcribe_url( url="https://dpgr.am/spacewalk.wav", model="nova-3", diarize=True, # tag each word with a speaker id smart_format=True, # punctuated_word for cleaner output punctuate=True, ) words = response.results.channels[0].alternatives[0].words or [] # Per-word: speaker, timestamps, confidence for w in words: speaker = getattr(w, "speaker", None) text = w.punctuated_word or w.word print(f"[speaker {speaker}] {text} ({w.start:.2f}s–{w.end:.2f}s, conf={w.confidence:.2f})") # Group consecutive words by speaker into utterances from itertools import groupby for speaker, group in groupby(words, key=lambda w: getattr(w, "speaker", None)): text = " ".join((w.punctuated_word or w.word) for w in group) print(f"Speaker {speaker}: {text}") ``` Per-word fields available on each entry: | Field | Type | Description | |---|---|---| | `word` | `str` | Lowercase token | | `punctuated_word` | `str \| None` | Token with smart-formatted casing/punctuation (when `smart_format=True`) | | `start`, `end` | `float` | Audio timestamps in seconds | | `confidence` | `float` | 0.0–1.0 confidence | | `speaker` | `int \| None` | Speaker id (when `diarize=True`); `None` if diarization disabled | | `speaker_confidence` | `float \| None` | Speaker-id confidence | For a higher-level breakdown, set `utterances=True` to get pre-grouped speaker turns at `response.results.utterances`. Set `paragraphs=True` for a `paragraphs` view organised by speaker turn boundaries. ## Quick start — WSS subset (diarize / redact / entities only) ```python import threading from deepgram.core.events import EventType with client.listen.v1.connect(model="nova-3", diarize=True, redact=["pii"]) as conn: conn.on(EventType.MESSAGE, lambda m: print(m)) threading.Thread(target=conn.start_listening, daemon=True).start() for chunk in audio_chunks: conn.send_media(chunk) conn.send_finalize() ``` ## Key parameters `summarize`, `topics`, `intents`, `sentiment`, `detect_language`, `diarize`, `redact`, `custom_topic`, `custom_topic_mode`, `custom_intent`, `custom_intent_mode`, `detect_entities`, plus all the standard STT params (`model`, `language`, `encoding`, `sample_rate`, ...). `redact` is typed as `Optional[str]` in the current generated SDK (`src/deepgram/listen/v1/media/client.py`). Pass a single redaction mode such as `"pci"`, `"pii"`, `"numbers"`, or `"phi"`. Multi-mode redaction at the transport level is supported by sending `redact` as a repeated query parameter — check `src/deepgram/types/listen_v1redact.py` for the current type and fall back to raw query-param construction (or multiple calls) if you need several modes. The earlier `Union[str, Sequence[str]]` override is no longer carried in `.fernignore`. ## API reference (layered) 1. **In-repo reference**: `reference.md` — "Listen V1 Media" (REST params include all analytics flags), "Listen V1 Connect" (WSS-supported subset). 2. **OpenAPI (REST)**: https://developers.deepgram.com/openapi.yaml 3. **AsyncAPI (WSS)**: https://developers.deepgram.com/asyncapi.yaml 4. **Context7**: library ID `/llmstxt/developers_deepgram_llms_txt`. 5. **Product docs**: - https://developers.deepgram.com/docs/stt-intelligence-feature-overview - https://developers.deepgram.com/docs/summarization - https://developers.deepgram.com/docs/topic-detection - https://developers.deepgram.com/docs/intent-recognition - https://developers.deepgram.com/docs/sentiment-analysis - https://developers.deepgram.com/docs/language-detection - https://developers.deepgram.com/docs/redaction - https://developers.deepgram.com/docs/diarization ## Gotchas 1. **`summarize` on `/v1/listen` accepts a boolean OR the string `"v2"`.** Use `"v2"` to pin the current summarization model; `True` also works (maps to the default model). `/v1/read` is the reverse — it accepts boolean only. If you need summarization on already-transcribed text, see `deepgram-python-text-intelligence`. 2. **Sentiment / topics / intents / summarize / detect_language are REST-only.** Don't pass them on WSS — they'll be ignored or rejected. 3. **English-only** for sentiment / topics / intents / summarize. 4. **Not all models support all overlays.** Flux / Base models have restrictions. Stick to `nova-3` unless you have a reason. 5. **Redaction values** are `pci`, `pii`, `phi`, `numbers`, etc. — not arbitrary strings. 6. **`custom_topic` / `custom_intent` need a mode** (`"extended"` or `"strict"`). 7. **Diarization is noisy on short / low-quality audio.** Expect speaker churn on <30s clips. ## Example files in this repo - `examples/15-transcription-advanced-options.py` — smart_format, punctuate, diarize - `tests/wire/test_listen_v1_media.py` — wire test covering intelligence params ## Related skills - `deepgram-python-speech-to-text` — same endpoint, plain transcription - `deepgram-python-text-intelligence` — same analytics, text input - `deepgram-python-conversational-stt` — Flux for turn-taking - `deepgram-python-voice-agent` — interactive assistants ## Central product skills For cross-language Deepgram product knowledge — the consolidated API reference, documentation finder, focused runnable recipes, third-party integration examples, and MCP setup — install the central skills: ```bash npx skills add deepgram/skills ``` This SDK ships language-idiomatic code skills; `deepgram/skills` ships cross-language product knowledge (see `api`, `docs`, `recipes`, `examples`, `starters`, `setup-mcp`).