| name | youtube-content |
| description | Process, summarize, and extract information from YouTube videos. Covers metadata lookup, transcript extraction, and content analysis for wrestling analysis and general video research. |
YouTube Content Processing
Used by WWE and other profiles to process YouTube video links shared in conversation.
Two-tier approach
Tier 1 — Quick metadata (oembed, no auth needed)
Fastest path for getting title, author, thumbnail type. Use when the user just asks "what is this video about?"
yt-dlp --skip-download --print "%%(title)s\n%%(uploader)s\n%%(duration)s" "URL"
Fallback when yt-dlp fails (auth block, rate limit):
Use the YouTube oembed API — no cookies, no auth, works server-side:
import urllib.request, json
oembed = 'https://www.youtube.com/oembed?url=' + url + '&format=json'
req = urllib.request.Request(oembed, headers={'User-Agent': 'Mozilla/5.0'})
resp = urllib.request.urlopen(req, timeout=10)
data = json.loads(resp.read())
The oembed endpoint returns: title, author_name, author_url, type, height, width, thumbnail_url.
Tier 2 — Full transcript + analysis (requires yt-dlp with cookies or fallback)
When the user wants a detailed summary, breakdown, or analysis of a specific video.
yt-dlp approach (auth-gated):
- Install:
pip install yt-dlp
- Basic usage:
yt-dlp --skip-download --dump-json "URL"
- Auth: YouTube may require
--cookies-from-browser or manual cookie export.
- If blocked by Sign-in wall, fall back to alternative approaches.
Alternative transcript approaches:
- YouTube's internal captions API can sometimes be accessed without auth for public videos.
- SearXNG (via researchit profile) can find discussion/recaps of the video.
- Manual viewing is not possible server-side without a browser.
Workflow for wrestling videos
- Get metadata via oembed (fast, no auth).
- If user asks for analysis and yt-dlp fails, explain the auth limitation and offer to:
- Search for discussions/recaps about the video topic
- Analyze the video from title + channel context
- Record key takeaways in the profile bank (wwe-profile for wrestling content).
Tier 3 — Web search fallback (when oembed is not enough but yt-dlp is blocked)
When the user asks for more depth (transcript, analysis, explanation) and all direct YouTube access is blocked on cloud IPs:
Approach: Use DuckDuckGo HTML search with the video title to find discussions, descriptions, and reviews of the video content.
curl -sL "https://html.duckduckgo.com/html/?q=<search-query>" \
-H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
Then extract result snippets with regex patterns:
- Pattern:
class="result__snippet">(.*?)</a> or class="result__snippet"[^>]*>(.*?)</(?:a|div)
- Clean HTML tags and decode HTML entities with Python:
re.sub(r'<[^>]+>', '', text) and html.unescape()
Search results often include:
- TikTok/Reddit posts discussing the video content
- Blog/article snippets that describe the video's thesis
- Forum discussions with context
Combine multiple search result snippets to reconstruct the video's content and thesis. This is NOT a transcript but gives enough context for a meaningful summary.
Combined workflow when YouTube is blocked:
oembed → title + channel (always works)- If user wants more depth → DuckDuckGo search with video title
- Synthesize from search snippets → give user a contextual summary
- Store learnings in the relevant profile bank (e.g. wwe-profile)
What does NOT work on cloud IPs (OCI, AWS, GCP)
All of these WILL fail from cloud provider IPs — do not attempt unless the user provides auth:
yt-dlp — always "Sign in to confirm you're not a bot"youtube-transcript-api — raises RequestBlocked- Invidious instances — cloud IP blocks (all instances return empty/error)
- Direct YouTube page fetch returns only generic placeholder description
Only two things work without auth from cloud IPs:
- oembed API (title + author only)
- DuckDuckGo HTML search (indirect context via search snippets)
Pitfalls
- yt-dlp WILL hit "Sign in to confirm you're not a bot" on YouTube without cookies. Do not retry — fall back to oembed immediately.
- youtube-transcript-api WILL raise
RequestBlocked from cloud IPs. Do not attempt from OCI/AWS/GCP.
- The oembed API does NOT return video description, tags, or full metadata — just title, author, thumbnail.
- Do NOT attempt to use the YouTube Data API without first establishing a Composio connection for the YouTube toolkit.
- Python
yt-dlp library is available after pip install yt-dlp but YouTube auth requirements apply there too.
- DuckDuckGo HTML search snippets are short (~200 chars) and may not capture the video's full argument. Synthesize carefully and caveat the user if the picture is incomplete.
Verification
After getting metadata, confirm to the user: video title, channel name, brief contextual assessment based on title/channel.
