social-media-intelligence

# Social Media Intelligence > This skill integrates financial-intelligence collection methods and quantitative applications across Twitter/X, Telegram, Discord, and Reddit. > Inspired by `himself65/finance-skills` modules such as `discord-reader`, `telegram-reader`, and `twitter-reader`. --- ## 1. Overview of the Four Major Financial Social Platforms ### 1.1 Twitter/X — The FinTwit Ecosystem **Core roles** | Role Type | Representative Account Traits | Signal Value | |---------|------------|---------| | Sell-side analyst | Institutional backing, dense posting around earnings | Medium, somewhat lagging | | Fund manager | Holdings views, industry judgment | High, but mixed with subjective opinion | | Macro commentator | Fed interpretation, macro-data reaction | High, a good sentiment barometer | | Crypto KOL | On-chain interpretation, project endorsement | Highly volatile, high manipulation risk | | Retail noise | Meme spread, herd sentiment | Contrarian signal value at extremes | **Core FinTwit circles** - `$TICKER` cashtag system directly maps discussion to the asset - Earnings-season sentiment patterns before and after reports - Real-time reaction speed to policy / macro events, often 15-60 minutes ahead of traditional media --- ### 1.2 Telegram — The Core Venue for Crypto Intelligence **Channel types** | Channel Type | Content Traits | How to Use | |---------|---------|---------| | Signal channels | Specific buy/sell levels, stop-loss / take-profit | Use as a sentiment thermometer, not for blind copy-trading | | Research push channels | Institutional PDF reports, on-chain data | Aggregate information and extract key numbers | | Macro flash channels | Real-time interpretation of FOMC, CPI, etc. | Event-driven signals | | Official project channels | Tokenomics updates, partnership announcements | Potential alpha, but requires filtering | | Whale alert channels | Large on-chain transfer alerts | Capital-flow signal | --- ### 1.3 Discord — Quant Communities and Project Ecosystems **Important community types** - Quant / DeFi research communities such as Degen Spartan and Messari Research - Official crypto project Discords with governance discussion and development progress - Trader communities focused on options flow and on-chain analysis - NFT / GameFi projects with floor-price alerts and activity monitoring **Distinctive value of Discord** - Community activity directly reflects project health - Developer channels such as `#dev` and `#build` show implementation activity - Governance participation indicates the willingness of token holders to stay involved --- ### 1.4 Reddit — A Barometer of Retail Sentiment **Core subreddits** | Subreddit | Core User Base | Main Signal | |-------|---------|---------| | r/wallstreetbets | Retail options traders | Meme-stock heat, abnormal options chatter | | r/investing | Value-oriented retail investors | Long-horizon sentiment, ETF flow | | r/cryptocurrency | Crypto retail | BTC / ETH cycle sentiment | | r/stocks | General stock discussants | Earnings-season sentiment | | r/options | Options-strategy community | Unusual IV-related topics | --- ## 2. Data Collection Methods ### 2.1 Twitter/X Data Collection **Tooling options** ```python # Option A: Official API v2 (paid, basic tier starts at $100/month) # Best for: production environments where compliance is the priority from tweepy import Client client = Client(bearer_token=os.getenv("TWITTER_BEARER_TOKEN")) # Search tweets discussing a cashtag over the last 7 days def fetch_cashtag_tweets(ticker: str, max_results: int = 100) -> list[dict]: """Collect Twitter discussion data for a given ticker. Args: ticker: Ticker symbol such as AAPL or BTC max_results: Max number of returned tweets, between 10 and 100 Returns: List of tweets, each containing id / text / created_at / public_metrics """ query = f"${ticker} -is:retweet lang:en" tweets = client.search_recent_tweets( query=query, max_results=max_results, tweet_fields=["created_at", "public_metrics", "author_id"], ) return [t.data for t in tweets.data or []] # Option B: ntscraper (unofficial, free, rate-limited) # Best for: research / historical backtesting # pip install ntscraper from ntscraper import Nitter scraper = Nitter() tweets = scraper.get_tweets("$AAPL", mode="term", number=50) ``` **Data schema (Twitter JSON Schema)** ```json { "platform": "twitter", "collected_at": "2026-03-29T08:00:00Z", "query": "$AAPL", "items": [ { "id": "tweet_id_string", "text": "tweet text", "created_at": "ISO8601 timestamp", "author": { "id": "user_id", "username": "handle", "followers_count": 50000, "verified": false }, "metrics": { "like_count": 120, "retweet_count": 45, "reply_count": 23, "quote_count": 8 }, "sentiment_score": null, "tags": ["$AAPL", "#earnings"] } ] } ``` **Suggested collection frequency** - Earnings season / major events: real time, poll every 5 minutes - Routine monitoring: hourly - Historical backfill: daily batch --- ### 2.2 Telegram Data Collection **Tooling** ```python # Telethon — official MTProto client, requires API_ID + API_HASH # pip install telethon from telethon.sync import TelegramClient from telethon import functions API_ID = int(os.getenv("TELEGRAM_API_ID")) API_HASH = os.getenv("TELEGRAM_API_HASH") async def fetch_channel_messages( channel_username: str, limit: int = 200, offset_date: datetime | None = None, ) -> list[dict]: """Collect historical messages from a Telegram channel. Args: channel_username: Channel username without @, e.g. "whale_alert" limit: Maximum number of messages offset_date: Start time to backtrack from Returns: List of messages containing id / text / date / views / forwards """ async with TelegramClient("session", API_ID, API_HASH) as client: messages = [] async for msg in client.iter_messages( channel_username, limit=limit, offset_date=offset_date ): if msg.text: messages.append({ "id": msg.id, "text": msg.text, "date": msg.date.isoformat(), "views": getattr(msg, "views", 0), "forwards": getattr(msg, "forwards", 0), }) return messages ``` **Data schema (Telegram JSON Schema)** ```json { "platform": "telegram", "channel": "whale_alert", "collected_at": "2026-03-29T08:00:00Z", "items": [ { "id": 12345, "text": "message text", "date": "ISO8601 timestamp", "views": 85000, "forwards": 320, "has_media": false, "reply_to_msg_id": null, "sentiment_score": null } ] } ``` **Suggested collection frequency** - Whale-alert / flash channels: real-time push, webhook mode - Signal channels: every 30 minutes - Research channels: daily --- ### 2.3 Discord Data Collection **Tooling** ```python # discord.py — official Bot API, requires Bot Token + server invitation permission # pip install discord.py import discord from discord.ext import commands async def fetch_channel_history( channel_id: int, limit: int = 500, after: datetime | None = None, ) -> list[dict]: """Collect message history from a Discord channel. Args: channel_id: Discord channel ID limit: Maximum number of messages, capped at 500 per request after: Start timestamp to backtrack from Returns: List of messages containing id / content / timestamp / author / reactions """ bot = commands.Bot(command_prefix="!") messages = [] @bot.event async def on_ready(): channel = bot.get_channel(channel_id) async for msg in channel.history(limit=limit, after=after): messages.append({ "id": str(msg.id), "content": msg.content, "timestamp": msg.created_at.isoformat(), "author": { "id": str(msg.author.id), "name": msg.author.name, "bot": msg.author.bot, }, "reaction_count": sum(r.count for r in msg.reactions), "attachments": len(msg.attachments), }) await bot.close() await bot.start(os.getenv("DISCORD_BOT_TOKEN")) return messages ``` **Data schema (Discord JSON Schema)** ```json { "platform": "discord", "guild_id": "server_id_string", "channel_id": "channel_id_string", "channel_name": "general-trading", "collected_at": "2026-03-29T08:00:00Z", "items": [ { "id": "message_id_string", "content": "message text", "timestamp": "ISO8601 timestamp", "author": { "id": "user_id_string", "name": "username#1234", "roles": ["Member", "Whale"], "bot": false }, "reaction_count": 42, "thread_count": 3, "sentiment_score": null } ] } ``` **Suggested collection frequency** - Active trading communities: hourly - Official project channels: every 4 hours - Governance channels: daily --- ### 2.4 Reddit Data Collection **Tooling** ```python # PRAW — official Reddit Python wrapper, free API # pip install praw import praw def fetch_subreddit_posts( subreddit_name: str, mode: str = "hot", limit: int = 100, time_filter: str = "day", ) -> list[dict]: """Collect hot posts and related metadata from a subreddit. Args: subreddit_name: Subreddit name, e.g. "wallstreetbets" mode: Sort mode: "hot" / "new" / "top" / "rising" limit: Maximum number of posts time_filter: Time filter for top mode, e.g. "hour" / "day" / "week" Returns: List of posts containing id / title / score / comments / created_utc """ reddit = praw.Reddit( client_id=os.getenv("REDDIT_CLIENT_ID"), client_secret=os.getenv("REDDIT_CLIENT_SECRET"), user_agent="vibe-trading/1.0", ) subreddit = reddit.subreddit(subreddit_name) posts = [] fetch_fn = { "hot": subreddit.hot, "new": subreddit.new, "top": lambda limit: subreddit.top(time_filter=time_filter, limit=limit), "rising": subreddit.rising, }[mode] for post in fetch_fn(limit=limit): posts.append({ "id": post.id, "title": post.title, "selftext": post.selftext[:500], # truncated body "score": post.score, "upvote_ratio": post.upvote_ratio, "num_comments": post.num_comments, "created_utc": post.created_utc, "url": post.url, "flair": post.link_flair_text, }) return posts ``` **Data schema (Reddit JSON Schema)** ```json { "platform": "reddit", "subreddit": "wallstreetbets", "collected_at": "2026-03-29T08:00:00Z", "items": [ { "id": "post_id", "title": "post title", "selftext": "body summary (500 chars)", "score": 12500, "upvote_ratio": 0.94, "num_comments": 847, "created_utc": 1743206400.0, "flair": "YOLO", "mentioned_tickers": ["GME", "AMC"], "sentiment_score": null } ] } ``` **Suggested collection frequency** - r/wallstreetbets around the market open: every 30 minutes - r/investing / r/stocks: every 4 hours - r/cryptocurrency: hourly --- ### 2.5 Compliance and Privacy Notes **Must comply with** - Twitter API terms: do not resell data to third parties; obey rate limits such as the basic-tier 500,000 tweets/month allowance - Telegram personal messages must not be collected; only public channels / groups are in scope - Discord must be accessed through the official Bot API; self-bots violate ToS and may get banned - Reddit PRAW rate limit: 60 requests/minute for authenticated users **Data storage rules** - Store user IDs in masked form such as hashes; do not retain raw usernames - Store raw text locally only and do not expose it through public APIs - Periodically purge raw data older than 30 days and keep only aggregated metrics --- ## 3. Sentiment Quantification Methodology