reddit-product-viability

name	reddit-product-viability
description	Scrape and analyze Reddit for real user signals about product viability, pain severity, willingness to pay, and competitor saturation. Validate product ideas before building by systematically analyzing discussions, complaints, feature requests, and purchasing behavior across relevant subreddits. Integrates with Firecrawl for scraping, Supabase for storage, and Superset for trend visualization.

Reddit Product Viability Research

When to Use This Skill

Use this skill when you need to:

• Validate product ideas before investing development time
• Assess market demand through real user conversations
• Identify pain points and severity across target segments
• Evaluate willingness to pay based on user discussions
• Analyze competitor saturation and gaps in solutions
• Discover feature requests and unmet needs
• Monitor product-market fit signals over time
• Research SaaS alternative opportunities (like SAP Concur, Ariba alternatives)

Core Capabilities

Product Viability Validation Framework

Systematically evaluate four critical dimensions:

1. Real Demand Signals

   • Volume of discussions about the problem
   • Frequency of complaints and pain points
   • Emotional intensity in user posts
   • Problem persistence over time

2. Pain Severity Assessment

   • Impact on users' work/life
   • Workarounds currently being used
   • Time/money currently wasted
   • Urgency of need for solution

3. Willingness to Pay

   • Current spending on alternatives
   • Budget discussions and constraints
   • "Shut up and take my money" signals
   • Pricing tolerance indicators

4. Competitor Saturation

   • Existing solutions mentioned
   • User satisfaction with alternatives
   • Gap analysis (unfulfilled needs)
   • Market positioning opportunities

Technical Implementation

• Reddit API + Firecrawl - Scrape subreddits, threads, comments
• Supabase Storage - Store posts with deduplication
• NLP Analysis - Sentiment, entity extraction, topic modeling
• Superset Dashboards - Visualize trends and insights
• Notion Integration - Track validation findings
• Scheduled Monitoring - Daily/weekly trend analysis

Prerequisites

Required Access

• Reddit API key (free tier: 100 requests/minute)
• Firecrawl API key (self-hosted or paid)
• Supabase project with pgvector
• Superset instance for visualization

Optional Integrations

• OpenAI API for GPT-4 analysis
• Perplexity API for research enhancement
• Notion for findings documentation

Python Dependencies

praw  # Reddit API wrapper
firecrawl-py
supabase-py
pandas
numpy
transformers  # For sentiment analysis

Implementation Patterns

Product Validation Prompt Template

✅ Product Viability — Reddit Insight Prompt

Goal: Validate real demand, pain severity, willingness to pay, and competitor 
saturation for [PRODUCT_IDEA].

Scrape and analyze Reddit for real user signals about the following product idea:

**Product Idea:** [Your product concept]

**Target Subreddits:**
- r/[relevant_sub1]
- r/[relevant_sub2]
- r/[relevant_sub3]

**Analysis Timeframe:** Past [6/12/24] months

**Key Questions to Answer:**

1. **Real Demand:**
   - How many users discuss this problem?
   - How often does it come up?
   - What triggers discussions about it?
   - Is the problem persistent or seasonal?

2. **Pain Severity:**
   - What impact does the problem have?
   - What workarounds are users trying?
   - How much time/money is being wasted?
   - What's the urgency level?

3. **Willingness to Pay:**
   - What are users currently spending on alternatives?
   - What's their budget range?
   - Are there "shut up and take my money" signals?
   - What pricing would be acceptable?

4. **Competitor Saturation:**
   - Which solutions are mentioned?
   - What are users' complaints about alternatives?
   - What gaps exist in current solutions?
   - Where's the market positioning opportunity?

**Output Format:**
- Quantitative metrics (post volume, sentiment scores)
- Qualitative insights (user quotes, pain points)
- Competitor analysis matrix
- Recommended next steps
- Risk factors and red flags

Reddit Scraping Script

# reddit_viability_scraper.py
import praw
from firecrawl import FirecrawlApp
from supabase import create_client
from datetime import datetime, timedelta
import pandas as pd
from transformers import pipeline

class RedditViabilityAnalyzer:
    def __init__(self, supabase_url, supabase_key, reddit_client_id, reddit_secret):
        # Initialize Reddit client
        self.reddit = praw.Reddit(
            client_id=reddit_client_id,
            client_secret=reddit_secret,
            user_agent='ProductViabilityBot/1.0'
        )
        
        # Initialize Supabase
        self.supabase = create_client(supabase_url, supabase_key)
        
        # Initialize sentiment analyzer
        self.sentiment_analyzer = pipeline("sentiment-analysis")
    
    def scrape_subreddit(self, subreddit_name, keywords, timeframe_months=6):
        """
        Scrape subreddit for product validation signals
        """
        subreddit = self.reddit.subreddit(subreddit_name)
        posts = []
        
        # Calculate timeframe
        cutoff_date = datetime.now() - timedelta(days=timeframe_months * 30)
        
        # Search for keywords
        for keyword in keywords:
            results = subreddit.search(
                keyword,
                sort='relevance',
                time_filter='year',
                limit=100
            )
            
            for post in results:
                if datetime.fromtimestamp(post.created_utc) >= cutoff_date:
                    # Extract post data
                    post_data = {
                        'id': post.id,
                        'title': post.title,
                        'text': post.selftext,
                        'score': post.score,
                        'num_comments': post.num_comments,
                        'created_utc': post.created_utc,
                        'url': post.url,
                        'subreddit': subreddit_name,
                        'keyword': keyword,
                        'scraped_at': datetime.now().isoformat()
                    }
                    
                    # Get top comments
                    post.comments.replace_more(limit=0)
                    comments = []
                    for comment in post.comments.list()[:10]:  # Top 10 comments
                        comments.append({
                            'text': comment.body,
                            'score': comment.score,
                            'created_utc': comment.created_utc
                        })
                    
                    post_data['comments'] = comments
                    posts.append(post_data)
        
        return posts
    
    def analyze_demand_signals(self, posts):
        """
        Analyze volume, frequency, and intensity of demand signals
        """
        df = pd.DataFrame(posts)
        
        analysis = {
            'total_posts': len(df),
            'avg_score': df['score'].mean(),
            'avg_comments': df['num_comments'].mean(),
            'total_engagement': df['score'].sum() + df['num_comments'].sum(),
            'posts_per_month': len(df) / 6,  # Assuming 6 month timeframe
            'top_posts': df.nlargest(5, 'score')[['title', 'score', 'url']].to_dict('records')
        }
        
        return analysis
    
    def analyze_pain_severity(self, posts):
        """
        Analyze pain points and their severity
        """
        pain_indicators = [
            'frustrated', 'annoying', 'waste of time', 'terrible',
            'awful', 'nightmare', 'ridiculous', 'broken', 'useless'
        ]
        
        high_pain_posts = []
        for post in posts:
            text = f"{post['title']} {post['text']}".lower()
            pain_score = sum(1 for indicator in pain_indicators if indicator in text)
            
            if pain_score > 0:
                high_pain_posts.append({
                    'title': post['title'],
                    'pain_score': pain_score,
                    'score': post['score'],
                    'url': post['url']
                })
        
        high_pain_posts.sort(key=lambda x: x['pain_score'], reverse=True)
        
        return {
            'high_pain_posts_count': len(high_pain_posts),
            'avg_pain_score': sum(p['pain_score'] for p in high_pain_posts) / len(high_pain_posts) if high_pain_posts else 0,
            'top_pain_posts': high_pain_posts[:10]
        }
    
    def analyze_willingness_to_pay(self, posts):
        """
        Analyze pricing discussions and budget indicators
        """
        price_keywords = [
            'price', 'cost', 'expensive', 'cheap', 'free', 'subscription',
            'monthly', 'yearly', 'budget', 'afford', 'worth', 'pay'
        ]
        
        pricing_posts = []
        for post in posts:
            text = f"{post['title']} {post['text']}".lower()
            if any(keyword in text for keyword in price_keywords):
                pricing_posts.append({
                    'title': post['title'],
                    'text': post['text'],
                    'url': post['url'],
                    'score': post['score']
                })
        
        return {
            'pricing_discussion_count': len(pricing_posts),
            'pricing_posts': pricing_posts[:10]
        }
    
    def analyze_competitor_saturation(self, posts):
        """
        Identify competitors and satisfaction levels
        """
        # This would need to be customized per use case
        competitors = {}
        
        for post in posts:
            text = f"{post['title']} {post['text']}".lower()
            
            # Extract competitor mentions (simplified)
            # In practice, use NER or custom extraction
            for comment in post.get('comments', []):
                # Analyze satisfaction with mentioned tools
                sentiment = self.sentiment_analyzer(comment['text'][:512])[0]
                
                # Track competitor mentions and sentiment
                # (Simplified - would need more sophisticated NER)
        
        return competitors
    
    def store_in_supabase(self, posts, analysis):
        """
        Store posts and analysis in Supabase
        """
        # Store raw posts
        for post in posts:
            self.supabase.table('reddit_posts').upsert({
                'post_id': post['id'],
                'title': post['title'],
                'text': post['text'],
                'score': post['score'],
                'num_comments': post['num_comments'],
                'created_at': datetime.fromtimestamp(post['created_utc']).isoformat(),
                'url': post['url'],
                'subreddit': post['subreddit'],
                'keyword': post['keyword'],
                'scraped_at': post['scraped_at']
            }).execute()
        
        # Store analysis summary
        self.supabase.table('viability_analysis').insert({
            'analyzed_at': datetime.now().isoformat(),
            'demand_signals': analysis['demand'],
            'pain_severity': analysis['pain'],
            'pricing_insights': analysis['pricing'],
            'competitor_analysis': analysis['competitors']
        }).execute()
    
    def generate_report(self, analysis):
        """
        Generate human-readable viability report
        """
        report = f"""
# Product Viability Analysis Report

Generated: {datetime.now().strftime('%Y-%m-%d %H:%M')}

## 1. Demand Signals ✅

- **Total Posts Analyzed:** {analysis['demand']['total_posts']}
- **Average Engagement:** {analysis['demand']['avg_score']:.1f} upvotes per post
- **Discussion Frequency:** {analysis['demand']['posts_per_month']:.1f} posts/month
- **Total Community Engagement:** {analysis['demand']['total_engagement']} interactions

**Top Discussions:**
"""
        for post in analysis['demand']['top_posts'][:3]:
            report += f"\n- [{post['title']}]({post['url']}) ({post['score']} upvotes)"
        
        report += f"""

## 2. Pain Severity 🔥

- **High-Pain Posts:** {analysis['pain']['high_pain_posts_count']}
- **Average Pain Score:** {analysis['pain']['avg_pain_score']:.2f}/10

**Most Painful Issues:**
"""
        for post in analysis['pain']['top_pain_posts'][:3]:
            report += f"\n- [{post['title']}]({post['url']}) (Pain: {post['pain_score']}, Score: {post['score']})"
        
        report += f"""

## 3. Willingness to Pay 💰

- **Pricing Discussions:** {analysis['pricing']['pricing_discussion_count']} posts mention pricing

## 4. Competitor Analysis 🎯

(Detailed competitor breakdown would go here)

## Recommendations

Based on the analysis:

1. **Market Validation:** {'STRONG' if analysis['demand']['total_posts'] > 50 else 'WEAK'}
2. **Pain Point Severity:** {'HIGH' if analysis['pain']['high_pain_posts_count'] > 10 else 'MODERATE'}
3. **Suggested Next Steps:**
   - Interview top posters for deeper insights
   - Build MVP focusing on highest pain points
   - Test pricing with {analysis['demand']['total_posts'] // 10} potential users

## Risk Factors ⚠️

- Monitor for seasonal trends
- Validate across multiple subreddits