open-source-tool-evaluation

name	open-source-tool-evaluation
description	Evaluate, configure, and run open-source GitHub tools in the local environment. Covers real dependency auditing, hidden system requirements, API configuration, and zero-cost local LLM integration.
trigger	- User asks to evaluate, install, run, or integrate an open-source GitHub project - User shares GitHub repo links and asks "does this work?" / "can you run this?" - Any task involving cloning a repo and making it operational - Auditing what a repo actually needs to run vs what its README claims

Open-Source Tool Evaluation

Core principle: Requirements.txt + README ≠ runnable. Always verify with real system probes.

1. Clone & Inspect (2 min)

mkdir -p ~/repos-analysis && cd ~/repos-analysis
git clone --depth 1 <repo-url>
cd <repo>

Check immediately:

• requirements.txt, pyproject.toml, package.json — dependency lists
• .env.example, config.yaml — configuration templates
• docker-compose.yml, Dockerfile — containerization
• README.md — claimed setup steps (treat as aspirational, not factual)

2. Real Dependency Audit (5 min) — CRITICAL

Never trust requirements.txt alone. Run these checks:

# Python packages
pip list | grep -iE "<key-package>"
python3 -c "import <package>" 2>/dev/null && echo "OK" || echo "MISSING"

# System-level deps (common hidden blockers)
# - weasyprint → needs libpango, libcairo, libgdk, gtk+
# - playwright → needs chromium browser download (~150MB)
# - torch → ~500MB-2GB download
# - postgres/mysql → needs running service

dpkg -l | grep -iE "pango|cairo|gdk|gtk"  # Debian/Ubuntu
which psql && psql --version              # PostgreSQL
docker ps | grep -i postgres              # PG container
ls ~/.cache/ms-playwright/                # Playwright browsers

# API keys / credentials
grep -E "API_KEY|SECRET|TOKEN|PASSWORD" .env.example 2>/dev/null | wc -l

3. Check What's Already Running

User rule: verify existing state BEFORE cloning or installing anything.

# Check if repo already cloned
ls -la ~/repos-analysis/<repo>/ 2>/dev/null && echo "EXISTS"

# Databases
docker ps | grep -iE "postgres|mysql|redis|mongo"
systemctl status postgresql 2>/dev/null

# Services on common ports
ss -tlnp | grep -E ":5000|:8000|:3000|:8080|:11434"

# Local Ollama (zero-cost LLM)
curl -s http://localhost:11434/api/tags | python3 -c "import sys,json; [print(m['name']) for m in json.load(sys.stdin).get('models',[])]"

# Check for existing containers with this app's name
docker ps -a | grep -iE "<app-name>"

4. Configure for Local Environment

4a. TrendRadar + Ollama (Zero-Cost Pattern)

cd <trendradar-clone>
pip install pyyaml fastmcp feedparser litellm json-repair pytz

# Edit config/config.yaml
#   ai.model: "ollama/<model-name>"
#   ai.api_key: "ollama"
#   ai.api_base: "http://localhost:11434"
#   notification.enabled: false   # (optional, for testing)

RUN_MODE=once python3 -m trendradar

Known working Ollama models: deepseek-coder-v2:16b, qwen3.5:35b, llama3.1:70b

Network note: Chinese platforms (toutiao, baidu, weibo, zhihu, douyin) are accessible from Vietnam without proxy.

For Docker deployments accessing Ollama on WSL host:

# host.docker.internal doesn't always work in WSL
WSL_IP=$(hostname -I | awk '{print $1}')
# Use http://$WSL_IP:11434 instead of http://host.docker.internal:11434

4b. BettaFish (Heavyweight)

Requires: PostgreSQL (running), 10 API keys, Playwright Chromium, torch, sentiment models (~3-5GB total). See references/bettafish-specs.md.

BettaFish Docker fix: The container crashes with exit 127 if .env is mounted as a directory instead of a file. Fix:

docker stop bettafish && docker rm bettafish
docker run -d --name bettafish -p 5000:5000 -v "$(pwd)/.env:/app/.env" ...
# NOT: -v ./bettafish-dir:/app (this makes /app/.env a directory)

5. Run & Verify

# Run once (don't start cron/daemon for testing)
RUN_MODE=once <command>

# Capture output
timeout 300 <command> 2>&1 | tee /tmp/run.log

# Verify outputs
ls -la output/ final_reports/ logs/
du -sh output/

6. Verify HTTP Endpoints

User rule: always provide concrete URLs, not "it's running."

# Test the main endpoint
curl -s http://localhost:<port>/ | head -5

# Verify specific routes
curl -s http://localhost:<port>/api/health 2>/dev/null || echo "No health endpoint"

# For web UIs, extract the title to confirm it loaded
curl -s http://localhost:5000/ | grep -o "<title>.*</title>"

# Report to user:
#   "Main UI:    http://localhost:5000"
#   "API:        http://localhost:5000/api"
#   "Status:     ✅ Running (title: '微舆')"

6. Report to User

Present in this format:

| Component | Status | Notes |
|-----------|--------|-------|
| Python deps | ✅/❌ | which are missing |
| Database | ✅/❌ | running? port? |
| System libs | ✅/❌ | GTK, Cairo, etc. |
| API keys | ❌ | count needed |
| Estimated setup time | X min | |
| Estimated running cost | $X/day | or $0 for Ollama |

Pitfalls

• Playwright: pip install playwright ≠ working. Must run playwright install chromium separately.
• Torch: CPU vs GPU versions differ wildly in size. Check requirements.txt comments.
• WeasyPrint: Fails silently if system GTK libs missing. Install libpango1.0-dev libcairo2-dev first.
• Chinese platforms: Usually accessible from SE Asia, but RSS feeds and some APIs may need proxy.
• LiteLLM + Ollama: Use format ollama/<model> with api_key: "ollama" and api_base: "http://host:11434".
• "All 11 platforms succeeded" in logs doesn't mean analysis was useful — check the HTML report content.

References

• references/bettafish-specs.md — Full BettaFish architecture, API keys, hidden blockers
• references/trendradar-ollama-setup.md — TrendRadar + Ollama zero-cost configuration, network notes, MCP setup