feature-verification

# Feature Verification Systematic 4-phase process for verifying new features work correctly across all layers: database, code deployment, runtime execution, and end-to-end functionality. ## When to Use - After implementing new features - When features "should work" but don't behave as expected - When verifying complex integrations - After deployments to production - When debugging deployment-related issues ## Core Methodology Follow these 4 phases in order. Each phase builds on the previous one. ### Phase 1: Root Cause Investigation Gather evidence from all layers before forming hypotheses. **Layer 1: Database Schema** - Verify tables and columns exist - Check data types and constraints - Confirm indexes and relationships - Test RLS policies **Layer 2: Code Deployment** - Verify all files are deployed - Check deployment version number - Confirm imports are correct - Validate environment variables **Layer 3: Runtime Execution** - Add logging to track execution paths - Verify functions are being called - Check async operations complete - Review error logs **Layer 4: End-to-End Testing** - Test complete user workflows - Verify data persistence - Check related features still work - Validate error handling **Detailed checklists:** See `references/verification-checklist.md` ### Phase 2: Pattern Analysis Compare expected vs actual behavior to identify patterns. **Questions to ask:** - What should happen vs what actually happens? - Are there similar working examples to compare against? - Is this issue consistent or intermittent? - What changed recently that might affect this? **Look for patterns:** - Configuration differences between working/broken features - Timing issues (async operations) - Permission/authentication issues - Deployment-specific problems ### Phase 3: Hypothesis and Testing Form testable hypotheses based on evidence, then test systematically. **Hypothesis template:** ``` Hypothesis: [Description of suspected root cause] Test: [How to verify this hypothesis] Evidence: [What evidence supports/refutes this] Conclusion: ✅ CONFIRMED / ❌ RULED OUT / ❓ INCONCLUSIVE ``` **Test each hypothesis:** - Start with most likely causes - Rule out possibilities systematically - Document evidence for each test - Don't skip to solutions without testing **Common root causes:** See `references/common-deployment-issues.md` ### Phase 4: Implementation and Verification Fix identified issues and verify the fixes work. **For each fix:** 1. Document what you're changing and why 2. Implement the fix 3. Redeploy if necessary 4. Retest the feature 5. Verify related features still work **Make fixes robust:** - Add logging for future debugging - Handle edge cases - Update documentation - Consider long-term improvements ## Deliverables Create a verification report documenting: 1. What was tested 2. Evidence gathered from each layer 3. Patterns identified 4. Hypotheses tested 5. Root causes found 6. Fixes implemented 7. Recommendations **Use template:** `templates/verification_report_template.md` ## Best Practices **Evidence Over Assumptions** - Gather data before forming theories - Document what works AND what doesn't - Test hypotheses systematically **Bottom-Up Verification** - Start with database (foundation) - Move to code deployment - Check runtime execution - Test end-to-end last **Work Backwards on Failures** - If end-to-end fails, check runtime - If runtime fails, check deployment - If deployment fails, check code - If code fails, check schema **Document Everything** - Create detailed verification reports - Record evidence at each layer - Document solutions for future reference - Share knowledge with team ## Example: Verifying Conversation Threading **Phase 1: Investigation** ``` Layer 1 (Database): ✅ wade_conversations table exists ✅ wade_messages table exists ✅ All columns present Layer 2 (Deployment): ✅ wade-brain v47 deployed ✅ conversation.ts included ⚠️ JWT verification enabled (should be disabled) Layer 3 (Runtime): ❌ Messages not being stored ❌ last_extraction_at not updating Layer 4 (End-to-End): ❌ Conversation history not working ``` **Phase 2: Pattern Analysis** - Expected: Messages stored after each exchange - Actual: No messages in database - Pattern: JWT verification blocking requests **Phase 3: Hypothesis Testing** ``` Hypothesis: JWT verification is rejecting service role key Test: Disable JWT and retry Evidence: After disabling, requests succeed Conclusion: ✅ CONFIRMED ``` **Phase 4: Implementation** - Disabled JWT verification via dashboard - Redeployed function - Retested: ✅ Messages now storing correctly - Documented: JWT must be disabled for internal services ## Supporting Resources - `references/verification-checklist.md` - Layer-by-layer verification checklists - `references/common-deployment-issues.md` - Known issues and solutions - `templates/verification_report_template.md` - Standard report format ## Related Skills - **systematic-debugging** - When verification reveals issues or bugs, use this skill to identify root causes and implement fixes ## Tips **For Supabase Edge Functions:** - Always verify JWT settings after deployment - Check environment variables are set - Include all files in deployment payload - Test with actual API calls, not just code review **For Database Changes:** - Verify migrations applied successfully - Check RLS policies don't block operations - Test with service role key - Query directly to confirm data structure **For Async Operations:** - Add logging to track completion - Update state before async operations - Use await for critical operations - Handle errors explicitly **For Silent Failures:** - Add descriptive logging with prefixes - Update status before/after operations - Re-throw errors when appropriate - Check logs even when "no errors" reported