data-viz

name	data-viz
description	Build world-class data visualizations for the Empathy Ledger platform — force-directed graphs, network maps, geographic overlays, temporal flows, and interactive dashboards using d3, recharts, leaflet, and framer-motion.

Data Visualization & Analysis

Build publication-quality, interactive data visualizations that transform the platform's entire dataset into compelling visual narratives. Think New York Times interactive features, not dashboard widgets.

Usage

/data-viz [visualization-type] [data-scope]

Examples:

• /data-viz network — Force-directed graph of storyteller-story-theme relationships
• /data-viz geographic — Layered map with story density, cultural territories, movement
• /data-viz temporal — Time-flow visualization of story creation, themes over time
• /data-viz themes — Thematic constellation / cluster visualization
• /data-viz impact — Cultural impact radial chart
• /data-viz all — Full insights dashboard with all visualization types
• /data-viz explore — Audit all available data and recommend visualizations

When to Use

• Building new visualizations for the /insights page or any public page
• Analyzing platform data to find patterns, gaps, or stories in the data
• Creating grant-ready or presentation-ready visual outputs
• Exploring relationships between stories, storytellers, themes, and places
• Building interactive explorations for community stakeholders

Philosophy

Data as storytelling, not surveillance. Every visualization should answer: "What truth does this reveal about the movement?" Not "how many clicks did we get."

Visualizations should:

• Celebrate cultural diversity, not flatten it
• Show connections and relationships, not rankings
• Reveal patterns that support the mission
• Be beautiful enough to present to funders, elders, and communities

Available Libraries

Library	Version	Best For
d3	^7.9.0	Force graphs, custom SVG, complex interactions
recharts	^3.2.0	Standard charts (bar, line, area, pie)
chart.js / react-chartjs-2	^4.5.0 / ^5.3.0	Canvas-based charts, radar, polar
leaflet / react-leaflet	^1.9.4 / ^5.0.0	Geographic maps, markers, overlays
framer-motion	^12.23.12	Animations, transitions, scroll effects

Note: For Leaflet, always use dynamic(() => import(...), { ssr: false }) — Leaflet needs window.

Design System

Token	Hex	Use
Ochre	#D4A373	Primary accent, warmth, heritage
Terracotta	#D97757	Energy, action, highlights
Sage	#6B8E72	Growth, nature, secondary
Forest	#2D5F4F	Depth, grounding, contrast
Charcoal	#2C2C2C	Text, dark backgrounds
Cream	#F8F6F1	Light backgrounds, paper texture

Extended palette for multi-series charts:

const VIZ_PALETTE = [
  '#D4A373', // ochre
  '#2D5F4F', // forest
  '#D97757', // terracotta
  '#8B6F47', // bark
  '#5B8A72', // sage-deep
  '#C4956A', // sand
  '#3D7A5F', // eucalypt
  '#A0522D', // sienna
  '#7B9E87', // lichen
  '#B8860B', // goldenrod
]

Font: Georgia (serif) for titles and large numbers. System sans for labels.

Data Sources (Complete Inventory)

Core Tables

Table	Key Fields for Viz	Rows (approx)
stories	themes, story_type, created_at, reading_time_minutes, language, location, word_count, cultural_sensitivity_level, privacy_level, status	113
storytellers	cultural_background, location, languages_spoken, is_elder, is_active, latitude, longitude	77
transcripts	key_quotes, themes, ai_summary, language, duration, word_count, ai_processing_status	~62
media_assets	media_type, file_type, culturally_sensitive, elder_approved, project_code	~284
organizations	type, location, indigenous_controlled	8
projects	status, act_project_code, organization_id	9
galleries	cultural_theme, cultural_sensitivity_level, photo_count	varies

Relationship Tables (for network graphs)

Junction	Connects	Use
project_storytellers	Projects <-> Storytellers	Network hubs
project_galleries	Projects <-> Galleries	Media clusters
gallery_media_associations	Galleries <-> Media	Asset networks
syndication_consent	Stories <-> Sites	Distribution flow

Rich JSON Fields

Field	Location	Contains
stories.themes	stories table	[{name: string}] — thematic tags
transcripts.key_quotes	transcripts table	[{text, theme, ...}] — extractable quotes
transcripts.themes	transcripts table	AI-extracted themes
storytellers.languages_spoken	storytellers table	Language array
media_assets.cultural_tags	media_assets table	Cultural tag array

Visualization Catalog

1. Force-Directed Network Graph (d3)

What: Interactive node-link diagram showing relationships between storytellers, stories, themes, and projects.

Nodes:

• Storytellers (large circles, colored by cultural_background)
• Stories (medium circles, colored by dominant theme)
• Themes (small circles, fixed palette)
• Projects (square nodes, hub positions)

Links:

• Storyteller → Story (wrote)
• Story → Theme (tagged with)
• Storyteller → Project (member of)
• Story → Story (shared themes = proximity)

Interactions:

• Hover: highlight connected nodes, show tooltip
• Click: expand node details panel
• Drag: reposition nodes
• Zoom: d3.zoom()
• Filter: toggle node types, filter by project/theme

Implementation pattern:

'use client'
import { useEffect, useRef } from 'react'
import * as d3 from 'd3'

// Use d3.forceSimulation() with:
// - forceLink() for edges
// - forceManyBody() for repulsion
// - forceCenter() for centering
// - forceCollide() for non-overlap
// Render to SVG, not canvas (for CSS/interaction)

Data query:

-- Nodes: storytellers + their stories + extracted themes
SELECT s.id, s.display_name, s.cultural_background,
       st.id as story_id, st.title, st.themes,
       ps.project_id
FROM storytellers s
JOIN stories st ON st.storyteller_id = s.id
LEFT JOIN project_storytellers ps ON ps.storyteller_id = s.id

2. Thematic Constellation (d3)

What: Radial/cluster layout where themes are central attractors and stories orbit around their themes. Stories with multiple themes create bridges between clusters.

Layout: d3.forceRadial() with theme nodes at fixed radial positions, stories attracted to their theme(s).

Visual effect: Galaxy/constellation metaphor — each theme is a "star system" with stories as planets. Multi-theme stories create visible connections between systems.

3. Geographic Story Map (Leaflet + d3)

What: Multi-layer interactive map showing:

• Layer 1: Storyteller locations (circle markers, sized by story count)
• Layer 2: Cultural territory overlays (color-coded regions)
• Layer 3: Story density heatmap
• Layer 4: Connection arcs between storytellers in same project

Implementation: Leaflet for base map + tile layer. d3 SVG overlay for arcs and custom markers.

Key data: storytellers.latitude, storytellers.longitude, storytellers.location (geocode if needed).

4. Temporal Flow (recharts or d3)

What: Animated timeline showing:

• Story creation over time (area chart)
• Theme emergence and evolution (stacked area)
• Storyteller onboarding (step chart)
• Project milestones (annotated points)

Interactions: Scrub timeline, hover for monthly details, toggle between cumulative and per-month views.

5. Cultural Diversity Sunburst (d3)

What: Nested ring chart: Organization → Project → Cultural Background → Individual contribution count.

Implementation: d3.partition() with d3.arc() rendering.

6. Theme Co-occurrence Matrix (d3)

What: Heatmap showing which themes appear together in stories. Reveals thematic relationships — e.g., "Land & Place" frequently co-occurs with "Family & Heritage."

Data: Cross-tabulate stories.themes — for each story with 2+ themes, increment the co-occurrence count for each pair.

7. Quote Wall / Stream (framer-motion)

What: Flowing, animated display of unattributed quotes from transcripts.key_quotes. Quotes drift across screen, grouped by theme color. Click to pause and read.

8. Impact Radial (d3 or chart.js)

What: Radar/polar chart showing platform impact across dimensions:

• Cultural preservation (elder reviews, sensitivity tags)
• Geographic reach (countries, locations)
• Community engagement (storytellers, stories)
• Media richness (photos, video, audio)
• Data sovereignty (consent rate, privacy controls)

9. Language Tree (d3)

What: Dendrogram or tree layout showing language diversity. Root → Language Family → Language → Speaker count. Highlights endangered/Indigenous languages.

10. Syndication Flow (d3 Sankey)

What: Sankey diagram showing content flow: Stories → Projects → Syndication Sites → External Domains. Width = volume of content flowing through each path.

API Endpoint

All visualizations pull from /api/public/analytics (already built).

For more complex visualizations needing raw relationship data, create specialized endpoints:

GET /api/public/analytics/network    → nodes + edges for force graph
GET /api/public/analytics/geographic → geocoded storyteller positions
GET /api/public/analytics/temporal   → time-series data
GET /api/public/analytics/themes     → co-occurrence matrix

Privacy Rules (NON-NEGOTIABLE)

1. No names in any public visualization — aggregate only
2. Cultural backgrounds: Only show groups with 3+ members
3. Geography: Country/region level only, never specific addresses
4. Quotes: Strip any attribution, show text + theme only
5. No individual node identification in network graphs — use anonymous labels like "Storyteller A" or just cultural background
6. Network graphs: Never reveal specific storyteller-story connections publicly

For authenticated/admin visualizations: Can show names and details with appropriate auth checks.

Component Architecture

All visualization components should follow this pattern:

// src/components/public/viz/[VizName].tsx
'use client'

interface [VizName]Props {
  data: /* typed data from API */
  className?: string
  interactive?: boolean  // default true
}

export function [VizName]({ data, className, interactive = true }: [VizName]Props) {
  // 1. Container ref for d3 rendering
  // 2. useEffect for d3 bindings (cleanup on unmount!)
  // 3. Responsive: use ResizeObserver or recharts ResponsiveContainer
  // 4. Loading state while d3 initializes
  // 5. Mobile: graceful degradation (simpler viz or static fallback)
}

File organization:

src/components/public/viz/
  ForceNetwork.tsx        — Force-directed graph
  ThematicConstellation.tsx — Radial theme clusters
  GeographicStoryMap.tsx  — Multi-layer Leaflet map
  TemporalFlow.tsx        — Timeline area chart
  CulturalSunburst.tsx    — Nested ring chart
  ThemeMatrix.tsx          — Co-occurrence heatmap
  QuoteStream.tsx         — Animated quote flow
  ImpactRadial.tsx        — Radar impact chart
  LanguageTree.tsx        — Language dendrogram
  SyndicationFlow.tsx     — Sankey flow diagram

Build Workflow

1. Query the data — write the API endpoint or extend /api/public/analytics
2. Prototype in isolation — build the component standalone with demo data
3. Wire to real data — connect to API, add loading/error states
4. Add to /insights page — or create a dedicated /insights/[viz-type] route
5. Mobile test — ensure graceful degradation
6. npm run build — verify compilation

Demo Data Strategy

Every visualization must work with demo data when real data is unavailable:

const DEMO_DATA = { /* realistic demo */ }

export function MyViz({ data }: Props) {
  const displayData = data && data.length > 0 ? data : DEMO_DATA
  const isDemo = !data || data.length === 0
  // Show subtle "Preview data" badge when demo
}

Related Skills

• /project-explorer — Audit what data exists for any project
• /data-viz explore — Auto-discover visualization opportunities from current data
• /cultural-review — Required review for any storyteller-facing visualization