liveview-skills-index

id	live_view_index
name	LiveView Skills Index
description	Skills for opening and driving agent-controllable visualization components in the Pantheon UI sidebar — interactive viewers the agent can open, control, and read back. Viewers: Vitessce (spatial / single- cell omics), Viv (bioimage / microscopy), volume3d (3D image volumes — MIP/ISO), spatial3d (3D spatial transcriptomics), Mol*, IGV, Gosling, Cytoscape, MSA, RDKit, phylotree, plus agent-generated apps.
tags	["live-view","visualization","vitessce","viv","bioimage","spatial","single-cell","interactive"]

LiveView — Agent-Controllable UI Components

A LiveView is an interactive component the agent opens in the Pantheon UI's right sidebar, then drives and observes through the live_view tools. Unlike a static plot image, a LiveView is live: the agent changes its state and the user sees it update; the user interacts with it and the agent reads the result back.

Load the relevant skill file before building a visualization.

Architecture — only the LiveView SDK runtime is built in. Every viewer is a plugin: a folder skills/live_view/<name>/ holding <name>.md (this guide), adapter.js (a setup(lv, root) module), and an optional demo.json. open_live_view either resolves a named viewer plugin (view_type="vitessce") or loads an agent-generated component (view_type="custom" + module_url). Adding a viewer = dropping a new <name>/ folder here; no app code changes.

Available skills

Vitessce — spatial / single-cell / imaging data

Open a Vitessce browser to explore spatial transcriptomics, single-cell, and microscopy-imaging datasets: spatial scatterplots, gene-expression coloring, heatmaps, cell-set selection, image layers.

Skill file: vitessce/vitessce.md

When to use:

• Visualizing spatial transcriptomics (10x Visium, Xenium, MERFISH, …)
• Single-cell data with embeddings (UMAP/t-SNE) the user should explore
• Cell segmentations as interactive objects — click/hover a cell, colour by type or gene. (Just viewing boundaries on an image → use Viv; Vitessce does not render clean boundaries.)
• Spatial omics where cells/sets/embeddings matter, not just the image

Viv — bioimage / microscopy viewer

Open a Viv viewer for high-resolution, multiplexed bioimaging — OME-TIFF and OME-Zarr (OME-NGFF): multichannel fluorescence, microscopy, IF/IMC/ CODEX, whole-slide images. Channel colors, contrast, pan/zoom, overview.

Skill file: viv/viv.md

When to use:

• The data is an image — OME-TIFF / OME-Zarr microscopy
• Multichannel fluorescence the user wants to recolour / adjust
• Cloud-hosted or local bioimages (served via serve_local_data)
• Overlaying a cell segmentation / showing cell boundaries on an image (boundaries as an extra channel — the preferred way to view a mask)

volume3d — 3D volumetric image (MIP / iso-surface)

Open a GPU volume renderer for a 3D scalar image — OPT / light-sheet / micro-CT / MRI scans, confocal z-stacks, segmentation probability maps. Ray-casts as a maximum-intensity projection (MIP) or an iso-surface (ISO); rotate, switch mode, tune the threshold. The input is a 3D TIFF / NIfTI / array — the skill converts it to an OME-NGFF Zarr pyramid.

Skill file: volume3d/volume3d.md

When to use:

• A dense 3D image volume the user wants to see in 3D (a flat 2D image → Viv)
• OPT / light-sheet / micro-CT / MRI / confocal z-stack; a 3D mask or density field
• "render / rotate this 3D scan", "show the iso-surface", "MIP of this stack"

spatial3d — 3D spatial transcriptomics (cell point cloud)

Open a 3D point-cloud viewer for single-cell spatial data — every cell a point at its (x, y, z), coloured by a categorical obs column (cell type / cluster) or by a gene's expression, in 3D / 2D / UMAP. Scales to ~1M+ cells. The input is an AnnData (.h5ad) with 3D obsm['spatial'] — the skill converts it to a spatial Zarr.

Skill file: spatial3d/spatial3d.md

When to use:

• 3D spatial transcriptomics (Stereo-seq / MERFISH / Slide-seq / 3D reconstructions), or any AnnData with 3D obsm['spatial']
• The cells live in a 3D volume and the 3D arrangement matters (vs. Vitessce, which is 2D scatterplots) — colour cells by type or by gene
• "show the cells in 3D", "colour the embryo by ", "where is "

Mol* — 3D molecular structures

Open a Mol* viewer for 3D macromolecular structures — proteins, nucleic acids, complexes — from the RCSB PDB, the AlphaFold DB, or a local .pdb / .cif file. Rotate, zoom, inspect; AlphaFold models colour by pLDDT.

Skill file: molstar/molstar.md

When to use:

• Showing a protein / nucleic-acid 3D structure (experimental or predicted)
• Visualising an AlphaFold prediction
• Any .pdb / .cif structure file (see also the structural_biology skill for obtaining / predicting structures)

IGV — genome browser (tracks on a reference)

Open an IGV.js genome browser to view BAM/CRAM alignments, VCF variants, BED/GFF annotations, bigWig coverage — on a reference genome (hg38, mm10, custom FASTA, …). Pan, zoom, jump to a gene or locus.

Skill file: igv/igv.md

When to use:

• "Look at this region in a genome browser" — RNA-seq pileups, ChIP/ATAC peaks, variant calling, splice junctions, CRISPR-screen hits
• Any BAM/CRAM/VCF/BED/GFF/bigWig on a reference genome
• A gene symbol or coordinate range to display

Gosling — designed genomic figures (Vega-Lite-like grammar)

Open a Gosling.js view for designed genomic visualisations — circular ideograms, multi-track / multi-sample layouts, comparative dual-genome views, custom encodings. Driven by a declarative JSON spec.

Skill file: gosling/gosling.md

When to use:

• Circular chromosome ideograms / circos-style plots
• Designed multi-track or sample-faceted genomic figures
• Comparative dual-genome / synteny visualisations
• Anything that's hard to build with matplotlib but easy with a grammar
• Use IGV instead if the task is to look at a BAM / VCF / BED file at a locus; the two viewers do not overlap.

Cytoscape — biological networks & pathways

Open a Cytoscape.js view for interactive networks — protein-protein interactions, signalling / metabolic / regulatory pathways, ontology graphs. Nodes + edges as JSON, built-in layouts (cose, breadthfirst, circle, dagre, ...), CSS-like stylesheet by selectors.

Skill file: cytoscape/cytoscape.md

When to use:

• PPI networks (STRING, BioGRID, IntAct)
• Signalling / metabolic / regulatory pathways
• Gene-regulatory networks (TF → target)
• Any graph the user benefits from interactively (drag, hover, zoom)

MSA — multiple sequence alignment viewer

Open a multiple-sequence-alignment view (EBI Nightingale's <nightingale-msa>) for protein or DNA alignments. Standard colour schemes (clustal, taylor, hydro, zappo, ...), configurable tile sizes.

Skill file: msa/msa.md

When to use:

• Display a pre-computed alignment from MAFFT / MUSCLE / Clustal / MMseqs2
• Compare orthologs at a functional site / domain
• Pair with phylotree for tree + alignment side-by-side

RDKit — 2D small-molecule depictions

Render 2D depictions of small molecules from SMILES / MOL block using RDKit-JS (WebAssembly build). Complements molstar (3D macromolecules) — RDKit is the canonical 2D view for drugs, metabolites, organics.

Skill file: rdkit/rdkit.md

When to use:

• A SMILES (or list) to view as a 2D structure
• Pull drugs / metabolites from ChEMBL / PubChem / DrugBank → render
• Display the substrate / product of a reaction
• Highlight substructures or specific atoms on a molecule

Phylotree — phylogenetic trees

Open a phylotree.js view for an interactive phylogenetic tree from a Newick string. Linear or radial layout, branch-length-scaled; rerooting, ladderise, clade collapse built in.

Skill file: phylotree/phylotree.md

When to use:

• Show a phylogeny from IQ-TREE / RAxML / FastTree / MrBayes / BEAST
• Inspect / collapse / reroot a clade interactively
• Pair with msa for tree + alignment side-by-side

Generate a custom LiveView app

Write your own interactive component with the LiveView SDK when no existing viewer fits — a bespoke dashboard, custom plot, or tailored data view that the agent can still open, drive, and observe.

Skill file: live-view-app.md

When to use:

• The data / interaction doesn't match a ready-made viewer
• You need a tailored view of analysis output
• You want a custom interactive control surface for the user

The live_view tools

Tool	Purpose
open_live_view(view_type, title, state, module_url?)	Open a viewer plugin (e.g. view_type="vitessce") or a custom component (view_type="custom" + module_url); returns view_id
serve_local_data(path)	Expose a workspace file/dir over HTTP+CORS; returns a fetchable URL
serve_endpoint(name, path, config?)	Expose a lightweight Python HTTP endpoint over the same CORS data server; returns a fetchable /api/<name>/ URL
manage_endpoints(action, name?)	Manage endpoints: action="list" lists all, action="info" checks one, action="unregister" removes one
live_view_update(view_id, patch)	Deep-merge a partial-state patch (drive it)
live_view_set_state(view_id, state)	Replace the whole state
live_view_get_state(view_id)	Read state, status, and diagnostics — incl. the user's own edits
live_view_call(view_id, action, args)	Invoke a component-defined action
live_view_screenshot(view_id)	Render the view to an image — observe_images it to see it
list_live_views() / close_live_view(view_id)	List / close

Use serve_local_data when the browser should fetch a file you already wrote. Use serve_endpoint when the browser should fetch computed data. It is not specific to Gosling: any LiveView can use the returned URL if its config accepts data URLs, and custom LiveView apps can call fetch(url) directly. Examples: Gosling CSV/JSON/BED tracks, Cytoscape graph JSON, MSA alignment text, a custom dashboard's computed table, or a future tileset-style API.

Endpoint modules must export async def handle(request), def build(): return handler, or def build(config): return handler, where the handler returns an aiohttp.web.Response / StreamResponse. The frontend and endpoint coordinate runtime parameters through ordinary HTTP: path tail (/api/name/<tail>), query params, headers, or POST JSON. config is only for registration-time JSON-serializable constants such as fixed paths or sample names; use request parameters for interactive controls, and files plus serve_local_data for large arrays or binary data. Keep handlers light: precompute heavy results before serving, or run complex services in a separate process and proxy them in.

Use manage_endpoints to inspect or clean up registered endpoints:

• manage_endpoints("list") — list all active endpoints with their URLs
• manage_endpoints("info", "track_name") — check if a specific endpoint exists
• manage_endpoints("unregister", "old_track") — remove an endpoint that's no longer needed

Registering a new endpoint with the same name replaces the previous handler. Endpoint handlers run in the data server's thread and should return quickly; any exceptions are logged but clients receive only a generic error message.

Workflow: open_live_view → verify (live_view_get_state for diagnostics, live_view_screenshot to see it — status: ready does NOT mean it rendered correctly) → drive with live_view_update → live_view_get_state before the next move. Never treat reading back your own live_view_update value as verification.