event-logger-nodes-workflow

name	event-logger-nodes-workflow
description	Implementation workflow for the newspack-event-logger-nodes application — adding job handlers, service CI verbs, dashboard React trees, topology files, and application Node subclasses. Use whenever the change lives under newspack-event-logger-nodes/ rather than the substrate runtime.
argument-hint	[handler / endpoint / dashboard / node]

Event Logger Nodes Workflow

The application built on the newspack-nodes runtime. For substrate changes (Node, Router, Topic, Partition, Worker, Fleet, REPL, Tee, Tail, Consumer), use the nodes-workflow skill in newspack-nodes instead.

AGENTS.md carries the architecture decisions and key files; this skill is the procedural companion.

When to Use

• Adding or changing a job handler (anything filtered onto newspack_nodes/job_handlers or newspack_nodes/remote_job_handlers)
• Adding a service CI verb / endpoint (per-namespace verbs on App\*_CI_Node classes — REST is the substrate's command-protocol surface)
• Touching the React dashboards (any tree under src/)
• Adding an application Node subclass (RequestBuilder-style processor)
• Modifying topology files under topologies/
• Changes to Log_Manager, Job_Intake, Flame_Builder_Node, Stats_Store, Remote_Job_Rewrite_Node, Discovery_Collector_Node

Phases

Phase 1: Confirm the layer

Application code may know about requests, jobs, flames, hubs, spokes, dashboards. Work generic enough that any future plugin on the substrate would benefit belongs in newspack-nodes.

Quick test: would a non-event-logger consumer of newspack-nodes ever want this? If yes → substrate. If no → here.

Phase 2: Implement

Adding a job handler

1. Define the callable. JobWorker handles per-job try/catch; don't wrap.
2. Filter onto the right list (a handler can register on either or both):
   • newspack_nodes/job_handlers — dispatched for k:"job" entries on every node's own JobWorker. Use when the work runs locally on the node that produced the entry.
   • newspack_nodes/remote_job_handlers — dispatched on the hub for k:"remote_job" entries (the rewritten product of spoke-aggregated k:"job" lines). Use when the work runs centrally on the hub after aggregation. Register under both when local and aggregated copies need the same name but different logic — e.g. the local handler runs unconditionally, the hub handler filters by a per-entry attribute.
3. Validate inputs at the handler boundary. The substrate limits size (32MB per job — canonical \Newspack_Nodes\Job_Intake::MAX_JOB_SIZE, which Job_Router_Node derives from) but never content.
4. Size discipline: if the payload could exceed 4KB, write via \Newspack_Nodes\Job_Intake::queue( $handler_name, $parameters ) (the class moved to the substrate) instead of Log_Manager. $parameters passes through to the handler; the optional 3rd arg is a hash_to_partition routing key. Job_Intake is the auto-locked large-write path.

For a timer-driven local handler, read the sibling newspack-cache-cozy plugin's Cache_Cozy_Tick_Node: a Timer_Node that hitchhikes the _router heartbeat, enqueues a job every interval, and registers its handler on newspack_nodes/job_handlers. This plugin ships none of its own; its Timer_Node subclasses (Request_Builder_Node, Request_Flight_Node, Discovery_Collector_Node) rotate caches or mint commands instead.

Adding a service CI verb

Endpoints are verbs on a Service CI (App\*_CI_Node). The substrate's command-protocol REST surface exposes them at /wp-json/newspack-nodes/v1/command (POST one or more envelopes per request, dispatched against the addressed node) and /wp-json/newspack-nodes/v1/messages/stream (GET SSE). No per-plugin REST controllers remain.

1. Pick the right service CI class (files under includes/app/class-*-ci-node.php) — this plugin owns FIVE: Discovery_CI_Node, Logger_CI_Node, Events_CI_Node, Performance_CI_Node, Rules_CI_Node. (status/settings/aggregator are substrate-owned CIs; the old servers CI was replaced by the substrate vault CI — add those verbs in newspack-nodes, not here.) All five mount on newspack_nodes/request_graph_ready in newspack-event-logger-nodes.php (make_node Rules_CI rules, etc.).
2. Add a verb entry in that CI's node_schema()['commands'] array — name, description, args (per-arg name/type/required/optional default), and an inline handler closure. There is no per-schema permission_callback field; gate inside the handler instead (step 4).
3. Handler signature is static function ( Command_Interpreter_Node $self, array $args, array $envelope = [] ): array. $args is a token array (list<string> argv), not a space-joined string — the substrate migrated the {name, arguments} command envelope to the token-array contract. Parse positional and option args via Command_Args::parse( self::arg_strings( $args ) ); arg_strings is the base Command_Interpreter_Node helper that normalizes the token array to list<string>. A verb wanting the payload as one blob reads self::arg_strings( $args )[0] — how rules save/upsert take their JSON argument. All five live ELN CIs type $self as Command_Interpreter_Node; no registry-injected CIs remain. node_schema() is static, so state arrives through $self.
4. Capability gate: call self::require_manage_options() first — a protected static helper on Service_CI_Node throwing a for non-admins, which the interpreter's central catch (step 5) turns into a reply along the FROM trail. The env tag the substrate sets pre-dispatch excludes worker requests.

Touching the per-URL logging ruleset (v0.28.0)

A per-URL ruleset replaced the seven global logging settings (log_urls/skip_urls/log_events/custom_events/significant_events/auto_disable_threshold/auto_protect_time_threshold): an ordered list of Rules (includes/class-rule.php), each a URL pattern (prefix /x or exact /x?) with a log/skip action and — for log rules — its own hooks, custom events, significant events, and auto-tune thresholds. Log_Manager builds a Rule_Matcher (includes/class-rule-matcher.php) per request and resolves the ONE governing rule (longest-prefix-wins, case-insensitive; no match ⇒ skip ⇒ zero hooks bound, no memcache). Empty means empty — there is no implicit / log-all baseline.

• Durable state lives in Rule_Set (includes/class-rule-set.php): the rule LIST rides the autoloaded newspack_event_logger_nodes_rules option; a heavy rule's hooks (past INLINE_HOOK_LIMIT = 100) tier out to a non-autoloaded newspack_event_logger_nodes_rule_hooks_<id> option mirrored into memcache (evlog:rules:hooks:<id>, TTL 3600, warmed on miss). Every write MUST go through Rule_Set::save() so inline↔pointer tiering and orphan reconcile hold — never raw update_option.
• A rule's id is Rule_Set::id_for($pattern) (the pattern's Log_Manager::url_hash()) — one id per pattern; client-supplied ids are ignored.
• Editing goes through the Rules_CI_Node service CI (rules shell-name; verbs list/save/upsert/delete/reset at _http/rules). The full rules-editor UI (src/rules/RulesAdmin) mounts from src/settings/index.js into the settings page's #event-logger-rules-editor "Logging Rules" container, not a separate submenu. The performance dashboard (src/overview/PerformanceDashboard.js) reuses src/rules/RuleEditModal for an inline "Log this URL" quick-add on the URL-detail view (the upsert path), not the full editor.

Adding a React dashboard / page

The v0.8.0 substrate-canonical pattern: every dashboard mounts the substrate's exospine, builds its node graph from substrate JS primitives, and exposes a view node React subscribes to.

1. Source under src/{tree-name}/. Build via wp-scripts (npm run build).
2. The plugin's main file maps ?page=<slug> to a React tree; add the slug to the page_to_tree map.
3. Use @wordpress/element (not direct React) and @newspack-nodes/runtime for substrate JS nodes (mountExospine, SseIn, HttpOut, Heartbeat, RemoteLink).
4. Hook layout per dashboard. Two valid mountExospine call forms:
   • Bare — const { interpreter, teardown: teardownSpine } = mountExospine(); returns the request-scope interpreter and a teardown directly. Use only when the tree has no overlay/reinit (useHookCatalogGraph is the one current example).
   • Build-callback — const { teardown } = mountExospine( build ); where build receives { interpreter }, wires the graph, and returns { teardown }. The substrate snapshots Core and rebuilds via Core.reinit() on "Reset Graph". Every reinit-capable dashboard (request-log, performance, gyroscope, error-log) uses this form.
   • Mount only the substrate boundary nodes the dashboard needs:
     • _sse (SseIn — EventSource ingress) — required for live-stream dashboards (request log, gyroscope, error log).
     • _http (HttpOut — POST /command boundary; it lazily defaults its own transport from the localized NewspackNodesData, so nothing is injected) — required for any command-fanout dashboard.
     • _heartbeat (Heartbeat — SSE slot keep-alive; target = '_http/workers' — pokes the substrate's workers/heartbeat verb, which calls SSE_Slot_Pool::touch). Required whenever _sse is mounted.
   • A command/reply dashboard with no live stream — e.g. overview/performance, which polls and issues on-demand commands — needs only _http plus the view node(s). A pure live-stream dashboard needs _sse + _heartbeat + a transform/view chain.
   • All mounted nodes set . Steer flow with / — no bespoke chains.

Adding an application Node subclass

1. Create includes/class-{name}.php with class Foo_Node extends \Newspack_Nodes\Node (every node class ends in _Node; shell-name = class minus _Node). Override fill().
2. No registration — the plugin registers the Newspack_Event_Logger_Nodes\ namespace once, so make_node('Foo') resolves \Newspack_Event_Logger_Nodes\Foo_Node and the palette scans the classmap. Just composer dump-autoload -o after adding the file.
3. A .tsl topology wires it: make_node Foo foo instantiates the node, connect_node foo next-step wires the sink, cmd foo:config <verb> [args…] runs a config verb. The substrate's Topology_Loader interprets the script per partition.

Adding a CLI command (wp nodes <verb>)

1. Live under includes/cli/class-<verb>-command.php. Register in newspack-event-logger-nodes.php inside the WP_CLI block.
2. Validate inputs at the boundary, and refuse rather than coerce — Command_Args::option_int() returns null for a malformed flag so each layer can report it in its own voice.
3. Make blocking work injectable. If the command reads stdin in a loop, calls sleep between iterations, or polls a file, take the resource or iteration-count as a parameter so tests can drive it deterministically. Two distinct seams in Reqgrep_Command:
   • process_stdin( $stream = null ) — stream-injection seam; defaults to null (resolving STDIN inside), tests pass a php://memory resource.
   • follow_mode( int $max_iterations = PHP_INT_MAX ) — iteration-cap seam; production passes the default, tests pass a small number. Keep them separate — don't fold the cap into the stdin reader.

Removing dead code

The deferred-loader pattern in newspack-event-logger-nodes.php (require_once chain run on plugins_loaded priority 11) loads every class in this plugin before anything constructs them, so class_exists() guards around in-plugin instantiation are dead branches. Delete any you find while editing; don't leave them as "defensive". The same applies to in-substrate class_exists() guards inside newspack-nodes. Out-of-plugin and optional-dependency guards (e.g. class_exists( 'Memcached' ) for the PHP extension) stay.

Type flags

• VALUE is an array (entry hash, request object, flame data) → set TYPE = TM_STRUCT.
• VALUE is a string (raw line, formatted text) → set TYPE = TM_BYTESTREAM.
• Producer and consumer must agree. Consumers reading array VALUE gate on TM_STRUCT.

Phase 3: Test

# Unit + integration tests (require newspack-nodes activated too).
# Always pass --enforce-time-limit so a hung test (readline without a TTY,
# infinite drain loop) aborts at the per-test budget instead of stalling the
# suite. Tests that legitimately sleep mark their class `#[Medium]`.
cd tests && ../vendor/bin/phpunit --enforce-time-limit

# Filter to a specific test file or method.
cd tests && ../vendor/bin/phpunit --enforce-time-limit --filter RequestBuilderTest

# Coverage HTML/Clover. Refuses to run as root (Log_Manager refuses root;
# the suite instantiates it, surfacing as 37 spurious failures).
# Invoke via `docker exec -u bend <container> bash tests/run-coverage.sh`.
tests/run-coverage.sh

Phase 4: Reload running workers

Workers cache loaded classes for their process lifetime (~10 min, substrate-controlled). After deploying, restart the relevant worker groups so the new bytecode lands:

# Restart all worker types in one shot.
wp nodes restart all

# Or per topology — the basename of the `.tsl` file (run `wp nodes types`
# first to enumerate cataloged topologies). Current topologies: combined,
# request-builder, job-router, flame-builder, performance, aggregator, hub-control.
wp nodes restart combined       
wp nodes restart request-builder
wp nodes restart job-router     
wp nodes restart flame-builder  
wp nodes restart aggregator        # hub-only; errors out on spokes (substrate validates against active topologies)
wp nodes restart hub-control       # hub-only single-instance settings-sync + discovery control plane

The substrate registers every worker-CLI verb (wp nodes {types,run,restart,status} and {ls,cli}). This plugin registers two, in the WP_CLI block of newspack-event-logger-nodes.php: wp nodes reqgrep (Reqgrep_Command — application-aware firehose filter) and wp nodes ruleset-bench (Ruleset_Bench_Command — dev-only per-URL-ruleset matcher benchmark, off the request hot path).

Phase 5: Live-verify

For changes touching the request-logging pipeline:

# Hit any URL on the site to generate firehose entries.
curl -sk "<site>/" -o /dev/null -w "HTTP %{http_code}\n"

# reqgrep with --recent shows the most-recent segment forward.
wp nodes reqgrep --recent | head -10

# reqgrep can also follow live (Ctrl-C to stop).
wp nodes reqgrep --follow

For dashboard changes: open the page and verify the panels render. The browser DevTools network tab shows REST traffic. Telemetry dashboards land at /wp-admin/admin.php?page=event-logger-* (event-logger-overview, event-logger-errors, event-logger-gyroscope, event-logger-requests); the Settings / hook-catalog tree (settings) is served at page=newspack-event-logger-nodes. The aggregator admin page is substrate-owned (newspack-nodes), not routed here.

For job handler changes: queue a job via the legitimate caller, wait, check wp nodes status for job-workers heartbeat, optionally reqgrep for the rid.

Patterns That Trip People Up

• Hub vs spoke: there is no operator hub toggle — both enable_workers (v0.5.0) and enable_aggregator were retired (tests/unit/RetiredConfigKeysTest.php guards them). Hub-mode derives from whether the aggregator topology is in the substrate's topologies list (and hub-control for settings/discovery fan-out). Settings fan-out is the substrate Settings_Sync_Node graph in hub-control; Auto_Tuner_Node::persist is a plain update_option. Missing consumers (no hub-control, no per-spoke HTTP_Out wired) are the structural gate — a recorded settings event is tailed and dropped.
• Memcache is required — Stats_Store (driven by Flame_Builder_Node), SSE slot rate limiting, and worker-position publishing all use it. Running locally without memcache, the stats path goes fail-soft (no data on dashboards) and the SSE slot pool fails closed (429).
• Salt rotation orphans keys but doesn't flush them — workers keep writing to the OLD salt until they respawn. After Stats_Store::flush_all(), restart workers to take effect immediately.
• Application nodes resolve by namespace prefix (no registry): make_node Flame_Builder (in a .tsl topology) → \Newspack_Event_Logger_Nodes\Flame_Builder_Node via the registered Newspack_Event_Logger_Nodes\ prefix. The .tsl shell-name is the class minus _Node (make_node Flame_Builder, Job_Router, Request_Builder, …).

After You Land

• Update AGENTS.md if the change altered an architecture decision or key file
• If you added a job handler that crosses the hub/spoke boundary, document which side is intended (job_handlers vs remote_job_handlers)
• Push to GitHub via the plugin's own remote (this is its own git repo)

Related Skills

• event-logger-nodes-debugging — dashboards, log paths, memcache schema, hub/spoke routing
• event-logger-nodes-review — application contract checklist
• nodes-workflow (in newspack-nodes) — for substrate-level changes