with one click
webstatus-workers
// Use when working with the webstatus notification pipeline, event producer, push delivery, or push workers (e.g., Email, Webhooks), and Pub/Sub subscribers.
// Use when working with the webstatus notification pipeline, event producer, push delivery, or push workers (e.g., Email, Webhooks), and Pub/Sub subscribers.
Use when creating or modifying Go backend API endpoints, modifying Spanner database schemas, or working with OpenAPI and Spanner mappers.
Use when modifying the ANTLR search grammar, adding new search terms, or working with the query parser and builder.
Use when writing, modifying, or debugging Playwright end-to-end (E2E) tests for webstatus.dev.
Use when modifying the frontend SPA, working with TypeScript, Lit web components, Shoelace components, or frontend tests.
Use when working on Go data ingestion workflows, scheduled Cloud Run jobs, or adding new scrapers for BCD, WPT, or other data sources.
Use when upgrading toolchain versions (Go, Node.js, Terraform, Playwright) or updating the DevContainer and Github CI configurations.
| name | webstatus-workers |
| description | Use when working with the webstatus notification pipeline, event producer, push delivery, or push workers (e.g., Email, Webhooks), and Pub/Sub subscribers. |
This skill provides architectural and implementation details for the workers/ directory in webstatus.dev, which houses the event-driven notification pipeline.
For a detailed breakdown of the notification pipeline architecture, including the event producer, push delivery dispatcher, and email sender, see references/architecture.md.
For guidance on adding future integrations (like Webhooks), see references/how-to-add-a-worker.md.
Workers must use the shared structs in lib/workertypes/types.go for all incoming and outgoing messages. This ensures that a change in the Spanner schema or API doesn't immediately break the worker pipeline (decoupling).
Strict Rules for Data Sync:
SearchName or user PII) to core data-sync pub/sub event schemas (e.g., FeatureDiffEvent, RefreshSearchCommand). Core events should represent the raw state change via canonical IDs (SearchID, EventID). Any display data needed for emails or UI should be fetched from Spanner at the last possible moment (Delivery/Render phase).FeatureDiff, the Added array represents features that newly matched the search query since the last run. It does NOT mean the feature just entered the web platform. Do not attempt to calculate baseline or status transitions against a "zero state" for features in the Added array, as this falsely reports them as "becoming" newly/widely available when they merely entered the search results.From.Status, To.Status, To.Version, and To.Date perfectly match before combining them.EventSummary, use a custom struct instead of a simple slice of strings, even if it has only one field initially. This prevents breaking serialization when adding fields in the future.Message in favor of Code enum) to preserve design intent for future maintainers.lib/backendtypes) to house shared simple types and error codes to break cyclic dependencies between generic orchestrators and versioned data handlers.v1.FeatureDiffSnapshot) must use the types and enums defined within that specific schema's package (e.g., v1.QueryErrorCode) rather than importing internal backendtypes. This ensures the worker pipeline remains fully insulated from backend internal changes.executionData).lib/workertypes/comparables, lib/workertypes, and storage blobtypes/v1), understand that this is an intentional application of the Multi-Layer Enum Strategy. This ensures service boundaries are respected and we decouple internal pipeline logic from the external storage schema. We use exhaustive switch statements (enforced by the linter) to map between these layers, ensuring that adding a new error code forces updates across all layers without silent failures or brittle type casting.spanner:9010) and Pub/Sub (pubsub:8060).FRONTEND_BASE_URL locally is usually http://localhost:5555.All workers must be decoupled from GCP-specific SDKs.
interface EmailSender) in the worker's package.struct PubSubSender) in lib/gcppubsub/gcppubsubadapters.generic.OptionallySet[T] pattern when defining blob structures or canonical in-memory state to handle forward/backward compatibility gracefully (quiet rollouts).slog for logging and bubbles up transient errors using errors.Join(event.ErrTransientFailure, err) to NACK the message for retry.make precommit to execute the full suite of Go tests, formatting, and linting.make go-lint to lint all Go code using golangci-lint.go.work), to run tests quickly for a single package without running the whole suite, execute go test from within the specific module directory:
cd workers/<worker_name> && go test -v ./...
When you add a new worker, update the notification pipeline, or change data flow:
docs/ARCHITECTURE.md to reflect the new pipeline step.GEMINI.md to ensure I am aware of the changes.When adding a new worker, you MUST ensure that the manual instance counts are updated across all environments.
worker_manual_instance_counts variable in infra/variables.tf is an object type. This means that if you add a new field (e.g., webhook) to the object definition, every environment's .tfvars file (e.g., infra/.envs/staging.tfvars, infra/.envs/prod.tfvars) MUST be updated to include that field, or Terraform execution will fail.