| name | pipelines-as-code |
| description | Use this skill when working with Materialize's alloy log-processing or metric-processing pipelines — anything under `packages/alloy-pipelines/`. Triggers include: building, modifying, reviewing, or rendering pipelines; working with the embedded JSONSchemas under `packages/mzmon-lib/schemas/alloy/`; deciding between typed blocks and the `raw:` escape hatch; extending the schemas to add a new component, stage, or attribute; debugging `mz-monitoring-build gen-pipelines` output; writing Materialize-specific log-processing patterns (level normalization, structured metadata, label families, drop/limit conventions); writing or reviewing alloy stages, loki.process pipelines, discovery.kubernetes, loki.relabel rules, or sources like loki.source.file / loki.source.journal. Also use it whenever someone refers to "log processing", "metric processing", or anything alloy-shaped, even if they don't use the word "pipeline".
|
Pipelines as Code
This skill is the entry point for the Materialize pipelines-as-code project. Stable conventions live in the repo docsite under docs/content/reference/internal/pipelines/ — this file is intentionally slim and links into the docsite at heading-level granularity. The non-link content below is the state snapshot: what currently exists, what's in flight, and what's queued.
Audience reminder
The pipelines themselves target alloy (the binary), so authoring decisions favor contributors rather than end users. There is no panel-description-voice equivalent — readability of the YAML and the rendered .alloy is the goal.
The docsite reference pages target repo contributors (SRE, Field Engineering, CloudOps, Database Engineers) and AI agents reading this skill.
Where to find what
| Looking for… | Read |
|---|
Pipeline model, strict-attributes policy, raw: escape rules, how to extend the schema | Authoring |
| Log pipeline conventions, label families, retention | Logging (stub) |
| Metrics pipeline conventions | Metrics (stub) |
Frequently needed deep links into Authoring:
Schema and code map
packages/alloy-pipelines/ ← YAML inputs
├── gateway.yaml – gateway processing pipeline
├── gateway-dest-stub.yaml – gateway default egress tail (swappable)
└── agent.yaml
packages/mzmon-lib/schemas/alloy/ ← validation schemas (embedded into the binary)
├── mzmon-alloy.schema.yaml – entry
├── top.schema.yaml – description / logging / livedebugging / blocks
├── loki.schema.yaml – loki.*, stage.* $defs, shared `rule` $def
├── discovery.schema.yaml – discovery.*, cross-ref'ing `rule` from loki
├── prometheus.schema.yaml – prometheus.*, cross-ref'ing `rule` from loki + `target` from discovery
├── otelcol.schema.yaml – otelcol.processor.{batch,memory_limiter,attributes,groupbyattrs,filter,transform} + output/action/conditions/statements sub-blocks
└── common/ – shared fragments. $id tail MUST match the file path
├── raw.schema.yaml – {raw: <block>} escape hatch + block primitives
├── attribute.schema.yaml – attributeValue (anyOf: literal | expression | …)
└── expression.schema.yaml – sys.env / function / operator / ref expression
(each fragment's `$id` + the `ID_*` const in validate.rs + the relative `$ref`s must agree)
packages/mzmon-lib/src/alloy/ ← AST, render, validate, pipeline (Rust)
├── ast.rs – Block / AttributeValue / Expression / Expressable<T>
├── render.rs – write_to + alloy-fmt-canonical formatting
├── validate.rs – embedded schemas + jsonschema validator + hints
├── pipeline.rs – Pipeline::from_yaml_str (YAML → Value → validate → typed)
├── test_support.rs – assert_renders (oracle: pipes through `alloy fmt`)
└── components/ ← typed sugar; tests colocated with impl
├── top.rs – LoggingBlock, LiveDebuggingBlock
├── capsule.rs – LogsReceiver, MetricsReceiver, OtelcolConsumer, RelabelRules, TargetEntry
(+ string_map, logs_receiver_list,
metrics_receiver_list, otelcol_consumer_list, target_list helpers)
├── loki.rs – LokiEchoBlock, LokiSourceJournalBlock, ...
├── relabel.rs – RelabelRule + RelabelSubBlock (shared by *.relabel)
├── discovery.rs – DiscoveryKubernetesBlock, DiscoveryRelabelBlock,
│ KubernetesSubBlock variants
├── prometheus.rs – Prometheus{Echo,Relabel,Scrape,ReceiveHttp,RemoteWrite,
OperatorPodMonitors,OperatorServiceMonitors}Block
(+ endpoint/http/clustering/basic_auth/tls_config/
selector/scrape sub-blocks)
└── otelcol.rs – OtelcolProcessor{Batch,MemoryLimiter,Attributes,GroupByAttrs,Filter,Transform}Block
(+ shared output block, attributes action block, Ottl type,
Processor/Attributes/Filter/Transform sub-block enums)
packages/mz-monitoring-build/ ← CLI: `mz-monitoring-build gen-pipelines`
Build/test:
make pipelines # render YAML → .alloy + run `alloy validate` per target
cargo test -p mzmon-lib # unit + oracle tests (uses `alloy fmt` if present)
cargo clippy -p mzmon-lib --all-targets # lints
cargo fmt # the team runs this in pre-commit; run it before
# committing any Rust changes
Current Pipeline State
This section captures the live state so the next session has something concrete to start from. Update it when state changes meaningfully (new pipeline, new typed component, schema gap closed, etc).
Pipeline inventory
| YAML | Rendered to | Status |
|---|
packages/alloy-pipelines/gateway.yaml | charts/.../pipelines/gateway.alloy | implemented — carries logs AND metrics. Logs: port of the reference staging-gateway.alloy inputProcessor (level/timestamp/per-service field normalization, per-level rate limits, final label shaping) + ingress (loki.source.api, loki.source.kubernetes_events, otelcol.receiver.otlp→otelcol.exporter.loki bridge — all typed), sampled debug tap. Metrics: processed primarily in otelcol, converted to Prometheus only at the write. prometheus.receive_http + prometheus.operator.{podmonitors,servicemonitors} (clustering on, env-driven scrape defaults) → otelcol.receiver.prometheus "inputBridge" (prom→OTLP; the OTLP receiver's metrics join here too) → otelcol.processor.filter "inputMetricProcessor" (rule-free passthrough today; the metric filter/transform tiers land here) → otelcol.processor.memory_limiter "outputMemoryLimiter" (refuse at 75%) → otelcol.processor.batch "outputBatch" → otelcol.processor.filter "egress" seam → otelcol.exporter.prometheus "outputBridge" (OTLP→prom, add_metric_suffixes=false) → prometheus.relabel "egress" → prometheus.remote_write. NB: the metrics filter is inputMetricProcessor; inputProcessor is the logs loki.process. Both log and metric egress seams supplied downstream (see below) |
packages/alloy-pipelines/gateway-dest-stub.yaml | charts/.../pipelines/gateway-dest-stub.alloy | build-time egress stand-in ONLY — the egress seams (loki.process "egress" for logs, otelcol.processor.filter "egress" for metrics, plus the post-bridge prometheus.relabel "egress") + default loki.write/prometheus.remote_write "destination". NOT what deploys: at install the destination is rendered by charts/.../templates/_alloy_helpers.tpl (Helm, from .Values.pipeline.{logging,metrics}.gateway.destination.*), which — only the pre-validate jobs () check it. The stub exists so can jointly (it dangles the refs alone). Keep it in step with the helper, but the helper is source of truth. Metrics helper auth: /// (sigv4 = AMP + IRSA) |
Both pass alloy validate. agent.yaml is a faithful port of the reference staging-agent.alloy (packages/ref-alloy-pipelines/, not checked in).
There are ZERO raw: blocks left in any pipeline. Every component in agent.yaml, gateway.yaml, gateway-metrics.yaml, and gateway-dest-stub.yaml is typed and schema-validated. raw: remains in the schemas as the escape hatch — the last oneOf branch of every sub-block list, and available for any component — but nothing currently uses it. Adding an untyped component should be a deliberate, temporary choice, not the default.
The conversion was verified by rendering before and after: agent.alloy, gateway-metrics.alloy, and gateway-dest-stub.alloy came out byte-identical, and gateway.alloy differs only in logs/metrics ordering inside one output block (the typed struct's canonical order vs. YAML order — semantically irrelevant to alloy, see the attribute-order note below). Both configs were also load-tested under a real alloy run, since alloy validate cannot see capsule/type mismatches.
Agent log sidechannel (gateway.yaml): the gateway tails the agent DaemonSet's own logs from the Kubernetes API — discovery.kubernetes "agent_pods" (typed, server-side selectors.label = app.kubernetes.io/name=alloy-agent) → discovery.relabel "agent_pods" (typed) → loki.source.kubernetes "agent_logs" → loki.process.inputProcessor. Points that are load-bearing rather than stylistic:
- The agent no longer tails its own logs —
discovery.relabel.local_pods in agent.yaml drops targets whose app.kubernetes.io/name is alloy-agent. The sidechannel is always on, so self-collection would deliver every line twice. The two changes are a pair.
clustering { enabled = true } on the tail is required, not an optimization. The gateway runs 2+ replicas and each would otherwise tail every agent pod.- RBAC is already covered by the alloy subchart's default
rbac.rules (pods, pods/log, namespaces), which the chart does not override.
- The
{app="alloy"} selector in inputProcessor was dead. app comes from app.kubernetes.io/name, which the subchart aliases make alloy-agent/alloy-gateway, so the reference deployment's bare alloy never matched — meaning alloy's own logs were never logfmt-parsed and the loki.echo drop never ran. Now {app=~"alloy(-agent|-gateway)?"}. That drop is what stops the echo tap's re-emitted pod logs being ingested a second time via the sidechannel.
- Verified end-to-end against a real API server (kind), not just
alloy validate: discovery → relabel → tail delivered live log lines, confirming targets = discovery.relabel.X.output is the right capsule shape for loki.source.kubernetes.
Container metrics come from the kubelet, not from a cAdvisor we run. gateway.yaml: discovery.kubernetes role=node → discovery.relabel (node-label allowlist) → prometheus.scrape "kubelet_cadvisor" (https, /metrics/cadvisor, SA bearer token, clustered) → the otelcol metrics pipeline. Everything below was measured on a live GKE cluster, not reasoned about:
- An in-agent
prometheus.exporter.cadvisor cost ~750Mi per node against a 200Mi logs-only envelope, and OOM-crashlooped the DaemonSet (180-224 restarts). A heap profile attributed ~34% to cAdvisor housekeeping and ~31% to the scrape loop reading it. The kubelet already computes those stats for eviction and the metrics API, so running our own computed them twice per node.
- The completeness argument for running our own does not survive measurement. A GKE kubelet serves 69 distinct
container_* metrics vs 70 from an in-process cAdvisor; the only real loss is container_health_state, which nothing queries. All 23 container_* metrics this repo's queries reference are present. Where GKE dashboards look sparse it is the collector scraping a subset, not the kubelet withholding data.
- The kubelet serves the
process-collector family too (container_file_descriptors, container_sockets, container_threads, container_ulimits_soft, container_processes). Those are why the agent briefly ran hostPID: true; that privilege — and the /proc/<pid>/environ exposure it grants alongside uid 0 — is no longer needed.
- The kubelet labels series
namespace / pod / container directly, so none of the container_label_io_kubernetes_* remapping an in-process cAdvisor needs applies.
- TLS: the in-cluster CA verifies the kubelet certificate on GKE (measured), so
ca_file + the SA token is enough and insecure_skip_verify stays false. Other distributions may need it true, and the failure is silent — hence GATEWAY_KUBELET_TLS_INSECURE.
clustering { enabled = true } on the scrape is required, not an optimization: the gateway runs many replicas and each would otherwise scrape every node (~6.7k series/node).- RBAC needs nothing added — the alloy subchart's default
clusterRules already grant nodes, nodes/metrics, nodes/proxy.
GOMEMLIMIT is set on both roles (~80% of the memory limit). Go's GC has no knowledge of a cgroup limit, so it grows the heap toward a ceiling the kernel enforces by killing the process; telling the runtime about it converts an OOM-kill into GC pressure. Keep it in step with the limit whenever either moves.
Top-level loki. components*: loki.echo, loki.process, loki.relabel, loki.source.journal, loki.source.file (incl. the file_match sub-block), loki.source.api (+http server), loki.source.kubernetes (+clustering), loki.source.kubernetes_events, loki.write (+endpoint).
Top-level discovery. components*: discovery.kubernetes, discovery.relabel.
Top-level prometheus. components*: prometheus.echo, prometheus.relabel, prometheus.scrape, prometheus.receive_http, prometheus.remote_write, prometheus.operator.podmonitors, prometheus.operator.servicemonitors. Scoped to the stable, in-cluster surface — see Metrics for the per-component raw:-deferred list (remote_write endpoint auth/TLS/queue tuning, operator client, scrape oauth2/authorization, selector match_expression, receive_http tls).
prometheus.* sub-blocks: clustering (enabled, shared by scrape + operators), basic_auth/tls_config (on scrape), http (receive_http server), endpoint (remote_write — url + scalars + a raw-only nested blocks for auth/queue), selector (operator — match_labels + raw-only nested blocks for match_expression), scrape (operator defaults), and the shared rule (cross-file $ref to loki.schema.yaml#/$defs/rule, reusing the RelabelRule sugar).
prometheus.scrape's scrape_interval/scrape_timeout are Expressable<GoDuration> via the existing expressionOrDuration $def (the same one operatorScrape uses), so the interval can be env-driven — it is the dominant cost lever on a per-node cAdvisor. That widened the struct enough that ComponentBlock::PrometheusScrape is now Boxed for clippy::large_enum_variant, like memory_limiter.
Top-level prometheus.exporter.* components: prometheus.exporter.cadvisor (the only one so far). Its collector allowlists are ExpressableList rather than Expressable, and its to_block emits attributes in declaration order rather than grouped by type, so store_container_labels renders next to the allowlist it gates.
Top-level otelcol. components*: otelcol.processor.{batch,memory_limiter,attributes,groupbyattrs,filter,transform}. filter is now wired into gateway.yaml (the metrics pipeline is otelcol-primary — see the inventory row above); the rest are schema+sugar only so far. memory_limiter's limit knobs (check_interval/limit/spike_limit/limit_percentage/spike_limit_percentage) are all Expressable for env-driven sizing (its ComponentBlock variant is Boxed — five Expressable fields made it the outsized variant, clippy::large_enum_variant). Receivers and exporters are now typed too: otelcol.receiver.{otlp,prometheus} and otelcol.exporter.{otlp,loki,prometheus}. Sub-blocks: grpc/http (one shared server struct — the variant names the block, so its ToBlock is hand-written rather than the dispatch macro; an empty grpc: {} is meaningful, since presence is what enables the listener), client + nested tls on the OTLP exporter. Still deferred to raw:: connectors, the per-processor debug_metrics block, exporter sending_queue/retry_on_failure, and (on filter) the deprecated traces/metrics/logs blocks.
OTTL (filter/transform) is carried by a dedicated Ottl newtype (ast.rs, a sealed LiteralScalar so fields are Vec<Expressable<Ottl>> — literal OTTL string OR an expression). We do NOT model OTTL's grammar/function library; statements render verbatim as normal (escaped) alloy strings. filter's *_conditions blocks and transform's *_statements/statements blocks share one Rust struct each (FilterConditionsBlock, TransformStatementsBlock), so their ToBlock is hand-written (not the macro) to pass each block name.
GOTCHA — otelcol needs alloy >= 1.17 and a label on every component. Confirmed by a real alloy validate (the schema/renderer agreeing is NOT proof — these aren't wired into a pipeline, so make pipelines doesn't validate them): (1) filter's *_conditions blocks are 1.17+ only — older alloy has just the deprecated traces/metrics/logs and errors "unrecognized block name"; we pin 1.17.1. (2) EVERY otelcol component must have a label or alloy load fails "must have a label" — so label is a required field on the otelcol structs and schema (required: [label]), deliberately diverging from the loki/prometheus families which leave label optional.
otelcol.* sub-blocks: output (shared — metrics/logs/traces lists of the OtelcolConsumer capsule; bare refs), action (attributes), *_conditions (filter: context + conditions), *_statements (transform: context/statements/conditions/error_mode), statements (transform inferred: trace/metric/log). Sub-block enums: ProcessorSubBlock, AttributesSubBlock, FilterSubBlock, TransformSubBlock.
loki.process stages (and stage.match body, recursively): stage.match, stage.drop, stage.limit, stage.regex, stage.replace, stage.template, stage.logfmt, stage.json, stage.timestamp, stage.labels, stage.static_labels, stage.label_drop, stage.structured_metadata, stage.structured_metadata_drop, stage.sampling, stage.cri (empty, no attributes), stage.tenant (label/source/value, each Expressable<String>).
discovery.kubernetes sub-blocks: selectors (incl. field/label as Expressable<String>), attach_metadata (node + namespace). Other sub-blocks (e.g. namespaces) use raw:.
*.relabel sub-blocks: rule (shared $def used by loki.relabel, discovery.relabel, and prometheus.relabel via cross-file $ref to loki.schema.yaml#/$defs/ruleBlock; the operator components' rule refs #/$defs/rule directly).
Literal-or-expression fields use Expressable<T> (ast.rs): a field typed Expressable<f64|String|bool> (or Option<…>) accepts either a scalar literal or an inline expression object ({env}, {function}, {operator}, {ref}). In use on stage.limit rate/burst, stage.drop older_than, stage.static_labels values, selectors field/label, and the operator scrape block's default_scrape_interval/default_scrape_timeout. Schema side: anyOf: [{type: <scalar>}, {$ref: common/expression.schema.yaml}] (safe — scalar vs object are disjoint). Scalars only — never Expressable<map/object> (a literal map would collide with the expression object, the same overlap that forced anyOf in the raw attributeValue). This is an actively-expanding pattern; adopt per-field as needed.
ExpressableList (ast.rs) is the list analog of Expressable, which stays sealed to scalars. The sealing exists because a literal map is shape-indistinguishable from the expression object; a literal list is not (array vs object are disjoint), so the untagged dispatch is unambiguous with Literal first. Used by cAdvisor's collector allowlists, which are wired to env vars via encoding.from_json.
Fields widened to accept expressions during the raw-block conversion: RelabelRule.replacement (stamping node from HOSTNAME), remoteWriteEndpoint.url and lokiWriteEndpoint.url (env-coalesced destinations), prometheus.scrape.scrape_interval/scrape_timeout, loki.source.api's listen_port, and the otelcol client.endpoint / server.endpoint.
Every sub-block oneOf ends with a raw: branch — the escape hatch is non-negotiable in the design.
Only the attributes we routinely use are documented per component — not exhaustive vs. alloy upstream. Strict additionalProperties: false is enforced; undocumented attributes are rejected at validation time with a hint pointing at the raw: escape vs. schema-extension choice. See Authoring §The strict-attributes / raw-escape policy.
Rust sugar deserialization status
The ComponentBlock enum in pipeline.rs dispatches to typed sugar structs (via #[serde(rename = "loki.echo")] etc.). Each typed component has a pub struct …Block { fields } + an impl ToBlock that normalizes to the generic Block AST for rendering. Sub-blocks follow the same pattern via per-component sub-block enums (KubernetesSubBlock, RelabelSubBlock). The ToBlock enum impls are generated by the impl_to_block_dispatch! macro (ast.rs, pub(crate), $crate:: paths) — list every variant in the invocation; a missed one is a compile error.
Done: every component in the typed-schema coverage list above round-trips through the typed path. No pending sugar work.
Capsule types (components/capsule.rs) make the bare-ref invariant structural: LogsReceiver (forward_to), MetricsReceiver (forward_to on prometheus.* — the metrics analog of LogsReceiver, list-wrapped via metrics_receiver_list), RelabelRules (relabel_rules), and TargetEntry (targets, untagged Ref(Identifier) | Literal(IndexMap); prometheus.scrape reuses it). targets refs are list-valued and must NOT be bare-array-wrapped. A discovery.* export (discovery.kubernetes.pods.targets, discovery.relabel.x.output) is a list(discovery.Target); a literal is a single discovery.Target. targets = [ <ref> ] is list(list(Target)) and alloy rejects it at load (conversion from '[]discovery.Target' is not supported) — even though alloy validate accepts it. So target_list (capsule.rs) emits: a single ref directly (targets = discovery.x.targets), literals-only as an array ([{…}]), and multiple/mixed via array.concat(ref, …, [{…}]). (Earlier this list-wrapped and "verified against alloy validate" — a false green that broke a live agent; see the alloy validate gotcha below.) forward_to/relabel_rules are NOT affected — those refs are single capsules, so list-wrapping (forward_to) or direct assignment (relabel_rules) is correct. Schema side: $defs/target (discovery.schema.yaml, generic) vs $defs/fileTargetEntry (loki.schema.yaml, required: [__path__]) — strictness lives in the schema, the Rust type stays generic. A new capsule type is a newtype + From<&T> for AttributeValue via Expression::name_to_ref + a schema $def — OtelcolConsumer (the otelcol output block's metrics/logs/, list-wrapped via ) is the worked example.
Load-bearing invariants
These are non-obvious things that must stay true — flagged here so reorders or refactors don't silently break them. Detail in Authoring §Load-bearing invariants.
AttributeValue variant order: Null/Bool/Number/String/Array must come before Expression. Serde's untagged struct deserializer accepts a sequence by positional-field assignment, so ["a", "b"] would misroute to Expression { raw: Some("a"), env: Some("b") }. Regression-tested in ast.rs.#[serde(deny_unknown_fields)] on Expression: keeps generic objects ({mapping: ...}) from silently matching Expression with all heads None. Also regression-tested.- Ref-valued attributes render as bare refs, never quoted strings — and this is now enforced by type: declare the field with a capsule type (
Vec<LogsReceiver>, Option<RelabelRules>, Vec<TargetEntry>) and convert via the capsule helpers / Expression::name_to_ref. Do NOT hand-build Expression { ref_name: ... } in new to_block impls; if a new capsule kind is needed, add it to components/capsule.rs.
raw: escape is always the last oneOf branch in every sub-block list. Adding a new typed branch goes before raw.- A nested block list is
Vec<RawOnlySubBlock> (or a typed sub-block enum), NEVER Vec<Block>. The schemas express a nested list as $ref: common/raw.schema.yaml — the externally-tagged {raw: {component, …}} wrapper — while Vec<Block> deserializes the unwrapped {component, …}. Schema-valid YAML then passes validation and dies in serde with missing field 'component', so the escape hatch reads as documented and is unusable. Four fields shipped that way in the raw-block conversion, because no pipeline exercises those escapes and nothing else notices. RawOnlySubBlock lives in ast.rs; pipeline.rs::raw_escape_round_trips_in_every_raw_only_list covers every such list and fails with that exact error if one regresses.
- The two drift independently and the mismatch is silent in both directions: an expression-capable schema over a plain field accepts YAML that then fails serde, and the reverse quietly rejects valid config. Widen (or narrow) both together.
Cleanup / refactor candidates
write_expression uses a rendered_expr flag: the idiomatic shape is a tuple-match over (&env, &raw, &function, &ref_name, &operator), which encodes "exactly one head set" structurally. The flag pattern bit us once (forgotten assignments); a refactor would prevent recurrence.- Renderer block-attribute alignment (task #15): current rule "any multi-line value disables alignment for the whole group" is too aggressive; alloy fmt aligns in more cases. Concretely, the only divergences in the rendered
agent.alloy vs alloy fmt are (a) rule blocks where a single-line attr (action, separator, replacement) sits in a group that also contains the multi-line source_labels array, and (b) loki.source.file's targets (single-line) next to the multi-line forward_to array. Everything else is canonical. Two ways out (an open decision): fix the alignment rule, or post-process rendered output through alloy fmt (we have alloy in CI now). Until then, assert_renders will reject tests that exercise those non-canonical shapes.
- Schema drift watch:
description blocks link to canonical alloy docs. When alloy upstream renames a field or adds one we use, the schema description and properties need a corresponding update. There's no automation here; it's a manual sweep when bumping alloy versions.
with_capacity + push loops in to_block impls (~5 sites): idiomatic form is self.blocks.iter().map(ToBlock::to_block).collect::<Result<Vec<_>>>()?. Cosmetic; sweep opportunistically.- Capsule newtype fields are
pub (LogsReceiver(pub Identifier)): if ref-path validation is ever added, switch to private field + fn new() -> Result<Self>.
Queued work (non-binding — directions, not commitments)
Rough backlog from recent sessions; shapes may change. In loose priority:
- Fold agent logs into the OTLP exporter: the agent now has
otelcol.exporter.otlp "gateway" for metrics while logs still go out via loki.write "gateway". Unifying them is tracked separately.
replace_map sugar on discovery.relabel/loki.relabel: a {source_label: target_label} map expanding to one action: replace rule each, in source order (relies on the preserve_order already enabled). Covers only the 1:1 replace case — multi-source / separator / replacement rules (e.g. job, __path__) stay explicit. Biggest readability win for the relabel-heavy blocks.- Renderer alignment vs
alloy fmt (the open decision above): fix the rule, or post-process through alloy fmt.
$comment / inline-comment rendering: the schema already declares $comment/commentedValue, but Block has no comment field and the renderer drops them — pure plumbing. Doc-gen from comments is a stretch follow-on.- Ref-resolution pass:
forward_to/targets/relabel_rules are free strings; nothing checks the referenced component exists (alloy validate catches it, but later + with worse messages).
- Reusable
Expressable schema $defs (numberOrExpr/stringOrExpr): fields currently inline the anyOf; a couple of named defs would DRY it. Cosmetic.
- Golden snapshot test for the full rendered
agent.alloy, once the above settle (so it doesn't churn).
- CI freshness: the
pipelines job in .github/workflows/test.yaml asserts committed .alloy matches a fresh render — keep rendered output committed.
A few decisions are deliberately deferred: typing gateway.yaml's ingress components (see Cleanup); loki.write remote/auth config (it's exceptional — shared by agents + gateways, may target remote sinks; the gateway sink currently renders basic_auth-free with a GATEWAY_LOKI_DEST env-coalesced url); and a real (build- or runtime-) parameterization mechanism for numeric knobs like stage.limit rates (current encoding.from_json env coercion is provisional).
