| name | tenzir |
| description | Answer questions using the Tenzir documentation. Use whenever the user asks about TQL syntax, pipeline operators, functions, data parsing or transformation, normalization, OCSF mapping, enrichment, lookup tables, contexts, packages, nodes, platform setup, deployment, configuration, integrations with tools like Splunk, Kafka, S3, Elasticsearch, or any other Tenzir feature. Also use when the user asks how to collect, route, filter, aggregate, or export security data with Tenzir, or needs help writing or debugging TQL pipelines, even if they don't mention 'Tenzir' explicitly but are clearly working in a Tenzir context. Also use for questions about Tenzir the company or product, release notes and changelog, blog posts, and solution use cases. |
Tenzir Documentation Map
The security data pipeline platform.
How to use this skill
Navigate the documentation based on the type of question:
Always read the relevant page before answering. Prefer TQL examples from the
documentation over inventing syntax.
Beyond the docs
This skill bundles only the documentation. For anything else about
Tenzir, fetch these live pages on demand instead of answering from
memory:
Answer patterns
Operator syntax question — "How does where work?"
→ Read where, explain the syntax, show the doc's TQL examples.
Integration question — "How do I send data to Splunk?"
→ Read Splunk, provide the pipeline example from the page.
Task question — "How do I parse syslog?"
→ Read Parse delimited text and
read_syslog. Combine the guide's approach
with the operator reference.
Guides are practical step-by-step explanation to help you achieve a specific goal.
Get Started
Drowning in logs, alerts, and rigid tools?
Learn how to migrate legacy TQL pipelines to Tenzir v6, adopt the new execution patterns, and use temporary compatibility controls while you update.
This guide shows you how to install the Tenzir CLI to run pipelines locally or deploy a persistent node.
You can run a pipeline via the platform, on the command line using the tenzir binary, or as code via the configuration file.
This guide shows you how to control pipeline lifecycles through the app or API.
When something goes wrong, the answers are often in the node’s logs, metrics, and diagnostics.
When a field comes out null, holds the wrong value, or has a type you didn’t expect, the cause is almost always visible in a warning or in the data one step upstream.
When a pipeline can’t keep up, you usually notice the symptoms first: a dashboard lags, a downstream system receives data late, or a backlog builds up.
When a node’s memory usage keeps climbing, you want to know whether it will level off or end in an out-of-memory kill, and which pipeline is responsible.
When the disk fills up on a machine that runs a node, you want to know whether the node is responsible, which data takes the space, and how to cap it.
When a node shows as offline in the Tenzir Platform but the process is still running, the node can’t reach the platform’s control plane.
Setup
Build your own AI Workbench by bringing an AI agent and configuring it with Tenzir’s agent skills.
The Tenzir Node is the vehicle to run pipelines.
The Tenzir Platform acts as a fleet management control plane for Tenzir Nodes.
This guide shows you how to create, configure, and delete organizations in the Tenzir Platform.
This guide shows you how to invite people to your organization, manage existing members, and understand the role-based permission model.
This guide shows you how to create, view, and delete workspaces that belong to an organization.
Workspaces in the platform logically group nodes, secrets, and dashboards.
You can pre-define dashboards for your static workspaces.
An ephemeral node is ideal for temporary or auto-scaling deployments.
Work with Data
This guide provides an overview of data collection in TQL.
This guide shows you how to parse text streams into structured events.
This guide shows you how to parse binary data formats into structured events.
This guide shows you how to extract structured data from string fields using TQL’s parsing functions.
Filtering and selecting are fundamental operations when working with data streams.
Transforming values is a fundamental part of data processing.
String manipulation is essential for cleaning, formatting, and transforming text data.
Time is fundamental in data analysis.
Lists (arrays) contain ordered sequences of values.
Records (objects) contain key-value pairs.
Real-world data is rarely flat.
Data comes in many formats.
This guide shows you how to mask sensitive fields such as IP addresses, email addresses, account identifiers, and credentials.
This guide provides an overview of data normalization in TQL.
Store reference data in lookup tables and apply it to OCSF events
Add asset inventory context to OCSF endpoint and user fields
Add threat intelligence to OCSF enrichments, observables, and OSINT fields
Add AI-generated summaries and labels to OCSF events in Tenzir pipelines
Run Sigma detection rules on parsed Windows Event Logs and OCSF process events
When working with data streams, you often need to control which events flow through your pipeline.
The deduplicate operator provides a powerful mechanism to remove duplicate events in a pipeline.
This guide shows you how to send data to various destinations using TQL output operators.
This guide shows you how to make pipeline data available to external consumers by starting an HTTP server.
This guide shows you how to connect pipelines using publish and subscribe operators.
This guide shows you how to fan out an event stream into subpipelines with each and group.
This guide shows you how to distribute events across multiple destinations using the loadbalance operator.
This guide shows you how to aggregate event streams with summarize and window.
Tenzir keeps track of metrics about node resource usage, pipeline state, and runtime performance.
Importing (or ingesting) data can be done by running a pipeline that ends with the import output operator.
Exporting (or querying) data can be done by running a pipeline that begins with the export input operator.
When you write a pipeline, you often reference field names.
This guide shows you how to transform data already stored in a node.
Build
Write your own package
This guide shows you how to create a package from scratch.
This guide shows you how to add tests to your package.
This guide shows you how to create user-defined operators (UDOs) for your package.
This guide shows you how to add deployable pipelines to your package.
This guide shows you how to add enrichment contexts to your package.
This guide shows you how to define package-wide constants in a constants.tql file and reference them as pkg::$name from the package’s own operators and pipelines, as well as from any pipeline that uses the package.
This guide shows you how to make packages configurable with inputs.
This guide shows you how to manage changelog entries and publish releases with tenzir-ship.
This guide shows you how to publish your package.
This guide shows you how to run existing integration tests with the tenzir-test framework.
This guide shows you how to create integration tests with the tenzir-test framework.
This guide shows you how to start fixtures in standalone mode without running tests.
This guide shows you how to create a fixture, wire it into the test harness, and use it from a test.
Runners tell tenzir-test how to execute a discovered file.
This guide shows you how to configure tenzir-test project hooks for setup and cleanup tasks that belong next to your tests.
Contribute
In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to make participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation.
The following diagram visualizes our branching model:
Security is a serious matter for us.
Install TQL language extensions for syntax highlighting in your editor
Tenzir uses CMake as build system with a C++23 compiler.
Tutorials are learning-oriented lessons that take you through a series of steps to complete a project.
Fundamentals
This tutorial teaches you to write TQL that is clear, efficient, and maintainable.
This tutorial teaches you how packages bundle pipelines, operators, contexts, and examples.
In this tutorial you’ll learn how to map events to Open Cybersecurity Schema Framework (OCSF).
Analytics
In this tutorial, you will learn how to use pipelines to plot data as charts.
Explanations are big-picture explanations of higher-level Tenzir concepts.
Architecture
This page explains Tenzir’s deployment architecture, which separates data processing from management through a layered design.
A Tenzir pipeline is a chain of operators that represents a dataflow.
A node is a running process that manages and executes pipelines.
The platform provides fleet management for nodes.
The Tenzir Query Language (TQL) is a dataflow language designed for processing of unstructured byte-streams and semi-structured events.
Concepts
This page explains how to configure the Tenzir CLI and Node.
Operators accept secrets as parameters for sensitive values, such as authentication tokens, passwords, or even URLs.
Enrichment means adding contextual data to events.
This page explains how packages bundle pipelines, operators, contexts, and examples into a deployable unit.
Help
This page defines central terms in the Tenzir ecosystem.
This page answers frequently asked questions about Tenzir.
The reference has nitty-gritty technical descriptions of how Tenzir works.
Language (TQL)
This page explains TQL’s type system, which provides strong typing with automatic inference.
Expressions form the computational core of TQL.
TQL programs are a sequence of statements.
TQL programs compose statements into complete data processing workflows that can execute.
Components
The below example configuration ships with every Tenzir package.
The Tenzir Helm chart deploys one or more tenzir-node instances on a Kubernetes cluster.
These settings configure the Tenzir Platform.
The Tenzir Platform command-line interface (CLI) allows you to interact with the Tenzir Platform from the command line to manage organizations, workspaces, and nodes.
Tools
The tenzir-test harness discovers and runs integration tests for pipelines, fixtures, and custom runners.
tenzir-ship ships as a Python package that requires Python 3.12 or later.
Security
Tenzir’s AWS operators authenticate with AWS using the AWS SDK’s default credential chain, an OIDC web identity token, or static credentials.
API
This reference is generated from the OpenAPI specification.
This reference is generated from the OpenAPI specification.
post/ping
post/user/list-nodes
post/user/create-node
post/user/rename-node
post/user/delete-node
post/user/get-node-token
post/user/generate-client-config
post/user/generate-download-url
post/user/create-demo-node
post/user/retire-demo-node
post/user/proxy
get/user/node-proxy/{tenantid}/{nodeid}/{httppath}
post/user/node-proxy/{tenantid}/{nodeid}/{httppath}
put/user/node-proxy/{tenantid}/{nodeid}/{httppath}
delete/user/node-proxy/{tenantid}/{nodeid}/{httppath}
patch/user/node-proxy/{tenantid}/{nodeid}/{httppath}
head/user/node-proxy/{tenantid}/{nodeid}/{httppath}
options/user/node-proxy/{tenantid}/{nodeid}/{httppath}
post/user/proxy-cached
post/serve
post/user/dashboard/store
post/user/dashboard/get
post/serve-multi
post/user/dashboard/list
post/user/dashboard/delete
post/user/alert/add
post/pipeline/create
post/pipeline/delete
post/user/alert/list
post/user/alert/delete
post/pipeline/launch
post/pipeline/list
post/pipeline/reset-ttl
post/pipeline/update
post/user/secrets/add-external-store
post/user/secrets/select-store
post/user/secrets/delete-external-store
post/user/secrets/list-stores
post/user/secrets/add
post/user/secrets/update
post/user/secrets/remove
post/user/secrets/list
post/user/delete-tenant
post/user/rename-tenant
post/user/switch-tenant
get/user/health
post/user/get-login-info
get/admin/webapp-key
post/admin/create-tenant
post/admin/generate-user-key
post/admin/spawn-node
get/admin/global-tenant-list
post/admin/force-delete-tenant
post/admin/check-connectivy
post/admin/update-tenant
post/admin/add-auth-function
post/admin/delete-auth-function
Operators
Tenzir comes with a wide range of built-in pipeline operators.
Functions
Functions appear in expressions and take positional and/or named arguments, producing a value as a result of their computation.
Node Index
Platform Index
Indexes
For the complete operator listing by category, read Operator Index.
For the complete function listing by category, read Function Index.
Integrations
Enrich DNS, web, and network telemetry with real-time threat intelligence.
Stream events through MSK, store and replay them in S3, ship them to CloudWatch or Amazon Security Lake in OCSF, and pull messages from SQS, all with first-class IAM integration.
Centralized logging for AWS resources, applications, and custom logs.
Ingest and process real-time data streams from Amazon Kinesis.
Publish and subscribe to Amazon's managed Kafka service.
Store, retrieve, and manage data objects in a scalable and durable cloud storage service.
Send OCSF events to Amazon Security Lake.
Send, receive, and manage messages between distributed applications.
Send and receive messages between systems using a standardized, protocol-based approach.
OpenText ArcSight is a SIEM and log management ecosystem.
Store, retrieve, and manage unstructured data at scale.
Real-time data streaming platform and event ingestion service.
Connect the Cisco security portfolio: firewall and IPS events from ASA and Firepower, DNS-layer telemetry from Umbrella, plus Duo, Secure Email, and Secure Endpoint.
Send structured events to ClickHouse tables.
Provides real-time event data including process, file, and network activity.
Index, search, and analyze data in a distributed and scalable manner.
Read from and write to files.
Collect, process, and forward logs and metrics from various sources to many sinks.
Transfer files between a client and a server using a reliable, connection-based protocol.
Connect Google Cloud, Google SecOps, and Google Workspace: stream through Pub/Sub, store in Cloud Storage, read Cloud Logging, and ship UDM events to SecOps.
Send events to Google's log management solution
Send, receive, and process messages between decoupled applications and services.
Store, retrieve, and manage data objects across a scalable, cloud-based environment.
Send events to Google SecOps
Collect, index, and analyze log data for monitoring and troubleshooting.
Transfer and receive data between clients and servers using a request-response protocol.
Send security events to IBM Security QRadar for SIEM analytics and incident investigation.
Publish, subscribe, store, and process streams of records in a distributed messaging system.
Connect Azure, Microsoft 365, and Windows: stream through Event Hubs, store in Blob Storage, ingest into Sentinel, and parse the full Windows telemetry surface.
Detect, prevent, and respond to security threats across endpoints and networks.
Microsoft Graph is the unified API for Microsoft 365, Microsoft Entra ID, and other Microsoft cloud services.
Provides audit logs for database activity, logins, and errors.
Collects Security, System, Application, and other critical OS logs.
Connects to MySQL over the network using the MySQL wire protocol.
Run a lake-native SIEM on ClickHouse for search, detections, cases, and AI-assisted triage.
NATS is a messaging system for services, edge deployments, and cloud-native applications.
Acquire packets from the network and process them with pipelines.
Search, analyze, and visualize data with a distributed, open-source platform.
Send metric events to Prometheus-compatible Remote Write receivers, including Prometheus, Grafana Mimir, Cortex, Thanos Receive, and VictoriaMetrics.
A rapidly growing cloud-native SIEM, tightly integrated with the Azure ecosystem and a strategic destination for Microsoft-centric organizations.
Send events to SentinelOne Singularity™ Data Lake
Store, analyze, and share structured and semi-structured data with a cloud-based warehouse.
Collect, index, and analyze machine-generated data for monitoring, searching, and troubleshooting.
Detect, log, and analyze network traffic for intrusion detection, prevention, and monitoring.
Send and receive Syslog over UDP, TCP, and other protocols.
Establish, maintain, and terminate reliable, connection-oriented communication over IP networks.
Send and receive lightweight, connectionless datagrams over IP networks.
Collect, monitor, and analyze endpoint data for digital forensics and incident response.
Collects security telemetry and runs detections.
Inspect, log, and analyze network traffic for security and operational insights.
Send, receive, and route messages asynchronously over various transport protocols.
Logs all web, firewall, and DNS traffic for users and locations.
