| name | tenzir |
| description | Answer questions using the Tenzir documentation. Use whenever the user asks about TQL syntax, pipeline operators, functions, data parsing or transformation, normalization, OCSF mapping, enrichment, lookup tables, contexts, packages, nodes, platform setup, deployment, configuration, integrations with tools like Splunk, Kafka, S3, Elasticsearch, or any other Tenzir feature. Also use when the user asks how to collect, route, filter, aggregate, or export security data with Tenzir, or needs help writing or debugging TQL pipelines, even if they don't mention 'Tenzir' explicitly but are clearly working in a Tenzir context. Also use for questions about Tenzir the company or product, release notes and changelog, blog posts, and solution use cases. |
Tenzir Documentation Map
The security data pipeline platform.
How to use this skill
Navigate the documentation based on the type of question:
Always read the relevant page before answering. Prefer TQL examples from the
documentation over inventing syntax.
Beyond the docs
This skill bundles only the documentation. For anything else about
Tenzir, fetch these live pages on demand instead of answering from
memory:
Answer patterns
Operator syntax question — "How does where work?"
→ Read where, explain the syntax, show the doc's TQL examples.
Integration question — "How do I send data to Splunk?"
→ Read Splunk, provide the pipeline example from the page.
Task question — "How do I parse syslog?"
→ Read Parse delimited text and
read_syslog. Combine the guide's approach
with the operator reference.
Guides are practical step-by-step explanation to help you achieve a specific goal.
Get Started
The Tenzir Platform is a web interface for managing pipelines and nodes.
This guide shows you how to control pipeline lifecycles through the app or API.
This guide shows you how to install the Tenzir CLI to run pipelines locally or deploy a persistent node.
Drowning in logs, alerts, and rigid tools?
You can run a pipeline via the platform, on the command line using the tenzir binary, or as code via the configuration file.
Learn how to migrate legacy TQL pipelines to Tenzir v6, adopt the new execution patterns, and use temporary compatibility controls while you update.
Setup
We provide several examples of possible platform deployment scenarios.
The default node configuration is optimized for most common scenarios.
The blob storage service exists for exchanging files between the platform and nodes.
You can pre-define dashboards for your static workspaces.
A PostgreSQL database stores the internal state of the platform.
The identity provider (IdP) handles authentication for the Tenzir Platform.
This guide shows you how to configure the three internal Tenzir services: the UI, Gateway, and Platform API.
This guide shows you how to configure a reverse proxy for the Tenzir Platform.
The Tenzir Platform provides a secret store for each workspace.
Set up TLS encryption for operators and platform connections
Workspaces in the platform logically group nodes, secrets, and dashboards.
Deploying a node means spinning it up in one of the supported runtimes.
This guide walks you through deploying the Tenzir Platform Sovereign Edition on AWS using CloudFormation.
This guide shows you how to invite people to your organization, manage existing members, and understand the role-based permission model.
This guide shows you how to create, view, and delete workspaces that belong to an organization.
This guide shows you how to create, configure, and delete organizations in the Tenzir Platform.
Build your own AI Workbench by bringing an AI agent and configuring it with Tenzir’s agent skills.
The Tenzir Node is the vehicle to run pipelines.
The Tenzir Platform acts as a fleet management control plane for Tenzir Nodes.
An ephemeral node is ideal for temporary or auto-scaling deployments.
Work with Data
This guide shows you how to aggregate event streams with summarize and window.
This guide shows you how to clean and normalize values in your data before mapping to a schema.
Tenzir keeps track of metrics about node resource usage, pipeline state, and runtime performance.
Data comes in many formats.
The deduplicate operator provides a powerful mechanism to remove duplicate events in a pipeline.
Add AI-generated summaries and labels to OCSF events in Tenzir pipelines
Add asset inventory context to OCSF endpoint and user fields
Add threat intelligence to OCSF enrichments, observables, and OSINT fields
Run Sigma detection rules on parsed Windows Event Logs and OCSF process events
Exporting (or querying) data can be done by running a pipeline that begins with the export input operator.
This guide shows you how to make pipeline data available to external consumers by starting an HTTP server.
This guide shows you how to fan out an event stream into subpipelines with each and group.
This guide shows you how to interact with HTTP APIs using fromhttp and tohttp operators.
Filtering and selecting are fundamental operations when working with data streams.
This guide shows you how to receive data directly from network sources using TQL.
Importing (or ingesting) data can be done by running a pipeline that ends with the import output operator.
This guide shows you how to distribute events across multiple destinations using the loadbalance operator.
String manipulation is essential for cleaning, formatting, and transforming text data.
This guide shows you how to map events to Microsoft Sentinel Advanced Security Information Model (ASIM) records in TQL.
This guide shows you how to map events to Splunk Common Information Model (CIM) fields in TQL.
This guide shows you how to map events to Elastic Common Schema (ECS) in TQL.
This guide shows you how to write OCSF mapping operators in TQL.
This guide shows you how to map events to Google SecOps Unified Data Model (UDM) records in TQL.
This guide shows you how to mask sensitive fields such as IP addresses, email addresses, account identifiers, and credentials.
This guide provides an overview of data normalization in TQL.
This guide provides an overview of data collection in TQL.
This guide shows you how to parse binary data formats into structured events.
This guide shows you how to parse text streams into structured events.
This guide shows you how to extract structured data from string fields using TQL’s parsing functions.
Real-world data is rarely flat.
This guide shows you how to send data to various destinations using TQL output operators.
Lists (arrays) contain ordered sequences of values.
Records (objects) contain key-value pairs.
When you write a pipeline, you often reference field names.
When working with data streams, you often need to control which events flow through your pipeline.
This guide shows you how to connect pipelines using publish and subscribe operators.
This guide shows you how to transform data already stored in a node.
Transforming values is a fundamental part of data processing.
Store reference data in lookup tables and apply it to OCSF events
Time is fundamental in data analysis.
Build
This guide shows you how to define package-wide constants in a constants.tql file and reference them as pkg::$name from the package’s own operators and pipelines, as well as from any pipeline that uses the package.
This guide shows you how to add enrichment contexts to your package.
Runners tell tenzir-test how to execute a discovered file.
This guide shows you how to create user-defined operators (UDOs) for your package.
This guide shows you how to add deployable pipelines to your package.
This guide shows you how to make packages configurable with inputs.
This guide shows you how to configure tenzir-test project hooks for setup and cleanup tasks that belong next to your tests.
This guide shows you how to create a package from scratch.
This guide shows you how to create a fixture, wire it into the test harness, and use it from a test.
Write your own package
This guide shows you how to manage changelog entries and publish releases with tenzir-ship.
This guide shows you how to publish your package.
This guide shows you how to start fixtures in standalone mode without running tests.
This guide shows you how to run existing integration tests with the tenzir-test framework.
This guide shows you how to add tests to your package.
This guide shows you how to create integration tests with the tenzir-test framework.
Contribute
Tenzir uses CMake as build system with a C++23 compiler.
In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to make participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation.
The following diagram visualizes our branching model:
Security is a serious matter for us.
Install TQL language extensions for syntax highlighting in your editor
Tutorials are learning-oriented lessons that take you through a series of steps to complete a project.
Fundamentals
This tutorial teaches you to write TQL that is clear, efficient, and maintainable.
In this tutorial you’ll learn how to map events to Open Cybersecurity Schema Framework (OCSF).
This tutorial teaches you how packages bundle pipelines, operators, contexts, and examples.
Analytics
In this tutorial, you will learn how to use pipelines to plot data as charts.
Explanations are big-picture explanations of higher-level Tenzir concepts.
Architecture
This page explains Tenzir’s deployment architecture, which separates data processing from management through a layered design.
The Tenzir Query Language (TQL) is a dataflow language designed for processing of unstructured byte-streams and semi-structured events.
A node is a running process that manages and executes pipelines.
A Tenzir pipeline is a chain of operators that represents a dataflow.
The platform provides fleet management for nodes.
Concepts
This page explains how to configure the Tenzir CLI and Node.
Enrichment means adding contextual data to events.
This page explains how packages bundle pipelines, operators, contexts, and examples into a deployable unit.
Operators accept secrets as parameters for sensitive values, such as authentication tokens, passwords, or even URLs.
Help
This page answers frequently asked questions about Tenzir.
This page defines central terms in the Tenzir ecosystem.
The reference has nitty-gritty technical descriptions of how Tenzir works.
Language (TQL)
Expressions form the computational core of TQL.
TQL programs compose statements into complete data processing workflows that can execute.
TQL programs are a sequence of statements.
This page explains TQL’s type system, which provides strong typing with automatic inference.
Components
The Tenzir Platform command-line interface (CLI) allows you to interact with the Tenzir Platform from the command line to manage organizations, workspaces, and nodes.
The below example configuration ships with every Tenzir package.
These settings configure the Tenzir Platform.
The Tenzir Helm chart deploys one or more tenzir-node instances on a Kubernetes cluster.
Tools
tenzir-ship ships as a Python package that requires Python 3.12 or later.
The tenzir-test harness discovers and runs integration tests for pipelines, fixtures, and custom runners.
Security
Tenzir’s AWS operators authenticate with AWS using the AWS SDK’s default credential chain, an OIDC web identity token, or static credentials.
API
This reference is generated from the OpenAPI specification.
This reference is generated from the OpenAPI specification.
post/ping
post/user/list-nodes
post/user/create-node
post/user/rename-node
post/user/delete-node
post/user/get-node-token
post/user/generate-client-config
post/user/generate-download-url
post/user/create-demo-node
post/user/retire-demo-node
post/user/proxy
get/user/node-proxy/{tenantid}/{nodeid}/{httppath}
post/user/node-proxy/{tenantid}/{nodeid}/{httppath}
put/user/node-proxy/{tenantid}/{nodeid}/{httppath}
delete/user/node-proxy/{tenantid}/{nodeid}/{httppath}
patch/user/node-proxy/{tenantid}/{nodeid}/{httppath}
head/user/node-proxy/{tenantid}/{nodeid}/{httppath}
options/user/node-proxy/{tenantid}/{nodeid}/{httppath}
post/user/proxy-cached
post/serve
post/user/dashboard/store
post/user/dashboard/get
post/serve-multi
post/user/dashboard/list
post/user/dashboard/delete
post/user/alert/add
post/pipeline/create
post/pipeline/delete
post/user/alert/list
post/user/alert/delete
post/pipeline/launch
post/pipeline/list
post/pipeline/reset-ttl
post/pipeline/update
post/user/secrets/add-external-store
post/user/secrets/select-store
post/user/secrets/delete-external-store
post/user/secrets/list-stores
post/user/secrets/add
post/user/secrets/update
post/user/secrets/remove
post/user/secrets/list
post/user/delete-tenant
post/user/rename-tenant
post/user/switch-tenant
get/user/health
post/user/get-login-info
get/admin/webapp-key
post/admin/create-tenant
post/admin/generate-user-key
post/admin/spawn-node
get/admin/global-tenant-list
post/admin/force-delete-tenant
post/admin/check-connectivy
post/admin/update-tenant
post/admin/add-auth-function
post/admin/delete-auth-function
Operators
Tenzir comes with a wide range of built-in pipeline operators.
Functions
Functions appear in expressions and take positional and/or named arguments, producing a value as a result of their computation.
Node Index
Platform Index
Indexes
For the complete operator listing by category, read Operator Index.
For the complete function listing by category, read Function Index.
Integrations
Enrich DNS, web, and network telemetry with real-time threat intelligence.
Stream events through MSK, store and replay them in S3, ship them to CloudWatch or Amazon Security Lake in OCSF, and pull messages from SQS, all with first-class IAM integration.
Centralized logging for AWS resources, applications, and custom logs.
Ingest and process real-time data streams from Amazon Kinesis.
Publish and subscribe to Amazon's managed Kafka service.
Store, retrieve, and manage data objects in a scalable and durable cloud storage service.
Send OCSF events to Amazon Security Lake.
Send, receive, and manage messages between distributed applications.
Send and receive messages between systems using a standardized, protocol-based approach.
OpenText ArcSight is a SIEM and log management ecosystem.
Store, retrieve, and manage unstructured data at scale.
Real-time data streaming platform and event ingestion service.
Connect the Cisco security portfolio: firewall and IPS events from ASA and Firepower, DNS-layer telemetry from Umbrella, plus Duo, Secure Email, and Secure Endpoint.
Send structured events to ClickHouse tables.
Provides real-time event data including process, file, and network activity.
Index, search, and analyze data in a distributed and scalable manner.
Read from and write to files.
Collect, process, and forward logs and metrics from various sources to many sinks.
Transfer files between a client and a server using a reliable, connection-based protocol.
Connect Google Cloud, Google SecOps, and Google Workspace: stream through Pub/Sub, store in Cloud Storage, read Cloud Logging, and ship UDM events to SecOps.
Send events to Google's log management solution
Send, receive, and process messages between decoupled applications and services.
Store, retrieve, and manage data objects across a scalable, cloud-based environment.
Send events to Google SecOps
Collect, index, and analyze log data for monitoring and troubleshooting.
Transfer and receive data between clients and servers using a request-response protocol.
Send security events to IBM Security QRadar for SIEM analytics and incident investigation.
Publish, subscribe, store, and process streams of records in a distributed messaging system.
Connect Azure, Microsoft 365, and Windows: stream through Event Hubs, store in Blob Storage, ingest into Sentinel, and parse the full Windows telemetry surface.
Detect, prevent, and respond to security threats across endpoints and networks.
Microsoft Graph is the unified API for Microsoft 365, Microsoft Entra ID, and other Microsoft cloud services.
Provides audit logs for database activity, logins, and errors.
Collects Security, System, Application, and other critical OS logs.
Connects to MySQL over the network using the MySQL wire protocol.
NATS is a messaging system for services, edge deployments, and cloud-native applications.
Acquire packets from the network and process them with pipelines.
Search, analyze, and visualize data with a distributed, open-source platform.
Send metric events to Prometheus-compatible Remote Write receivers, including Prometheus, Grafana Mimir, Cortex, Thanos Receive, and VictoriaMetrics.
A rapidly growing cloud-native SIEM, tightly integrated with the Azure ecosystem and a strategic destination for Microsoft-centric organizations.
Send events to SentinelOne Singularity™ Data Lake
Store, analyze, and share structured and semi-structured data with a cloud-based warehouse.
Collect, index, and analyze machine-generated data for monitoring, searching, and troubleshooting.
Detect, log, and analyze network traffic for intrusion detection, prevention, and monitoring.
Send and receive Syslog over UDP, TCP, and other protocols.
Establish, maintain, and terminate reliable, connection-oriented communication over IP networks.
Send and receive lightweight, connectionless datagrams over IP networks.
Collect, monitor, and analyze endpoint data for digital forensics and incident response.
Collects security telemetry and runs detections.
Inspect, log, and analyze network traffic for security and operational insights.
Send, receive, and route messages asynchronously over various transport protocols.
Logs all web, firewall, and DNS traffic for users and locations.
