# Security Documents Generator
Generate synthetic Security data for Elasticsearch and Kibana development, demos, and performance testing.
> Note: For Elasticsearch `8.18` and earlier, use tag `8.18-compatibility`.
## What this tool can generate
- Alerts and events
- Entity Store data (users, hosts, services, generic entities)
- Privileged User Monitoring datasets
- Detection rules and gap scenarios
- Risk engine datasets and ingest loads
- Cloud Security Posture (Elastic + third-party sources)
- Entity Store performance data + baseline comparison reports
## Requirements
- Node.js `24.18.0`
- Yarn `^1.22.22`
- Access to Elasticsearch and Kibana
## Install
```bash
yarn
```
## Configuration
On first run, the CLI creates `config.json` interactively if no valid config exists:
```bash
yarn start
```
You can authenticate with either:
- `username` + `password`
- `apiKey`
### Example `config.json` (API key)
```json
{
"elastic": {
"node": "https://example.es.us-west2.gcp.elastic-cloud.com",
"apiKey": "your-elastic-api-key"
},
"kibana": {
"node": "https://example.kb.us-west2.gcp.elastic-cloud.com:9243",
"apiKey": "your-kibana-api-key"
},
"serverless": false,
"eventIndex": "logs-testlogs-default"
}
```
### Example `config.json` (local development with self-signed certificates)
```json
{
"elastic": {
"node": "https://localhost:9200",
"apiKey": "your-elastic-api-key"
},
"kibana": {
"node": "https://localhost:5601",
"apiKey": "your-kibana-api-key"
},
"serverless": false,
"eventIndex": "logs-testlogs-default",
"allowSelfSignedCerts": true
}
```
> **Security note**: Only enable `allowSelfSignedCerts` for local development or testing environments. This option disables TLS certificate verification, which should never be used in production.
### Example `config.json` (basic auth)
```json
{
"elastic": {
"node": "http://localhost:9200",
"username": "elastic",
"password": "changeme"
},
"kibana": {
"node": "http://localhost:5601",
"username": "elastic",
"password": "changeme"
},
"serverless": false,
"eventIndex": "logs-testlogs-default"
}
```
### Environment variable overrides
Environment variables override `config.json` values:
- `ELASTIC_NODE`, `ELASTIC_USERNAME`, `ELASTIC_PASSWORD`, `ELASTIC_API_KEY`
- `KIBANA_NODE`, `KIBANA_USERNAME`, `KIBANA_PASSWORD`, `KIBANA_API_KEY`
- `SERVERLESS`
- `EVENT_INDEX`
- `EVENT_DATE_OFFSET_HOURS`
- `ALLOW_SELF_SIGNED_CERTS` - Set to `true` or `1` to disable TLS certificate verification (for local development only)
## CLI help
```bash
yarn start help
```
## Quick start recipes
### Alerts + risk score API timing
```bash
yarn start delete-alerts
yarn start generate-alerts -n 10000 -h 100 -u 100
yarn start test-risk-score
```
### Entity Store (interactive)
```bash
yarn start entity-store
```
### Privileged User Monitoring (quick)
```bash
yarn start privmon-quick --space default
```
### Cloud Security Posture demo data
```bash
yarn start csp --data-sources all --findings-count 50
```
### Correlated Organization Data
Interactive — prompts for size, productivity suite, and detection rules:
```bash
yarn start org-data
```
Non-interactive — skip all prompts by providing options directly:
```bash
yarn start org-data --size small --productivity-suite microsoft --integrations azure,entra_id
```
Enable detection rules without prompting:
```bash
yarn start org-data --size medium --detection-rules
```
Scale the dataset to an approximate total document count with `--doc-count`. The employee
population is sized so the enabled integrations produce roughly the requested number of documents
(overriding the size-based employee count):
```bash
yarn start org-data --integrations active_directory,okta --doc-count 100000
```
Each prompt is skipped individually when its flag is present. Omit any flag to be prompted for it:
| Flag | Values | Default (when omitted) |
| ---------------------- | ------------------------------------------- | ------------------------- |
| `--size` | `john_doe`, `small`, `medium`, `enterprise` | interactive prompt |
| `--productivity-suite` | `microsoft`, `google` | interactive prompt |
| `--detection-rules` | flag (boolean) | interactive prompt |
| `--integrations` | comma-separated list | all default integrations |
| `--all` | flag (boolean) | — |
| `--doc-count` | number | size-based employee count |
| `--name` | string | `Acme CRM` |
| `--space` | string | `default` |
| `--seed` | number | random |
> **Note on `--doc-count`**: the total is an approximation — it back-computes the employee count
> from the enabled integrations' per-employee document yield (e.g. Okta ≈ 3 docs/employee,
> Active Directory ≈ 1.35 docs/employee), so the actual count lands near the target rather than
> exactly on it. When `--doc-count` is set and `--size` is omitted, size defaults to `enterprise`
> (it only affects non-employee config such as hosts and cloud resources), so the command runs
> without prompting.
## Commands
Detailed command documentation is colocated with command code under `src/commands`.
| Command | Summary | Details |
| ---------------------------------------------------------------- | ----------------------------------------------------------------------------------- | -------------------------------------------------------- |
| `generate-alerts` | Generate synthetic detection alerts | `src/commands/documents/README.md` |
| `generate-events` | Generate synthetic events in `eventIndex` | `src/commands/documents/README.md` |
| `generate-graph` | Generate fake graph data | `src/commands/documents/README.md` |
| `delete-alerts` | Delete all generated alerts | `src/commands/documents/README.md` |
| `delete-events` | Delete all generated events | `src/commands/documents/README.md` |
| `entity-resolution-demo` | Load entity resolution demo dataset | `src/commands/entity_store/README.md` |
| `entity-store` | Interactive Entity Store generation flow | `src/commands/entity_store/README.md` |
| `quick-entity-store` | Quick non-interactive Entity Store setup | `src/commands/entity_store/README.md` |
| `clean-entity-store` | Clean Entity Store data | `src/commands/entity_store/README.md` |
| `risk-score-v2` | End-to-end Entity Store V2: entities, alerts, risk engine, criticality, watchlists | `src/commands/entity_store/README.md` |
| `generate-entity-maintainers-data` | Entity Store V2 maintainer data: risk history, anomaly behaviours, relationships | `src/commands/entity_store/README.md` |
| `leads` | Generate AI-driven threat hunting leads (interactive, requires inference connector) | `src/commands/lead_generation/` |
| `test-risk-score` | Run risk score API test call | `src/commands/misc/README.md` |
| `generate-entity-ai-insights` | Generate vulnerabilities, misconfigurations and anomalous behavior for entities. | `src/commands/misc/README.md` |
| `generate-asset-criticality` | Generate asset criticality assignments | `src/commands/misc/README.md` |
| `generate-legacy-risk-score` | Install and generate legacy risk score data | `src/commands/misc/README.md` |
| `single-entity` | Create one entity with optional setup flows | `src/commands/misc/README.md` |
| `privileged-user-monitoring` | Interactive privileged user monitoring dataset generation | `src/commands/privileged_user_monitoring/README.md` |
| `privmon-quick` | Fast privileged user monitoring generation | `src/commands/privileged_user_monitoring/README.md` |
| `rules` | Generate detection rules and events | `src/commands/rules/README.md` |
| `delete-rules` | Delete detection rules | `src/commands/rules/README.md` |
| `risk-engine ingest` | Generate and ingest risk-engine data in batches | `src/commands/risk_engine/README.md` |
| `esql-stress-test` | Stress test ESQL queries | `src/commands/risk_engine/README.md` |
| `painless-stress-test` | Stress test scripted metric risk scoring | `src/commands/risk_engine/README.md` |
| `create-risk-engine-data` | Build risk engine perf data file | `src/commands/risk_engine/README.md` |
| `create-risk-engine-dataset` | Build named risk engine perf datasets | `src/commands/risk_engine/README.md` |
| `upload-risk-engine-dataset` | Upload all files from a perf dataset directory | `src/commands/risk_engine/README.md` |
| `upload-risk-engine-data-interval` | Repeatedly upload risk engine data file | `src/commands/risk_engine/README.md` |
| `create-perf-data` | Create Entity Store perf JSONL data file | `src/commands/entity_store_perf/README.md` |
| `upload-perf-data` | Upload perf data once | `src/commands/entity_store_perf/README.md` |
| `upload-perf-data-interval` | Upload perf data repeatedly at intervals | `src/commands/entity_store_perf/README.md` |
| `create-baseline` | Extract and save baseline metrics from logs | `src/commands/baseline_metrics/README.md` |
| `list-baselines` | List saved baseline metric files | `src/commands/baseline_metrics/README.md` |
| `compare-metrics` | Compare a run against baseline metrics | `src/commands/baseline_metrics/README.md` |
| `generate-cloud-security-posture` (`csp`) | Generate CSP findings across sources | `src/commands/generate_cloud_security_posture/README.md` |
| `generate-correlated-organization-data` (`org-data`) | Generate correlated organization security integration data | `src/commands/org_data/` |
| `generate-correlated-organization-data-quick` (`org-data-quick`) | Quick correlated organization data generation with defaults | `src/commands/org_data/` |
### Quick command list
- **Documents**
- `generate-alerts`, `generate-events`, `generate-graph`, `delete-alerts`, `delete-events`
- **Entity Store**
- `entity-resolution-demo`, `entity-store`, `quick-entity-store`, `clean-entity-store`, `risk-score-v2`, `generate-entity-maintainers-data`
- **Risk and Security utilities**
- `test-risk-score`, `generate-entity-ai-insights`, `generate-asset-criticality`, `generate-legacy-risk-score`, `single-entity`
- **Privileged User Monitoring**
- `privileged-user-monitoring` (`privmon`), `privmon-quick` (`quickmon`)
- **Rules**
- `rules`, `delete-rules`
- **Risk engine**
- `risk-engine ingest`, `esql-stress-test`, `painless-stress-test`, `create-risk-engine-data`, `create-risk-engine-dataset`, `upload-risk-engine-dataset`, `upload-risk-engine-data-interval`
- **Entity Store performance**
- `create-perf-data`, `upload-perf-data`, `upload-perf-data-interval`
- **Baselines and metrics**
- `create-baseline`, `list-baselines`, `compare-metrics`
- **Cloud Security Posture**
- `generate-cloud-security-posture` (`csp`)
- **Correlated Organization Data**
- `generate-correlated-organization-data` (`org-data`), `generate-correlated-organization-data-quick` (`org-data-quick`)
- **Threat hunting leads**
- `leads`
## AI assistant skills
Skills are instruction files that give an AI assistant focused context for a specific task in this repo. Two formats are included:
- **Claude Code skills** (`.claude/commands/`) — slash commands auto-discovered when the repo is open in [Claude Code](https://claude.ai/code). Type `/skill-name` to invoke.
- **Agent skills** (`.agents/skills/`) — reusable instruction files you can reference manually in any AI assistant that supports attaching context files.
### `/seed-ea` (Claude Code)
Helps you populate every Entity Analytics page in Kibana without needing to remember which command covers which page. Describe what you want to populate and the skill gives you the exact `yarn start` command — or runs it for you.
Requires [Claude Code](https://claude.ai/code). See [SEED_EA_SKILL.md](SEED_EA_SKILL.md) for full documentation.
Located at `.claude/commands/seed-ea.md`.
### `/update-org-data-integrations` (agent skill)
Updates existing integrations or creates new ones for the `org-data` command, using upstream field
definitions and reference sample events from Elastic repos as the source of truth. Recommended to
use in **Plan mode** (or equivalent) so the agent proposes changes before applying them.
Located at `.agents/skills/update-org-data-integrations/SKILL.md`.
## Performance and baselines
Perf-data generation and baseline workflows are documented in command-specific pages:
- `src/commands/entity_store_perf/README.md`
- `src/commands/baseline_metrics/README.md`