# NeMo Anonymizer
[](https://opensource.org/licenses/Apache-2.0)
[](https://www.python.org/downloads/)
**Detect and replace sensitive entities in text using LLM-powered workflows.**
---
## What can you do with Anonymizer?
- **Detect entities** using GLiNER-PII and LLM-based augmentation and validation
- **Replace with 4 strategies** — LLM-generated substitute, redact, annotate, or hash (deterministic, local)
- **Preview results** before full runs with `display_record()` visualization
---
## Quick Start
### 1. Install
```bash
pip install nemo-anonymizer
```
> [!NOTE]
> This project will download and install additional third-party open source
> software projects. Review the license terms of these open source projects
> before use.
Or install from source:
```bash
git clone https://github.com/NVIDIA-NeMo/Anonymizer.git
cd Anonymizer
make install
```
### 2. Set up model providers
By default, Anonymizer expects a compatible local GLiNER2 endpoint for entity detection and uses
text LLMs hosted on [OpenRouter](https://openrouter.ai) for augmentation and validation.
The tutorial notebooks can start the local detector automatically. You can also bring your own models
via custom provider configs.
Use of OpenRouter is subject to its own terms of service and privacy practices, which are separate
from and independent of the NeMo Framework library. Review those terms and the data policies of the
models you select before sending sensitive data to the default endpoint.
```bash
export OPENROUTER_API_KEY="your-openrouter-api-key"
```
### 3. Anonymize text
#### CLI
> Tip: All examples below use `uv run` to invoke commands. If you prefer, activate the venv with `source .venv/bin/activate` and run commands directly.
```bash
DATA_URL="https://raw.githubusercontent.com/NVIDIA-NeMo/Anonymizer/refs/heads/main/docs/data/NVIDIA_synthetic_biographies.csv"
# Preview on a small sample
uv run anonymizer preview --source $DATA_URL --text-column biography --replace redact --num_records 3
# Full run with output file
uv run anonymizer run --source $DATA_URL --text-column biography --replace redact --output result.csv
# Validate config without running
uv run anonymizer validate --source $DATA_URL --text-column biography --replace hash
```
Run `anonymizer --help` or `anonymizer <subcommand> --help` for all options.
#### Python API
```python
from anonymizer import Anonymizer, AnonymizerConfig, AnonymizerInput, Redact
DATA_URL = "https://raw.githubusercontent.com/NVIDIA-NeMo/Anonymizer/refs/heads/main/docs/data/NVIDIA_synthetic_biographies.csv"
# Uses Anonymizer's bundled model providers (see src/anonymizer/config/default_model_configs/providers.yaml)
anonymizer = Anonymizer()
config = AnonymizerConfig(replace=Redact())
preview = anonymizer.preview(
config=config,
data=AnonymizerInput(source=DATA_URL, text_column="biography"),
num_records=3,
)
# Visualize with entity highlights and replacement map
preview.display_record()
# Most important columns only
preview.dataframe
# Full pipeline trace, including internal underscore-prefixed columns
preview.trace_dataframe
```
For custom model endpoints, pass a providers YAML:
```python
anonymizer = Anonymizer(model_providers="path/to/model_providers.yaml")
```
For services and other applications that already hold text in memory, pass
typed records instead of writing a temporary CSV or Parquet file:
```python
from anonymizer import TextRecord, TextRecordsInput
data = TextRecordsInput(
records=[
TextRecord(id="turn-1", text="Contact Alice at alice@example.com"),
TextRecord(id="turn-2", text="Bob works at Acme"),
],
data_summary="Application observability messages",
)
result = anonymizer.run(config=config, data=data)
# Stable public output: IDs, transformed text, and validated entity spans.
print(result.dataframe[["id", "text_replaced", "final_entities"]])
```
IDs and input order are retained for records that complete the pipeline. A
dropped record uses the same caller-provided ID in `result.failed_records`.
`trace_dataframe` is for pipeline diagnostics.
## Language And Regional Coverage
Anonymizer has been tested most extensively on English-language data. Multilingual quality has not yet been evaluated systematically across languages, domains, and models.
Although testing so far has been primarily in English, the supported entity set is not limited to U.S.-specific identifiers. Detection and anonymization can also apply to international formats such as non-U.S. phone numbers, addresses, legal references, and national or regional identification numbers, though coverage will vary by language, region, and model configuration.
If you are working with another language, we encourage you to experiment on a small sample first with `preview()`, validate detected entities and transformed output carefully, and adjust your model providers and model configs as needed.
---
## Replacement Strategies
| Strategy | Output for `"Alice"` (first_name) | Configurable |
|----------|----------------------------------|-------------|
| **Substitute** | `Maya` | `instructions` |
| **Redact** | `[REDACTED_FIRST_NAME]` | `format_template` |
| **Annotate** | `<Alice, first_name>` | `format_template` |
| **Hash** | `<HASH_FIRST_NAME_3bc51062973c>` | `format_template`, `algorithm`, `digest_length` |
```python
from anonymizer import Redact, Annotate, Hash, Substitute
# LLM-generated contextual replacements
AnonymizerConfig(replace=Substitute())
# Constant redaction
AnonymizerConfig(replace=Redact(format_template="****"))
# Annotation with entities tagging
AnonymizerConfig(replace=Annotate(format_template="<{text}-|-{label}>"))
# Deterministic hash with short digest
AnonymizerConfig(replace=Hash(algorithm="sha256", digest_length=8))
```
---
## Using the Agent Skill
This repo ships an Agent Skill at [`skills/anonymizer/`](skills/anonymizer/SKILL.md) that elicits your dataset's privacy requirements, helps choose Rewrite or Replace with an appropriate strategy, and drafts a runnable script for you to iterate on.
Install via [skills.sh](https://skills.sh):
```bash
npx skills add NVIDIA-NeMo/Anonymizer
```
After installation, invoke it with `/anonymizer` in an agent that supports slash-style skill calls, or describe what you want to anonymize and let it auto-trigger.
---
## Development
```bash
make install-dev # Install with dev dependencies
make test # Run tests
make coverage # Run with coverage report
make format-check # Lint + format check (read-only)
anonymizer --help # CLI usage
make install-pre-commit # Install pre-commit hooks
```
---
## Requirements
- Python 3.11+
- [NeMo Data Designer](https://github.com/NVIDIA-NeMo/DataDesigner) (installed as dependency)
- A compatible local GLiNER2 endpoint (the tutorial notebooks can start one automatically)
- [OpenRouter API key](https://openrouter.ai/settings/keys) for the default text LLM provider, or custom model endpoints
---
## Telemetry and Privacy
NeMo Anonymizer collects anonymous run-level telemetry to help prioritize product improvements. One event is sent per `Anonymizer.run()` / `Anonymizer.preview()` call, containing only technical metadata: the replacement strategy in use, models used, model hosts (e.g. `nvidia-build`, `openrouter`, `other`), input-record counts, run duration, and failure attribution by pipeline step. **No user data, record contents, prompts, or model outputs are collected.** See the [Telemetry and Privacy docs](https://nvidia-nemo.github.io/Anonymizer/latest/#telemetry-and-privacy) for the full field list.
You may opt out of telemetry at any time:
- **For one CLI invocation**: pass `--no-emit-telemetry`
```bash
uv run anonymizer run --source data.csv --text-column text --replace redact --no-emit-telemetry
```
- **In the SDK**: set `emit_telemetry=False` on `AnonymizerConfig`
```python
config = AnonymizerConfig(replace=Redact(), emit_telemetry=False)
```
- **For the current shell**: set the environment variable
```bash
export NEMO_TELEMETRY_ENABLED=false
```
Aggregate usage data (such as which models are most popular) will be shared back with the community. It is not used to track any individual user behavior.
**Use of third-party endpoints, including OpenRouter and NVIDIA Build:** Anonymizer can be configured to use various inference endpoints, including [OpenRouter](https://openrouter.ai), [build.nvidia.com](https://build.nvidia.com), or local model servers. If you choose to use a third-party endpoint, that endpoint's own terms of service and privacy practices apply independently of this library. Any opt-out you exercise within Anonymizer does not extend to data collection by your chosen endpoint.
---
## License
Apache License 2.0 — see [LICENSE](LICENSE) for details.