extracting-pii-entities

name	extracting-pii-entities
description	Detect PHI/PII spans in clinical text with OpenMed's extract_pii without altering the text. Use when the user wants to find names, dates, MRNs, phone numbers, addresses, SSNs, or other identifiers and get their offsets and labels (not redact them), inspect what would be removed before de-identifying, route spans to a custom redactor, normalize labels to a canonical taxonomy, or filter by confidence and language. Covers extract_pii, the PIIEntity fields, CANONICAL_LABELS / normalize_label, and how it differs from deidentify. Pairs before reidentifying-text and deidentifying-clinical-text.
license	Apache-2.0
metadata	{"project":"OpenMed","category":"openmed-core","pairs":"adjacent","version":"1.0"}

Extracting PII Entities

openmed.extract_pii finds PHI/PII spans and returns them without changing the text. Use it when you need to see the identifiers — to audit, route to a custom redactor, or decide a policy — rather than produce redacted output. It runs on-device.

When to use

• You want the spans and labels of identifiers, with the original text intact.
• You need a preview of what deidentify would act on before committing.
• You are feeding detected spans into a downstream redactor (your own, Presidio, or deidentify).
• You want to normalize model labels to a stable canonical taxonomy.

If you instead want redacted/masked output directly, use deidentifying-clinical-text (openmed.deidentify). If you need reversible masking, see reidentifying-text.

extract_pii vs deidentify

	extract_pii	deidentify
Changes the text?	No	Yes (mask/remove/replace/hash/shift)
Returns	PredictionResult (spans)	DeidentificationResult (redacted text)
Default threshold	0.5	0.7 (safety-biased)
Use for	detection, audit, routing	producing safe output

Install

pip install "openmed[hf]"

Quick start

import openmed

note = "Patient John Doe (MRN 00481726), DOB 1970-01-15, phone [phone]."

result = openmed.extract_pii(note, confidence_threshold=0.5)

for ent in result.entities:
    print(f"{ent.label:10} {ent.text!r:18} {ent.confidence:.2f} [{ent.start}:{ent.end}]")

extract_pii(...) returns a PredictionResult. Its .entities are PIIEntity objects (synthetic example fields shown):

ent.text            # the identifier surface string, e.g. "617-555-0142"
ent.label           # detected label, e.g. "PHONE"
ent.confidence      # model score in [0, 1]   (NOTE: .confidence, not .score)
ent.start / ent.end # character offsets into the original note
ent.canonical_label # label mapped to OpenMed's canonical taxonomy (if set)
ent.entity_type     # same as label

The text is unchanged — result.text is your original input.

Signature & key parameters

openmed.extract_pii(
    text,
    model_name="OpenMed/OpenMed-PII-SuperClinical-Small-44M-v1",  # default EN model
    confidence_threshold=0.5,     # raise for precision, lower for recall
    use_smart_merging=True,       # merge fragmented spans into whole units
    lang="en",                    # en es pt fr de it nl hi te ar tr ja
    loader=None,                  # reuse a ModelLoader across calls
)

• use_smart_merging=True (default) reassembles fragmented predictions into complete units (a full phone number, a full date) — keep it on.
• lang selects the language-appropriate default model and regex patterns. Pass the right language; do not run the English model on non-English text. Use openmed.get_default_pii_model(lang) to confirm coverage.

Normalize labels to the canonical taxonomy

Different models may emit slightly different label spellings. Normalize them to OpenMed's canonical set so downstream logic is stable:

import openmed
from openmed import CANONICAL_LABELS, normalize_label

result = openmed.extract_pii("Email [e-mail]; SSN 123-45-6789.")

for ent in result.entities:
    canon = ent.canonical_label or normalize_label(ent.label)
    assert canon in CANONICAL_LABELS or canon == "OTHER"
    print(ent.text, "->", canon)

CANONICAL_LABELS is a frozenset of UPPER_SNAKE_CASE labels (e.g. PERSON, DATE, PHONE, EMAIL, SSN, ID_NUM, LOCATION). normalize_label(label) accepts messy inputs ("FIRSTNAME", "first_name", "B-EMAIL") and maps unknown labels to OTHER rather than raising.

Feed spans to a downstream redactor

extract_pii gives you offsets; you decide the action. A simple offset-based redactor (replace highest-offset first so positions stay valid):

import openmed

note = "Patient John Doe, MRN 00481726, seen 2024-03-02."
result = openmed.extract_pii(note, confidence_threshold=0.6)

redacted = note
for ent in sorted(result.entities, key=lambda e: e.start, reverse=True):
    redacted = redacted[:ent.start] + f"[{ent.label}]" + redacted[ent.end:]

print(redacted)   # Patient [PERSON], MRN [ID_NUM], seen [DATE].

For production redaction, masking strategies, and policy profiles, hand the work to openmed.deidentify instead of hand-rolling — it adds a safety sweep and date-shifting (see deidentifying-clinical-text).

Hand-off to / from OpenMed

• To deidentifying-clinical-text: once you have reviewed the spans, call openmed.deidentify(note, method="mask", policy="hipaa_safe_harbor") to produce safe output — it re-detects with a higher default threshold for safety.
• To reidentifying-text: if you need reversibility, use openmed.deidentify(..., keep_mapping=True) and store the mapping securely.
• To Presidio / custom anonymizers: extract_pii spans (label, start, end) translate cleanly into other recognizers' result formats; OpenMed also ships an Anonymizer (openmed.Anonymizer) for richer surrogate generation.

Edge cases & gotchas

• Attribute is .confidence, not .score. PIIEntity extends EntityPrediction.
• Detection is not redaction. extract_pii never changes text — if a caller expected redacted output, they want deidentify.
• Threshold trade-off: 0.5 favors recall (good for finding PHI to review). For removing PHI, prefer deidentify's safety-biased 0.7 default.
• Language matters: wrong lang silently lowers recall. Verify with get_default_pii_model(lang).
• No raw PHI in logs/audit. Record offsets, labels, and hashes — never the identifier text. Use synthetic data in examples and tests.
• Local-first. No cloud calls in PHI workflows; models run on-device after a one-time download.

Standards & references

• HIPAA Safe Harbor 18 identifiers: 45 CFR §164.514(b)(2) — https://www.hhs.gov/hipaa/for-professionals/privacy/special-topics/de-identification/
• OpenMed canonical labels: openmed.CANONICAL_LABELS / openmed.normalize_label.
• OpenMed PII models: https://huggingface.co/OpenMed