create-preprocessor-scripts

name	create-preprocessor-scripts
description	Use when adding or updating kphtools IDA preprocessor finder scripts, config.yaml symbol entries, LLM_DECOMPILE reference YAMLs, generated reference annotations, or merged finder coverage for ntoskrnl symbols.
disable-model-invocation	true

Create Preprocessor Scripts

Create or update kphtools preprocessor finders under ida_preprocessor_scripts/, with matching config.yaml entries and annotated LLM reference YAML when needed.

When to Use

• Adding or changing an ida_preprocessor_scripts/find-XXX.py finder.
• Adding or changing config.yaml modules[].skills or modules[].symbols entries.
• Generating ida_preprocessor_scripts/references/<module>/<FuncName>.<arch>.yaml.
• Adding annotations to generated reference YAML for LLM_DECOMPILE.
• Merging related symbols into one find-A-AND-B.py finder.

Required Inputs

Collect only what the task needs:

• Symbol names and output YAML names.
• Category: struct_offset, gv, or func.
• data_type, module name, architecture, aliases, and dependencies if present.
• Discovery source: direct PDB/name metadata, NtAPI syscall signature bytes, FUNC_XREFS caller-side byte signatures, or an LLM_DECOMPILE reference function.
• Target struct/member pairs, global variable names, or function names.
• Desired YAML fields for each output artifact.

Workflow

1. Determine Targets and Grouping

• Prefer existing kphtools patterns over inventing a new finder shape.
• Merge symbols when they share the same reference function and can be located from the same evidence.
• Split symbols when they use different reference functions, form dependency chains, or need different discovery logic.
• Merged finder names use find-A-AND-B-AND-C.py; the config.yaml skill name must match the filename without .py.
• When a merge replaces old finders and the current task authorizes cleanup, delete stale ida_preprocessor_scripts/find-Old.py files and stale .claude/skills/find-Old/ directories. Also remove old config.yaml skill entries.

2. Write the Finder Script

Place scripts at:

ida_preprocessor_scripts/find-<SkillName>.py

There are two distinct caller shapes depending on discovery method.

NtAPI Signature Pattern

Use when the target is a Windows NT system call and a unique byte signature from the syscall stub is known. Import _extract_ntapi and call preprocess_ntapi_symbols directly — do not use preprocess_common_skill for this pattern.

from __future__ import annotations

from ida_preprocessor_scripts import _extract_ntapi

TARGET_FUNCTION_NAMES = ["NtAlpcCreatePortSection"]

NTAPI_SIGNATURES = {
    "NtAlpcCreatePortSection": ["BE A7 FB 31 06 00 00 00"],
}

GENERATE_YAML_DESIRED_FIELDS = {
    "NtAlpcCreatePortSection": ["func_name", "func_rva"],
}


async def preprocess_skill(session, skill, symbol, binary_dir, pdb_path, debug, llm_config):
    return await _extract_ntapi.preprocess_ntapi_symbols(
        session=session,
        skill=skill,
        symbol=symbol,
        binary_dir=binary_dir,
        pdb_path=pdb_path,
        debug=debug,
        target_function_names=TARGET_FUNCTION_NAMES,
        ntapi_signatures=NTAPI_SIGNATURES,
        generate_yaml_desired_fields=GENERATE_YAML_DESIRED_FIELDS,
    )

Module-level variables for this pattern: TARGET_FUNCTION_NAMES, NTAPI_SIGNATURES, GENERATE_YAML_DESIRED_FIELDS.

The signature bytes come from the NT syscall stub (e.g. mov eax, <syscall_number> encodes the system call index). Each signature is a list to allow multiple candidates across OS versions.

NT API finders produce a reference YAML (e.g. NtAlpcCreatePortSection.yaml) consumed as expected_input by downstream finders. They do not get a symbols entry in config.yaml because they are reference anchors, not kphtools output symbols.

Examples: ida_preprocessor_scripts/find-NtSecureConnectPort.py, ida_preprocessor_scripts/find-NtAlpcCreatePortSection.py.

PDB / Name / LLM Pattern

Use ida_preprocessor_common.preprocess_common_skill for all other cases (struct offsets, global variables, named functions, LLM_DECOMPILE):

async def preprocess_skill(session, skill, symbol, binary_dir, pdb_path, debug, llm_config):
    return await preprocess_common_skill(
        session=session,
        skill=skill,
        symbol=symbol,
        binary_dir=binary_dir,
        pdb_path=pdb_path,
        debug=debug,
        llm_config=llm_config,
        ...
    )

Module-level variables for this pattern:

• Struct offsets: TARGET_STRUCT_MEMBER_NAMES, STRUCT_METADATA, GENERATE_YAML_DESIRED_FIELDS.
• Global variables: TARGET_GLOBALVAR_NAMES, GV_METADATA, GENERATE_YAML_DESIRED_FIELDS.
• Functions (direct): TARGET_FUNCTION_NAMES, FUNC_METADATA, GENERATE_YAML_DESIRED_FIELDS.
• Functions (by caller xrefs): TARGET_FUNCTION_NAMES, FUNC_XREFS, GENERATE_YAML_DESIRED_FIELDS — pass func_xrefs=FUNC_XREFS.
• LLM_DECOMPILE: LLM_DECOMPILE, passed as llm_decompile_specs=LLM_DECOMPILE.

FUNC_XREFS Pattern

Use when the target function cannot be found by name or PDB metadata, but its callers contain unique byte sequences. IDA locates functions that have at least one caller matching all specified xref criteria.

from __future__ import annotations

import ida_preprocessor_common as preprocessor_common

TARGET_FUNCTION_NAMES = ["PerfDiagInitialize"]

FUNC_XREFS = [
    {
        "func_name": "PerfDiagInitialize",
        "xref_strings": [],           # caller must reference these ASCII strings
        "xref_unicode_strings": [],   # caller must reference these Unicode strings (prefix FULLMATCH: for exact match)
        "xref_gvs": [],               # caller must reference these global variable names
        "xref_signatures": ["49 49 14 67", "32 51 59 48"],  # caller must contain one of these byte patterns
        "xref_funcs": [],             # caller must call these functions
        "exclude_funcs": [],          # discard candidates whose callers call these functions
        "exclude_strings": [],
        "exclude_unicode_strings": [],
        "exclude_gvs": [],
        "exclude_signatures": [],
    },
]

GENERATE_YAML_DESIRED_FIELDS = {
    "PerfDiagInitialize": ["func_name", "func_rva"],
}


async def preprocess_skill(session, skill, symbol, binary_dir, pdb_path, debug, llm_config):
    return await preprocessor_common.preprocess_common_skill(
        session=session,
        skill=skill,
        symbol=symbol,
        binary_dir=binary_dir,
        pdb_path=pdb_path,
        debug=debug,
        llm_config=llm_config,
        func_names=TARGET_FUNCTION_NAMES,
        func_xrefs=FUNC_XREFS,
        generate_yaml_desired_fields=GENERATE_YAML_DESIRED_FIELDS,
    )

Key rules for FUNC_XREFS:

• xref_signatures is a list of alternative byte patterns — a caller qualifies if it contains any one of them.
• All non-empty lists within a single entry are ANDed: the caller must satisfy every specified criterion.
• Multiple entries in FUNC_XREFS cover multiple target functions in the same finder.
• Leave unused lists as [] rather than omitting the key — the full dict shape is required.

Examples:

• Direct struct offset: ida_preprocessor_scripts/find-EpObjectTable.py.
• Global variable: ida_preprocessor_scripts/find-PspCreateProcessNotifyRoutine.py.
• Function (direct): ida_preprocessor_scripts/find-ExReferenceCallBackBlock.py.
• Function (FUNC_XREFS, xref_signatures only): ida_preprocessor_scripts/find-AlpcpInitSystem.py, ida_preprocessor_scripts/find-PerfDiagInitialize.py.
• Function (FUNC_XREFS, xref_unicode_strings + xref_signatures): ida_preprocessor_scripts/find-AlpcpInitSystem.py.
• Single-symbol LLM_DECOMPILE struct offset: ida_preprocessor_scripts/find-AlpcHandleTableLock.py (ref: AlpcAddHandleTableEntry).
• Merged LLM_DECOMPILE struct offsets (two targets, same ref): ida_preprocessor_scripts/find-AlpcHandleTable-AND-AlpcPortContext.py (ref: AlpcpCreateClientPort).
• Merged LLM struct offsets: ida_preprocessor_scripts/find-AlpcAttributes-AND-AlpcAttributesFlags-AND-AlpcCommunicationInfo-AND-AlpcOwnerProcess-AND-AlpcConnectionPort-AND-AlpcServerCommunicationPort-AND-AlpcClientCommunicationPort.py.

3. Update config.yaml

Check first: search config.yaml for the skill name before editing — the entry may already exist and only need verification, not addition.

For the target module:

• Add one skills entry whose name exactly matches the script basename.
• Add every produced YAML under expected_output.
• Only add a symbols entry when the user explicitly requests it. Do not add symbols entries by default — many finders produce reference anchors (e.g. function RVAs) consumed only as expected_input by downstream finders, not as kphtools output symbols.
• Avoid duplicate symbol entries.
• If the finder depends on prior outputs, add expected_input so dependency artifacts are produced first.
• When renaming or merging a finder, remove stale skill entries for old scripts.

4. Generate and Annotate Reference YAML

Check first: run Glob ida_preprocessor_scripts/references/<module>/<FuncName>*.yaml before generating — the file may already exist.

Generate reference YAML with generate_reference_yaml.py:

uv run python generate_reference_yaml.py -func_name=ExReferenceCallBackBlock

Omitting -auto_start_mcp and -binary uses the already-running IDA MCP session, which is the normal case. -auto_start_mcp requires -binary and is only needed when no IDA instance is running.

Useful flags: -func_name, -module, -arch, -auto_start_mcp, -binary, -debug.

Output path:

ida_preprocessor_scripts/references/<module>/<FuncName>.<arch>.yaml

Looking up struct member offsets via IDA MCP: use only the idc module — ida_struct is not available. Example:

import idc
tid = idc.get_struc_id("_EPROCESS")
off = idc.get_member_offset(tid, "SectionObject")
print(f"offset: {hex(off)} = {off}")

Rules:

• Generate reference YAML sequentially. Do not parallelize IDA/MCP reference generation.
• For LLM_DECOMPILE, annotation is mandatory. Add comments in procedure and/or disasm_code that make the target unambiguous.
• Check existing annotations before adding: the procedure section may already contain the needed comments (e.g., from a previous task on the same file). Only add what is missing.
• Annotate all visible struct member accesses in the function (not just the primary target). Surrounding context — e.g., Handles at 0x0 and TotalHandles at 0x10 alongside the target Lock at 0x8 — helps the LLM correctly identify and bound the target offset.
• Annotation format in disasm_code: append ; 0xXX = YY = Struct->Member to the relevant instruction line.
• Annotation format in procedure: append // 0xXX = YY = Struct->Member inline at the end of the relevant statement.
• Decimal value should omit LL suffix when the offset is small and unambiguous (e.g., 0x8 = 8 = ... not 0x8 = 8LL = ...).
• Preserve generated structure; only add human/LLM guidance comments needed for reliable extraction.

5. Validate

Run dump_symbols.py to verify the finder is loaded and recognized:

uv run dump_symbols.py -debug > /tmp/dump_symbols_out.txt 2>&1

Then check the output for the skill name:

grep -E "preprocess status for find-<SKILL_NAME>" /tmp/dump_symbols_out.txt

Expected results (either is a pass):

• find-<SkillName>: success — skill ran and produced output.
• find-<SkillName>: absent_ok — skill is recognized; outputs were cached from a prior run.

Any other status (e.g., failed, skill name absent from output) indicates a misconfiguration.

Also run:

• YAML parse check for config.yaml and changed reference YAML files.
• git diff --check.

6. Commit Changes

Check the current branch:

git branch --show-current

• If on main: commit to a dev branch (e.g., dev or a new dev-<feature> branch).
• If on any other branch: commit directly to the current branch.

Stage only relevant files (finder scripts, reference YAMLs, config.yaml). Do not stage .claude/ tooling directories.

Common Mistakes

• Script filename and config.yaml skill name do not match.
• NtAPI finder uses preprocess_common_skill instead of _extract_ntapi.preprocess_ntapi_symbols.
• NtAPI finder result YAML gets a symbols entry — it should not; it is a reference anchor for downstream expected_input, not a kphtools output symbol.
• Function finder (LLM_DECOMPILE or direct) result YAML gets a symbols entry without user request — do not add symbols entries unless the user explicitly asks.
• expected_output omits one artifact from a merged finder.
• Symbol exists in script output but not in config.yaml symbols.
• Old finder scripts or old skill entries remain after an authorized merge.
• FUNC_XREFS finder passes func_metadata=FUNC_METADATA instead of func_xrefs=FUNC_XREFS — these are different kwargs and the wrong one silently finds nothing.
• FUNC_XREFS dict omits required keys (e.g. exclude_signatures) — always include the full dict shape with empty lists for unused fields.
• xref_signatures contains byte patterns that are too short and match unrelated callers — use patterns of at least 4 bytes; prefer 6+ for uniqueness.
• LLM_DECOMPILE exists but is not passed as llm_decompile_specs.
• Reference YAML lacks annotations, so the LLM returns wrong or incomplete offsets.
• Raw LLM output contains non-strict YAML such as repeated top-level keys or a truncated member name; fix the parser/prompt or reference annotation rather than accepting an empty parsed result.
• Reference generation is run in parallel and IDA/MCP sessions interfere with each other.

Completion Checklist

• Finder script exists and follows local patterns.
• config.yaml skill entry, expected outputs, symbols, and dependencies are consistent.
• Reference YAML exists for LLM_DECOMPILE and includes target annotations.
• Stale merged-away scripts/skill entries are removed when cleanup was authorized.
• dump_symbols.py -debug shows preprocess status for find-<SkillName>/<SymbolName>: success for the new skill.
• Changes committed to the current branch (or a dev branch if on main).