fuzzer-run-triage

name	fuzzer-run-triage
description	Triage SafeRE Jazzer fuzzer runs performed manually by the user: locate Surefire/Jazzer logs, distinguish latest run artifacts from persistent corpus and older saved inputs, extract AssertionError findings, summarize self-contained repro details, and prepare GitHub issue material without rerunning fuzzing.

Fuzzer Run Triage

Goal

When the user has run a SafeRE Jazzer fuzzer by hand, find what that run discovered and present the actionable bug findings. For normal SafeRE crosscheck divergences, focus first on the AssertionError text in the Surefire XML and raw console logs. The assertion should contain the self-contained semantic repro: regex, flags, operation, input/state when applicable, and SafeRE vs JDK behavior. Do not start from crash-* files unless the assertion text is incomplete, the console log was truncated, or exact Jazzer replay is required.

Do not assume target/fuzz-reproducers exists: Jazzer JUnit usually writes libFuzzer crash-* inputs to the fuzzer's input resource directory. Maven/Surefire writes XML summaries under target/surefire-reports, and SafeRE's helper script records raw stdout/stderr under target/fuzz-logs.

First Checks

1. Identify the fuzzer class from the user's command or question, for example CharacterClassExpressionFuzzer.

2. Inspect the latest matching Surefire files:

   • safere-fuzz/target/surefire-reports/TEST-org.safere.fuzz.<Fuzzer>.xml
   • safere-fuzz/target/surefire-reports/org.safere.fuzz.<Fuzzer>.txt

3. Inspect helper-script console logs if present. These are often the best source for complete == Java Exception: java.lang.AssertionError blocks and may also include artifact_prefix, Test unit written, Base64, timeout, and saved-input path lines that Surefire XML omits:

   find safere-fuzz/target/fuzz-logs -path '*/<Fuzzer>.log' \
     -printf '%TY-%Tm-%Td %TH:%TM:%TS %p\n' | sort
   

4. Check file mtimes before trusting artifacts:

   stat safere-fuzz/target/surefire-reports/TEST-org.safere.fuzz.<Fuzzer>.xml
   stat safere-fuzz/target/surefire-reports/org.safere.fuzz.<Fuzzer>.txt
   stat safere-fuzz/target/fuzz-logs/<run-id>/<Fuzzer>.log
   

5. Extract findings from the XML and raw console log. Treat AssertionError divergence blocks as the primary repro source. Treat Base64 and Test unit written lines as exact replay provenance, not as a substitute for a self-contained semantic repro:

   rg -n '== Java Exception|AssertionError|CrosscheckException|PatternSyntaxException|divergence|DEDUP_TOKEN|libFuzzer crashing input|Test unit written|artifact_prefix|reproducer_path|Java reproducer written' \
     safere-fuzz/target/surefire-reports/TEST-org.safere.fuzz.<Fuzzer>.xml
   rg -n '== Java Exception|AssertionError|CrosscheckException|PatternSyntaxException|divergence|DEDUP_TOKEN|libFuzzer: timeout|Test unit written|artifact_prefix|Base64:|reproducer_path|Java reproducer written' \
     safere-fuzz/target/fuzz-logs/<run-id>/<Fuzzer>.log
   

Where Artifacts Live

• Surefire logs: safere-fuzz/target/surefire-reports/

  • Mixed by fuzzer class in one directory.
  • The same fuzzer's .xml and .txt files are overwritten by newer runs.
  • Other fuzzer reports may be older and unrelated.
  • XML may omit raw libFuzzer lines such as artifact_prefix, Test unit written, and Base64.

• Raw helper-script console logs: safere-fuzz/target/fuzz-logs/<run-id>/<Fuzzer>.log

  • Created by safere-fuzz/scripts/run-fuzz-test.sh.
  • Preserve the combined stdout/stderr stream while still showing output live in the terminal.
  • Prefer these logs for complete AssertionError repro text.
  • Use these logs for mapping == Java Exception blocks to crash-*, slow-unit-*, and timeout-* files only when exact Jazzer input replay or provenance is needed.
  • If a user provides a manually saved console log, inspect it the same way as these files.

• Generated corpus: safere-fuzz/.cifuzz-corpus/org.safere.fuzz.<Fuzzer>/<method>/

  • Persistent coverage corpus, intentionally accumulated across runs.
  • Not a clean list of current-run bugs.

• Jazzer JUnit crash inputs: safere-fuzz/src/test/resources/org/safere/fuzz/<Fuzzer>Inputs/<method>/crash-*

  • Saved libFuzzer inputs for JUnit fuzz findings. Despite the crash-* name, SafeRE crosscheck divergences are usually AssertionErrors, not JVM crashes.
  • These are exact replay artifacts, not the preferred source for writing issues when assertion text already contains a self-contained repro.
  • These are persistent checked-in seed locations, so use mtimes and the Surefire log to tell new crashes from old ones.

• Compiled test resource copies: safere-fuzz/target/test-classes/org/safere/fuzz/<Fuzzer>Inputs/<method>/

  • Build output copies. Do not treat these as source-of-truth new findings.

• Standalone Java reproducers: path from -Djazzer.reproducer_path=...

  • In Jazzer JUnit mode these often do not exist, even when bugs were found.
  • If present, inspect them, but absence is not evidence that no bugs were found.

Finding New Saved Inputs

Find the fuzzer method and input resource directory. For most SafeRE fuzzers the method name is visible in the Surefire testcase name and in the resource path:

rg -n '<testcase name=' safere-fuzz/target/surefire-reports/TEST-org.safere.fuzz.<Fuzzer>.xml
find safere-fuzz/src/test/resources/org/safere/fuzz/<Fuzzer>Inputs -type f -name 'crash-*' \
  -printf '%TY-%Tm-%Td %TH:%TM:%TS %p\n' | sort

If the user gives the approximate run time, compare crash mtimes to that time. If they do not, compare crash mtimes to the Surefire XML mtime and the sun.java.command or testcase elapsed time inside the XML.

Only do this saved-input search when the logs do not contain enough assertion detail to reproduce, when the user asks for exact replay, or when a timeout/crash needs the raw unit. If the assertion already has a complete regex/flags/input/operation repro, the saved input is optional provenance.

Replaying Candidates on the Current Branch

When the user wants bugs verified or filed, replay candidates against the current working tree and current branch. Do not treat old console output as proof that a bug still exists.

1. Install the current SafeRE module and compile the current fuzz test classes first:

   mvn -pl safere -DskipTests install -q
   mvn -pl safere-fuzz -DskipTests test-compile -q
   

   Do not skip the safere install step before standalone safere-fuzz replays. A command such as mvn -pl safere-fuzz ... surefire:test does not rebuild reactor dependencies, so it can resolve an older org.safere:safere artifact from the local Maven repository and report bugs that are already fixed on the current branch.

2. Prefer direct semantic repros from the assertion text when they are complete. Replay saved inputs one at a time only when you need to confirm the exact Jazzer unit or recover omitted fuzzer choices. Isolate the copied test resource directory under target/test-classes to avoid stale build-output inputs and get an exact saved-input-to-finding mapping:

   TARGET=safere-fuzz/target/test-classes/org/safere/fuzz/<Fuzzer>Inputs/<method>
   SOURCE=safere-fuzz/src/test/resources/org/safere/fuzz/<Fuzzer>Inputs/<method>
   rm -f "$TARGET"/*
   cp "$SOURCE/<crash-or-timeout-file>" "$TARGET/"
   mvn -pl safere-fuzz -Dtest=<Fuzzer> -Dsurefire.failIfNoSpecifiedTests=false surefire:test
   

3. Restore the copied test resources from source after isolated replay:

   rm -f "$TARGET"/*
   cp "$SOURCE"/* "$TARGET"/
   

4. Record whether the candidate still reproduces, the current assertion output, and any exact saved-input source or Base64 if used. If it no longer reproduces on the current branch, say so and do not file a bug for it.

5. Deduplicate reproducible findings by semantic bug class before filing. Multiple crash-* files or Base64 units may represent one parser rule, matcher invariant, or Unicode-boundary bug.

6. Preserve durable repro material before relying on it in an issue. Local crash-*, slow-unit-*, timeout-*, console-log, and /tmp paths are useful provenance, but they are not durable bug report content.

Interpreting Results

Use assertion logs first:

• The XML usually contains the readable exception text: regex, flags, input, SafeRE result, JDK result, stack trace, and dedup tokens.
• Raw console logs usually contain the most complete AssertionError blocks and can also contain saved-input paths and Base64 units.
• crash-* files are the inputs Jazzer can replay through the fuzz target, but they may not be human-readable and should not be necessary for ordinary divergence issues if the assertion is well formed.
• A .txt report with Failures: 0, Errors: 0 does not prove no bugs were found when jazzer.keep_going is in use. Always inspect the XML and raw console log for == Java Exception, DEDUP_TOKEN, Test unit written, and libFuzzer: timeout.

If an AssertionError does not include enough information to write a self-contained regression test or GitHub issue, tell the user explicitly. Offer to file a separate bug against the fuzz harness/assertion reporting, because SafeRE fuzz assertions should make ordinary divergences reproducible without consulting local crash-* files.

Classify findings before filing:

• compile divergence: SafeRE accepts/rejects a pattern differently from JDK.
• match divergence: SafeRE and JDK compile but produce different match results.
• API sequence divergence: matcher/split/replace/stateful operation differs.
• crash/hang/stack overflow: SafeRE throws unexpectedly, times out, or exceeds stack limits.

Deduplicate by semantic class, not only exact string. If two findings are variants of the same parser rule or matcher invariant, discuss whether to file one broader issue or multiple focused issues.

Reporting to the User

Summarize:

• Which run files were inspected, with mtimes if recency matters.
• Whether the Surefire report is a latest per-fuzzer report or mixed with older reports.
• Whether a raw target/fuzz-logs/<run-id>/<Fuzzer>.log file, or a manually saved console log, was inspected.
• Each finding's regex/flags/input/operation and SafeRE vs JDK behavior.
• Whether the assertion text was sufficient for a self-contained repro.
• The corresponding crash-* files or Base64 units only if needed for exact replay, incomplete assertion text, timeout/crash triage, or provenance.
• Any slow-unit-*, timeout-*, or Base64 units for hangs/timeouts.
• Whether target/fuzz-reproducers exists, and why absence may be normal.

When asked to file issues:

1. Search existing issues for exact repro substrings and the semantic class.
2. Write the issue body to a temp file and use gh issue create --body-file.
3. Make the issue self-contained. Do not rely on local or ephemeral paths such as target/surefire-reports, target/fuzz-logs, /tmp/..., or local crash-* files being available later.
4. Include enough durable repro material to recreate the bug:
   • minimized regex, flags, input, operation, and observed SafeRE/JDK behavior;
   • a small Java snippet when practical;
   • for incomplete assertions, timeouts, crashes, or binary-only repros, include Base64 and/or hex bytes in the issue body, or attach/upload the input artifact if GitHub supports it for the workflow being used;
   • if the project owner wants persisted reproducers in-repo, add them deliberately as tracked test resources or regression tests rather than depending on untracked local files.
5. Local paths may be included only as provenance, clearly secondary to the self-contained repro data.
6. Do not close existing issues unless all items are resolved.