Identify computational bottlenecks, analyze parallel scaling, estimate memory requirements, and generate optimization recommendations for materials simulations — parse timing logs to find dominant phases (solver, assembly, I/O), evaluate strong and weak scaling efficiency, profile memory from mesh and field parameters, and detect bottlenecks with actionable fix suggestions. Use when a simulation is running slower than expected, investigating MPI scaling efficiency, planning HPC resource allocation, deciding whether to tune the preconditioner or reduce I/O frequency, or estimating if a problem fits in available RAM, even if the user only says "my simulation is too slow" or "how many nodes do I need."
Identify computational bottlenecks, analyze parallel scaling, estimate memory requirements, and generate optimization recommendations for materials simulations — parse timing logs to find dominant phases (solver, assembly, I/O), evaluate strong and weak scaling efficiency, profile memory from mesh and field parameters, and detect bottlenecks with actionable fix suggestions. Use when a simulation is running slower than expected, investigating MPI scaling efficiency, planning HPC resource allocation, deciding whether to tune the preconditioner or reduce I/O frequency, or estimating if a problem fits in available RAM, even if the user only says "my simulation is too slow" or "how many nodes do I need."
allowed-tools
Read, Write, Grep, Glob
metadata
{"author":"HeshamFS","version":"1.2.2","security_tier":"medium","security_reviewed":true,"tested_with":["claude-code"],"last_evaluated":"2026-06-24","eval_cases":5,"last_reviewed":"2026-06-23","standards":["Amdahl (1967), serial-fraction speedup law S(N)=1/(f+(1-f)/N)","Gustafson (1988), scaled speedup law S(N)=N-f(N-1)","Gropp, Lusk & Skjellum (1999), Using MPI (parallel scaling and communication)","Saad (2003), Iterative Methods for Sparse Linear Systems (solver/preconditioner choice)","Karypis & Kumar (1998), METIS multilevel graph partitioning (load balancing)"]}
Performance Profiling
Goal
Provide tools to analyze simulation performance, identify bottlenecks, and recommend optimization strategies for computational materials science simulations.
Requirements
Python 3.10+
No external dependencies (uses Python standard library only)
Works on Linux, macOS, and Windows
Inputs to Gather
Before running profiling scripts, collect from the user:
Input
Description
Example
Simulation log
Log file with timing information
simulation.log
Scaling data
JSON with multi-run performance data
scaling_data.json
Simulation parameters
JSON with mesh, fields, solver config
params.json
Available memory
System memory in GB (optional)
16.0
Decision Guidance
When to Use Each Script
Need to identify slow phases?
├── YES → Use timing_analyzer.py
│ └── Parse simulation logs for timing data
│
Need to understand parallel performance?
├── YES → Use scaling_analyzer.py
│ └── Analyze strong or weak scaling efficiency
│
Need to estimate memory requirements?
├── YES → Use memory_profiler.py
│ └── Estimate memory from problem parameters
│
Need optimization recommendations?
└── YES → Use bottleneck_detector.py
└── Combine analyses and get actionable advice
Choosing Analysis Thresholds
Metric
Good
Acceptable
Poor
Phase dominance
<30%
30-50%
>50%
Parallel efficiency
>0.80
0.70-0.80
<0.70
Memory usage
<60%
60-80%
>80%
Script Outputs (JSON Fields)
All scripts wrap their payload in a top-level object with two keys: inputs and results. The fields below live under results.
The detector applies per-type dominance thresholds: solver/assembly/general phases are flagged above 50% of runtime; I/O phases above 30%. Any flagged phase above 70% is reported as high severity.
Scenario
Meaning
Action
Solver >50% (high >70%)
Solver-dominated
Tune preconditioner, check tolerance
Assembly >50%
Assembly-dominated
Cache matrices, vectorize, parallelize
I/O >30%
I/O-dominated
Reduce frequency, use parallel I/O
Balanced (below thresholds)
Well-balanced
Look for algorithmic improvements
Scaling Analysis
Efficiency
Meaning
Action
>0.80
Excellent scaling
Continue scaling up
0.70-0.80
Good scaling
Monitor at larger scales
0.50-0.70
Poor scaling
Investigate communication/load balance
<0.50
Very poor scaling
Reduce processor count or redesign
Memory Profile
Usage
Meaning
Action
<60% available
Safe
No action needed
60-80% available
Moderate
Monitor, consider optimization
>80% available
High
Reduce resolution or increase processors
>100% available
Exceeds capacity
Must reduce problem size
The estimate follows the three-term formula Total = Field + Solver Workspace + Matrix Storage (see references/profiling_guide.md). Matrix storage and solver workspace depend on solver.type:
iterative (default): sparse matrix (default 7-point stencil, override via solver.stencil_nnz) plus workspace vectors.
direct: sparse matrix scaled by a conservative fill-in factor (solver.fillin_factor, default 10) to reflect factorization fill-in — a direct solver estimates far more memory than an iterative one for the same mesh.
matrix-free: no assembled matrix; workspace vectors only.
The estimate is intentionally conservative so a "will it fit in RAM?" decision does not silently under-estimate.
Error Handling
Error
Cause
Resolution
Log file not found
Invalid path
Verify log file path
No timing data found
Pattern mismatch
Provide custom pattern with --pattern
At least 2 runs required
Insufficient data
Provide more scaling runs
Missing required parameters
Incomplete params
Add mesh and fields to params file
Optimization Strategies by Bottleneck Type
Solver Bottlenecks
Use algebraic multigrid (AMG) preconditioner
Tighten solver tolerance if over-solving
Consider direct solver for small problems
Profile matrix assembly vs solve time
Assembly Bottlenecks
Cache element matrices if geometry is static
Use vectorized assembly routines
Consider matrix-free methods
Parallelize assembly with coloring
I/O Bottlenecks
Reduce output frequency
Use parallel I/O (HDF5, MPI-IO)
Write to fast scratch storage
Compress output data
Scaling Bottlenecks
Investigate communication overhead
Check for load imbalance
Reduce synchronization points
Use asynchronous communication
Consider hybrid MPI+OpenMP
Memory Bottlenecks
Reduce mesh resolution
Use iterative solver (lower memory than direct)
Enable out-of-core computation
Increase number of processors
Use single precision where appropriate
Verification checklist
Before trusting a profiling result or acting on a recommendation, record the concrete evidence below:
Confirmed timing_analyzer.py actually matched entries — results.phases is non-empty and results.total_time > 0; if a custom --pattern was used and results.message/suggested_patterns appeared, the pattern was fixed and re-run (an empty phases list silently looks like a fast simulation).
Cross-checked that the sum of phases[].percentage is ~100% and that named phases cover the wall-clock time — unaccounted-for time means missing log lines, not a balanced run.
For scaling claims, used >=2 runs spanning a real processor range and recorded results.average_efficiency and results.efficiency_threshold_processors from scaling_analyzer.py; verified the --type (strong vs weak) matches how the runs were generated (fixed total size vs fixed work-per-rank).
Recorded the memory breakdown from memory_profiler.py (field_memory_gb, solver_workspace_gb, matrix_storage_gb, total_memory_gb) and confirmed solver.type (iterative / direct / matrix-free) matches the real solver — a direct solve carries the ~10x fill-in factor and a wrong type makes the "fits in RAM?" answer unsafe.
Checked results.warnings and compared total_memory_gb (and per_process_gb) against the actual --available-gb; treated >80% as the documented "high" band, not a pass.
For each bottleneck_detector.py recommendation, confirmed the driving bottleneck (its category, value, and threshold) is consistent with the timing/scaling/memory inputs that were actually supplied — recommendations only reflect the JSON files passed via --timing/--scaling/--memory.
After implementing an optimization, re-ran the relevant analyzer and recorded the before/after value to confirm the bottleneck actually moved (re-profile step of the workflow).
Common pitfalls & rationalizations
Tempting shortcut
Why it's wrong / what to do
"timing_analyzer.py returned no bottlenecks, so the run is balanced."
An empty/low result is often a pattern mismatch — phases may be empty or partial. Verify total_time matches wall-clock and that phase percentages sum to ~100% before concluding "balanced".
"Two runs scaled fine, so it scales."
Two points only give an average efficiency; they cannot reveal where efficiency falls off. Add more processor counts and check efficiency_threshold_processors, and confirm you used the correct --type (strong vs weak).
"Iterative vs direct is just a flag; memory is about the same."
memory_profiler.py applies a conservative ~10x fill-in factor for direct and stores no matrix for matrix-free. Setting the wrong solver.type can under-estimate RAM by an order of magnitude — set it to the real solver.
"It fits in --available-gb total, so we're fine."
The relevant number for an MPI run is per_process_gb against per-node/per-rank RAM, and >80% of total already triggers a warning. Check the per-process figure and the warnings list, not just the total.
"I/O is under 50%, so I/O isn't the bottleneck."
I/O is flagged at the lower 30% threshold, not 50%. A 30-50% I/O phase is a real bottleneck the detector reports — reduce output frequency or use parallel I/O.
"The recommendation says tune the preconditioner, so the solver is the problem."
Recommendations are only as complete as the JSON you passed in. If --scaling/--memory were omitted, those bottlenecks are simply invisible — feed all available analyses before trusting the priority ranking.
Security
Input Validation
User-supplied --pattern regex values are validated for length (500 chars max) and rejected if they contain constructs prone to catastrophic backtracking (ReDoS)
Scaling data entries are validated for finite time values, integer processor counts, and bounded run count (10,000 max)
available_gb is validated as a positive finite number; mesh dimensions and field parameters are validated as positive integers
--type (scaling type) is validated against a fixed allowlist (strong, weak)
All loaded JSON files must have an object (dict) as root element
File Access
timing_analyzer.py reads a single log file specified by --log; log files are capped at 500 MB and rejected before parsing
scaling_analyzer.py, memory_profiler.py, and bottleneck_detector.py read JSON files capped at 100 MB
Phase names extracted from log files are truncated to 200 characters and stripped of control characters to prevent prompt-injection payloads from propagating into agent context
No scripts write to the filesystem; all output goes to stdout
Tool Restrictions
Read: Used to inspect script source, references, simulation logs, and result files
Write: Used to save profiling reports or optimization recommendations; writes are scoped to the user's working directory
Grep/Glob: Used to locate log files, result files, and search references
The skill's allowed-tools excludes Bash to prevent the agent from executing arbitrary commands when processing untrusted simulation logs or result files
Safety Measures
No eval(), exec(), or dynamic code generation
All subprocess calls use explicit argument lists (no shell=True)
Reduced tool surface (no Bash) limits the agent to read/write operations only
Phase names and diagnostic strings are sanitized before inclusion in output to prevent injection
Limitations
Log parsing: Depends on pattern matching; may miss unusual formats
Scaling analysis: Requires at least 2 runs for meaningful results
Memory estimation: Approximate; actual usage may vary
Recommendations: General guidance; may need domain-specific tuning
References
references/profiling_guide.md - Profiling concepts and interpretation
