Standardmäßig ist der Prompt ausgewählt, der zuerst die Quelle prüft. Sie können zu einem direkten Befehl wechseln oder eine lokale Kopie herunterladen.
Quelldateien prüfen
Lesen Sie SKILL.md und alle von SkillsMP angezeigten Begleitdateien, bevor Sie sich für eine Installation entscheiden.
Mit Codex oder Claude installieren Kopieren Sie diesen Prompt, fügen Sie ihn in Codex, Claude oder einen anderen Assistant ein und lassen Sie die Skill-Seite prüfen und installieren.
Ein direkter Befehl überspringt den Prüf-Prompt. Prüfen Sie die Quelle, bevor Sie ihn ausführen.
Continuous benchmarking guidance for detecting performance regressions in CI pipelines. Covers baseline file management
with BenchmarkDotNet JSON exporters, GitHub Actions workflows for artifact-based baseline comparison, regression
detection patterns with configurable thresholds, and alerting strategies for performance degradation.
Version assumptions: BenchmarkDotNet v0.14+ for JSON export, GitHub Actions runner environment. Examples use
actions/upload-artifact@v4 and actions/download-artifact@v4.
Scope
Baseline file management with BenchmarkDotNet JSON exporters
GitHub Actions workflows for artifact-based baseline comparison
Regression detection with configurable thresholds
Alerting strategies for performance degradation
Out of scope
BenchmarkDotNet setup and benchmark class design -- see [skill:dotnet-benchmarkdotnet]
Performance architecture patterns -- see [skill:dotnet-performance-patterns]
Profiling tools (dotnet-counters, dotnet-trace, dotnet-dump) -- see [skill:dotnet-profiling]
OpenTelemetry metrics and distributed tracing -- see [skill:dotnet-observability]
Composable CI/CD workflow design -- see [skill:dotnet-gha-patterns]
Cross-references: [skill:dotnet-benchmarkdotnet] for benchmark class setup and JSON exporter configuration,
[skill:dotnet-observability] for correlating benchmark regressions with runtime metrics changes,
[skill:dotnet-gha-patterns] for composable workflow patterns (reusable workflows, composite actions, matrix builds).
Baseline File Management
BenchmarkDotNet JSON Export
BenchmarkDotNet's JSON exporter produces machine-readable results for automated comparison. Configure the exporter in
benchmark classes:
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Exporters.Json;
[JsonExporterAttribute.Full]
[MemoryDiagnoser]
publicclassCriticalPathBenchmarks
{
[Benchmark(Baseline = true)]
publicvoidProcessOrder() { /* ... */ }
[Benchmark]
() { }
}
```text
Or configure via custom config all benchmark classes:
```csharp
BenchmarkDotNet.Configs;
BenchmarkDotNet.Exporters.Json;
BenchmarkDotNet.Jobs;
BenchmarkDotNet.Running;
config = ManualConfig.Create(DefaultConfig.Instance)
.AddJob(Job.ShortRun)
.AddExporter(JsonExporter.Full)
.WithArtifactsPath();
BenchmarkSwitcher.FromAssembly((Program).Assembly).Run(, config);
```json
{
: ,
: [
{
: ,
: {
: ,
: ,
: ,
:
},
: {
: ,
: ,
: ,
:
}
}
]
}
```text
Key fields regression comparison:
| Field | Purpose |
| ----------------------------------- | ----------------------------------------------- |
| `Statistics.Mean` | ; must update deliberately | Small benchmark suites, stable hardware |
| GitHub Actions artifacts | No repo bloat; automatic retention | -day retention; cross-workflow access requires tokens | Large benchmark suites, shared runners |
| ; cross-repo sharing | Extra infrastructure; credential management | Multi-repo benchmark comparison |
This skill focuses the **GitHub Actions artifact** strategy the . For composable workflow patterns
reusable actions, see [skill:dotnet-gha-patterns].
---
```yaml
name: Benchmarks
:
pull_request:
paths:
-
-
workflow_dispatch:
permissions:
contents: read
actions: read
jobs:
benchmark:
runs-: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup .NET
uses: actions/setup-dotnet@v4
:
dotnet-version:
- name: Run benchmarks
run: dotnet run -c Release --project benchmarks/MyBenchmarks.csproj -- --exporters json
- name: Upload benchmark results
uses: actions/upload-artifact@v4
:
name: benchmark-results-${{ github.sha }}
path: benchmarks/BenchmarkDotNet.Artifacts/results/
retention-days:
```text
This workflow downloads the baseline a previous run compares against current results:
```yaml
name: Benchmark Regression Check
:
pull_request:
paths:
-
-
permissions:
contents: read
actions: read
env:
BENCHMARK_PROJECT: benchmarks/MyBenchmarks.csproj
RESULTS_DIR: benchmarks/BenchmarkDotNet.Artifacts/results
jobs:
benchmark:
runs-: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup .NET
uses: actions/setup-dotnet@v4
:
dotnet-version:
- name: Download baseline results
uses: actions/download-artifact@v4
:
name: benchmark-baseline
path: ./baseline-results
--error:
id: download-baseline
- name: Run benchmarks
run: dotnet run -c Release --project ${{ env.BENCHMARK_PROJECT }} -- --exporters json
- name: Compare baseline
: steps.download-baseline.outcome ==
shell: bash
run: |
-euo pipefail
python3 scripts/compare-benchmarks.py \
--baseline ./baseline-results \
--current \
--threshold \
--output benchmark-comparison.md
- name: Upload current results baseline
: github. ==
uses: actions/upload-artifact@v4
:
name: benchmark-baseline
path: ${{ env.RESULTS_DIR }}/
retention-days:
overwrite:
- name: Upload comparison report
: steps.download-baseline.outcome ==
uses: actions/upload-artifact@v4
:
name: benchmark-comparison-${{ github.sha }}
path: benchmark-comparison.md
retention-days:
```markdown
**Key design decisions:**
- `--error: ` baseline download handles first-run (no baseline exists yet)
- Baseline only updated `main` branch merges to prevent PR branches polluting the baseline
- `overwrite: ` replaces the previous baseline artifact
For converting these inline workflows reusable `workflow_call` patterns, see [skill:dotnet-gha-patterns].
---
Compare current benchmark results against baseline percentage thresholds. A regression flagged the current
mean exceeds the baseline mean more than the configured threshold:
```python
= {}
= json.load(f)
bm data.(, []):
name = bm[]
benchmarks[name] = {
: bm[][],
: bm[][],
: bm[][],
: bm.(, {}).(, ),
}
= load_benchmarks(baseline_dir)
current = load_benchmarks(current_dir)
regressions = []
name, curr current.items():
name baseline:
= baseline[name]
[] == :
time_change_pct = ((curr[] - []) / []) *
alloc_change = curr[] - []
time_change_pct > threshold_pct:
regressions.append({
: name,
: [],
: curr[],
: time_change_pct,
: alloc_change,
})
regressions
__name__ == :
import argparse
parser = argparse.ArgumentParser(description=)
parser.add_argument(, =True, help=)
parser.add_argument(, =True, help=)
parser.add_argument(, type=, =,
help=)
parser.add_argument(, =, help=)
= parser.parse_args()
regressions = compare(.baseline, .current, .threshold)
; small regressions are signal |
| GitHub Actions shared runners | % | Shared runners introduce % variance noisy neighbors |
| Self-hosted runners | % | More stable than shared, but still monitor variance |
**Calibrate thresholds empirically:** Run the same benchmark suite times your CI environment without code
changes. The maximum observed variance sets your noise floor. == && github.event_name ==
uses: actions/github-script@v7
:
script: |
fs = require();
body = fs.readFileSync(, );
github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: body
});
```text
Exit non-zero status the comparison script to fail the GitHub Actions job. This prevents merging PRs that
introduce performance regressions:
```yaml
- name: Check regressions
: steps.download-baseline.outcome ==
shell: bash
run: |
-euo pipefail
python3 scripts/compare-benchmarks.py \
--baseline ./baseline-results \
--current \
--threshold
```text
For status checks branch protection integration benchmark gates, see [skill:dotnet-gha-patterns].
For -term trend analysis beyond single-PR comparison, upload results to a persistent store track metrics over
time:
| Approach | Tool | Complexity |
| ---------------------------------- | --------------------------------------------- | ------------------------------------- |
| GitHub Actions artifacts | Built-, -day retention | Low -- artifact download/upload only |
| GitHub Pages benchmark-action | `benchmark-action/github-action-benchmark@v1` | Medium -- auto-generates trend charts |
| External time-series DB | InfluxDB, Prometheus + Grafana | High -- full observability stack |
The simplest approach most projects the artifact-based baseline comparison shown skill. Graduate to trend
tracking you need historical regression analysis across many releases.
---
Full benchmark runs take + minutes. Use `Job.ShortRun` CI to reduce iteration counts retaining regression
detection capability:
```csharp
BenchmarkDotNet.Configs;
BenchmarkDotNet.Jobs;
:
{
{
AddJob(Job.ShortRun
.WithWarmupCount()
.WithIterationCount()
.WithInvocationCount());
AddExporter(BenchmarkDotNet.Exporters.Json.JsonExporter.Full);
}
}
```json
Apply conditionally based environment:
```csharp
config = Environment.GetEnvironmentVariable()
? CiConfig()
: DefaultConfig.Instance;
BenchmarkRunner.Run<CriticalPathBenchmarks>(config);
```text
Run only critical-path benchmarks CI to reduce pipeline duration:
```bash
dotnet run -c Release --project benchmarks/MyBenchmarks.csproj -- \
--filter *Critical* --exporters json
```bash
```csharp
[]
[]
[]
{
[]
{ }
}
[]
[]
{
[]
{ }
}
```text
Run `Critical` benchmarks every PR; run `Extended` benchmarks a nightly schedule.
```yaml
name: {{ github.run_number }}
path: benchmarks/BenchmarkDotNet.Artifacts/results/
retention-days:
```text
For scheduled workflow patterns matrix builds across TFMs, see [skill:dotnet-gha-patterns].
---
**Primary approach:** Use Serena symbol operations efficient code navigation:
**Find definitions**: `serena_find_symbol` instead of text search
**Understand structure**: `serena_get_symbols_overview` organization
**Track references**: `serena_find_referencing_symbols` impact analysis
**Precise edits**: `serena_replace_symbol_body` clean modifications
**When to use Serena vs traditional tools:**
- ✅ **Use Serena**: Navigation, refactoring, dependency analysis, precise edits
- ✅ **Use Read/Grep**: Reading full files, pattern matching, simple text operations
- ✅ **Fallback**: If Serena unavailable, traditional tools work fine
**Example workflow:**
```text
Read: src/Services/OrderService.cs
Grep:
serena_find_symbol:
serena_get_symbols_overview:
```
**Use `Job.ShortRun` CI, `Job.Default`** -- benchmark jobs run many iterations statistical
precision, taking + minutes per benchmark . CI pipelines need faster feedback `ShortRun` ( warmup,
iteration).
**Set threshold above measured noise floor** -- shared CI runners introduce % timing variance noisy
neighbors. A % threshold shared runners produces positives. Calibrate running the same code multiple
times measuring variance.
**Use allocation changes hard gates** -- allocation counts are deterministic unaffected runner noise. A
zero-to-nonzero allocation change always a real regression, unlike timing variations.
**Only update baselines main branch** -- PR branches can update the baseline, a regression one PR becomes
the baseline, masking it subsequent comparisons.
**Always ` -euo pipefail` bash steps** -- without `pipefail`, a regression detection script that exits
non-
public
void
ProcessOrderOptimized
/* ... */
for
using
using
using
using
var
// fewer iterations for CI speed
"./benchmark-results"
typeof
args
### JSON Export Structure
The exported JSON file (`*-report-full.json`) contains structured benchmark results:
```json
for json_file inPath(results_dir).glob("*-report-full.json"):
withopen(json_file) as f:
data
for
in
get
"Benchmarks"
"FullName"
"mean"
"Statistics"
"Mean"
"median"
"Statistics"
"Median"
"stddev"
"Statistics"
"StandardDeviation"
"allocated"
get
"Memory"
get
"BytesAllocatedPerOperation"
0
return
benchmarks
def compare(baseline_dir: str, current_dir: str, threshold_pct: float) -> list:
"""Compare current results against baseline. Returns list of regressions."""
baseline
Set the threshold above this noise floor (typically 2x the
observed variance).
### Allocation Regression Detection
Memory allocation regressions are more reliable signals than timing regressions because allocations are deterministic
(not affected by noisy neighbors):
```python
# Add to the compare function:
if alloc_change > 0:
regressions.append({
"name": name,
"type": "allocation",
"baseline_alloc": base["allocated"],
"current_alloc": curr["allocated"],
"alloc_change": alloc_change,
})
```text
Use allocation changes as a **hard gate** (zero tolerance fornew allocations in zero-alloc paths) and timing changes as
a **soft gate** (warning with threshold).
---
## Alerting Strategies
### PR Comment with Regression Summary
Post benchmark comparison results as a PR comment for reviewer visibility:
```yaml
- name: Comment PR with results
if: steps.download-baseline.outcome
'success'
'pull_request'
with
const
'fs'
const
'benchmark-comparison.md'
'utf8'
await
### Fail the Build on Regression
with
from
for
if
'success'
set
"${{ env.RESULTS_DIR }}"
10
# Script exits non-zero if regressions found -- fails the job
required
and
with
### Trend Tracking
long
and
in
90
with
for
is
in
this
when
## CI-Specific BenchmarkDotNet Configuration
### ShortRun for CI Speed
10
-30
in
while
using
using
public
class
CiConfig
ManualConfig
publicCiConfig()
3
5
1
on
var
"CI"
is
not
null
new
### Filtering Benchmarks for CI
in
# Run only benchmarks in the "Critical" category
BenchmarkCategory("Critical")
MemoryDiagnoser
JsonExporterAttribute.Full
public
class
CriticalPathBenchmarks
Benchmark
publicvoidProcessOrder()
/* ... */
BenchmarkCategory("Extended")
MemoryDiagnoser
public
class
ExtendedBenchmarks
Benchmark
publicvoidRareCodePath()
/* ... */
on
on
### Nightly Benchmark Schedule
Nightly Benchmarks (Full Suite)
on:
schedule:
- cron: '0 3 * * *' # 3 AM UTC daily
workflow_dispatch:
jobs:
benchmark-full:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: '8.0.x'
- name: Run full benchmark suite
run: dotnet run -c Release --project benchmarks/MyBenchmarks.csproj -- --exporters json
# No --filter: runs all benchmarks including Extended category
- name: Upload full results
uses: actions/upload-artifact@v4
with:
name: benchmark-full-$
90
and
## Code Navigation (Serena MCP)
for
1.
2.
for
file
3.
for
4.
for
# Instead of:
"public void ProcessOrder"
# Use:
"OrderService/ProcessOrder"
"src/Services/OrderService.cs"
## Agent Gotchas
1.
in
not
default
for
10
-30
class
with
3
5
2.
5
-10
from
5
on
false
by
and
3.
as
and
by
is
4.
from
if
in
new
from
5.
set
set
in
zero in a pipeline (e.g., `script | tee`) does not fail the GitHub Actions step.
6. **Handle missing baselines gracefully** -- the first CI run has no baseline to compare against. Use
`continue-on-error: true` on the baseline download step and skip comparison when no baseline exists.
7. **Export JSON, not just Markdown** -- Markdown reports are human-readable but not machine-parseable for automated
regression detection. Always include `[JsonExporterAttribute.Full]` or `JsonExporter.Full` in the config.
