build-performance-analysis

# Build Performance Analysis with vcperf.exe ## Execution order A single **workflow**, always run in order. ### Workflow — always run, in order **Treat each session as independent.** Re-capture traces, JSON analysis, and baseline measurements fresh each session instead of reusing data from a previous session — the one-time trace-permission grant persists. 1. [Prerequisites: Trace Permission](#prerequisites-trace-permission) — grant ETW rights. Run **before** any workspace exploration (project enumeration, file listing, code search). 2. [Core Workflow](#core-workflow) + [The /jsonAnalysis Flag](#the-jsonanalysis-flag) 3. [Baseline Measurement](#baseline-measurement) — 5 uninstrumented clean rebuilds of the **pre-change** state; average for a stable baseline. **Make no code or project edits during this step**. 4. [Remediation Strategies](#remediation-strategies) — the **first** step where you edit anything; apply changes informed by Steps 2–3. 5. [Measuring Final Performance Improvement](#measuring-final-performance-improvement) — 5 uninstrumented clean rebuilds of the post-change state; compare against the baseline. If a PCH or template change shows a neutral or regressed delta, also run [Iterative Build Measurement](#iterative-build-measurement). ## Which vcperf.exe executable should I use? The `sessionStart` hook installs `vcperf.exe` under `%LOCALAPPDATA%\vcperf\build-perf-cpp` (multiple versions may coexist). Resolve the **newest** version once and reuse it: ```powershell $vcperfPkg = Get-ChildItem "$env:LOCALAPPDATA\vcperf\build-perf-cpp" -Directory -Filter 'Microsoft.Cpp.vcperf.*' -ErrorAction SilentlyContinue | Sort-Object { ($_.Name -replace '^Microsoft\.Cpp\.vcperf\.', '' -split '\.' | ForEach-Object { ([long]$_).ToString().PadLeft(20, '0') }) -join '.' } -Descending | Select-Object -First 1 $vcperf = if ($vcperfPkg) { (Get-ChildItem $vcperfPkg.FullName -Recurse -Filter vcperf.exe | Where-Object DirectoryName -like '*\x64' | Select-Object -First 1).FullName } ``` **Authentication**: If `$vcperf` resolves empty, read `references/feed-auth.md` and follow it before proceeding. ### Invocation rule (read before running any vcperf command) Every `vcperf` command below runs the resolved `$vcperf` path directly in PowerShell (`& $vcperf ...`). Both rules are mandatory: 1. **Use the absolute `$vcperf` path, never a bare `vcperf`.** Bare `vcperf` resolves off `PATH` to the older Visual-Studio-bundled copy (may lack `/jsonAnalysis`), not the one under `%LOCALAPPDATA%\vcperf\build-perf-cpp`. 2. **Judge success by the output file, not `$LASTEXITCODE`.** PowerShell's `& $vcperf` can surface an internal COM `HRESULT` (e.g. `0x80040013`) in `$LASTEXITCODE` and render output as blank lines, so a run that actually succeeded can look failed. After a `/stop` or `/stopnoanalyze`, confirm the expected `.etl`/`.json` file was written (`Test-Path`) instead of trusting the exit code. The command blocks in [Prerequisites](#prerequisites-trace-permission) and [Core Workflow](#core-workflow) below apply both rules — follow their exact form. ## What is vcperf? vcperf.exe captures and analyzes C++ build traces using the C++ Build Insights SDK. It records ETW events during MSVC compilations and produces either ETL files (for Windows Performance Analyzer) or structured JSON reports. ## Prerequisites: Trace Permission `vcperf /start` writes ETW providers that require **either**: - the invoking process to be **elevated (Administrator)**, OR - a one-time **`vcperf /grantusercontrol`** run elevated for the current user — after which non-admin invocations may use `/start /noadmin` (reduced data: no CPU sampling). The CLI agent runs unelevated and **must solve this itself** — do not ask the user to "open an elevated prompt and run X by hand". Run on first use. Once Step 4 verifies the grant, Step 4b persists a per-user marker file so **future sessions skip Trace Permission Steps 2–4 entirely**. Within a session, also cache the Step 1 result in working memory. **Skip entirely for `vcperf /analyze` and `vcperf /stop` against an existing trace** — only `/start` needs the grant. ### Step 1 — Check whether rights already exist Two signals; either is sufficient. **Run Check A first; skip Check B if A is true.** **Check A — process is elevated:** ```powershell $isAdmin = ([Security.Principal.WindowsPrincipal]::new( [Security.Principal.WindowsIdentity]::GetCurrent()) ).IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator) ``` If `$isAdmin` is `$true` → skip to Step 5. **Check B — agent-persisted grant marker** (only when Check A is false): ```powershell $grantMarker = Join-Path $env:LOCALAPPDATA 'vcperf\grant-verified' $hasGrantMarker = Test-Path -LiteralPath $grantMarker -PathType Leaf ``` If `$hasGrantMarker` is `$true`, the agent already granted and verified rights in a previous session — proceed without re-prompting. **If Check A OR Check B is true → you have rights. Skip to Step 5.** ### Step 2 — Ask the user before triggering UAC (REQUIRED) If neither check is true, you MUST get explicit consent before launching an elevated process. **Use the `ask_user` tool — never just print prompt text in chat.** ``` ask_user( question = "vcperf needs additional permissions to execute a trace. May I run a one-time elevated `vcperf /grantusercontrol`? A UAC prompt will appear.", choices = ["Yes (Recommended)", "No"] ) ``` If the user answers **No**, stop the workflow. Report that vcperf cannot run without trace permission and let the user decide how to proceed. Do NOT fall through to Step 3, do not retry, and do not silently downgrade to a non-vcperf measurement. ### Step 3 — Grant rights elevated Launch an elevated PowerShell that runs `vcperf /grantusercontrol` and wait for it. The grant produces no output artifact, so Step 4 verifies it functionally: ```powershell # $vcperf must already hold the absolute path (see resolution block above). # /grantusercontrol writes no output file, so Step 4 verifies it functionally; # we don't depend on vcperf's exit code here (see Invocation rule). try { Start-Process powershell.exe ` -ArgumentList '-NoProfile','-Command',"& '$vcperf' /grantusercontrol" ` -Verb RunAs -Wait $grantCancelled = $false } catch [System.ComponentModel.Win32Exception] { if ($_.Exception.NativeErrorCode -eq 1223) { # Win32 error 1223 = ERROR_CANCELLED — user dismissed the UAC prompt. $grantCancelled = $true } else { throw } } $grantCancelled ``` - A UAC dialog will appear. If the user cancels it, `Start-Process -Verb RunAs` throws `Win32Exception` with `NativeErrorCode = 1223` — treat that as "user declined" and stop (same outcome as a "No" in Step 2). - Otherwise the elevated grant ran. Its exit code is not reliable (see Invocation rule), so **Step 4 is the authoritative check** that the grant actually worked — proceed there. ### Step 4 — Verify the grant worked Do not trust Step 3 alone — run a functional probe. Use `/stopnoanalyze` (not `/stop`); the probe only needs to prove `/start /noadmin` succeeded, and `/stop` would waste seconds analyzing an empty session. ```powershell $probe = 'PermProbe_' + [guid]::NewGuid().ToString('N').Substring(0,8) $probeEtl = Join-Path $env:TEMP "$probe.etl" Remove-Item $probeEtl -ErrorAction SilentlyContinue # Success signal is whether the probe ETL is produced, not $LASTEXITCODE # (see Invocation rule). A working /start + /stopnoanalyze writes the ETL; # if /start was denied, no ETL appears. $startOutput = (& $vcperf /start /noadmin $probe *>&1 | Out-String).Trim() $stopOutput = (& $vcperf /stopnoanalyze $probe $probeEtl *>&1 | Out-String).Trim() $started = Test-Path $probeEtl if (-not $started) { $probeOutput = @($startOutput, $stopOutput) | Where-Object { $_ } Write-Error ("vcperf permission probe failed.`n" + ($probeOutput -join "`n")) } Remove-Item $probeEtl -ErrorAction SilentlyContinue $started ``` If `$started` is `$true`, the grant is verified — proceed. If `$false`, the grant did not take effect — surface the failure with the vcperf output and stop. Do not silently retry. **Do NOT run Step 4b on a failed verify.** ### Step 4b — Persist the verified grant (so future sessions skip Steps 2–4) Run only **after** Step 4 returned `$started = $true`. Write the per-user **marker file** that Check B reads: ```powershell $grantMarker = Join-Path $env:LOCALAPPDATA 'vcperf\grant-verified' New-Item -Path (Split-Path $grantMarker -Parent) -ItemType Directory -Force | Out-Null $payload = @{ grantedUtc = (Get-Date).ToUniversalTime().ToString('o'); vcperf = $vcperf } | ConvertTo-Json -Compress [IO.File]::WriteAllText("$grantMarker.tmp", $payload, [Text.Encoding]::UTF8) Move-Item -LiteralPath "$grantMarker.tmp" -Destination $grantMarker -Force ``` - Check presence of marker file to determine if grant is persisted. - On every subsequent session, Check B returns `$true` and the agent skips straight to Step 5. ### Step 5 — Invoke vcperf with the right flag When starting the tracing session: - **always** pass `/noadmin` to `vcperf /start`, or the start will fail with a privilege error. - **Stale-grant recovery.** If `/start /noadmin` fails with a privilege error while Check B was `$true`, the cached grant is stale (the underlying ETW grant was revoked outside the agent). Recover by running Steps 2–4 directly — **skip Step 1**. ## Core Workflow There are three phases: **start** a trace, **build** your project, then **stop** and **analyze** to produce results. > Below, `vcperf` is shorthand for the resolved `$vcperf` absolute path run directly in PowerShell (`& $vcperf ...`), per the [Invocation rule](#invocation-rule-read-before-running-any-vcperf-command). Never run a bare `vcperf` (it hits the VS-bundled copy on `PATH`), and judge success by the output file rather than `$LASTEXITCODE`. ### 1. Start a tracing session ``` vcperf /start [/noadmin] [/nocpusampling] [/level1 | /level2 | /level3] <sessionName> ``` | Flag | Purpose | |------|---------| | `/noadmin` | Run without admin (less data) | | `/nocpusampling` | Skip CPU sampling | | `/level1` | Basic events | | `/level2` | Adds additional detail | | `/level3` | Maximum detail, includes template instantiation data | Use `/level3` when you plan to analyze templates. ### 2. Build your project Run your normal build command while the trace is active. Any MSVC-driven build works — MSBuild, CMake (Ninja or MSBuild generator), or direct `cl.exe`. The *compiler* must be MSVC; the *build driver* is your choice. When building solution files, look for either `.sln` or `.slnx`. #### Build configuration matters (Debug vs Release) vcperf only reports work the compiler actually performs. **Debug** builds skip most optimizer passes, so the `Functions` section of the JSON output will be sparse or empty — you will not see backend codegen, inlining, or optimization hotspots. Frontend phases (`IncludedFiles`, `Templates`) are still captured normally because parsing and template instantiation happen regardless of configuration. For meaningful function-level analysis, build in **Release** (or another optimized configuration such as `RelWithDebInfo`). As a rule, profile the configuration you intend to ship; before choosing Debug, confirm with the user that frontend-only data is enough for their question, and call out the missing `Functions` data in your report. ### 3. Stop and analyze ``` vcperf /stop [/templates] <sessionName> output.etl vcperf /stop [/templates] <sessionName> output.etl /jsonAnalysis output.json vcperf /stop [/templates] <sessionName> /timetrace output.json ``` Or capture raw first, then analyze offline: ``` vcperf /stopnoanalyze <sessionName> raw.etl vcperf /analyze [/templates] raw.etl output.etl /jsonAnalysis output.json ``` ### 4. Avoid timing gaps — run start/build/stop as one shell command `vcperf` records ETW events from `/start` until `/stop`. **Any wall-clock gap between `/start`, the build, and `/stop` is captured as noise** — model thinking, tool-call round-trips, prompts, and shell startup all inflate `BuildDurationMs` and can shift hotspot rankings. From a CLI agent or any interactive workflow, **chain `/start → build → /stop` into a single shell command**. Do not issue them as three separate tool calls — the latency between calls (often several seconds each) contaminates the trace. **PowerShell with MSBuild (single invocation — recommended):** ```powershell # $vcperf holds the absolute path resolved earlier. Run as one scriptblock so # /start -> build -> /stop execute back-to-back with no timing gaps. Judge trace # success by the output files (Test-Path), not $LASTEXITCODE (see Invocation rule). $buildExit = & { & $vcperf /start /noadmin /level3 MySession | Out-Host msbuild Project.sln /m /t:Rebuild /p:Configuration=Release | Out-Host $exitCode = $LASTEXITCODE & $vcperf /stop /templates MySession out.etl /jsonAnalysis out.json | Out-Host $exitCode } $traceOk = (Test-Path out.etl) -and (Test-Path out.json) ``` **PowerShell with CMake (single invocation):** ```powershell $buildExit = & { & $vcperf /start /noadmin /level3 MySession | Out-Host cmake --build build --target clean | Out-Host cmake --build build --parallel | Out-Host $exitCode = $LASTEXITCODE & $vcperf /stop /templates MySession out.etl /jsonAnalysis out.json | Out-Host $exitCode } $traceOk = (Test-Path out.etl) -and (Test-Path out.json) ``` `$buildExit` is the build tool's own exit code (reliable — an ordinary process), telling you whether the build succeeded. `$traceOk` confirms vcperf actually produced the trace. The `;` sequence never short-circuits, so `/stop` always runs even if the build fails. **cmd.exe alternative** (if you prefer `&&` short-circuit semantics; substitute the full path for `%VCPERF%`): ```cmd "%VCPERF%" /start /noadmin /level3 MySession && cmake --build build --target clean && cmake --build build --parallel && "%VCPERF%" /stop /templates MySession out.etl /jsonAnalysis out.json ``` `&&` skips `/stop` if the build fails; replace the `&&` before the final `"%VCPERF%" /stop` with `&` to emit the trace anyway. The same rule applies to incremental measurements (see [Iterative Build Measurement](#iterative-build-measurement)): the touch step may be a separate command, but `/start → build → /stop` must be one chained invocation. ## The /jsonAnalysis Flag Produces a structured JSON report alongside the ETL output. **Syntax:** `/jsonAnalysis <path>.json` — must appear **after** the ETL output path. **Constraints:** - Cannot be combined with `/timetrace` (mutually exclusive). - Pair with `/templates` to include template data. **Example:** ``` vcperf /stop /templates MyBuild output.etl /jsonAnalysis analysis.json ``` ## JSON Output Schema See [references/json-schema.md](references/json-schema.md) for the full field-by-field schema of the JSON analysis output, including `IncludedFiles`, `Functions`, `Templates`, and `IncludeTree` sections. --- ## Baseline Measurement Establish a baseline **before** applying the candidate change. A single build measurement is noisy — file system caches, antivirus, background processes, and thermal throttling all introduce variance, so always run a 5-build sample.