resolve-real-project-diff

name

resolve-real-project-diff

description

How to run `resolve_real_project_diff` (in the `borzoi` LSP crate): the whole-project name-resolution differential over **one** `dotnet restore`d F# project. It drives the LSP's full runtime chain — Compile order + assets-file assembly closure (real referenced DLLs read into an `AssemblyEnv`) + the `resolve_project` fold — and gates every divergence and alt-binder to zero against FCS's `uses-project` oracle. Use to answer "can the LSP correctly analyse this specific project?", validate the runtime resolution chain (including imported-assembly target identity) against a new project, or triage a real-world resolution divergence. This is the single-project, real-assembly gate — not the corpus-wide, per-file-isolation worklist sweeps (see resolve-divergence-sweep / parser-divergence-sweep).

Whole-project resolution differential vs FCS

crates/lsp/tests/all/resolve_real_project_diff.rs drives the real LSP runtime chain end-to-end over one on-disk F# project and diffs every name use against FCS:

1. Compile order + #if defines from the workspace's .fsproj evaluation (SemanticState::parses_for_project).
2. The project's composed reference set and the AssemblyEnv built from it, taken from one cache entry (SemanticState::env_reference_dlls_for_project
   • assembly_env_for_project): assets-file package and framework DLLs, each F# <ProjectReference>'s built output DLL, and the C# sidecar's metadata DLLs.
3. resolve_project folds cross-file + assembly resolution over the lot.

FCS (fcs-dump uses-project) is the oracle: it type-checks the same Compile-ordered files as one project, reading exactly that reference set and nothing else (--noframework). This is the sharpest single test for "does dependency resolution actually work on a real project", because it exercises the whole chain, not a unit slice.

Both sides reading the same set is what makes the numbers mean anything: an oracle that also resolved its own SDK binds names our AssemblyEnv cannot, we defer, and a deferral is not a divergence — the gate passes while the two sides read different worlds. The price is that an incomplete set stops the project type-checking, so the report counts oracle errors and gates them to zero.

This is the single-project gate, not a corpus sweep. It points at one restored project, reads its real referenced assemblies, and gates divergences and alt-binders to zero — so it can adjudicate imported-assembly target identity, which a per-file run cannot. If instead you want the corpus-wide worklist of what we still get wrong or defer — resolved per-file, in isolation, with an empty AssemblyEnv, as a measurement rather than a gate — that is [[resolve-divergence-sweep]] (name resolution) or parser-divergence-sweep (the parser layer below it).

Prerequisites

• The target project must have been dotnet restored — the test reads <project_dir>/obj/project.assets.json. The LSP reads only that standard location (no BaseIntermediateOutputPath support), so a project that relocates obj/ — e.g. the F# compiler's artifacts/obj/ layout — will not resolve.
• Its F# <ProjectReference>s must have been built: the composed set names each referenced project's output DLL. An unbuilt one is absent from both sides, and the oracle-error gate fails rather than reporting a number.
• Run under nix develop (the harness builds/drives fcs-dump). The first run builds fcs-dump via dotnet build -c Release, so budget several minutes.

Choosing a project

The oracle is faithful only for projects that are:

• SDK-default framework — a non-default <FrameworkReference> (Microsoft.AspNetCore.App, WindowsDesktop) is out of scope (FCS isn't handed it, so those framework symbols diverge).
• multi-file with imported-assembly uses — the test gates on cross_file_match > 0 and asm_match > 0, so a single-file project or one with only local references is rejected as vacuous. (A <TargetFramework> in the SDK-default set, no duplicate Compile basenames.)

A non-default <LangVersion> is supported (threaded to FCS as --langversion), unless the pin needs an SDK newer than the oracle's. A .fsi-bearing project is supported too: sema folds a signature file into an inert slot, so its own uses read as gaps while the implementation side gates the signature surface (a cross-file use of a sig-exposed val resolves to the .fsi ident).

Quick candidate scan for a restored, multi-file project:

# multi-file (>1 Compile), no FrameworkReference, restored:
for f in $(find ~ -maxdepth 6 -name '*.fsproj' 2>/dev/null); do
  d=$(dirname "$f")
  [ -f "$d/obj/project.assets.json" ] || continue
  grep -q FrameworkReference "$f" && continue
  n=$(grep -c 'Compile Include' "$f")
  [ "$n" -gt 1 ] && echo "$n  $f"
done

Running

BORZOI_PROJECT_FSPROJ=/abs/path/to/Foo/Foo.fsproj \
  nix develop -c cargo test -p borzoi --test all \
  resolve_real_project_diff:: -- --ignored --nocapture

#[ignore]d by default; skips with guidance if BORZOI_PROJECT_FSPROJ is unset. Note the --test all <group>:: filter form (one test binary per crate — see AGENTS.md); --test resolve_real_project_diff does not resolve.

Reading the result

The report line tallies (see report):

resolve-real-project <path>: <N> in-proj match (<M> cross-file) | <A> asm match | <D> diverge | <B> alt-binder | <G> gaps | <E> oracle errors

• in-proj / asm match — uses where our resolution equals FCS (in-project binder, or (assembly simple name, full name) for imported symbols).
• gaps — Deferred/unmodelled uses. Expected; counted, not gated.
• divergences + alt-binders — both gated to zero. A divergence is a wrong/Unresolved resolution where FCS resolved concretely; an alt-binder is a same-named binder at the wrong range/file (a wrong-shadow go-to-def). Each gated site is printed as "<file>":<range> "<text>" -> FCS <x>, we gave <y>.
• oracle errors — FCS's own error diagnostics, gated to zero. Non-zero means the reference set we handed it is incomplete: the affected uses carry no target, so they leave the comparison rather than failing it, and every count above describes a project FCS could not check.

BORZOI_PROJECT_SAMPLE=<n> raises the per-bucket print cap (default 40) — worth setting when triaging, since "40 of one known class" and "40 of one known class plus 96 of something else" print identically.

A divergence here is a sema (name-resolution) finding, not necessarily a dependency-resolution one: if the "we gave" side names a symbol in a referenced assembly at all, that assembly was resolved and read — the disagreement is about resolution precedence. A genuine dependency-resolution failure shows up instead as asm_match == 0, an empty env, or the vacuity assertion firing.

An imported symbol's name is adjudicated by the comparator shared with borzoi-corpus-diff (borzoi_sema::test_support, behind its test-support feature): FCS's rendered name or the structural declaration our own resolution certifies. That is what stops ImmutableArray<byte>.Empty — FCS printing the enclosing generic type with its arguments, which our full names never carry — being scored as a wrong target.

Validated (zero divergences, zero alt-binders) against WoofWare.{WeakHashTable, LiangHyphenation, Expect, PawPrint.Domain}. WoofWare.PawPrint's main library reaches zero divergences across 41k in-project + 13.6k imported uses but still holds 6 alt-binders: an [<AutoOpen>] module's active patterns in a referenced project's assembly are not in scope from the enclosing namespace, so a parameterised case's argument binds as a pattern binder. PawPrint.Domain originally surfaced one String qualifier-precedence divergence (the FSharp.Core String module picked over System.String for String.Equals) — a sema precedence bug, fixed by making module-qualified member lookup follow FCS's in-module search domain (module_qualified_occupied; pinned by crates/sema's resolve_string_qualifier_repro and resolve_qualifier_precedence_diff).

What this test does not cover

• The in-house NuGet resolver (borzoi-nuget) is not wired into the runtime (Slice 8 of docs/nuget-restore-plan.md outstanding); resolution depends entirely on a pre-existing dotnet restore. This test therefore only exercises the assets-file path, never the offline-resolve fallback.
• C# <ProjectReference>s go through the sidecar; an F# project behind a C# boundary is a known under-resolution.