vstest-build-test

name	vstest-build-test
description	Build, test, and validate changes in the vstest repository. Use when building vstest projects, running unit tests, smoke tests, or acceptance tests, or when deploying locally built vstest.console for manual testing.

Building and Testing vstest

Pre-Build: Environment Setup

Before building, verify the .dotnet toolchain matches the current OS. The repo bootstraps its own .NET SDK into .dotnet/.

Detect OS vs .dotnet Mismatch

Run this check before every first build in a session:

# Determine current OS
OS=$(uname -s)   # "Linux", "Darwin" (macOS), or contains "MINGW"/"MSYS" (Windows/Git Bash)

if [ -d ".dotnet" ]; then
  if [ "$OS" = "Linux" ] || [ "$OS" = "Darwin" ]; then
    # On Linux/macOS the dotnet binary must be an ELF/Mach-O executable, not .exe
    if [ -f ".dotnet/dotnet.exe" ] && [ ! -f ".dotnet/dotnet" ]; then
      echo "MISMATCH: .dotnet contains Windows binaries but OS is $OS"
      rm -rf .dotnet .packages artifacts
      echo "Cleaned .dotnet, .packages, and artifacts for fresh bootstrap"
    fi
  else
    # On Windows the dotnet binary should be dotnet.exe
    if [ -f ".dotnet/dotnet" ] && [ ! -f ".dotnet/dotnet.exe" ]; then
      echo "MISMATCH: .dotnet contains Linux/macOS binaries but OS is Windows"
      rm -rf .dotnet .packages artifacts
      echo "Cleaned .dotnet, .packages, and artifacts for fresh bootstrap"
    fi
  fi
fi

After cleanup (or if .dotnet doesn't exist), the build script automatically downloads the correct SDK version from global.json.

Build

Platform Commands

Action	Windows	Linux / macOS
Restore + Build	./build.cmd	./build.sh
Restore only	./restore.cmd	./restore.sh
Build + Pack	./build.cmd -pack	./build.sh --pack
Release config	./build.cmd -c Release -pack	./build.sh -c Release --pack
Single project	./build.cmd -project <csproj>	./build.sh --projects <csproj>

Full Build (Recommended)

For projects with many cross-project dependencies (e.g., HtmlLogger, TrxLogger, vstest.console):

# Linux / macOS
./build.sh --pack

# Windows
./build.cmd -pack

This produces NuGet packages under artifacts/packages/Debug/Shipping/.

Single Project Build

For isolated projects with few dependencies:

# Linux / macOS
./build.sh --projects <path-to-csproj>

# Windows
./build.cmd -project <path-to-csproj>

Warning: This does NOT work for projects like HtmlLogger that have many transitive dependencies. Use --pack / -pack instead.

Test

Unit Tests (Default)

# Linux / macOS
./test.sh

# Windows
./test.cmd

Specific Project(s)

-projects / --projects takes a resolvable path or glob — it is passed through Resolve-Path, so a bare project nickname or category (e.g. smoke, htmllogger) fails with Cannot find path. Point it at the csproj(s):

# Windows
./test.cmd -projects "test\**\*HtmlLogger*\*.csproj"

# Linux / macOS
./test.sh --projects "test/**/*HtmlLogger*/*.csproj"

For a single project you can also build+test its csproj directly with the bootstrapped SDK:

# Windows
./.dotnet/dotnet.exe test test/Microsoft.TestPlatform.Extensions.HtmlLogger.UnitTests/*.csproj -c Debug

# Linux / macOS
./.dotnet/dotnet test test/Microsoft.TestPlatform.Extensions.HtmlLogger.UnitTests/*.csproj -c Debug

Test Categories (smoke / integration / performance / compatibility)

These are switches handled by eng/build.ps1 — NOT -projects values:

# Windows
./test.cmd -smokeTest           # TestCategory=Smoke (a subset of integration tests)
./test.cmd -integrationTest     # full acceptance / integration suite
./test.cmd -performanceTest
./test.cmd -compatibilityTest

# Linux / macOS: only --integrationTest and --performanceTest exist, because test.sh
# calls arcade's eng/common/build.sh directly and never goes through eng/build.ps1.
./test.sh --integrationTest     # includes smoke; there is no -smokeTest switch here

-smokeTest and -integrationTest are mutually exclusive (smoke is a subset); passing both throws.

Filter by Test Name

On Windows use the -filter parameter. Do not pass --filter inside TestRunnerAdditionalArguments — eng/build.ps1 explicitly throws if you do.

On Linux/macOS test.sh has no -filter parameter, so the filter goes in as an MSBuild property. The single quotes are required, otherwise bash interprets &, |, and the inner quotes.

# Windows
./test.cmd -integrationTest -filter "FullyQualifiedName~MyScenario"

# Linux / macOS
./test.sh --integrationTest --property:'TestRunnerAdditionalArguments=--filter "FullyQualifiedName~MyScenario"'

See CONTRIBUTING.md for more filtering examples.

Running tests locally (DOTNET_ROOT gotcha)

Test executables are built against the repo's preview TFM (e.g. net11.0), and integration and smoke tests additionally launch test-asset apphosts built the same way. An apphost resolves its shared runtime from DOTNET_ROOT, falling back to the machine-wide install (C:\Program Files\dotnet), which usually lacks the preview runtime — so it fails instantly with "You must install or update .NET to run this application." This hits plain unit test runs too, not only integration and smoke tests.

• test.sh (Linux/macOS) sets DOTNET_ROOT to the repo .dotnet automatically.
• test.cmd (Windows) does not — set it yourself before running:

$env:DOTNET_ROOT = "$PWD\.dotnet"
${env:DOTNET_ROOT(x86)} = "$PWD\.dotnet\dotnet-sdk-x86"   # only if x86 test hosts run
$env:DOTNET_MULTILEVEL_LOOKUP = "0"
./test.cmd -smokeTest

Manual Validation with vstest.console

After building with --pack / -pack, validate vstest.console changes by unzipping the built package:

1. Locate the package: artifacts/packages/Debug/Shipping/Microsoft.TestPlatform.<version>-dev.nupkg
2. Unzip it (.nupkg files are ZIP archives)
3. Run the local vstest.console against a test project

Alternative: Direct Artifact Paths

• xplat (netcoreapp): artifacts/<Configuration>/netcoreapp1.0/vstest.console.dll
• Windows desktop: artifacts/<Configuration>/net46/win7-x64/vstest.console.exe

Test Categories

Category	Speed	What it tests	How to run
Unit tests	Fast	Individual units	./test.cmd / ./test.sh (default)
Smoke tests	Slow	P0 end-to-end scenarios	-smokeTest switch
Acceptance / integration	Slowest	Extensive coverage	-integrationTest switch

Troubleshooting

• OS mismatch errors: If you see SDK load failures, run the mismatch detection script above to clean and re-bootstrap.
• Toolset version <version> has not been restored.: test.cmd / test.sh do not restore. Build once (./build.cmd -c Release) first, or pass -restore -build to the test command.
• Tests fail instantly on Windows with "You must install or update .NET to run this application": test.cmd does not set DOTNET_ROOT, so the preview-TFM apphosts look in C:\Program Files\dotnet (which lacks the preview runtime). Set $env:DOTNET_ROOT = "$PWD\.dotnet" before running — see "Running tests locally".
• If build fails asking for .NET 4.6 targeting pack, install it from Microsoft Downloads
• Enable verbose diagnostics: see docs/diagnose.md
• For debugging, add Debugger.Launch at process entry points (testhost.exe, vstest.console.exe)