synapseml-local-setup

Set up and validate SynapseML locally in WSL or Linux. Use when an agent needs SynapseML working locally, runs sbt compile/test, sees Java 21, Scala 2.12 compiler-bridge, bad constant pool index, Spark, or local validation failures.

Jump to install

Repository: microsoft/SynapseML
Last source activity: May 5, 2026 at 22:37
Detected SKILL.md language: English
Stars: 5,242
Forks: 868

Install options

The review-first prompt is selected by default. You can switch to a direct command or download a local copy.

Review the source files

Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.

Back to search Back to installation

name	synapseml-local-setup
description	Set up and validate SynapseML locally in WSL or Linux. Use when an agent needs SynapseML working locally, runs sbt compile/test, sees Java 21, Scala 2.12 compiler-bridge, bad constant pool index, Spark, or local validation failures.
compatibility	Linux/WSL with bash, git, rg, sbt, and JDK 11 installed. Designed for the SynapseML repo.

SynapseML Local Setup

Use this skill before any local SynapseML build, compile, or test validation.

Important

Always use an explicit SynapseML repo path.
Do not run SynapseML SBT with the machine default Java 21. Use JDK 11: /usr/lib/jvm/java-11-openjdk-amd64
Java 21 can fail before project code compiles with bad constant pool index: 0 while building Scala 2.12 compiler-bridge_2.12.
Compile commands are safe. Some cognitive service tests create, write, list, or delete real Azure resources. Inspect before running those tests and ask for approval if live resources are involved.

Workflow

1. Diagnose the repo and toolchain

Run scripts/synapseml-doctor.sh:

scripts/synapseml-doctor.sh --repo <synapseml-repo>

Capture:

Git branch and dirty state.
Default Java version.
JDK 11 availability.
sbt version and SynapseML Scala/Spark versions.

2. Compile with JDK 11

Run scripts/synapseml-sbt.sh:

scripts/synapseml-sbt.sh --repo <synapseml-repo> -- cognitive/Test/compile

Expected result:

sbt welcome line says Java 11.
core and cognitive main/test classes compile.
Command exits with [success].

3. Run a safe local smoke test

Run scripts/synapseml-smoke-test.sh:

scripts/synapseml-smoke-test.sh --repo <synapseml-repo>

Expected result:

One local Spark test runs.
Output includes All tests passed.

4. Inspect PR-specific tests before running them

Before running service tests, run scripts/check-live-service-tests.sh:

scripts/check-live-service-tests.sh --path <test-file-or-directory>

If it reports live-service hooks, ask the user before running that suite. Do not create or delete Azure Search indexes just to test a PR.

5. Run targeted tests only after safety review

Use the JDK 11 wrapper for any targeted SBT command:

scripts/synapseml-sbt.sh --repo <synapseml-repo> -- '<module>/testOnly <SuiteName> -- -z "<test filter>"'

If tests fail before compiling project code, load references/troubleshooting.md.

Known-good baseline

On 2026-05-05, this setup was validated with:

JDK: /usr/lib/jvm/java-11-openjdk-amd64
Java: openjdk version "11.0.30"
SynapseML: Scala 2.12.17, Spark 3.5.0, sbt 1.10.11
Compile: sbt 'cognitive/Test/compile' succeeded.
Smoke test: UDFTransformerSuite filtered test succeeded.