| name | libsync-build-and-env |
| description | Use when setting up libsync from a fresh clone; when make deps, poetry install, or npm install fails; when ModuleNotFoundError: No module named 'qobuz' or 'tidal' appears; when docker build fails or bakes unexpected SDK code; when make dev serves a 404 UI or make dev-frontend requests hit the wrong port; when the sdks/qobuz_api_client submodule is empty, dirty, or needs a pin bump; or when local Python/Node/Poetry versions differ from CI and behavior diverges. |
libsync: build and environment runbook
This skill recreates the libsync development environment from scratch and documents every known build trap. All paths are relative to the repository root. Facts marked with a date were verified on that date against the repo; re-verification commands are at the bottom.
1. From clone to running
Definitions used throughout:
- Poetry: the Python dependency manager that owns the backend virtualenv (
pyproject.toml + poetry.lock).
- The submodule: the git submodule at
sdks/qobuz_api_client, which contains both the qobuz and tidal Python SDK packages.
- Editable install:
pip install -e <path>, which links a package into the virtualenv from a source directory instead of copying it.
Run these commands in order:
git clone --recursive git@github.com:arthursoares/libsync.git
cd libsync
make deps
poetry install
cd frontend && npm install && cd ..
make dev
make deps does exactly three things (Makefile, as of 2026-07-03):
git submodule update --init --recursive
poetry run pip install -e sdks/qobuz_api_client/clients/python
poetry run pip install -e sdks/qobuz_api_client/clients/python/tidal
The SDKs are not Poetry dependencies. They are editable pip installs into the Poetry venv. pyproject.toml says this explicitly in a comment; the SDKs carry their own dependencies (mutagen, aiolimiter, pycryptodomex, aiofiles).
Install-order note: the README runs make deps before poetry install; CI (.github/workflows/pytest.yml) runs poetry install first and then the two SDK pip installs. Both orders work because poetry run auto-creates the venv on first use. Do not "fix" one to match the other without reason.
Smoke checks after setup:
make test
poetry run python -c "import qobuz, tidal; print('SDKs OK')"
which ffmpeg
If make test floods the terminal with DEBUG log blocks, that is configured behavior (log_cli = true, log_level = DEBUG in pyproject.toml), not a failure.
2. Toolchain matrix
| Tool | CI / Docker pin | Local constraint in repo | Version file? |
|---|
| Python | 3.12 (pytest.yml, docker/Dockerfile python:3.12-slim) | pyproject.toml: >=3.10 <4.0 | No .python-version |
| Node | 20 (pytest.yml, node:20-alpine) | README says "Node 20+"; no engines field in frontend/package.json | No .nvmrc |
| Poetry | 1.8.0 (snok/install-poetry@v1 with version: 1.8.0) | none | none |
| ffmpeg | apt-installed in Docker only | README lists it as a local requirement; nothing installs it | n/a |
Traps in this matrix:
- Known inconsistency (as of 2026-07-03): the CI Poetry pin looks stale.
poetry.lock declares lock-version = "2.1" and its header says "generated by Poetry 2.3.2" — that is the Poetry 2.x lockfile format, while CI pins Poetry 1.8.0. CI is currently green, so either the action resolves differently or 1.8.0 tolerates the lock; this has not been confirmed either way. If CI dependency installs start failing mysteriously, suspect this first. Do not regenerate poetry.lock with a different Poetry major version as a drive-by.
- Local machines drift freely. Because no
.python-version / .nvmrc / .tool-versions exists, your local Python/Node/Poetry can be years newer than CI. pyproject.toml allows any Python >=3.10 <4.0, so this usually works, but when local and CI results disagree, check versions before debugging code.
- ffmpeg is a silent runtime dependency. The Tidal SDK downloader calls
shutil.which("ffmpeg") to remux DASH-delivered FLAC-in-MP4 into real .flac files (sdks/qobuz_api_client/clients/python/tidal/tidal/downloader.py). If ffmpeg is missing, downloads do not fail — files are silently left as MP4-wrapped FLAC in .flac-named files, which strict FLAC tools choke on. Nothing in make deps or poetry install installs it. Check with which ffmpeg; install with brew install ffmpeg on macOS. Docker images already include it.
3. The submodule (sdks/qobuz_api_client)
- What it is: a git submodule pinned by gitlink (a commit hash recorded in the parent repo's tree) to
https://github.com/arthursoares/qobuz_tidal_api_client.git. The path sdks/qobuz_api_client deliberately keeps the old repo name after the upstream repo was renamed to qobuz_tidal_api_client — do not rename the path.
- Current pin: commit
5e66211 ("feat(tidal): PKCE OAuth + DASH manifest support + folder label fixes") (as of 2026-07-03, v0.0.6).
- Update procedure (per CLAUDE.md):
git -C sdks/qobuz_api_client pull, re-run make deps, run make test, then commit the submodule bump in the parent repo through the normal PR flow.
Failure modes:
| Symptom | Cause | Fix |
|---|
make deps pip installs fail; sdks/qobuz_api_client/ is empty | Cloned without --recursive, submodule never initialized | git submodule update --init --recursive, then re-run make deps |
docker build fails at the COPY sdks/... step, or the image lacks SDKs | Same — the Dockerfile COPYs the submodule from the host checkout; there is no in-Dockerfile clone fallback (the Dockerfile comment says so) | Initialize the submodule on the host first |
ModuleNotFoundError: No module named 'qobuz' (or 'tidal') in pytest or at runtime | Editable installs are invisible to Poetry: poetry sync (Poetry 2.x), poetry install --sync, or any venv rebuild silently removes them | Re-run make deps |
make deps fails on a checkout of an old libsync commit — the pinned SDK commit does not exist upstream | The SDK repo's history has been rewritten; the parent repo records at least one re-pin after a rewrite (commit 52837cd "re-pin qobuz_api_client submodule after SDK history rewrite") plus a rename/make-public bump (89e8599). Old gitlinks may point at commits that no longer exist upstream. | Check out a libsync commit whose pin exists, or fetch the SDK repo's current head and accept the skew |
Rule of thumb: after any operation that touches the Poetry venv, re-run make deps. It is idempotent and cheap.
4. Makefile target anatomy
Every target, verified against the Makefile (as of 2026-07-03):
| Target | What it actually does | Notes |
|---|
deps | Submodule init + two editable SDK installs | See section 1 |
test | poetry run pytest tests/ -q --tb=short | 154 tests, no credentials needed |
test-unit / test-all | Same as test but -v | Identical to each other; both verbose |
build-frontend | cd frontend && npm run build | Output goes to frontend/build |
build-local | build-frontend, then rm -rf backend/static && cp -r frontend/build backend/static | Hazard: if the frontend build fails, backend/static may already be deleted. The backend only mounts static files when the directory exists (backend/main.py, static-serving block), so the API still runs but every UI route 404s. Fix: re-run make build-local after fixing the build error. |
dev | build-local, mkdir -p data, then uvicorn on :8080 with --factory --reload and env vars STREAMRIP_DB_PATH=data/streamrip.db STREAMRIP_DOWNLOADS_PATH=~/Downloads/Music | The primary dev loop |
dev-backend | Same uvicorn line, no frontend rebuild | Use when only touching backend code |
dev-frontend | cd frontend && npm run dev (Vite dev server on :5173) | Broken by design gap — see below |
build-docker | docker build -f docker/Dockerfile -t streamrip . | Image tag keeps the legacy name deliberately |
run-docker | Runs the image with ~/Downloads/Music:/music, named volume streamrip-data:/data, STREAMRIP_DB_PATH=/data/streamrip.db | |
docker | build-docker + run-docker | |
lint | poetry run ruff check backend/ | Currently FAILS on clean main due to local-vs-CI ruff version skew — do not blindly --fix. Details live in the sibling skill libsync-validation-and-qa. |
make dev-frontend does not work out of the box (as of 2026-07-03). The frontend API client hardcodes BASE = '/api' with relative fetches (frontend/src/lib/api/client.ts), and frontend/vite.config.ts contains no proxy configuration — it is just defineConfig({ plugins: [sveltekit()] }). So API requests from the Vite dev server hit :5173 instead of the backend on :8080, and there is no CORS setup either. The Makefile comment "proxied to a running backend" and the matching README line are stale. The reliable loop is make dev (full) or make dev-backend (backend-only, reusing the last-built backend/static). Whether a Vite proxy is planned is an open question — treat any fix as a change needing the normal PR flow.
Environment variables the backend reads — exactly two (verified by grepping backend/ for os.environ/os.getenv, as of 2026-07-03): STREAMRIP_DB_PATH (default data/streamrip.db) and STREAMRIP_DOWNLOADS_PATH (default /music). Both are only fallbacks: a downloads_path value in the SQLite config table overrides the env var. The per-source dedup DBs are derived from STREAMRIP_DB_PATH's directory: downloads.db for Qobuz (legacy name kept on purpose), downloads-<source>.db for others (backend/services/download.py, _download_album area). Full config catalog: see sibling skill libsync-config-and-flags.
5. Docker build
docker/Dockerfile is a two-stage build (as of 2026-07-03):
- Stage 1 (
node:20-alpine): npm ci from frontend/package*.json, then npm run build.
- Stage 2 (
python:3.12-slim): apt-installs ffmpeg; copies pyproject.toml, poetry.lock, README.md, backend/, and sdks/qobuz_api_client/clients/python/ (renamed inside the image to ./sdks/qobuz_api_client_py/); runs pip install poetry, poetry config virtualenvs.create false, poetry install --only main, then pip install websockets ./sdks/qobuz_api_client_py ./sdks/qobuz_api_client_py/tidal; copies the stage-1 frontend build into ./backend/static/. Exposes 8080; CMD is uvicorn with --factory.
Traps:
- The build is not fully lockfile-driven.
poetry install --only main uses poetry.lock, but the follow-up pip install websockets is unpinned, and the SDKs are installed from whatever the host's submodule checkout contains at build time.
- A local
docker build bakes in uncommitted submodule edits. The COPY takes the submodule working tree as-is, including local modifications (see section 6 — the worktree is currently dirty). If you need a reproducible image, verify git -C sdks/qobuz_api_client status --short is clean first, or use the CI-published image from GHCR.
- Uninitialized submodule = broken build. Same as section 3: initialize on the host before
docker build.
- Two compose files exist:
docker/docker-compose.yml (builds locally, named volume streamrip-data) and docker-compose.example.yml at repo root (pulls ghcr.io/arthursoares/libsync:latest, bind-mounts ./music and ./data). Running and operating containers is covered by sibling skill libsync-run-and-operate.
CI's docker-publish job is a required merge check — a Dockerfile breakage blocks all merges to main. Never route around this; fix the build.
6. Known local-state facts (dated 2026-07-03 — verify before relying on them)
- The SDK submodule working tree carries 31 modified files (
git -C sdks/qobuz_api_client status --short shows 31 M entries, +65/−80 lines) while the gitlink pin itself is clean at 5e66211. Spot-checked diffs are lint-only churn (import re-sorting, f-string→plain-string); not every file was audited. Because the SDKs are editable installs, these uncommitted edits ARE live in the running backend and would be baked into a local docker build. Do not reset or commit these files without maintainer say-so (inferred — confirm with maintainer; they may be in-flight upstream work). The parent-repo git status shows this as m sdks/qobuz_api_client (lowercase m = dirty content, not a changed pin).
data/ contains legacy zombie DBs. Current contents: streamrip.db (the live config/library DB), downloads.db (live Qobuz dedup DB), plus downloads_albums.db and failed.db, which no code in backend/ or tests/ references (verified by grep) — they are leftovers from the deleted CLI era. Deleting data/ entirely wipes local credentials and library state; what is safe to delete is the domain of sibling skill libsync-run-and-operate.data/ is gitignored (both data/ and *.db patterns in .gitignore), so none of this can leak into commits.
When NOT to use this skill
- Running, deploying, releasing, or deciding what data artifacts are safe to delete → libsync-run-and-operate.
- Lint/format version-skew details, pre-push checklist, what CI actually gates → libsync-validation-and-qa.
- Config keys, env-var precedence, and DB-config overrides in depth → libsync-config-and-flags.
- A build that used to work and now fails for non-environment reasons → libsync-debugging-playbook.
- How to get a change (e.g. a Vite proxy fix, a Poetry pin bump) merged → libsync-change-control.
- SDK internals, auth flows, quality tiers → qobuz-tidal-domain-reference.
Provenance and maintenance
Re-verify each drift-prone fact before trusting it:
make deps contents and all Makefile targets: cat Makefile- README setup order and requirements:
sed -n '24,100p' README.md
- CI pins (Python 3.12, Poetry 1.8.0, Node 20):
grep -n "python-version\|node-version\|version:" .github/workflows/pytest.yml
- poetry.lock format vs CI Poetry pin:
head -1 poetry.lock && grep -n "lock-version" poetry.lock
- Python constraint and SDK-install comment:
grep -n "python =" pyproject.toml && sed -n '22,27p' pyproject.toml
- Absence of version files:
ls .python-version .nvmrc .tool-versions frontend/.nvmrc 2>&1
- No
engines field: grep -n engines frontend/package.json || echo "no engines field"
- Submodule pin and remote:
git submodule status && cat .gitmodules
- Submodule worktree dirtiness:
git -C sdks/qobuz_api_client status --short | wc -l
- SDKs still importable in the venv:
poetry run python -c "import qobuz, tidal; print('OK')"
- Test count/time:
time make test
- Dockerfile anatomy (COPY-from-host, unpinned websockets):
cat docker/Dockerfile
- ffmpeg presence and its use in the Tidal SDK:
which ffmpeg; grep -n 'shutil.which("ffmpeg")' sdks/qobuz_api_client/clients/python/tidal/tidal/downloader.py
- No Vite proxy / hardcoded
/api base: cat frontend/vite.config.ts && grep -n "BASE" frontend/src/lib/api/client.ts
- Backend env vars (should remain exactly two):
grep -rn "os.environ\|os.getenv" backend/
- Zombie DBs unreferenced:
ls data/ && grep -rn "downloads_albums\|failed.db" backend/ tests/ || echo "unreferenced"
