| name | traigent-setup-quickstart |
| description | Install, set up, and get first value from the Traigent SDK for LLM optimization. The cold-start path: use when the user is new to traigent, wants their first run, has no dataset yet, or wants to install traigent, set up their first optimization, create an evaluation dataset, or get started with @traigent.optimize. Covers pip install, API-key setup, mock mode, a linear first-value walkthrough, and running a first optimization. |
| license | Apache-2.0 |
| metadata | {"author":"Nimrod","version":"1.0.17"} |
Traigent Quickstart
When to Use
Use this skill when:
- Setting up Traigent for the first time in a new project
- Installing the SDK and configuring the environment
- Creating a first
@traigent.optimize decorated function
- Building an evaluation dataset in JSONL format
- Verifying that the installation works correctly
- Running optimization in mock mode for development
Cold Start — First Value, One Step at a Time
When there's no prior run to look at, do not open with menus, methodology, or the
advanced sections below. Detect cold start — the user is new to Traigent, has never
viewed a run, and has no dataset or decorated function ready — and walk them to their
first visible result one action at a time: one question or one command, wait, then
the next. Never dump the whole pipeline at once.
The linear path:
- Find the agent to optimize. Ask for (or detect from the project) the one function
that calls an LLM. Just that — don't discuss knobs yet.
- Find or make the dataset. If they already have labeled examples, hand off to
traigent-dataset-curate for the one canonical dataset contract. If they have
nothing ready, use the bundled fallback below — no dataset needed to see value.
- Mock dry-run first. Always run keyless mock mode before anything paid (the "Your
First Optimization" example below is exactly this). Show the ranked table so they see
the loop work at zero cost and zero egress.
- Confirm the real run explicitly. A real (paid) run happens only after the user
sees the mock result and says go — and after the cost-gate approval. Never jump from
mock straight to spend.
- View it in the portal. Once a real run launches, watch the run start, watch the
rows appear on the portal, and have the user open and inspect the run.
Where the plan comes from — honesty rule. When it's time to decide what to tune,
ask the Traigent service for the run plan via the traigent-analyze-guidance skill (its
traigent plan CLI / get_optimization_plan MCP tool). Present the one plan the
service returns — not a menu you invented. There is no onboarding/phase parameter in
the SDK or CLI today, so don't pass one or imply the client picks a phase. If the service
can't return a plan, say so plainly and fall back to a generic, conservative knob
family — model + temperature only, explicitly labeled as a generic fallback. Never
encode task-specific ordering (scout/pivot/routing) in these docs; the service owns that.
Tasks like text2SQL, RAG, classification, and extraction may be named as things the
service plans for — but the plan logic stays server-side.
No agent or dataset yet? Start from references/first-value-fallback.md — a complete
mock-first first-value path that needs neither, then gates any real-provider spend behind
explicit approval and a cost_limit cap.
Once the user has seen a first result, hand off to the lifecycle skills rather than
duplicating them here: traigent-dataset-curate (real data), traigent-setup-decorator
(a real decorator), and traigent-analyze-guidance (what to do after a run). Do not surface
advanced playbooks during cold start.
Installation
This is the Python SDK skill. Building in JavaScript/TypeScript instead? Use the
traigent-js skill — it covers the native JS/TS optimizer (the @traigent/sdk package)
with a different install and API. The rest of this page assumes Python.
Basic Install
The fast path is the core floored install — sufficient for the entire keyless mock
quickstart because litellm ships in core. Install optional extras only after the mock
run succeeds.
pip install "traigent>=0.19"
python - <<'PY'
import importlib.metadata as md
import traigent
v = md.version("traigent")
if v == "0.0.1" or not hasattr(traigent, "optimize"):
raise SystemExit(
f"Bad Traigent install: traigent {v}. "
'You likely got the PyPI placeholder — reinstall with: python -m pip install --upgrade "traigent>=0.19"'
)
print(f"traigent {v} OK")
PY
Warning: pip printing traigent 0.0.1 does not provide the extra ... is FATAL — you installed the placeholder package; reinstall with python -m pip install --upgrade "traigent>=0.19".
Literal First Run (execution-only agents)
Run this entire block in the foreground and wait for it to finish. Do not split it,
background it, or continue after a failed command. The final expected line is
TRAIGENT-DRY-RUN-OK.
#!/usr/bin/env bash
set -euo pipefail
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade --timeout 60 --retries 5 "traigent>=0.19"
python - <<'PY'
import importlib.metadata as md
import traigent
v = md.version("traigent")
if v == "0.0.1" or not hasattr(traigent, "optimize"):
raise SystemExit(
f"Bad Traigent install: traigent {v}. "
'You likely got the PyPI placeholder — reinstall with: python -m pip install --upgrade "traigent>=0.19"'
)
print(f"traigent {v} OK")
PY
cat > ticket_eval.jsonl <<'JSONL'
{"input": "I was charged twice for my subscription", "output": "billing"}
{"input": "Please update the email address on my account", "output": "account"}
{"input": "The API returns a 500 error on POST requests", "output": "technical"}
{"input": "What are your business hours?", "output": "general"}
{"input": "My invoice has the wrong tax ID", "output": "billing"}
{"input": "I cannot reset my password", "output": "account"}
JSONL
cat > ticket_classifier.py <<'PY'
import traigent, litellm
from traigent.testing import enable_mock_mode_for_quickstart
enable_mock_mode_for_quickstart()
def mock_demo_accuracy(output, expected, config=None, **_):
cfg = config or traigent.get_config() or {}
base = 0.88 if cfg.get("model") == "gpt-4o" else 0.68
return max(0.0, base - 0.04 * float(cfg.get("temperature", 0.0)))
@traigent.optimize(
eval_dataset="ticket_eval.jsonl",
objectives=["accuracy"],
configuration_space={
"model": ["gpt-4o-mini", "gpt-4o"],
"temperature": [0.0, 0.7],
},
metric_functions={"accuracy": mock_demo_accuracy},
offline=True,
)
def classify_ticket(query: str) -> str:
config = traigent.get_config()
response = litellm.completion(
model=config["model"],
temperature=config["temperature"],
messages=[
{"role": "system", "content": "Classify the ticket as: billing, technical, account, or general."},
{"role": "user", "content": query},
],
)
return response.choices[0].message.content
results = classify_ticket.optimize_sync(max_trials=4, algorithm="grid")
print(f"Stop reason: {getattr(results, 'stop_reason', None)}")
print(f"Best config: {results.best_config}")
assert results.trials, "no trials ran"
assert not getattr(results, "failed_trials", []), f"failed trials: {results.failed_trials}"
assert results.best_config is not None, "no best config selected"
print("TRAIGENT-DRY-RUN-OK")
PY
export TRAIGENT_OFFLINE_MODE=true
export LITELLM_LOCAL_MODEL_COST_MAP=True
export TRAIGENT_OFFLINE_MODE=true
export LITELLM_LOCAL_MODEL_COST_MAP=True
python ticket_classifier.py
This block is generated from references/literal-quickstart.sh — edit that file, not the block.
Use a virtual environment (recommended)
Install into a project virtualenv — it's standard Python practice and it's the friction-free
path here. A fresh venv is enough; you do not need --system-site-packages.
python -m venv .venv && source .venv/bin/activate
python -m pip install --upgrade "traigent>=0.19"
pip install "traigent>=0.19" resolves from PyPI and pulls litellm (a core dependency) along with
it, so the keyless mock path — which intercepts litellm.completion(...) — works immediately,
with no extra install. Only the LangChain / OpenAI / Anthropic adapter clients live in the
integrations extra (below).
Why a venv instead of system Python? On modern Debian/Ubuntu/Fedora the system interpreter
is marked externally managed (PEP 668), so a bare pip install into it is refused with
error: externally-managed-environment. The venv above avoids that entirely.
With Optional Extras
Install these after the keyless mock run succeeds.
pip install "traigent[recommended]>=0.19"
pip install "traigent[integrations]>=0.19"
pip install "traigent[analytics]>=0.19"
pip install "traigent[all]>=0.19"
pip install "traigent[enterprise]>=0.19"
See references/installation-extras.md for the full table of extras and their contents.
Requirements
Get Your Traigent API Key
Backend-connected features (the default cloud smart optimizer, dataset synthesis, analytics dashboards, the CI gate, and portal result history) all require TRAIGENT_API_KEY. There are two ways to obtain it:
Portal key (experiments-scoped)
- Sign up at the Traigent portal and create a project.
- In your project settings, go to API Keys and click Create key.
- This issues a
user-type key scoped to experiments:read experiments:write — sufficient for SDK optimizations and analytics.
export TRAIGENT_API_KEY="uk_..."
CLI device-authorization key (project-scoped)
The CLI device-flow issues a project-scoped sk_-prefixed key with broader permissions (quota, dataset management, full project access). Use this when you need project-level operations beyond experiments.
Run traigent auth login in your terminal — it opens a browser for OAuth device authorization. The key is written to ~/.traigent/credentials. Then export it:
export TRAIGENT_API_KEY="sk_..."
Which key to use? The portal experiments-scoped key is sufficient for most optimization workflows. Use the device-flow key for quota management, cross-project access, or when the CLI reports permission errors.
For the standard path, set TRAIGENT_API_KEY once, omit algorithm and offline, and let Traigent use the default cloud smart optimizer with portal result sync. Use algorithm="grid" or "random" only when you explicitly want local search; use offline=True only when zero egress is required.
Prereq for real (non-offline) runs: set TRAIGENT_API_KEY.
The SDK defaults to the cloud backend (https://portal.traigent.ai) when
TRAIGENT_BACKEND_URL is unset — no env var is needed for the standard cloud path.
Set TRAIGENT_BACKEND_URL only to target a dev or self-hosted backend
(e.g. http://localhost:5000). Exception: the traigent next-steps and traigent plan
CLI commands default their --backend-url flag to http://localhost:5000, so for cloud
use pass the flag or set the env var explicitly.
Portal-issued API keys use the uk_... prefix.
export TRAIGENT_API_KEY="uk_..."
export TRAIGENT_BACKEND_URL="https://portal.traigent.ai"
Key Hygiene — Reference Names, Never Values
Observed across multiple coding-agent CLI families (Codex, Claude, Gemini), not one tool's quirk —
treat every rule below as universal regardless of which agent is driving the session.
- Never print, echo, or log a key value.
echo $TRAIGENT_API_KEY, a bare env/printenv
dump, and set -x/bash -x wrapped around any key-touching command are all leaks — each writes
the raw secret to stdout, which most agent harnesses capture into the visible transcript and
often into a log file too.
- Reference keys only by env-var name (
TRAIGENT_API_KEY, OPENAI_API_KEY, ...) in commands,
code, and chat — never paste or reconstruct the value itself.
- Treat transcripts and logs as shareable artifacts. Assume anything printed to the terminal
or written to a log may be read, copied, or shared later; keep secrets out of both.
Environment Setup
Development Mode (Recommended for Getting Started)
Mock mode is the keyless dev path for provider calls — LLM calls are intercepted and return canned responses. Activate it in code:
from traigent.testing import enable_mock_mode_for_quickstart
enable_mock_mode_for_quickstart()
enable_mock_mode_for_quickstart() is the recommended activation path. It is hard-blocked when ENVIRONMENT=production and emits a once-per-process WARNING so a test that accidentally runs in a deployed system is loud and visible.
- Mock scope: only LiteLLM (
litellm.completion) and LangChain (ChatOpenAI, ChatAnthropic, etc.) calls are intercepted. Raw openai.OpenAI() / anthropic.Anthropic() clients are not intercepted — a function using a raw client will make real, billable calls in mock mode. Use LiteLLM in examples that must run keyless.
- No separate install needed for mock:
litellm ships with the SDK core (pip install "traigent>=0.19" pulls it), so litellm.completion(...) is interceptable the moment Traigent is installed — you do not need to pip install litellm yourself. (LangChain adapters do require pip install "traigent[integrations]>=0.19".)
- Mock ≠ offline. Mock stops LLM cost (calls are intercepted) — it does not stop backend egress. With
TRAIGENT_API_KEY set and the default offline=False, a "mock dry-run" is still sent to the Traigent backend and appears on your portal as a mock-data experiment (and counts against quota). For a fully local, private dry-run, also pass offline=True (or run with no key). enable_mock_mode_for_quickstart() alone does not make a run local.
- Real metrics read 0.0 under mock. Every intercepted call returns the same canned text, so exact/execution-match scorers score a uniform 0.0 across trials — expected in mock, not a broken pipeline (that is exactly why the example below wires a mock-only demo scorer).
Legacy Env-Var Path
The previous quickstart docs taught export TRAIGENT_MOCK_LLM=true. That env var still works in non-production environments for backward compatibility with existing fixtures, but it is hard-blocked when ENVIRONMENT=production (an OSError is raised at SDK import). Prefer the in-code API for new code.
Using a .env File
Traigent supports .env files via python-dotenv (included in the integrations extra). Create a .env file in your project root:
TRAIGENT_API_KEY=uk_... # portal key; use your sk_... key here if you used the CLI device flow
OPENAI_API_KEY=sk-...
TRAIGENT_DEBUG=1
Recommended: have the user paste keys into .env, never into the chat
When a user says "I have my key", do not ask them to type the secret into the
conversation — it would be captured in the agent transcript, logs, and context. Open the
.env file for them to paste into directly. This is both more secure (the raw key
never touches the chat) and better UX (they see exactly where it goes). Procedure:
-
Create the file from the project template if one exists (cp .env.example .env),
otherwise create a minimal .env with key names pre-filled and values blank, so the
user only pastes after each =:
TRAIGENT_API_KEY=
# provider key — fill the one(s) this project uses:
OPENAI_API_KEY=
# ANTHROPIC_API_KEY=
# Bedrock: AWS_ACCESS_KEY_ID= / AWS_SECRET_ACCESS_KEY= / AWS_REGION=
-
Always show the user the absolute path (e.g. /home/me/proj/.env). This is the
guaranteed fallback — they can open it in their own editor no matter what happens next.
-
Best-effort: pop the file open in a standalone editor window, launched detached.
Pick the launcher by OS; never wrap it in timeout:
- Linux:
setsid -f gnome-text-editor "$ENV" — or the first of
kate / gedit / xed / mousepad that exists; last resort xdg-open "$ENV".
- macOS:
open -t "$ENV" (opens the default text editor in its own window).
- Windows:
start "" notepad "%ENV%" (Notepad is always present), or
Start-Process notepad "$env:ENV" in PowerShell.
Pitfalls that look like success but aren't (verified the hard way):
- Don't open via the user's IDE (
code <file> / cursor <file>): it can spawn a
nested instance that crashes, and it hijacks whichever IDE window is focused — so
.env can pop up inside an unrelated project.
- Don't trust the launcher's exit code as "opened" — a crashed window can still exit 0.
Verify the editor process is actually alive (e.g.
pgrep) and ask the user to confirm
the window appeared.
-
Pick the provider key by detecting the vendor from the project (its
openai / anthropic / litellm / Bedrock imports or config). If the vendor is
ambiguous, undetectable, or the project uses multiple providers (e.g. OpenAI and
Bedrock), ask the user which provider(s) and label the matching key(s) in .env.
-
Wait for the user to paste and save. Confirm .env is in .gitignore.
-
Fallback: if no standalone editor opens (or the user says no window appeared), have
them open the printed path manually; only as a last resort use a terminal export VAR=...
(less private than the file).
Never echo, log, or read back the key value. .env must be git-ignored — never commit real keys.
Production Mode
For production, set your provider API keys and don't call enable_mock_mode_for_quickstart():
export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...
Before your first real run, verify your model IDs are live. Provider catalogs change — a
delisted or renamed ID causes a 404 or a degraded/unpriced trial. Preflight with
traigent models --provider <p> --check <model_id> (see the CLI Quick Reference below), or
query the provider's live catalog directly (e.g. curl -s https://openrouter.ai/api/v1/models
for OpenRouter). The traigent-setup-integrations skill covers multi-provider model verification.
See references/environment-variables.md for all available environment variables.
Your First Optimization
Always dry-run first. Before a real (paid) run, run in mock mode, review the cost estimate, and get explicit approval. See the traigent lifecycle skill for the mandatory dry-run-first / cost-approval workflow.
Real LLM runs require cost approval. A real (non-mock) optimization is blocked by a cost
gate. To confirm you accept the cost, set TRAIGENT_COST_APPROVED=true in the environment
(the verified path); some SDK versions also accept cost_approved=True in the
@traigent.optimize() decorator. The SDK prints an estimate
before any trial executes; the estimate may be high (fallback pricing is conservative), but the
gate is a safety confirmation — nothing runs until you approve.
Tiny Real Cost and KPI Probe
After the mock dry-run passes and before any full run, run one tiny real optimization: 1-2 dataset examples, minimal trials, and the cheapest candidate model. Check both surfaces before scaling up: results.total_cost must be neither None nor 0.0 with real calls (both mean cost is not wired), and each trial's metrics must contain the declared objectives with non-degenerate values (not all 0.0/all 1.0). If either surface fails, wire it first — see traigent-optimize-run → Cost Wiring Probe for the fix ladder (custom model pricing env vars, per-trial cost metrics, TRAIGENT_STRICT_COST_ACCOUNTING).
Objective naming rule: Default: at least one objective labeled accuracy (built-in objective or your metric_functions key). If accuracy doesn't apply to this problem, name the primary quality KPI after the product concept, for example valid_schema, and note why accuracy was skipped.
Here is a complete working example. This function classifies customer queries using an LLM, and Traigent will find the best model and temperature combination.
Note on mock scope: enable_mock_mode_for_quickstart() intercepts LiteLLM and LangChain calls. Raw openai.OpenAI() / anthropic.Anthropic() client calls are not intercepted — use litellm.completion() for a fully keyless dry-run (see references/installation-extras.md for traigent[integrations]).
Why the example includes a mock_demo_accuracy scorer: in mock mode every LLM call returns the same canned string, so a real accuracy metric scores every trial 0.0 — a discouraging all-zeros table that looks broken. The demo scorer below ignores the (canned) output and ranks trials by their config, so the keyless dry-run produces a meaningful table — the same approach the bundled traigent quickstart command uses. It is mock-only: delete it for a real run, where Traigent scores actual model output against your dataset labels. (Prefer not to keep a placeholder scorer in your own code? Just run traigent quickstart for the same ranked demo without writing one.)
import asyncio
import os
from pathlib import Path
os.environ["TRAIGENT_OFFLINE_MODE"] = "true"
os.environ["LITELLM_LOCAL_MODEL_COST_MAP"] = "True"
import litellm
import traigent
from traigent import Choices
from traigent.testing import enable_mock_mode_for_quickstart
enable_mock_mode_for_quickstart()
Path("eval_queries.jsonl").write_text(
"\n".join(
[
'{"input": "I was charged twice for my subscription", "output": "billing"}',
'{"input": "The API returns a 500 error on POST requests", "output": "technical"}',
'{"input": "What are your business hours?", "output": "general"}',
]
)
+ "\n",
encoding="utf-8",
)
def mock_demo_accuracy(output, expected, config=None, **_):
"""Mock-only demo scorer — DELETE this for a real (paid) run.
In mock mode every ``litellm.completion`` call returns the same canned
string, so a real accuracy metric would score every trial 0.0 (a
misleading all-zeros table). This placeholder ignores ``output`` and ranks
trials by their config so the keyless dry-run produces a meaningful table —
the same trick the bundled ``traigent quickstart`` demo uses. On a real run,
remove it and let Traigent score actual model output against your labels.
"""
cfg = config or traigent.get_config() or {}
base = 0.85 if cfg.get("model") == "gpt-4o" else 0.65
return max(0.0, base - 0.05 * float(cfg.get("temperature", 0.5)))
@traigent.optimize(
eval_dataset="eval_queries.jsonl",
objectives=["accuracy"],
algorithm="random",
offline=True,
model=Choices(["gpt-4o-mini", "gpt-4o"]),
temperature=Choices([0.0, 0.5, 1.0]),
metric_functions={"accuracy": mock_demo_accuracy},
)
def classify_query(query: str) -> str:
config = traigent.get_config()
response = litellm.completion(
model=config["model"],
temperature=config["temperature"],
messages=[
{"role": "system", "content": "Classify the query as: billing, technical, or general."},
{"role": "user", "content": query},
],
)
return response.choices[0].message.content
async def main():
results = await classify_query.optimize(max_trials=6)
print(f"Best config: {results.best_config}")
print(f"Best score: {results.best_score}")
print(f"Trials run: {len(results.trials)}")
classify_query.apply_best_config(results)
answer = classify_query("I can't log in to my account")
print(f"Classification: {answer}")
asyncio.run(main())
In a notebook (Jupyter/IPython/Colab)? asyncio.run() raises
RuntimeError: asyncio.run() cannot be called from a running event loop there — the notebook
already runs one. Use await main() directly in a cell, or the synchronous alternative below.
Synchronous Alternative
If you prefer synchronous execution:
results = classify_query.optimize_sync(max_trials=6)
expected is a scoring label — do not put it in your agent function's signature.
The evaluator calls your function with the example's input fields only (plus any
config-injected params). It then passes the function's output and the dataset's
expected / output field to your scoring_function or metric_functions. A
function that declares expected as a parameter will fail every trial with
TypeError: missing required argument: 'expected'.
def classify_query(query: str, expected: str) -> str: ...
def classify_query(query: str) -> str: ...
def score(output: str, expected: str) -> float: ...
Key Concepts
@traigent.optimize(...) -- Decorator that wraps your function for optimization. Define what parameters to tune in the decorator arguments.traigent.get_config() -- Call inside your function to retrieve the current trial's configuration. Works during optimization trials and after apply_best_config().func.optimize(max_trials=N) -- Run the optimization loop asynchronously. Returns an OptimizationResult.func.apply_best_config(results) -- Lock in the best configuration found so that subsequent calls use it.
You've run your first optimization — now make it robust. The decorator above is intentionally a local dry-run recipe: a small model + temperature space, algorithm="random", and offline=True. For a real optimization, graduate to the more robust traigent-setup-decorator skill — custom evaluators / metric_functions, injection mode, execution policy, and weighted objectives — then launch with traigent-optimize-run, which adds what a real run needs beyond the basic .optimize() call: cost limits (cap a paid sweep before it overruns), algorithm choice ("auto" for connected real runs; "grid"/"random" for explicit local/offline search; named smart selectors like bayesian/optuna are roadmap, not executable end-to-end), parallel trials, and quota-aware run sizing. That decorator-setup → run-optimization pair is the recommended path from "first run" to a production optimization.
Dataset Format
Traigent uses JSONL (JSON Lines) files for evaluation datasets. Each line must have an input field and an output field.
Example: eval_queries.jsonl
{"input": "I was charged twice for my subscription", "output": "billing"}
{"input": "The API returns a 500 error on POST requests", "output": "technical"}
{"input": "What are your business hours?", "output": "general"}
input -- The value passed to your function during evaluation.output -- The expected/ground-truth result used for scoring.
You can include additional fields for metadata, but input and output are required.
One canonical contract. This flat form is the simplest case of the single Traigent dataset
contract: input (or input_data) → your function's args, the gold key → the expected value,
and every other top-level key → example.metadata[...] (how a per-example side field like a
db_path reaches a scorer). Values may be nested dicts, and the gold key has accepted aliases
(output, expected, expected_output, answer, target, label). See traigent-dataset-curate
for the full contract and traigent-eval-build for an execution-scored recipe that reads a
metadata field.
Dataset path sandbox. The SDK enforces a TRAIGENT_DATASET_ROOT / CWD sandbox — an
eval-dataset path outside the sandbox root (default: the current working directory) is
rejected at load with a ConfigurationError ("Dataset path must reside under …"). Keep the JSONL under the directory you run from,
or set TRAIGENT_DATASET_ROOT to the directory containing your datasets.
Tips for Good Datasets
- Include at least 10-20 examples for meaningful optimization.
- Cover edge cases and diverse inputs.
- Ensure ground-truth
output values are consistent and well-defined.
- Reserve a holdout slice before tuning; never tune and validate on the same rows.
- For evaluation dataset creation beyond this minimal JSONL, use
traigent-dataset-curate.
Verify Installation
Check SDK info
traigent info
This prints the installed version, Python version, available integrations, and optimization defaults.
Verify from Python
import traigent
print(traigent.get_version_info())
Validate an evaluation dataset
traigent validate eval_queries.jsonl
CLI Quick Reference
Start here (keyless, no API key required):
traigent quickstart
traigent onboard
| Command | Description |
|---|
traigent quickstart | Run the bundled mock-mode demo (keyless, zero-setup, always works) |
traigent onboard | Guided setup for Traigent in this project (API key, project, env) |
traigent models | List/validate model IDs before a run, e.g. traigent models --provider anthropic --check claude-3-haiku-20240307 (model preflight; catalogs change) |
traigent recommend | Evidence-backed TVAR recommendations for your agent/task type |
traigent recommend-eval | Metric and evaluator recommendations for your task type |
traigent generate-config | Scaffold a full @traigent.optimize() config for your function |
traigent detect-tvars | Detect tuned-variable candidates in existing Python files |
traigent info | Show SDK version, environment, and integrations |
traigent algorithms | List available optimization algorithms |
traigent validate | Validate dataset files and configuration |
Next Steps
- Configure your decorator for a real optimization -- The example above is the minimal decorator. To make it optimization-ready -- custom evaluators /
metric_functions, injection mode, execution (algorithm/offline), and weighted objectives -- use the traigent-setup-decorator skill, then launch with traigent-optimize-run. This decorator-setup → run-optimization pair is the standard two-step cycle for going from "first run" to a real optimization.
- Dry-run before a real run -- See the
traigent lifecycle skill for the mandatory dry-run-first / cost-approval workflow before any paid execution.
- Mind your plan quota -- Cloud optimization is metered by
optimization_samples (~max_trials × dataset_size per run) and optimization_trials, separate from dollar cost. Check usage and size large runs to fit; see the traigent-optimize-run skill ("Quota & Run Sizing").
- Define parameter search spaces -- See the
traigent-optimize-config-space skill for Range, IntRange, Choices, LogRange, factory presets, and constraints.
- Choose an optimization algorithm -- For connected real runs, omit
algorithm or use "auto"; use "grid"/"random" for explicit local/offline search. traigent algorithms on SDK 0.20.0 lists local/runtime registry names and may omit auto and named smart selectors (Traigent/Traigent#1751), so do not treat that CLI output as the complete public selector contract. "bayesian" and "optuna" validate as known names but are not executable end-to-end as selector names yet (Traigent/Traigent#1752).
- Add multiple objectives -- Use
objectives=["accuracy", "cost", "latency"] for multi-objective optimization.
- Use framework integrations -- Install
traigent[integrations] for LangChain, OpenAI, and Anthropic adapters.
- Verify model IDs before a real run -- Catalogs change; run
traigent models --provider <p> --check <id> (or query the provider's live catalog) so a delisted/renamed ID doesn't cause a 404 or a degraded, unpriced trial. See traigent-setup-integrations.
Traigent Interaction Policy
Track an interaction profile and adapt to it. Persona (stable): control=delegate|guided|inspect,
expertise=se|ds|unknown. Mood (this session): pace=execute|balanced|explore. Default when
unknown: guided,se,balanced. Infer from explicit user statements first, then recent behavior;
an explicit correction wins immediately. Never store or send this profile anywhere by default.
Fetch the live profile (when available)
At session or skill start, if a configured Traigent client is available, seed the profile from the
backend with the skill name:
policy = None
try: policy = await client.get_interaction_policy(skill="<this skill>")
except Exception: pass
Treat the returned profile as the STARTING seed: its control/expertise/pace axes plus
question_budget, options_max, and jargon_level replace the static defaults below. Explicit user
corrections in-conversation ALWAYS override the seed. If the call is unavailable or
fallback_policy="static_v1", simply use the static defaults below; the SDK already fails soft.
- Always be concise.
- Match terminology to expertise. For
se: plain engineering words; define each Traigent or
statistics term once in plain language (no Bayesian / variance-decomposition / Pareto jargon
unless asked). For ds: compact optimization and statistical terms are fine.
- Presenting options: show at most 3, mark exactly one Recommended, and give one short
persona-appropriate trade-off per option.
- Autonomy. For
delegate or execute: pick the recommended reversible action and proceed, asking
only at hard gates. For guided: offer options with a recommendation at the key decisions. For
inspect or explore: give brief rationale or evidence before asking, and ask before branch
choices.
- Hard gates — always confirm regardless of persona: paid or provider model calls, sending data or
private content off the machine, destructive edits, decisions the Traigent service is meant to
return, and any missing fact the step truly requires.
- Always end by recommending the next Traigent skill or action to take.
- Never weaken Traigent safety: dry-run before any paid run; get explicit approval before real cost
or before any data leaves the machine; treat service-returned plans and next steps as
authoritative. Never put the persona profile or any private content into telemetry, run metadata,
experiment names, logs, or provenance files.
