traigent-analyze-results

name	traigent-analyze-results
description	Analyze and report Traigent optimization results from the terminal — without opening the portal's tabs. Use when a user asks to analyze a run, 'how did my run do?', 'analyze my latest run in project X', what the winner is, or to read result fields, reports, leaderboards, Pareto trade-offs, correlations, or parameter/example insights. Decision questions route to `traigent-analyze-guidance` for portal-tracked runs and `traigent-analyze-guidance` for offline/local runs. Also covers the local OptimizationResult object: reading results.best_config, comparing trials, checking stop_reason, calling apply_best_config(), accessing total_cost or total_tokens, or understanding why optimization stopped.
license	Apache-2.0
metadata	{"author":"Nimrod","version":"1.1.12"}

Analyzing Traigent Optimization Results

This skill makes optimization-result analysis terminal-first. Instead of navigating the portal's many tabs, ask in plain language ("how did my latest run in project X do?") and the skill calls the traigent-analytics MCP server, then narrates the answer: a headline, a confidence label, a few evidence bullets, and a portal deep-link as a fallback. It surfaces a deeper view (Pareto, leaderboard, correlation, parameter/example insights) only when the brief's evidence/action calls for it or you ask.

There are two surfaces, and this skill covers both:

• Cloud / portal run (terminal-first, the default below). A run that lives in the Traigent cloud — analyzed through the traigent-analytics MCP tools.
• In-process OptimizationResult object. The value optimize_sync() / await optimize() returns in your own Python process — analyzed field-by-field (the "Working with the local OptimizationResult" section).

When to Use

Use this skill when you want to understand a finished run. This covers:

• "Analyze my latest run in project X" / "how did my run do?"
• Getting the one-line verdict, confidence, and evidence summary for a run
• Pulling a focused drilldown (Pareto, leaderboard, correlations, parameter- or example-insights) directly via its registered tool, with the portal deep-link for interactive exploration
• Reading the best configuration and score
• Comparing individual trial results
• Understanding why optimization stopped (stop reasons)
• Checking cost and token usage
• Applying the best configuration for production use
• Reviewing optimization history across multiple runs

Terminal-First Analysis (MCP)

When the run lives in the Traigent cloud/portal, drive the analysis through the traigent-analytics MCP server. This skill orchestrates and narrates; it does not compute analytics and does not do any auth or tenant logic — the MCP server resolves the caller's tenant from the authenticated session and returns backend-produced analytics payloads. Treat every tool response as authoritative and never invent fields, numbers, rankings, or charts.

1. Collect explicit project + run context

Before calling anything, pin down which project and which run. Never assume a global "latest" — "latest" is only meaningful inside one project.

• If the user named a project but not a run, ask for the run (or confirm "the most recent run in project X" explicitly with them) before proceeding.
• If neither is given, ask which project first.

Keep these as opaque ids the user provides; the skill does not guess or enumerate them.

If the portal shows several optimization runs together as a cohort/group, treat that as browsing help only. The group is formed from source ids for runs that share the same agent and canonical dataset; it does not merge configurations, dedupe by tuned variables/objectives/config hashes, or create one analytics run. Analytics stay scoped to one explicit run_id, or to an explicit run_ids list when the user asks to compare runs (see the Multi-Run View section for the source-preserving cohort table).

2. Call the analytics brief first (progressive disclosure)

The keystone tool returns an already-computed backend brief. Call it first and lead with its headline, confidence, evidence, and any backend-reported action fields — do not open a drilldown yet.

analytics_get_run_decision_brief(
    project_id = "<the project the user named>",
    run_id     = "<the run the user named>",
    intent     = "iterate" | "deploy" | "debug" | "report",
)

Use deploy for reading deployment-relevant result fields, debug for "why is it stuck?", report for a summary/report request, and iterate only when you are inspecting the backend's analysis payload without deciding the next run.

Decision questions are out of scope for this read-only analysis skill. For portal-tracked runs, route open-ended next-step decisions to traigent-analyze-guidance (traigent next-steps RUN_ID --json); for offline/local runs or unavailable service payloads, route them to traigent-analyze-guidance.

The tool returns an ok flag and a decision_brief object. Narrate the brief in this order:

1. Headline — the one-sentence verdict, in plain language.
2. Confidence — the brief's confidence. Never upgrade a low/medium confidence into a stronger claim.
3. Evidence — the summary strings from the brief's evidence list.
4. Backend-reported action fields — recommended_action.kind, optional config_id, and why, if present. Report these fields as analysis output; do not turn them into this skill's decision.
5. Fallback — build a navigation-only portal link: https://portal.traigent.ai/p/<project_id>/runs/<run_id>.

3. Pull one drilldown only when the brief or user asks

The brief may include drilldowns. The single-run drilldown fetchers are registered (SDK >= 0.18.0.dev0), so a drilldowns[].tool that names one of the registered tools below can be called directly. Pull at most one extra tool per turn, and only when the user asks or the brief clearly calls for it. Use the portal deep-link for interactive exploration or for any view that has no registered tool.

The analytics tools this skill may call are:

• analytics_get_run_decision_brief(project_id, run_id, intent="iterate")
• analytics_get_run_report(project_id, run_id)
• analytics_get_project_overview(project_id)
• analytics_compare_runs(project_id, run_ids)
• analytics_list_experiment_groups(project_id, agent_id=None, dataset_id=None) — requires an SDK build that exposes the experiment-group analytics tools; if absent, fall back to analytics_compare_runs for explicit run_ids or the portal.
• analytics_get_experiment_group(project_id, group_id) — same experiment-group tool gate.
• analytics_list_experiment_group_configuration_runs(project_id, group_id) — same experiment-group tool gate.
• analytics_get_single_run_pareto(project_id, run_id, x_measure="cost", y_measure="quality", request_count=1)
• analytics_get_correlation_matrix(project_id, run_id, method="pearson", min_sample=3)
• analytics_get_run_leaderboard(project_id, run_id, objective="weighted", weights=None, constraints=None, request_count=1, limit=50)
• analytics_get_parameter_insights(project_id, run_id, target_measure="quality", min_trials=10, top_k=10)
• analytics_get_example_insights(project_id, run_id)
• analytics_render_chart(payload, kind, output_path=None) with kind in {run_pareto, run_correlations}

Do not call any other analytics tool name. The render tool does not fetch or compute a drilldown; it renders an already-fetched backend payload. Call it only when you already have a backend-produced run_pareto or run_correlations object from a registered tool response. If the payload is absent, use the portal deep-link instead.

Symptom / requested view	First surface (only if triggered / asked)	Reported signal / handoff
Clean winner	(none — headline is enough)	Report the winner and route promotion decisions to traigent-analyze-guidance or traigent-ci-safety-gate
Expensive winner / Pareto trade-off	analytics_get_single_run_pareto, then analytics_render_chart with kind="run_pareto" to draw it	Report the trade-off; route operating-point decisions to traigent-analyze-guidance for portal runs
Dominated winner / leaderboard	analytics_get_run_leaderboard	Report the dominating config; route the next-step decision to traigent-analyze-guidance
Low trials	(none — state low confidence)	Report low confidence; route more-trials decisions to traigent-analyze-guidance or traigent-analyze-guidance for offline/local runs
One knob dominates	analytics_get_parameter_insights	Report the dominant knob; route space changes to traigent-analyze-guidance or offline/local diagnosis to traigent-analyze-guidance
Flat scores	analytics_get_parameter_insights	Report flatness; route dataset/space decisions to traigent-analyze-guidance or offline/local diagnosis to traigent-analyze-guidance
Noisy examples	analytics_get_example_insights (safe projection)	Report the safe projection; route evaluator/data changes to traigent-analyze-guidance or offline/local diagnosis to traigent-analyze-guidance
Cost blowup	analytics_get_single_run_pareto (+ render kind="run_pareto")	Report cost evidence; route budget/guardrail decisions to traigent-analyze-guidance or traigent-ci-safety-gate

For the full tool contract (every tool's arguments and response shape and the geometry-vs-words rule), see references/mcp-analytics-tools.md. For choosing the next experiment once analysis names the problem, hand off portal-tracked runs to traigent-analyze-guidance and offline/local runs to traigent-analyze-guidance.

4. Multi-Run View (Cohort Table)

When the user asks for multi-run, history, "across my runs", or cohort results for one agent+dataset, use the experiment-group view. Requires an SDK build that exposes the experiment-group analytics tools; if they are absent, fall back to analytics_compare_runs for explicit run_ids or the portal.

Call the cohort tools in this order:

analytics_list_experiment_groups(project_id, agent_id=None, dataset_id=None)
analytics_get_experiment_group(project_id, group_id)
analytics_list_experiment_group_configuration_runs(project_id, group_id)

Each cohort tool is a thin reader over a read-only backend endpoint (viewer role, paginated where it lists):

• analytics_list_experiment_groups / GET /api/v1/experiment-groups — list cohorts, optionally filtered by agent_id and dataset_id query params.
• analytics_get_experiment_group / GET /api/v1/experiment-groups/{group_id} — one cohort's summary (404 when the group is not visible in your scope).
• analytics_list_experiment_group_configuration_runs / GET /api/v1/experiment-groups/{group_id}/configuration-runs — the cohort's configuration-run rows, source ids preserved.

Present one aggregated table labelled:

grouped by agent+dataset — rows remain individual source runs

Rows are configuration-runs across the cohort's runs. Include these columns when present: experiment_run_id, configuration_run_id, trial_number, key configuration parameters, key measures such as accuracy/score, cost, and latency, status, and timestamp. Keep source ids visible in every row; join on ids and never deduplicate by configuration hash.

This is a presentation aggregation over source rows, not a merged analytics identity. The portal GROUP is browsing help and never a merged analytics run. Per-run analytics still go through the single-run tools above.

Privacy: narrate findings, not raw example values

The example-insights drilldown (analytics_get_example_insights) is registered, but it is privacy-bounded: the backend returns scoring metadata only — coarse counts, cohort labels, redacted example refs, dataset-quality buckets, and templated recommendations. It must never expose proprietary difficulty, informativeness, ambiguity, or latent feature-vector values. Do not request, infer, or print such values, and do not paste raw per-example payloads into the conversation.

Tool availability

Registered (SDK >= 0.18.0.dev0): analytics_get_run_decision_brief, analytics_get_run_report, analytics_get_project_overview, analytics_compare_runs, analytics_get_single_run_pareto, analytics_get_correlation_matrix, analytics_get_run_leaderboard, analytics_get_parameter_insights, analytics_get_example_insights, and analytics_render_chart.

Experiment-group tools: analytics_list_experiment_groups, analytics_get_experiment_group, and analytics_list_experiment_group_configuration_runs. Requires an SDK build that exposes the experiment-group analytics tools; if absent, fall back to analytics_compare_runs for explicit run_ids or the portal.

The portal deep-link is a fallback for interactive exploration (hover / zoom / filter) or any view without a registered tool. Treat every tool response as authoritative; never fabricate output, charts, rankings, or field behavior, and do not call an unlisted analytics tool name.

Working with the local OptimizationResult

The rest of this skill covers the in-process OptimizationResult object returned by optimize_sync() (or await optimize()) when you analyze a run inside your own Python program rather than from the cloud. The same outcomes — best config, trials, stop reason, cost — read directly off the returned object.

Quick Results

After running optimization, the OptimizationResult object provides immediate access to the key outcomes:

import traigent

@traigent.optimize(
    eval_dataset="eval_data.jsonl",
    configuration_space={"model": ["gpt-4o-mini", "gpt-4o"], "temperature": [0.0, 0.5, 1.0]},
    objectives=["accuracy"],
    max_trials=10,
)
def classify(text):
    config = traigent.get_config()
    # ... LLM call using config ...
    return result

results = classify.optimize_sync()

# Top-level results
print(results.best_config)      # {"model": "gpt-4o", "temperature": 0.0}
print(results.best_score)       # 0.92 (float or None if no eligible trial)
print(results.stop_reason)      # "max_trials_reached"
print(results.duration)         # 45.3 (seconds, wall-clock)
print(results.status)           # OptimizationStatus.COMPLETED
print(results.algorithm)        # Name of optimization algorithm used
print(results.optimization_id)  # Unique ID for this run
print(results.objectives)       # ["accuracy"]
print(results.timestamp)        # datetime when optimization completed

Identifying a run in the portal. Runs are labeled by the experiment_name set on the decorator (@traigent.optimize(experiment_name=...)) or, if omitted, by the access-time TRAIGENT_EXPERIMENT_NAME env var, then the deterministic self-describing default "<func_name>[<obj1>,<obj2>,...][<knob1>,...]", then bare func.__name__ only when no objectives or knobs were registered. The current SDK has no tags/metadata argument. To make runs easy to find later, give each one a descriptive experiment_name before you run it. See traigent-setup-decorator -> "Naming and Labeling Runs".

best_score is None when no trial produced a valid score (e.g., all trials failed). Always check before comparing:

if results.best_score is not None:
    print(f"Best accuracy: {results.best_score:.2%}")
else:
    print("No successful trials produced a score")

Reading Trials

Each trial in results.trials is a TrialResult with full details about what happened:

for trial in results.trials:
    print(f"Trial {trial.trial_id}")
    print(f"  Config: {trial.config}")
    print(f"  Status: {trial.status}")        # TrialStatus enum
    print(f"  Duration: {trial.duration:.1f}s")
    print(f"  Metrics: {trial.metrics}")       # {"accuracy": 0.85, "latency": 1.2}
    print(f"  Successful: {trial.is_successful}")
    print(f"  Timestamp: {trial.timestamp}")

    # Safe metric access with default
    accuracy = trial.get_metric("accuracy", default=0.0)
    latency = trial.get_metric("latency", default=None)

    # Check for errors
    if trial.error_message:
        print(f"  Error: {trial.error_message}")

    # Trial metadata (additional context)
    if trial.metadata:
        print(f"  Metadata: {trial.metadata}")

Filtering Trials

Use the built-in properties to filter trials by outcome:

# Only successful trials
for trial in results.successful_trials:
    print(f"{trial.config} -> accuracy={trial.get_metric('accuracy')}")

# Only failed trials
for trial in results.failed_trials:
    print(f"FAILED: {trial.config} -> {trial.error_message}")

# Success rate
print(f"Success rate: {results.success_rate:.0%}")
# e.g., "Success rate: 80%"

Comparing Trial Configurations

Find which configuration parameters matter most:

# Sort trials by a specific metric
sorted_trials = sorted(
    results.successful_trials,
    key=lambda t: t.get_metric("accuracy", 0.0),
    reverse=True,
)

# Show top 3
for i, trial in enumerate(sorted_trials[:3], 1):
    print(f"#{i}: accuracy={trial.get_metric('accuracy'):.3f} config={trial.config}")

# Compare best vs worst
if len(sorted_trials) >= 2:
    best = sorted_trials[0]
    worst = sorted_trials[-1]
    for key in best.config:
        if best.config[key] != worst.config[key]:
            print(f"  {key}: best={best.config[key]}, worst={worst.config[key]}")

Configuration Insights

Use get_optimization_insights(results) for a first structured pass over top configurations, performance summary, parameter insights, and recommendations. Treat it as analysis input; deciding the next experiment belongs in traigent-analyze-guidance for portal-tracked runs or traigent-analyze-guidance for offline/local runs.

from traigent.utils.insights import get_optimization_insights

insights = get_optimization_insights(results)
print(insights.get("top_configurations", []))
print(insights.get("performance_summary", {}))
print(insights.get("parameter_insights", {}))
print(insights.get("recommendations", []))

Cost and Performance

Track what the optimization run cost in API spend and tokens:

# Total cost across all trials (None if not tracked)
if results.total_cost is not None:
    print(f"Total cost: ${results.total_cost:.4f}")

# Total tokens consumed (None if not tracked)
if results.total_tokens is not None:
    print(f"Total tokens: {results.total_tokens:,}")

# Aggregated experiment statistics
stats = results.experiment_stats
print(f"Total duration: {stats.total_duration:.1f}s")
print(f"Total cost: ${stats.total_cost:.4f}")
print(f"Unique configurations tested: {stats.unique_configurations}")
print(f"Average trial duration: {stats.average_trial_duration:.1f}s")
print(f"Cost per configuration: ${stats.cost_per_configuration:.4f}")
print(f"Trial counts: {stats.trial_counts}")
# Trial counts: {"total": 10, "completed": 8, "failed": 2, ...}

# Best metrics from the winning trial
print(f"Best metrics: {results.best_metrics}")
# {"accuracy": 0.92, "latency": 0.8}

None means not tracked, not local. results.total_cost / total_tokens are aggregated locally from per-trial metrics and read None only when no positive cost was captured (mock/offline runs, unpriced custom models — see traigent-optimize-run → Cost Wiring Probe). A real paid run — local or portal-tracked — should show a positive total_cost; treat None/0.0 with real calls as cost not wired, never as "expected for a local run". Per-trial: trial.get_metric("total_cost") is the trial total ("cost" is the per-example mean).

The Quality / Cost / Latency Trade-off (multi-objective)

After a multi-objective run (objectives=["accuracy", "cost"]), the single best_score no longer tells the whole story — you want the trade-off set (the Pareto frontier): the configurations where you cannot improve one objective without sacrificing another.

Get one aggregated row per configuration with to_aggregated_dataframe() (groups repeated samples of the same config and averages each metric), then filter to the non-dominated set:

df = results.to_aggregated_dataframe(primary_objective="accuracy")
# One row per unique config. Columns: config params + samples_count + each metric as its
# mean under its BARE name (e.g. "accuracy", "cost") + "duration" (mean seconds).
print(df.columns.tolist())  # confirm the exact metric column names for your run

# Non-dominated (Pareto) frontier: maximize accuracy, minimize cost.
def pareto_front(df, maximize="accuracy", minimize="cost"):
    keep = []
    for i, row in df.iterrows():
        dominated = (
            (df[maximize] >= row[maximize]) & (df[minimize] <= row[minimize])
            & ((df[maximize] > row[maximize]) | (df[minimize] < row[minimize]))
        ).any()
        if not dominated:
            keep.append(i)
    return df.loc[keep].sort_values(minimize)

frontier = pareto_front(df)
print(frontier[["accuracy", "cost", "duration"]])  # use your run's actual metric names

Each frontier row is a defensible operating point: pick the cheapest config that clears your accuracy bar, or the most accurate within your cost budget. (results.to_dataframe() gives the raw per-trial rows if you want to plot the full cloud behind the frontier.)

Find Your Run on the Portal

A run that actually reaches the backend syncs to the Traigent portal, where the same trade-off is rendered visually. That requires both offline=False (the default) and valid credentials (TRAIGENT_API_KEY): a run with no key can fall back to local-only execution and then is not portal-tracked. The portal Pareto/frontier view also requires >=2 objectives; a single-objective run shows an "add a second measure" hint there, not a blank frontier. To locate a synced run:

# The portal/backend identifiers (None when offline or local-fallback):
print(f"Portal experiment: {results.experiment_id}")  # backend experiment identifier
print(f"Portal link:       {results.cloud_url}")      # direct URL to the experiment on the portal
# (results.optimization_id is the SDK's local run id, not the portal identifier.)
# Open results.cloud_url, or go to https://portal.traigent.ai -> Experiments and find this run by
# its experiment_id (or resolved experiment_name) to read the rendered view.

An offline=True run, or a non-offline run that fell back to local (no key), is not on the portal — use the dataframe read above instead.

Verify the Run Actually Persisted (persistence_status)

A portal-tracked run (non-offline, experiment_id set) can finish all its trials locally but still fail to finalize on the backend — e.g. a network blip or backend 5xx during the final session-close call. The SDK retries that finalize call (3 attempts, exponential backoff) before giving up; if every attempt fails, it does not pretend the run is fine:

if results.metadata.get("persistence_status") == "failed":
    print("Backend finalize failed after retries — the portal session may be stuck RUNNING.")
    print(results.metadata.get("persistence_error"))  # the underlying exception, if any

When this is "failed", do not assume the run synced — the local OptimizationResult can look complete while the backend session is left RUNNING on the portal. Re-check the run on the portal (or run traigent local sync) before reporting a portal-tracked result as final. (Newer SDK builds — the fix merged as Traigent#1731 — also expose this as results.persistence_failed, a bool shorthand for the same check; check the metadata key directly if your installed SDK predates it.)

The portal may group runs that share the same agent and canonical dataset. Use that group only to find related source runs. Before applying or recommending a configuration, record the exact source experiment_id, experiment_run_id, and configuration_run_id shown by the portal/API. A grouped view does not make grouped configurations a single analytics run, and it does not imply equivalent configs have been merged across runs.

Stop Reasons

The stop_reason field tells you why optimization ended. This is critical for deciding whether to run more trials:

Stop Reason	Meaning	Action
"max_trials_reached"	Hit the max_trials limit	Increase max_trials if results are still improving
"max_samples_reached"	Hit the max samples/examples limit	Increase sample budget or reduce dataset size
"timeout"	Exceeded the timeout duration	Increase timeout or reduce config space
"cost_limit"	Hit the cost budget limit	Increase cost_limit or use cheaper models
"optimizer"	Optimizer decided to stop (search space exhausted)	Config space fully explored; results are final
"plateau"	No improvement detected	Results have converged; more trials unlikely to help
"user_cancelled"	User cancelled or declined cost approval	Review cost estimates, re-run if needed
"condition"	A generic stop condition triggered	Check convergence_info for details
"error"	Optimization failed due to an exception	Check failed trials for error messages
"vendor_error"	Provider error (rate limit, quota, service issue)	Check API keys, quotas, and provider status
"network_error"	Connectivity failure	Check network connection and retry
None	Stop reason not set	Typically means the run completed normally

if results.stop_reason == "max_trials_reached":
    print("Consider increasing max_trials for better results")
elif results.stop_reason == "plateau":
    print("Optimization converged - these are likely the best results")
elif results.stop_reason == "cost_limit":
    print(f"Budget exhausted at ${results.total_cost:.2f}")
elif results.stop_reason == "error":
    for trial in results.failed_trials:
        print(f"Error in trial {trial.trial_id}: {trial.error_message}")

Applying Best Config

After optimization, apply the winning configuration so your function uses it in production:

# Run optimization
results = classify.optimize_sync()

# Apply the best configuration
classify.apply_best_config(results)

# Now every call uses the optimized config
# traigent.get_config() inside the function returns results.best_config
response = classify("What category is this email?")

apply_best_config() sets the configuration so that subsequent calls to traigent.get_config() inside the decorated function return the best configuration from the optimization run. The applied config is also readable from outside the function via func.current_config on the OptimizedFunction instance:

classify.apply_best_config(results)
print(classify.current_config)  # {"model": "gpt-4o", "temperature": 0.5}

Config Access Lifecycle

When	API	Notes
During optimization trials	traigent.get_config()	Returns current trial config. Thread-safe via contextvars.
During optimization trials (strict)	traigent.get_trial_config()	Raises OptimizationStateError if not in active trial.
After apply_best_config()	traigent.get_config()	Returns the applied best config.
From optimization results	results.best_config	Dict with the best configuration found.
From the function object	func.current_config	Current config on the OptimizedFunction instance.

Safety Check Before Applying

Verify results before applying:

results = classify.optimize_sync()

if results.best_score is not None and results.best_score >= 0.85:
    classify.apply_best_config(results)
    print(f"Applied config with score {results.best_score:.2%}")
else:
    print(f"Score {results.best_score} below threshold, not applying")
    # Use a known-good default instead

This threshold check runs on the optimization/search slice — it gates whether to apply, not whether to promote. Promotion is a separate decision that requires candidate-vs-incumbent evaluation on the holdout slice (see traigent-ci-safety-gate).

Optimization History

Review results from previous optimization runs on the same function:

history = classify.get_optimization_history()

for past_result in history:
    print(f"Run {past_result.optimization_id}")
    print(f"  Algorithm: {past_result.algorithm}")
    print(f"  Best score: {past_result.best_score}")
    print(f"  Best config: {past_result.best_config}")
    print(f"  Trials: {len(past_result.trials)}")
    print(f"  Duration: {past_result.duration:.1f}s")
    print(f"  Stop reason: {past_result.stop_reason}")
    print(f"  Timestamp: {past_result.timestamp}")

Compare across runs to see if optimization is improving over time:

history = classify.get_optimization_history()
if len(history) >= 2:
    latest = history[-1]
    previous = history[-2]
    if latest.best_score is not None and previous.best_score is not None:
        improvement = latest.best_score - previous.best_score
        print(f"Improvement: {improvement:+.3f}")

Complete Example

End-to-end workflow: optimize, analyze, decide, apply.

import traigent

@traigent.optimize(
    eval_dataset="summarization_eval.jsonl",
    configuration_space={
        "model": ["gpt-4o-mini", "gpt-4o", "gpt-4-turbo"],
        "temperature": [0.0, 0.3, 0.7],
        "max_tokens": [256, 512, 1024],
    },
    objectives=["accuracy"],
    max_trials=15,
)
def summarize(text):
    config = traigent.get_config()
    # ... your LLM summarization logic ...
    return summary

# 1. Run optimization
results = summarize.optimize_sync()

# 2. Quick summary
print(f"Best config: {results.best_config}")
print(f"Best score: {results.best_score}")
print(f"Stop reason: {results.stop_reason}")
print(f"Duration: {results.duration:.1f}s")
print(f"Success rate: {results.success_rate:.0%}")

# 3. Cost analysis
if results.total_cost is not None:
    print(f"Total cost: ${results.total_cost:.4f}")
if results.total_tokens is not None:
    print(f"Total tokens: {results.total_tokens:,}")

# 4. Trial breakdown
print(f"\nTop 5 trials by accuracy:")
top_trials = sorted(
    results.successful_trials,
    key=lambda t: t.get_metric("accuracy", 0.0),
    reverse=True,
)[:5]
for trial in top_trials:
    print(f"  {trial.config} -> accuracy={trial.get_metric('accuracy'):.3f}")

# 5. Convergence check
if results.stop_reason == "plateau":
    print("\nOptimization converged naturally")
elif results.stop_reason == "max_trials_reached":
    print("\nMay benefit from more trials")

# 6. Apply if good enough
THRESHOLD = 0.80
if results.best_score is not None and results.best_score >= THRESHOLD:
    summarize.apply_best_config(results)
    print(f"\nApplied best config (score={results.best_score:.2%})")

    # Production usage
    output = summarize("Summarize this quarterly earnings report...")
else:
    print(f"\nScore {results.best_score} below threshold {THRESHOLD}, skipping apply")

Reference Files

• Terminal-First Analytics — MCP Tool Contract
• OptimizationResult and TrialResult API Reference
• Convergence Analysis Patterns

See Also

Skill	Use
traigent-analyze-guidance	Get the canonical next-step decision for a portal-tracked run.
traigent-analyze-guidance	Form a local next-iteration hypothesis for offline/local runs, unavailable service payloads, or service-flagged local evidence.
traigent-analyze-variable-importance	A deeper, local tuned-variable importance card when one_knob_dominates and you want the bootstrap-CI breakdown.
traigent-ci-safety-gate	Gate a clean_winner (or a cost guardrail for cost_blowup) before promoting it to production.

Traigent Interaction Policy

Track an interaction profile and adapt to it. Persona (stable): control=delegate|guided|inspect, expertise=se|ds|unknown. Mood (this session): pace=execute|balanced|explore. Default when unknown: guided,se,balanced. Infer from explicit user statements first, then recent behavior; an explicit correction wins immediately. Never store or send this profile anywhere by default.

Fetch the live profile (when available)

At session or skill start, if a configured Traigent client is available, seed the profile from the backend with the skill name:

policy = None
try: policy = await client.get_interaction_policy(skill="<this skill>")
except Exception: pass

Treat the returned profile as the STARTING seed: its control/expertise/pace axes plus question_budget, options_max, and jargon_level replace the static defaults below. Explicit user corrections in-conversation ALWAYS override the seed. If the call is unavailable or fallback_policy="static_v1", simply use the static defaults below; the SDK already fails soft.

• Always be concise.
• Match terminology to expertise. For se: plain engineering words; define each Traigent or statistics term once in plain language (no Bayesian / variance-decomposition / Pareto jargon unless asked). For ds: compact optimization and statistical terms are fine.
• Presenting options: show at most 3, mark exactly one Recommended, and give one short persona-appropriate trade-off per option.
• Autonomy. For delegate or execute: pick the recommended reversible action and proceed, asking only at hard gates. For guided: offer options with a recommendation at the key decisions. For inspect or explore: give brief rationale or evidence before asking, and ask before branch choices.
• Hard gates — always confirm regardless of persona: paid or provider model calls, sending data or private content off the machine, destructive edits, decisions the Traigent service is meant to return, and any missing fact the step truly requires.
• Always end by recommending the next Traigent skill or action to take.
• Never weaken Traigent safety: dry-run before any paid run; get explicit approval before real cost or before any data leaves the machine; treat service-returned plans and next steps as authoritative. Never put the persona profile or any private content into telemetry, run metadata, experiment names, logs, or provenance files.