latitude-telemetry

name

latitude-telemetry

description

Add or review Latitude Telemetry for LLM apps. Use for Latitude tracing, LLM observability, missing traces, OpenTelemetry/OTLP integration in TypeScript, Python, and other runtimes. Covers instrumenting a new project under an existing Latitude account (invoked directly) as well as the latitude-setup hand-off, discovering config via the Latitude MCP, and verifying that real traces land via the Latitude MCP, CLI, or API.

Latitude Telemetry

Add or review Latitude Telemetry without disrupting existing observability. Latitude is OpenTelemetry-based, so compose with the app's current OTel/Sentry/Datadog setup instead of replacing it.

Entry points

Two ways in; both do the same audit → instrument → verify work:

• Invoked directly — the common case. The user already has a Latitude account and API key and wants to instrument an app: their first project under that account, or an additional one. They may already know the project slug, or you infer it from the repo or discover it via the Latitude MCP/CLI. Source config as described below; do not run the zero-account bootstrap.
• Delegated from latitude-setup — the from-scratch, no-account path. That skill has already provisioned a temporary account and written LATITUDE_API_KEY/LATITUDE_PROJECT_SLUG to .env, so config is in place — skip the MCP discovery detour and go straight to audit → instrument.

Either way, the work isn't done until you have verified that real traces land in Latitude (workflow step 6).

First decision: redirect existing OTLP, or add the SDK?

Before installing anything, determine how the app already emits telemetry — the simplest correct integration is often no new dependency:

• The app already exports OpenTelemetry traces — a generic OTLP exporter, or a GenAI instrumentation like Vercel AI SDK telemetry (experimental_telemetry), OpenInference, Traceloop/OpenLLMetry, or hand-rolled OTel spans? Prefer repointing that exporter at Latitude rather than installing the SDK. It's a config-only change — set the OTLP traces endpoint to https://ingest.latitude.so/v1/traces and add the auth + project headers (see "Other targets → Generic OTLP" below). It preserves the app's existing span conventions and adds zero new instrumentation. Installing the Latitude SDK on top would re-instrument the same model calls and can duplicate or clobber those spans — so don't, unless the redirect can't carry something you need.
• The app has no LLM telemetry yet (or only vendor SDKs with no OTLP export)? Use the Latitude SDK (TypeScript / Python sections below), or attach a LatitudeSpanProcessor to an existing OTel provider.

For an app that already produces spans, the OTLP-redirect path is the lower-risk default. Confirm which case applies during the audit and state the chosen approach in the plan.

Workflow

1. Audit first

   • Identify languages, package managers, entry points, runtimes, deployment config, and env conventions.
   • Check the package registry for the latest Latitude SDK for the target language; use the current alpha if it is the latest release, and do not copy versions from examples.
   • Check for LATITUDE_API_KEY and either LATITUDE_PROJECT_SLUG or per-capture project routing. Look in env files, secret-manager references, deployment config, and CI. If missing, direct the user to add real values in the existing secret/config system; add placeholders only to examples/docs. If there is no Latitude account or key at all and the user wants to start from scratch, defer to the latitude-setup skill to provision one first (see "No account yet?" below).
   • Find existing telemetry: @opentelemetry/*, opentelemetry-*, dd-trace, @sentry/*, sentry-sdk, newrelic, Honeycomb, Jaeger/Tempo/OTLP exporters, LangSmith/Langfuse/Helicone/Phoenix/Traceloop, custom span processors, and OTEL_* env vars. If the app already exports OTLP, prefer redirecting it over installing the SDK (see "First decision" above). Otherwise existing SDKs usually initialize first; Latitude initializes second or attaches a LatitudeSpanProcessor to the existing provider.
   • Find LLM call sites: OpenAI chat/responses, Anthropic messages, Bedrock, Cohere, Together, Vertex/Google AI, Azure OpenAI, Vercel AI SDK generateText/streamText, LangChain, LlamaIndex, OpenAI Agents, LiteLLM, CrewAI, etc. Trace from route/job/CLI/agent entry points to the actual model calls. Note streaming paths; consume streams inside the capture boundary.
   • Note whether the app keeps long-term memory — state it persists and reloads across separate interactions (files, a database, a vector store, a key-value store, or a provider like Mem0/Zep/Supermemory), as distinct from within-request conversation history. If it does, memory observability is a strongly recommended add-on — see "Long-term memory?" below.
   • If the request is not clearly covered here, consult the Latitude docs (https://docs.latitude.so/llms.txt, especially telemetry/*) and/or the telemetry package implementation in github.com/latitude-dev/latitude-llm/packages/telemetry/*.

2. Group use cases

   • Group related prompts, tools, retrieval, and model calls by final goal, not by file.

No account yet? Zero-account CLI bootstrap

If the app has no Latitude account or API key and the user wants to start from scratch, defer to the latitude-setup skill before instrumenting. It uses the latitude CLI to bootstrap a temporary account with no signup — returning an API key, one project slug, and a browser link to claim ownership — writes LATITUDE_API_KEY and LATITUDE_PROJECT_SLUG into .env, then returns here to instrument against that projectSlug.

The bootstrap projectSlug is stable across latitude-setup's delete-and-recreate trace-cleanup step (same project name → same slug), so write LATITUDE_PROJECT_SLUG once; it never needs re-editing for cleanup.

This is a third way to source configuration values, alongside "already present in the app" and "discovered via the Latitude MCP" (below). Prefer whichever already applies, in order: existing values → Latitude MCP discovery (the user already has an account) → CLI bootstrap via latitude-setup (no account yet).

Latitude MCP-assisted configuration

Use this section when adding Latitude telemetry and configuration values are missing or ambiguous. The Latitude MCP is a remote OAuth-authenticated MCP server at https://api.latitude.so/v1/mcp that can expose Latitude workspace data and actions to the agent. It is not required for telemetry, but when it is connected it should be used to reduce user back-and-forth.

• Check MCP availability first. Before asking the user for Latitude project/API-key details, inspect the connected MCP tools/servers available in the current agent harness. If a Latitude MCP server is available and authenticated, use it to discover organization/project metadata and to help prepare telemetry configuration.
• Offer MCP installation as a preliminary clarification, before the plan. If the Latitude MCP is not connected, stop before presenting the implementation plan and ask whether the user wants to install/connect it so the agent can automatically discover projects and help fill configurable telemetry variables. Briefly explain that the Latitude MCP gives the agent OAuth-scoped access to their Latitude workspace, including projects, keys, traces, annotations, scores, searches, issues, datasets, and other Latitude resources; connected agents can be revoked under Settings → Keys → OAuth Keys. Do not install or configure the MCP without explicit approval. Do not bundle this MCP question into the implementation plan or approval request. If the user declines, continue with the normal manual configuration flow and then present the implementation plan.
• Use MCP to fill non-secret config. Prefer MCP-provided project data to identify the correct LATITUDE_PROJECT_SLUG when the user has already indicated, or the repo clearly implies, which Latitude project should receive traces. If multiple plausible projects exist, present the options and ask the user to choose one.
• Use MCP for secret creation/metadata only when safe. If the Latitude MCP exposes API-key management, use it only after user approval and only to create or identify the needed key metadata. Do not print real API key values in chat. Put secrets directly into the project's existing secret manager only when the harness/tooling supports doing so safely; otherwise add placeholders to env examples/docs and tell the user where to store the real value.
• Do not ask for values the MCP can answer. If the MCP can list projects, infer slugs, or confirm existing key names, do that before asking the user. Ask only for decisions MCP cannot know, such as which project should receive traces when ambiguous, whether to create a new API key, or where secrets should be stored.
• Keep MCP separate from app telemetry. MCP helps configure Latitude; it does not trace the target app's LLM calls. The app still needs the telemetry SDK or OTLP exporter configured with LATITUDE_API_KEY and LATITUDE_PROJECT_SLUG or equivalent OTLP headers.

Configuration values

When asking the user to provide config, explain what each value is and where to find it:

• LATITUDE_API_KEY: authenticates uploads to Latitude. Find or create it in Latitude under Settings → API Keys.
• LATITUDE_PROJECT_SLUG: chooses which Latitude project receives traces. In the Latitude app, open the project; the slug appears in the sidebar title section. It is the short project identifier, not the display name.
• Generic OTLP setups encode the same values as an OTLP traces endpoint of https://ingest.latitude.so/v1/traces plus headers Authorization=Bearer <api-key> and X-Latitude-Project=<project-slug>. Set them through whatever mechanism the app already uses — the variable names vary by app/framework (OTel's convention is OTEL_EXPORTER_OTLP_[TRACES_]ENDPOINT / …_HEADERS, but apps may use different names or configure the exporter in code), so match the app rather than assuming a fixed name.
• Quote any .env value that contains spaces. The Latitude CLI reads LATITUDE_API_KEY from .env with a strict parser that stops at the first unquoted spaced value — so an unquoted header/token value (e.g. one containing Bearer ) prevents the CLI from ever reading the key. Wrap such values in double quotes; the quotes are stripped by Node's --env-file and other loaders, so one quoted .env works everywhere. Details in latitude-cli → Authentication.

Never ask for real secret values in chat if the project has an existing secret manager. Ask where the user wants them stored, and add placeholders only to env examples/docs.

TypeScript

Install the latest @latitude-data/telemetry with the project's package manager. Initialize existing Sentry/Datadog/New Relic/Honeycomb/custom OTel first, then Latitude.

import OpenAI from "openai";
import { Latitude } from "@latitude-data/telemetry";

const latitude = new Latitude({
  apiKey: process.env.LATITUDE_API_KEY!,
  project: process.env.LATITUDE_PROJECT_SLUG!,
  instrumentations: { openai: OpenAI },
});

await latitude.ready; // optional, use when first-call coverage matters

Use the project's real env validation; the snippet only shows the SDK shape.

The set of supported instrumentations grows over time — treat any list here as a snapshot, not the source of truth. Look up the current supported keys and the exact per-provider/framework setup for the app's stack in the Latitude docs (https://docs.latitude.so, telemetry/*; llms.txt for an index) or the telemetry package (github.com/latitude-dev/latitude-llm/packages/telemetry/*) rather than relying on memory. Common keys at time of writing include openai, openai-agents, anthropic, bedrock, cohere, langchain, llamaindex, togetherai, vertexai, aiplatform. Pass the same SDK module object the app imports. For Anthropic and most namespace packages, prefer import * as AnthropicSDK from "@anthropic-ai/sdk" then instrumentations: { anthropic: AnthropicSDK }.

Special cases:

• Vercel AI SDK: initialize Latitude without instrumentations; set experimental_telemetry.isEnabled: true on each generateText, streamText, etc. call.
• Custom existing OTel: add new LatitudeSpanProcessor(apiKey, project) beside existing processors and call await registerLatitudeInstrumentations({ instrumentations, tracerProvider }).

Use capture(name, async () => { ... }, { userId, sessionId, tags, metadata, project }) at use-case boundaries. project overrides the constructor default for multi-project routing. capture() adds context to instrumented spans; it does not create LLM spans by itself.

For short-lived scripts/jobs, call await latitude.flush() or await latitude.shutdown() before exit. Do not call shutdown() per request in long-lived servers.

Python

Requires Python 3.11+. Install the latest latitude-telemetry with the project's package manager.

import os
import openai
from latitude_telemetry import Latitude

latitude = Latitude(
    api_key=os.environ["LATITUDE_API_KEY"],
    project=os.environ["LATITUDE_PROJECT_SLUG"],
    instrumentations={"openai": openai},
)

If an OpenTelemetry provider is already registered, Latitude(...) attaches to it. For custom setups, add LatitudeSpanProcessor to the existing provider and call register_latitude_instrumentations(instrumentations={...}, tracer_provider=provider).

As with TypeScript, the supported set changes — verify against the docs, don't trust this list. Check the current keys and per-provider examples for the app's stack in the Latitude docs (https://docs.latitude.so, telemetry/*) or the telemetry package. Common keys at time of writing include openai, openai-agents, anthropic, bedrock, cohere, langchain, llamaindex, togetherai, vertexai, aiplatform, plus Python-only ones such as aleph_alpha, crewai, dspy, google_generativeai, groq, haystack, litellm, mistralai, ollama, replicate, sagemaker, transformers, watsonx. Pass imported module objects, not string lists.

Use capture() as a wrapper with snake_case options, especially when context is per request:

from latitude_telemetry import capture

def run_agent(user_id: str, session_id: str):
    return capture(
        "support-agent-run",
        lambda: agent.run(),
        {"user_id": user_id, "session_id": session_id, "tags": ["support"]},
    )

For short-lived processes, call latitude.flush() or latitude.shutdown() before exit. Do not call shutdown() per request in long-lived services.

Other targets

• Generic OTLP / other languages: send traces to https://ingest.latitude.so/v1/traces with Authorization: Bearer <LATITUDE_API_KEY> and X-Latitude-Project: <LATITUDE_PROJECT_SLUG>. For full model/token/message details, ensure LLM spans follow OpenTelemetry GenAI semantic conventions (gen_ai.* attributes).
• Coding-agent / harness telemetry (Claude Code, OpenClaw, Pi, Hermes, …): this is separate from app SDK instrumentation — it traces an agent harness's own prompts/responses/tool I/O, not the target app's LLM calls. Keep the two separate, and ask before installing any hook/plugin, since prompts, responses, and tool I/O can be sent to Latitude. Offer --no-content (structural-only: timing, tokens, model/tool names, no content) where the integration supports it. New harnesses are added over time — check the docs (https://docs.latitude.so, telemetry/*) or packages/telemetry/* for the current set and exact flags. At time of writing:
  • Claude Code: npx -y @latitude-data/claude-code-telemetry install (full-content only).
  • OpenClaw: npx -y @latitude-data/openclaw-telemetry-cli install (supports --no-content).
  • Pi: npx -y @latitude-data/pi-telemetry install (supports --no-content).
  • Hermes (Nous Research's Python harness): pip install latitude-telemetry-hermes into the same Python that runs Hermes, then enable it by adding latitude to plugins.enabled in ~/.hermes/config.yaml (not via hermes plugins enable).

Long-term memory? Add memory observability

If the app has long-term memory — state the agent persists and reloads across separate interactions — it is highly recommended to also instrument its memory operations. That unlocks Latitude's Memory page: each store's contents, per-record change history and diffs, the tokens read and written per session, and the session behind every write. Memory operations ride this same telemetry pipeline, so there is no extra account or exporter setup — only memory-specific spans to add.

• What counts as long-term memory. State the agent writes down to use later and reads back in on a subsequent interaction — files (a memory/ dir, per-user notes), a memory-shaped database table it populates and queries, a vector store the agent both writes and semantically searches for recall, a key-value/Redis store of durable per-user state, or a provider like Mem0/Zep/Supermemory. It persists across sessions, runs, or users.
• What is NOT long-term memory (do not instrument as memory): the conversation/message history within a single request or session (that is the trace itself), prompt context and few-shot examples, a pure performance cache, and a read-only reference corpus the agent only queries and never writes (that is retrieval, not memory).
• How to tell: you already explored the codebase during the audit — look for a persistence boundary the agent crosses on its own initiative (functions like remember/recall/store/retrieve/upsert, a memory/ module, a vector-store client the agent both writes and queries, or a memory-provider SDK import).
• If it has long-term memory, read memory.md (bundled with this skill) once base tracing is verified, and follow it — it covers the store/record model, where and when to emit, the memory SDK helpers, and raw gen_ai.memory.* spans. If it does not, do nothing extra; there is no memory to observe.