with one click
llm
Guidelines for implementing LLM (Language Model) functionality in the application
Menu
Guidelines for implementing LLM (Language Model) functionality in the application
| name | llm |
| description | Guidelines for implementing LLM (Language Model) functionality in the application |
LLM-related code is organized in specific directories:
apps/web/utils/ai/ - Main LLM implementationsapps/web/utils/llms/ - Core LLM utilities and configurationsapps/web/__tests__/ - LLM-specific testsutils/llms/index.ts - Core LLM functionalityutils/llms/model.ts - Model definitions and configurationsutils/llms/use-cases.ts - Product use-case to model-role routingutils/usage.ts - Usage tracking and monitoringFor product features with a static model choice, use getModelForUseCase(emailAccount.user, LlmUseCase.FeatureName) from utils/llms/use-cases.ts. Keep direct getModel(user, modelType) calls for generic helpers where the model role is intentionally passed from upstream. When adding or changing a use case, update utils/llms/use-cases.test.ts.
Follow this standard structure for LLM-related functions:
import { z } from "zod";
import { createScopedLogger } from "@/utils/logger";
import { chatCompletionObject } from "@/utils/llms";
import type { EmailAccountWithAI } from "@/utils/llms/types";
import { createGenerateObject } from "@/utils/llms";
import { getModelForUseCase, LlmUseCase } from "@/utils/llms/use-cases";
export async function featureFunction(options: {
inputData: InputType;
emailAccount: EmailAccountWithAI;
}) {
const { inputData, user } = options;
if (!inputData || [other validation conditions]) {
logger.warn("Invalid input for feature function");
return null;
}
const system = `[Detailed system prompt that defines the LLM's role and task]`;
const prompt = `[User prompt with context and specific instructions]
<data>
...
</data>
${emailAccount.about ? `<user_info>${emailAccount.about}</user_info>` : ""}`;
const modelOptions = getModelForUseCase(
emailAccount.user,
LlmUseCase.FeatureName,
);
const generateObject = createGenerateObject({
userEmail: emailAccount.email,
label: "Feature Name",
modelOptions,
});
const result = await generateObject({
...modelOptions,
system,
prompt,
schema: z.object({
field1: z.string(),
field2: z.number(),
nested: z.object({
subfield: z.string(),
}),
array_field: z.array(z.string()),
}),
});
return result.object;
}
System and User Prompts:
Schema Validation:
Logging:
Error Handling:
withRetryInput Formatting:
Type Safety:
Code Organization:
AI-First Behavior:
Draft Attribution Versioning:
apps/web/utils/ai/reply/draft-attribution.ts DRAFT_PIPELINE_VERSIONSee llm-test.mdc
Cursor Cloud VM setup and service startup instructions for local development
Guidelines for testing the application with Vitest, including unit tests, integration tests (emulator), AI tests, and eval suites for LLM features
Simplify and refine recently modified code for clarity, consistency, and maintainability while preserving exact behavior. Use when asked to simplify, polish, refactor lightly, or clean up current-session changes before review or PR.
Commit changes and open a pull request with safe metadata
Review prompt and tool-description changes in a branch or PR. Use when asked to audit system prompts, prompt builders, tool descriptions, prompt inputs, or nearby evals/tests; summarize what changed; identify guidance that feels eval-shaped or too heavy for product use; and find repeated instructions between the main prompt and tool descriptions.
Add a new changelog entry to docs/changelog-entries/