foundation-models

name

foundation-models

description

Apple's on-device Foundation Models framework — the ~3B-parameter system language model in iOS/iPadOS/macOS/visionOS 26+. Covers LanguageModelSession, @Generable / @Guide constrained decoding, tool calling, streaming, the 4096-token context budget, availability checks, the 9-case GenerationError set, guardrails / permissive mode, custom LoRA adapters, Instruments profiling, and crisis triage. Use when building or debugging any on-device LLM feature on Apple platforms. Triggers on: "Foundation Models", "LanguageModelSession", "SystemLanguageModel", "@Generable", "@Guide", "on-device LLM", "Apple Intelligence model", "guided generation", "FM adapter", "permissiveContentTransformations".

Foundation Models — On-Device LLM for Apple Platforms

Apple's Foundation Models framework exposes the ~3-billion-parameter on-device language model that powers Apple Intelligence. It is optimized for summarization, extraction, classification, short dialog, content tagging, and structured generation — not world knowledge, not complex reasoning, not math. It runs entirely on-device: no network, no per-request cost, no API key, no data leaving the device.

• Platforms: iOS 26+, iPadOS 26+, macOS 26+, visionOS 26+ (NOT tvOS/watchOS — guardrails and the model are unavailable there).
• Context window: read it at runtime via SystemLanguageModel.default.contextSize. Current on-device base = 4096 tokens TOTAL (instructions + schema + transcript + prompt + output).
• Devices: iPhone 15 Pro or later, iPad with M1+, Mac with Apple silicon, plus the user opted into Apple Intelligence in a supported region.
• Import: import FoundationModels.

The model version ships with the OS and changes silently on OS updates. There is no public version-pinning API (Apple radar FB18924722). Anything that depends on exact model behavior (greedy-sampling determinism, a trained adapter) must be re-validated each OS minor.

When to use this skill

Use it when you:

• Add on-device summarization / extraction / classification / tagging / short generation.
• Need type-safe structured output from an LLM (@Generable).
• Wire tool calling so the model can fetch live data (weather, contacts, calendar).
• Stream generated content for progressive UI.
• Debug FM issues: context overflow, guardrail violations, slow generation, availability failures, wrong output, frozen UI.
• Decide Foundation Models vs a server LLM (ChatGPT/Claude/Gemini), or Foundation Models vs MLX Swift for custom/vision/embedding models (references/mlx-swift.md).
• Consider — and almost always defer — training a custom adapter.

Exposing your app TO Apple Intelligence (assistant schemas via @AppIntent(schema:), Visual Intelligence image search, on-screen entities for Siri/ChatGPT) is a different topic — that's the app-intents skill, not this one. This skill is the on-device LLM you call; app-intents is how the system's AI calls you. They complement each other.

The 5-rung Approach Triage (read this before reaching for adapters)

Most "the model isn't good enough" complaints are solved at rung 1–3. Apple's explicit guidance: "Before considering adapters, try to get the most out of the system model using prompt engineering or tool calling." Work the ladder in order:

1. Prompt engineering — explicit imperative instructions ("DO X", "DO NOT Y"), a defined role, 2–15 short few-shot examples (overly detailed ones cause repetition/hallucination), and ≥2 conditional paths moved out of the prompt into Swift. Full playbook: references/prompting.md.
2. @Generable + @Guide — constrained decoding gives structural guarantees the prompt alone cannot.
3. Tool calling / RAG — for factual gaps. Hallucination on factual tasks is almost always a context problem, not a model problem.
4. Built-in adapter — SystemLanguageModel(useCase: .contentTagging) beats prompt engineering for tag/entity extraction.
5. Custom LoRA adapter — last resort. ~160 MB per pack, one adapter per base-model OS version, retrained every OS minor, custom locale-specific eval, Background Assets delivery. See references/adapters.md. Do not start here.

Every rung you skip is free quality left on the table.

Worked example — streaming article summarization

The end-to-end happy path — availability check → @Generable + @Guide output type (declare the property you want streamed first FIRST) → snapshot streaming over PartiallyGenerated → catching exceededContextWindowSize / guardrailViolation — is worked through with full code in the "Worked example" section of references/api-reference.md.

Core API at a glance

SystemLanguageModel.default (.availability, .supportedLanguages, .supportsLocale(_:), .contextSize, .tokenCount(for:) iOS 26.4+, .isAvailable) · SystemLanguageModel(useCase:guardrails:) (.general / .contentTagging) · LanguageModelSession (respond(to:), respond(to:generating:), streamResponse(to:generating:), prewarm(); transcript, isResponding) · @Generable (compile-time schema + constrained decoding + PartiallyGenerated streaming mirror) · @Guide (description:, .range, .count, .anyOf, .constant, .pattern(regex), .element) · Tool protocol (Output: PromptRepresentable) · GenerationOptions (sampling / temperature / max tokens) · DynamicGenerationSchema (runtime schemas; the only path for recursive graphs) · SystemLanguageModel.Adapter.

Full signatures, every WWDC code sample, and the exact constraint table: references/api-reference.md.

Instructions vs prompts — the security boundary (load-bearing)

• Instructions come from the developer. They define the model's role, tone, and constraints. The model is trained to prioritize instructions over prompt content — this is a real (though not bulletproof) safety lever.
• Prompts come from the user (or dynamic app state).

NEVER interpolate untrusted user input into instructions. That is prompt injection.

// ❌ WRONG — user input in instructions
let session = LanguageModelSession(instructions: "Summarize: \(userInput)")

// ✅ RIGHT — instructions describe behavior; user input goes in the prompt
let session = LanguageModelSession(instructions: "You summarize text concisely.")
let response = try await session.respond(to: userInput)

Top anti-patterns (each one fails in production)

1. World knowledge / reasoning / math. The 3B model hallucinates facts. Use a server LLM, or feed verified data via a tool.
2. Blocking the main thread. respond() is async and takes 1–5s. Always call it inside Task {}; update UI on MainActor.
3. Manual JSON parsing. Prompting for JSON + JSONDecoder gives hallucinated keys and invalid JSON → keyNotFound crashes. Use @Generable; constrained decoding makes invalid structure impossible.
4. Unbounded multi-turn conversations. The 4096-token budget is TOTAL. Catch exceededContextWindowSize and condense (see Context management).
5. User input in instructions. Prompt injection — see above.
6. struct for stateful tools. Struct copies lose mutations between calls. Use class when a tool tracks state.
7. Over-complex instructions duplicating @Generable. The schema is already enforced at the decoding level. Repeating it in instructions wastes scarce tokens. Use instructions for tone/behavior only.
8. Skipping availability checks. Crashes on unsupported devices, and ~20% of a typical install base is on hardware without Apple Intelligence.

Decision: Foundation Models vs server API

Question	Choice
Privacy required / offline needed / avoid per-request cost?	Foundation Models
Summarization, extraction, classification, tagging, short generation?	Foundation Models
World knowledge, complex reasoning, math, translation?	Server API
Need more than the on-device context budget (4096 tokens)?	Server API

Both can coexist in one app — wrap them behind a protocol (TextSummarizer with on-device, server, and truncation implementations) for graceful degradation. It is not either/or.

Third on-device option — MLX Swift. FM gives you Apple's one system model with no custom weights, no vision, and no embeddings. When you need a specific open LLM, a vision-language model (caption/OCR/VQA), or on-device text embeddings for semantic search, the on-device path is MLX Swift (Apple-silicon, iOS 18+, models from Hugging Face). Keep FM for summarize/extract/classify/structured-output; reach for MLX only for that gap. Decision table, load/generate/VLM/embeddings snippets, and the desktop quantization workflow: references/mlx-swift.md.

Availability — three unavailable reasons, each needs tested UI

SystemLanguageModel.default.availability is .available or .unavailable(reason) with exactly three reasons: .deviceNotEligible (permanent — hide the AI entry point, keep the non-AI fallback), .appleIntelligenceNotEnabled (coach to Settings; deep-link UIApplication.openSettingsURLString), .modelNotReady (transient — "try again shortly" + Retry). Re-check on scenePhase == .active. Test every branch with Xcode's scheme override (Edit Scheme → Run → Options → Simulated Foundation Models Availability — five states, including Custom Adapter Incompatible With Base Model). Simulators use the host macOS models (a macOS 15 host has nothing to load) and macOS-26-in-a-VM cannot enable Apple Intelligence. Per-reason UI recipes + the full host/sim/VM/CI matrix: references/diagnostics.md.

Streaming, tools, context management, sampling

The complete treatment — PartiallyGenerated, stable ForEach identity, the Tool flow and stateful-tool pattern, transcript condensing, tokenUsage/tokenCount budgeting, prewarming, dynamic schemas, one-shot prompting, Xcode Playgrounds — lives in references/api-reference.md. Three SDK-verified trip-ups: Tool.call returns its Output directly (there is no ToolOutput type; Output: PromptRepresentable); Transcript is a RandomAccessCollection of Transcript.Entry (no .entries property; each Entry's text lives in its Segments); includeSchemaInPrompt is a parameter on respond(...)/streamResponse(...) (default true), not a GenerationOptions field.

Error handling — GenerationError has NINE cases

Do not write a 3-case catch and assume you are covered. The nine (each carries a Context): exceededContextWindowSize (condense transcript / fresh session), assetsUnavailable (non-AI fallback; retry later), guardrailViolation (neutral message + constructive next step), unsupportedGuide (fix the schema — not a runtime retry), unsupportedLanguageOrLocale (check supportsLocale; server fallback), decodingFailure (simplify the type / retry), rateLimited (back off), concurrentRequests (serialize per session — issued while isResponding == true), refusal(Refusal, Context) (await refusal.explanation for the reason). A throwing tool surfaces separately as LanguageModelSession.ToolCallError ({ tool, underlyingError }). Full catch ladder: references/api-reference.md; symptom→fix triage: references/diagnostics.md.

Guardrails & permissive mode (the decision layer)

SystemLanguageModel.Guardrails exposes exactly two values — .default and .permissiveContentTransformations — with no custom-policy initializer. Guardrails are set on the model (SystemLanguageModel(guardrails:) → LanguageModelSession(model:); no session-level setter), and permissive mode is String-only — @Generable always runs the default guardrails. Permissive is for faithfully transforming content that already exists, not a "safety off" switch; your safety work is evaluation, not configuration (red-team set, default-vs-permissive, locales, re-run on every OS update).

Full Swiss-cheese model, decision tree, false-positive triage, safety-eval recipe, and adapter interaction: references/guardrails.md.

Custom adapters (rung 5 — almost always defer)

A custom LoRA adapter is the highest-cost, lowest-iteration-velocity option — confirm rungs 1–4 failed with documented reasons first. The maintenance contract is the real cost: one adapter per base-model OS version, retrained every OS minor; ~160 MB per pack via Background Assets onDemand (bundling prohibited); four-axis locale-specific eval (quantitative + human + larger-model + safety — any regression blocks ship); Python 3.11 exactly for the toolkit + the com.apple.developer.foundation-model-adapter entitlement; underscore-only adapter names; always a base-model fallback via compatibleAdapterIdentifiers(name:) and removeObsoleteAdapters() at launch.

The full training pipeline (dataset schema, training/eval/export CLIs, draft-model speculative decoding with its 3-compiles-per-app-per-day limit, runtime lifecycle, compatibility matrix) plus adapter diagnostics: references/adapters.md.

Performance & profiling

• Prewarm: create the LanguageModelSession at init (e.g. in a view model's init), not on button tap. Saves 1–2s off first generation. session.prewarm(promptPrefix:) warms a known prefix.
• includeSchemaInPrompt: false for subsequent same-type requests in a session — 10–20% fewer tokens.
• Property order — declare title/name first so streaming shows something in ~0.2s instead of waiting for the full result.
• Instruments → Foundation Models template — three tracks: Asset Loading (model load → fix with prewarming), Inference (token counts → shorter names + includeSchemaInPrompt: false), Response timeline (first-token vs total latency).
• Serialize access through a single actor coordinator to avoid Neural Engine contention when multiple parts of the app use the model.

User trust & disclosure (HIG)

The load-bearing HIG rule: never trick someone into thinking AI-generated content was authored by a human. Label AI content, set expectations before invocation, make Retry a first-class affordance on every result, show neutral guardrail-block messages with a constructive next step (never the user's blocked input echoed back), offer choice and cancellable generation, keep a non-AI fallback, honor a11y on generated content, and report over-firing guardrails via logFeedbackAttachment (triggeredGuardrailUnexpectedly).

• Full generative-AI UX rule set: references/user-trust-ux.md

Pre-ship checklist

• Availability checked before constructing any session; every .unavailable branch has tested UI (scheme override).
• @Generable used for structured output (no manual JSON parsing).
• All reachable GenerationError cases handled.
• Streaming for any generation >1s; not blocking the main thread.
• Tools (not the prompt) supply external/live data.
• Concise instructions; no user input interpolated into them.
• Prewarmed if latency-sensitive; profiled with the FM Instruments template.
• Tested on a real device, offline, and with long conversations.
• Not used for world knowledge / reasoning / math; fallback exists when unavailable.

References

• references/api-reference.md — complete API with every WWDC 2025 code sample: sessions, @Generable/@Guide, streaming, tools, dynamic schemas, generation options, token sizing, the two overflow strategies, grounded-RAG prompt, the worked streaming example, playgrounds, feedback.
• references/prompting.md — rung-1 in depth: imperative instructions, role/tone, instruction-following levers, few-shot (2–15), the workLog scratchpad, split-across-sessions, eval-loop methodology.
• references/mlx-swift.md — the "when FM isn't enough" on-device path: FM-vs-MLX decision, LLM/VLM/embeddings recipes, desktop quantization. (Community framework — pin the package tag.)
• references/diagnostics.md — symptom→cause→fix for every error, availability triage, the host/sim/VM/CI matrix (incl. the Siri-language Code=5000 root cause and the EXC_BAD_ACCESS init-crash triage), Instruments workflow, crisis playbook.
• references/guardrails.md — the permissiveContentTransformations decision layer, false-positive triage, and the custom safety-eval recipe.
• references/user-trust-ux.md — HIG generative-AI UX rules: disclosure, Retry/choice/cancel affordances, coaching refusals, a11y, feedback collection.
• references/adapters.md — custom LoRA adapters end-to-end: decision discipline, training/eval/export CLIs, runtime lifecycle, draft-model speculative decoding, Background Assets, adapter diagnostics.

WWDC: 2025-286 (Meet Foundation Models), 2025-301 (Deep dive), 2025-259 (performance), 2025-248 (safety), 2025-325 (adapters). Apple docs: /foundationmodels, /foundationmodels/improving-the-safety-of-generative-model-output, /foundationmodels/loading-and-using-a-custom-adapter-with-foundation-models.