foundation-models

# Foundation Models Integrate Apple's on-device LLM into your apps for privacy-preserving AI features. Companion references: **safety-and-guardrails.md** (model limits, prompt design, the four-layer safety stack), **models-and-agents.md** (Private Cloud Compute, `LanguageModel` protocol, vision input, `DynamicProfile` agentic sessions — the iOS 27 wave), and **utilities-package.md** (Apple's open-source utilities package: OpenAI-compatible endpoints, just-in-time Skills, history compression). ## When This Skill Activates - User wants AI text generation features - User needs structured data from natural language - User asks about prompting or LLM integration - User wants to implement AI assistants or agentic features (tool loops, multi-profile sessions) - User needs content summarization or extraction - User asks about Private Cloud Compute, guardrails, or model safety ## Model Fit — Check Before Building The on-device model is ~3B parameters (2-bit quantized): built for **summarization, extraction, classification, tagging, revision, short chat** — not math, code generation, facts, or world knowledge (WWDC25 248). For capability boundaries, prompt-design rules, and the safety stack, read `safety-and-guardrails.md` first. For anything bigger, `PrivateCloudComputeLanguageModel` (32k context, reasoning) and third-party backends are in `models-and-agents.md`. ## Quick Start ### 1. Check Availability ```swift import FoundationModels struct IntelligentView: View { private var model = SystemLanguageModel.default var body: some View { switch model.availability { case .available: ContentView() case .unavailable(.deviceNotEligible): UnsupportedDeviceView() case .unavailable(.appleIntelligenceNotEnabled): EnableIntelligenceView() case .unavailable(.modelNotReady): ModelDownloadingView() case .unavailable(let reason): ErrorView(reason: reason) } } } ``` ### 2. Create a Session ```swift // Simple session let session = LanguageModelSession() // Session with instructions let session = LanguageModelSession(instructions: """ You are a helpful cooking assistant. Provide concise, practical advice for home cooks. """) ``` ### 3. Generate Response ```swift let response = try await session.respond(to: "What's a quick dinner idea?") print(response.content) ``` ## Prompt Engineering Best Practices ### The Instruction Formula Instructions set the model's persona and constraints. They're prioritized over prompts. ``` [Role] + [Task] + [Style] + [Safety] ``` **Example:** ```swift let instructions = """ You are a fitness coach specializing in home workouts. Help users create exercise routines based on their equipment and goals. Keep responses under 100 words and use bullet points for exercises. Decline requests for medical advice and suggest consulting a doctor. """ ``` ### Instruction Components | Component | Purpose | Example | |-----------|---------|---------| | **Role** | Define persona | "You are a travel expert" | | **Task** | What to do | "Help plan itineraries" | | **Style** | Output format | "Use bullet points, be concise" | | **Safety** | Boundaries | "Don't provide medical advice" | ### Effective Prompts Prompts are user inputs. Make them: | Principle | Bad | Good | |-----------|-----|------| | **Specific** | "Help with cooking" | "Suggest a 30-minute vegetarian dinner" | | **Constrained** | "Tell me about dogs" | "Describe Golden Retrievers in 3 sentences" | | **Focused** | "I need help with many things" | "What ingredients substitute for eggs in baking?" | ### Prompt Patterns **Question Pattern:** ```swift let prompt = "What are three ways to reduce food waste at home?" ``` **Command Pattern:** ```swift let prompt = "Create a weekly meal plan for a family of four, budget-friendly." ``` **Extraction Pattern:** ```swift let prompt = """ Extract the following from this email: - Sender name - Meeting date - Action items Email: \(emailContent) """ ``` **Transformation Pattern:** ```swift let prompt = "Rewrite this text to be more formal: \(casualText)" ``` ## Structured Output with @Generable Get typed Swift data instead of raw strings. ### Define Generable Types ```swift @Generable(description: "A recipe suggestion") struct Recipe { var name: String @Guide(description: "Cooking time in minutes", .range(5...180)) var cookingTime: Int @Guide(description: "Difficulty level", .options(["Easy", "Medium", "Hard"])) var difficulty: String @Guide(description: "List of ingredients", .count(3...15)) var ingredients: [String] @Guide(description: "Step-by-step instructions") var instructions: [String] } ``` ### @Guide Constraints | Constraint | Use Case | Example | |------------|----------|---------| | `.range(min...max)` | Numeric bounds | `.range(1...100)` | | `.options([...])` | Enum-like choices | `.options(["Low", "Medium", "High"])` | | `.count(n)` | Exact array length | `.count(5)` | | `.count(min...max)` | Array length range | `.count(3...10)` | ### Two Rules the Macro Hides (WWDC25 301) - **Don't re-describe your schema in the prompt.** The framework injects your `@Generable` type's details "in a specific format that the model has been trained on" — hand-written "respond in JSON with fields…" text duplicates it and wastes tokens. Constrained decoding masks invalid tokens per-step, so structural correctness is guaranteed, not prompted for. - **Property order is generation order.** "Properties are generated in the order they are declared on your Swift struct… you may find that the model produces the best summaries when they're the last property" (WWDC25 286). Put conditioning fields (context, inputs, reasoning) *before* the properties that should depend on them; put summaries last. This affects output quality *and* streaming animations. For schemas only known at runtime, build a `DynamicGenerationSchema` (supports `arrayOf:` and `referenceTo:` cross-references), validate with `GenerationSchema(root:dependencies:)` (throws on unresolved references), respond via `session.respond(to:schema:)`, and read untyped values with `response.content.value(String.self, forProperty: "question")`. ### Generate Structured Data ```swift let session = LanguageModelSession(instructions: """ You are a recipe assistant. Generate practical, home-cook friendly recipes. """) let recipe = try await session.respond( to: "Suggest a quick pasta dish", generating: Recipe.self ) print("Recipe: \(recipe.content.name)") print("Time: \(recipe.content.cookingTime) minutes") print("Ingredients: \(recipe.content.ingredients.joined(separator: ", "))") ``` ### Complex Nested Structures ```swift @Generable(description: "A travel itinerary") struct Itinerary { var destination: String @Guide(description: "Daily activities for the trip") var days: [DayPlan] } @Generable(description: "Activities for one day") struct DayPlan { var dayNumber: Int @Guide(description: "Morning activity") var morning: String @Guide(description: "Afternoon activity") var afternoon: String @Guide(description: "Evening activity") var evening: String } ``` ## Tool Calling Let the model call your code to access data or perform actions. ### Define a Tool ```swift struct WeatherTool: Tool { let name = "getWeather" // verb, short, no abbreviations let description = "Get current weather for a location" // ~one sentence @Generable struct Arguments { @Guide(description: "City name") var location: String } func call(arguments: Arguments) async throws -> ToolOutput { let weather = await WeatherService.shared.fetch(for: arguments.location) return ToolOutput("Temperature: \(weather.temp)°F, Conditions: \(weather.conditions)") } } ``` Rules from the deep dive (WWDC25 301): - **Name = verb, description = one sentence.** "These strings are put verbatim in your prompt. So longer strings means more tokens, which can increase the latency." No abbreviations, no implementation details. - **Arguments are `@Generable`** — guided generation guarantees valid arguments; nest `@Generable` enums to give the model a closed set of options. - **The session holds one instance for its whole lifetime** — tools may be stateful (e.g. track already-returned results to avoid repeats). - **Tools can be called in parallel within a single request** — tool state must be concurrency-safe. - Tool output lands in the transcript like model output — it consumes context window. ### Use Tools in Session ```swift let weatherTool = WeatherTool() let session = LanguageModelSession( instructions: "You help users plan outdoor activities based on weather.", tools: [weatherTool] ) // Model automatically calls tool when needed let response = try await session.respond( to: "Should I go hiking in San Francisco today?" ) ``` ### Tool Error Handling ```swift do { let response = try await session.respond(to: prompt) } catch let error as LanguageModelSession.ToolCallError { print("Tool '\(error.tool.name)' failed: \(error.underlyingError)") } catch { print("Generation error: \(error)") } ``` ## Snapshot Streaming Show responses as they generate for better UX. ### Stream to SwiftUI ```swift @Generable struct StoryIdea { var title: String @Guide(description: "A brief plot summary") var plot: String @Guide(description: "Main characters", .count(2...4)) var characters: [String] } struct StreamingView: View { @State private var partial: StoryIdea.PartiallyGenerated? @State private var isGenerating = false var body: some View { VStack(alignment: .leading) { if let partial { if let title = partial.title { Text(title).font(.headline) } if let plot = partial.plot { Text(plot) } if let characters = partial.characters { ForEach(characters, id: \.self) { char in Text("• \(char)") } } } Button("Generate Story Idea") { Task { await generateStory() } } .disabled(isGenerating) } } func generateStory() async { isGenerating = true defer { isGenerating = false } let session = LanguageModelSession() let stream = session.streamResponse( to: "Create a sci-fi story idea", generating: StoryIdea.self ) for try await snapshot in stream { partial = snapshot } } } ``` ## Multi-Turn Conversations Reuse sessions to maintain context. ```swift @Observable final class ChatViewModel { private var session: LanguageModelSession? var messages: [ChatMessage] = [] func startConversation() { session = LanguageModelSession(instructions: """ You are a helpful assistant. Remember context from earlier in our conversation. """) } func send(_ message: String) async throws { guard let session else { return } messages.append(ChatMessage(role: .user, content: message)) let response = try await session.respond(to: message) messages.append(ChatMessage(role: .assistant, content: response.content)) } } ``` ## Error Handling ⚠️ **`LanguageModelSession.GenerationError` is deprecated at iOS 27 and becomes _unavailable_ when you rebuild with Xcode 27.** This is a hard break, not a warning: Apple's deprecation note says apps built with Xcode 26 keep catching the old error until you rebuild, and you *must* move to the new types before submitting from Xcode 27. Its nine cases were split across **three** enums by concern. ```swift do { let response = try await session.respond(to: prompt) // ── LanguageModelError — model-level, backend-agnostic (any LanguageModel) ── } catch LanguageModelError.contextSizeExceeded(let e) { // e.contextSize / e.tokenCount — recover by condensing the transcript (below) } catch LanguageModelError.guardrailViolation { // Safety block: proactive features ignore silently; // user-initiated features explain + offer alternatives (see safety-and-guardrails.md) } catch LanguageModelError.refusal(let e) { // NOT a safety block — the model declined for its own reasons. // e.explanation (and .explanationStream) say why. Surface it; don't retry blindly. } catch LanguageModelError.rateLimited(let e) { // Server-backed models only. e.resetDate tells you when to retry, when known. } catch LanguageModelError.unsupportedLanguageOrLocale { // Also pre-check SystemLanguageModel.default.supportedLanguages before prompting } catch LanguageModelError.unsupportedCapability(let e) { // The backend doesn't do e.capability (tools / vision / reasoning / guided generation). // Capabilities are NOT uniform across models — see models-and-agents.md. } catch LanguageModelError.timeout { // ── LanguageModelSession.Error — you drove the session wrong ── } catch LanguageModelSession.Error.concurrentRequests { // Gate submit on session.isResponding — one request per session } catch LanguageModelSession.Error.transcriptMutationWhileResponding { // Mutate session.transcript only while isResponding == false // ── SystemLanguageModel.Error — on-device assets ── } catch SystemLanguageModel.Error.assetsUnavailable { // Model assets not on device — check availability first (see Quick Start) } catch { print("Unexpected error: \(error)") } ``` ### Migrating off `GenerationError` | iOS 26 `GenerationError` | iOS 27 replacement | |---|---| | `exceededContextWindowSize` | `LanguageModelError.contextSizeExceeded` — now carries `contextSize` **and** `tokenCount` | | `guardrailViolation` | `LanguageModelError.guardrailViolation` | | `refusal` | `LanguageModelError.refusal` | | `rateLimited` | `LanguageModelError.rateLimited` — now carries `resetDate` | | `unsupportedGuide` | `LanguageModelError.unsupportedGenerationGuide` (renamed) | | `unsupportedLanguageOrLocale` | `LanguageModelError.unsupportedLanguageOrLocale` |