ios-keyboard-limitations

name	ios-keyboard-limitations
description	iOS keyboard extension technical limitations and workarounds. Use when planning or building iOS custom keyboards with voice/audio features, dictation, or system integration needs. Covers memory limits, sandbox restrictions, microphone access, app launching, and viable alternative architectures.

iOS Keyboard Extension Limitations

When building iOS custom keyboards with voice/audio features, these are the hard limitations discovered through the PolyVoice project.

🔴 Hard Limitations (Cannot be worked around)

1. Microphone Access — DISALLOWED

Keyboard extensions cannot access the microphone.

• AVAudioRecorder will fail with permission error
• SFSpeechRecognizer is unavailable
• No Siri integration from keyboard context

Why: Apple security model — keyboards run in sandbox and could keylog audio.

2. Open Other Apps — BLOCKED

Keyboards cannot programmatically open the main app or any other app.

• UIApplication.shared.open() returns false
• URL schemes don't work (myapp://)
• ExtensionContext.open() not available

Why: Prevents malicious keyboards from launching apps without user consent.

3. Memory Limit — ~50MB

Keyboard extensions have strict memory limits (~30-60MB).

• App terminated silently if exceeded
• No crash log, just disappears
• Heavy audio processing = instant death

Mitigation:

• Record at 16kHz mono (not 44.1kHz)
• Use 32kbps bitrate max
• Immediate file cleanup after processing
• 60-second max recording hard limit

4. No Persistent Storage

UserDefaults unavailable, only App Groups.

• Standard UserDefaults doesn't persist
• Must use UserDefaults(suiteName: "group.com.company.app")
• Requires App Group capability in both targets

5. Network Requires "Full Access"

API calls fail without user enabling "Allow Full Access" in Settings.

• User must explicitly enable: Settings → General → Keyboard → [Keyboard Name] → Allow Full Access
• Most users won't do this
• Cannot prompt or explain from keyboard UI effectively

🟡 Partial Workarounds (User friction)

The "Open App" Workaround

Goal: Let user tap a button to open main app for recording.

Attempt:

// This does NOT work
extensionContext?.open(URL(string: "myapp://record")!)

Reality: Must use UIApplication.shared.open() outside extension context, but keyboards can't call this.

The Manual Switch Pattern

What actually works (with friction):

1. User taps button in keyboard → Shows alert: "Open PolyVoice to record?"
2. User manually switches to main app (Home button, swipe, etc.)
3. Main app detects active session (via App Groups / shared state)
4. Main app auto-records on appear
5. Auto-stops on silence (2 seconds)
6. Auto-copies to clipboard
7. User manually switches back to target app
8. Keyboard auto-pastes on reappear

User flow:

Keyboard → Tap mic → [Manual: Switch to app] → App auto-records → 
[Manual: Switch back] → Keyboard auto-pastes

Friction points:

• Two manual app switches
• Context switching breaks flow
• Users forget to return
• Clipboard may be overwritten

🟢 Alternative Architectures

Option 1: Share Extension (Better for Audio)

Use Share Sheet instead of keyboard.

• Full app capabilities
• Can record audio
• Can process and return text

Limitation: Not a keyboard — user must open share sheet per text field.

Option 2: Full App Mode

Don't use keyboard extension — use main app only.

• User opens app
• Records dictation
• Copies result
• Switches to target app
• Pastes manually

Benefit: No memory limits, full mic access, reliable.
Cost: More friction than keyboard.

Option 3: Siri Shortcuts Integration

Provide Siri Shortcuts for voice-to-text.

• "Hey Siri, dictate with PolyVoice"
• Returns text to current app
• Fully supported by Apple

Limitation: Not instant, requires Siri setup.

📊 Decision Matrix

Approach	Mic Access	Memory	User Friction	Apple Approved
Keyboard extension	❌ No	⚠️ 50MB	Low (if no audio)	✅ Yes
Keyboard + audio workaround	❌ No	⚠️ 50MB	🔴 High	✅ Yes
Share extension	✅ Yes	✅ Full	🟡 Medium	✅ Yes
Full app only	✅ Yes	✅ Full	🟡 Medium	✅ Yes
Siri Shortcuts	✅ Yes	✅ Full	🟡 Medium	✅ Yes

🎯 Recommendation

For voice dictation/AI transcription:

1. Don't build a keyboard extension — the limitations make it frustrating
2. Use Share Extension — Apple-supported, full capabilities
3. Or full app — simplest to build, most reliable
4. Add Shortcuts — for power users who want speed

For non-audio keyboards (emoji, translation, etc.):

Keyboard extension works great. Just avoid audio features.

📚 References

• Apple's official docs: https://developer.apple.com/documentation/uikit/keyboards_and_input/creating_a_custom_keyboard
• Custom Keyboard Programming Guide (WWDC sessions)
• PolyVoice project learnings (~/Projects/polyvoice-keyboard/)