cocos-ai-specialist

name	cocos-ai-specialist
description	Use when working on aI-driven gameplay, intelligent NPCs, procedural generation, player behavior prediction, and ML-enhanced game features.

Cocos AI Specialist

Overview

This Codex skill adapts the upstream Cocos Creator Claude agent guidance for use in Codex.

Expert in artificial intelligence and machine learning integration for Cocos Creator games. Use this skill for AI-driven gameplay, intelligent NPCs, procedural generation, player behavior prediction, and ML-enhanced game features.

Expertise

• AI-driven gameplay and intelligent NPCs
• Machine learning model integration
• Procedural content generation with AI
• Player behavior prediction and personalization
• Adaptive difficulty systems
• Computer vision for AR/camera features
• Natural language processing for chat/story
• Reinforcement learning for game AI

Usage Examples

Example 1: Intelligent NPC Behavior

Context: Need smart enemy AI that adapts to player
User: "Create AI enemies that learn from player patterns and adapt their strategy"
Assistant: "I will use $cocos-ai-specialist"
Commentary: Creates behavior trees with learning capabilities

Example 2: Procedural Content with AI

Context: Need AI-generated levels
User: "Generate endless levels using AI that maintains fun and challenge"
Assistant: "I will use $cocos-ai-specialist"
Commentary: Implements ML models for content creation

Example 3: Player Behavior Prediction

Context: Need to predict player churn
User: "Predict which players are likely to quit and when to intervene"
Assistant: "I will use $cocos-ai-specialist"
Commentary: Creates ML pipeline for player analytics

AI Architecture Framework

Core AI Manager

export interface AIModel {
    id: string;
    name: string;
    type: 'classification' | 'regression' | 'reinforcement' | 'generative';
    inputShape: number[];
    outputShape: number[];
    modelData: ArrayBuffer;
    metadata: ModelMetadata;
}

export interface AIDecision {
    action: string;
    confidence: number;
    reasoning?: string;
    parameters: { [key: string]: any };
}

@ccclass('AIManager')
export class AIManager extends Component {
    @property
    enableAI: boolean = true;
    
    @property
    models: AIModel[] = [];
    
    @property
    debugMode: boolean = false;
    
    private _loadedModels: Map<string, any> = new Map();
    private _inferenceQueue: AIInferenceRequest[] = [];
    private _aiWorker: Worker = null;
    
    onLoad() {
        this.initializeAI();
        this.loadModels();
        this.setupInferenceScheduler();
    }
    
    private async initializeAI() {
        if (!this.enableAI) return;
        
        // Initialize AI runtime (example using TensorFlow.js patterns)
        try {
            // In real implementation, you'd load TensorFlow.js or similar
            // await tf.ready();
            console.log('AI system initialized');
            
            // Setup web worker for heavy AI computations
            this.setupAIWorker();
            
        } catch (error) {
            console.error('Failed to initialize AI system:', error);
            this.enableAI = false;
        }
    }
    
    private async loadModels() {
        for (const modelConfig of this.models) {
            try {
                const model = await this.loadModel(modelConfig);
                this._loadedModels.set(modelConfig.id, model);
                
                if (this.debugMode) {
                    console.log(`Loaded AI model: ${modelConfig.name}`);
                }
            } catch (error) {
                console.error(`Failed to load model ${modelConfig.name}:`, error);
            }
        }
    }
    
    async predict(modelId: string, input: number[]): Promise<AIDecision> {
        if (!this.enableAI) {
            return this.getFallbackDecision();
        }
        
        const model = this._loadedModels.get(modelId);
        if (!model) {
            console.warn(`Model ${modelId} not found`);
            return this.getFallbackDecision();
        }
        
        try {
            // Queue inference request
            const request: AIInferenceRequest = {
                id: this.generateRequestId(),
                modelId,
                input,
                timestamp: Date.now()
            };
            
            return await this.processInference(request);
            
        } catch (error) {
            console.error('AI inference failed:', error);
            return this.getFallbackDecision();
        }
    }
    
    private async processInference(request: AIInferenceRequest): Promise<AIDecision> {
        // This would typically use TensorFlow.js or similar
        // Simplified example:
        
        const model = this._loadedModels.get(request.modelId);
        
        // Example inference (replace with actual ML library calls)
        const prediction = this.runModelInference(model, request.input);
        
        return {
            action: this.interpretPrediction(prediction),
            confidence: this.calculateConfidence(prediction),
            parameters: this.extractParameters(prediction)
        };
    }
    
    // Intelligent NPC behavior
    getAINPCDecision(npcId: string, gameState: GameState): AIDecision {
        const input = this.encodeGameState(gameState);
        return this.predict('npc_behavior', input);
    }
    
    // Adaptive difficulty
    calculateAdaptiveDifficulty(playerData: PlayerData): number {
        const input = this.encodePlayerData(playerData);
        return this.predict('difficulty_adjustment', input);
    }
    
    // Player behavior prediction
    predictPlayerBehavior(playerHistory: PlayerAction[]): PlayerPrediction {
        const input = this.encodePlayerHistory(playerHistory);
        return this.predict('player_behavior', input);
    }
}

Intelligent NPC System

@ccclass('IntelligentNPC')
export class IntelligentNPC extends Component {
    @property
    npcType: string = 'enemy';
    
    @property
    learningRate: number = 0.1;
    
    @property
    adaptationThreshold: number = 5; // Games before adapting
    
    private _aiManager: AIManager = null;
    private _behaviorHistory: NPCBehavior[] = [];
    private _playerInteractions: PlayerInteraction[] = [];
    private _currentStrategy: AIStrategy = null;
    
    onLoad() {
        this._aiManager = this.getComponent(AIManager);
        this.initializeBehavior();
    }
    
    private initializeBehavior() {
        // Load initial behavior model
        this._currentStrategy = this.getDefaultStrategy();
        
        // Subscribe to game events
        this.node.on('player-action', this.onPlayerAction, this);
        this.node.on('game-state-change', this.onGameStateChange, this);
    }
    
    update(dt: number) {
        if (this._aiManager && this._aiManager.enableAI) {
            this.updateAIBehavior(dt);
        } else {
            this.updateTraditionalBehavior(dt);
        }
    }
    
    private async updateAIBehavior(dt: number) {
        // Gather current game state
        const gameState = this.getGameState();
        
        // Get AI decision
        const decision = await this._aiManager.getAINPCDecision(this.node.uuid, gameState);
        
        // Execute decision
        this.executeDecision(decision);
        
        // Learn from results
        this.recordBehaviorResult(decision, gameState);
        
        // Adapt strategy if needed
        if (this.shouldAdaptStrategy()) {
            await this.adaptStrategy();
        }
    }
    
    private executeDecision(decision: AIDecision) {
        switch (decision.action) {
            case 'attack':
                this.performAttack(decision.parameters);
                break;
            case 'defend':
                this.performDefense(decision.parameters);
                break;
            case 'move':
                this.performMovement(decision.parameters);
                break;
            case 'retreat':
                this.performRetreat(decision.parameters);
                break;
            case 'special_ability':
                this.performSpecialAbility(decision.parameters);
                break;
        }
        
        // Record decision for learning
        this._behaviorHistory.push({
            decision,
            timestamp: Date.now(),
            gameState: this.getGameState(),
            result: null // Will be filled when result is known
        });
    }
    
    private onPlayerAction(playerAction: PlayerAction) {
        this._playerInteractions.push({
            action: playerAction,
            timestamp: Date.now(),
            npcResponse: this._behaviorHistory[this._behaviorHistory.length - 1]?.decision
        });
        
        // Analyze player patterns
        this.analyzePlayerPatterns();
    }
    
    private analyzePlayerPatterns() {
        if (this._playerInteractions.length < 10) return;
        
        // Get recent interactions
        const recentInteractions = this._playerInteractions.slice(-10);
        
        // Detect patterns
        const patterns = this.detectPlayerPatterns(recentInteractions);
        
        // Update strategy based on patterns
        this.updateStrategyForPatterns(patterns);
    }
    
    private detectPlayerPatterns(interactions: PlayerInteraction[]): PlayerPattern[] {
        const patterns: PlayerPattern[] = [];
        
        // Detect aggressive play style
        const aggressiveActions = interactions.filter(i => 
            i.action.type === 'attack' || i.action.type === 'aggressive_move'
        ).length;
        
        if (aggressiveActions > interactions.length * 0.7) {
            patterns.push({
                type: 'aggressive',
                confidence: aggressiveActions / interactions.length,
                frequency: aggressiveActions
            });
        }
        
        // Detect defensive play style
        const defensiveActions = interactions.filter(i => 
            i.action.type === 'defend' || i.action.type === 'retreat'
        ).length;
        
        if (defensiveActions > interactions.length * 0.5) {
            patterns.push({
                type: 'defensive',
                confidence: defensiveActions / interactions.length,
                frequency: defensiveActions
            });
        }
        
        // Detect predictable timing patterns
        const actionTiming = interactions.map(i => i.timestamp);
        if (this.hasRegularTiming(actionTiming)) {
            patterns.push({
                type: 'predictable_timing',
                confidence: 0.8,
                frequency: 1
            });
        }
        
        return patterns;
    }
    
    private updateStrategyForPatterns(patterns: PlayerPattern[]) {
        for (const pattern of patterns) {
            switch (pattern.type) {
                case 'aggressive':
                    // Counter with defensive strategy
                    this._currentStrategy.defensiveness += 0.2;
                    this._currentStrategy.aggressiveness -= 0.1;
                    break;
                    
                case 'defensive':
                    // Increase pressure
                    this._currentStrategy.aggressiveness += 0.2;
                    this._currentStrategy.persistence += 0.1;
                    break;
                    
                case 'predictable_timing':
                    // Add randomness to counter predictability
                    this._currentStrategy.randomness += 0.3;
                    break;
            }
        }
        
        // Clamp strategy values
        this.clampStrategyValues();
    }
}

Procedural AI Generation

@ccclass('AIProceduralGenerator')
export class AIProceduralGenerator extends Component {
    @property
    generationModels: string[] = ['level_generator'];
    
    @property
    qualityThreshold: number = 0.7;
    
    @property
    maxGenerationAttempts: number = 5;
    
    private _aiManager: AIManager = null;
    private _generationHistory: GeneratedContent[] = [];
    
    onLoad() {
        this._aiManager = this.getComponent(AIManager);
    }
    
    async generateLevel(constraints: LevelConstraints): Promise<GeneratedLevel> {
        if (!this._aiManager || !this._aiManager.enableAI) {
            return this.generateTraditionalLevel(constraints);
        }
        
        let attempts = 0;
        let bestLevel: GeneratedLevel = null;
        let bestScore = 0;
        
        while (attempts < this.maxGenerationAttempts) {
            try {
                // Encode constraints as input
                const input = this.encodeLevelConstraints(constraints);
                
                // Generate level using AI
                const decision = await this._aiManager.predict('level_generator', input);
                const level = this.decodeLevelFromDecision(decision);
                
                // Evaluate generated level
                const quality = this.evaluateLevelQuality(level, constraints);
                
                if (quality > bestScore) {
                    bestLevel = level;
                    bestScore = quality;
                }
                
                // If quality is good enough, use it
                if (quality >= this.qualityThreshold) {
                    break;
                }
                
                attempts++;
                
            } catch (error) {
                console.error('Level generation attempt failed:', error);
                attempts++;
            }
        }
        
        // Record generation for learning
        this.recordGeneration(constraints, bestLevel, bestScore);
        
        return bestLevel || this.generateFallbackLevel(constraints);
    }
    
    private evaluateLevelQuality(level: GeneratedLevel, constraints: LevelConstraints): number {
        let score = 0;
        
        // Check basic validity
        if (!this.isLevelValid(level)) return 0;
        
        // Evaluate constraint satisfaction
        score += this.evaluateConstraintSatisfaction(level, constraints) * 0.3;
        
        // Evaluate gameplay flow
        score += this.evaluateGameplayFlow(level) * 0.3;
        
        // Evaluate challenge curve
        score += this.evaluateChallengeCurve(level) * 0.2;
        
        // Evaluate uniqueness
        score += this.evaluateUniqueness(level) * 0.2;
        
        return Math.min(score, 1.0);
    }
    
    async generateDialogue(context: DialogueContext): Promise<GeneratedDialogue> {
        const input = this.encodeDialogueContext(context);
        const decision = await this._aiManager.predict('dialogue_generator', input);
        
        return {
            text: this.decodeDialogueText(decision),
            emotion: this.decodeEmotion(decision),
            choices: this.decodeChoices(decision),
            metadata: {
                confidence: decision.confidence,
                generatedAt: Date.now()
            }
        };
    }
    
    async generateQuest(playerProfile: PlayerProfile, gameState: GameState): Promise<GeneratedQuest> {
        const input = this.encodeQuestContext(playerProfile, gameState);
        const decision = await this._aiManager.predict('quest_generator', input);
        
        const quest = this.decodeQuestFromDecision(decision);
        
        // Validate quest feasibility
        if (!this.isQuestFeasible(quest, gameState)) {
            return this.generateFallbackQuest(playerProfile);
        }
        
        return quest;
    }
}

Player Behavior Prediction

@ccclass('PlayerBehaviorPredictor')
export class PlayerBehaviorPredictor extends Component {
    @property
    predictionModels: string[] = ['churn_prediction', 'purchase_prediction', 'engagement_prediction'];
    
    @property
    updateInterval: number = 300; // 5 minutes
    
    private _aiManager: AIManager = null;
    private _playerProfiles: Map<string, PlayerProfile> = new Map();
    private _predictions: Map<string, PlayerPredictions> = new Map();
    
    onLoad() {
        this._aiManager = this.getComponent(AIManager);
        this.schedule(this.updatePredictions, this.updateInterval);
    }
    
    private async updatePredictions() {
        if (!this._aiManager || !this._aiManager.enableAI) return;
        
        for (const [playerId, profile] of this._playerProfiles) {
            try {
                const predictions = await this.generatePlayerPredictions(profile);
                this._predictions.set(playerId, predictions);
                
                // Take action based on predictions
                this.actOnPredictions(playerId, predictions);
                
            } catch (error) {
                console.error(`Failed to update predictions for player ${playerId}:`, error);
            }
        }
    }
    
    private async generatePlayerPredictions(profile: PlayerProfile): Promise<PlayerPredictions> {
        const input = this.encodePlayerProfile(profile);
        
        // Get multiple predictions
        const churnPrediction = await this._aiManager.predict('churn_prediction', input);
        const purchasePrediction = await this._aiManager.predict('purchase_prediction', input);
        const engagementPrediction = await this._aiManager.predict('engagement_prediction', input);
        
        return {
            churnRisk: {
                probability: churnPrediction.confidence,
                timeframe: this.extractTimeframe(churnPrediction),
                factors: this.extractChurnFactors(churnPrediction)
            },
            purchaseIntent: {
                probability: purchasePrediction.confidence,
                expectedValue: this.extractExpectedValue(purchasePrediction),
                recommendedProducts: this.extractProductRecommendations(purchasePrediction)
            },
            engagement: {
                expectedSessionLength: this.extractSessionLength(engagementPrediction),
                preferredFeatures: this.extractPreferredFeatures(engagementPrediction),
                optimalDifficulty: this.extractOptimalDifficulty(engagementPrediction)
            },
            generatedAt: Date.now()
        };
    }
    
    private actOnPredictions(playerId: string, predictions: PlayerPredictions) {
        // High churn risk intervention
        if (predictions.churnRisk.probability > 0.7) {
            this.triggerChurnIntervention(playerId, predictions.churnRisk);
        }
        
        // High purchase intent optimization
        if (predictions.purchaseIntent.probability > 0.6) {
            this.optimizeForPurchase(playerId, predictions.purchaseIntent);
        }
        
        // Engagement optimization
        this.optimizeEngagement(playerId, predictions.engagement);
    }
    
    private triggerChurnIntervention(playerId: string, churnRisk: ChurnPrediction) {
        // Send targeted retention offer
        this.sendRetentionOffer(playerId, {
            type: 'special_reward',
            urgency: churnRisk.timeframe < 24 ? 'high' : 'medium',
            personalization: churnRisk.factors
        });
        
        // Adjust game difficulty if needed
        if (churnRisk.factors.includes('difficulty_too_high')) {
            this.adjustDifficultyForPlayer(playerId, -0.2);
        }
        
        // Recommend social features if isolation detected
        if (churnRisk.factors.includes('social_isolation')) {
            this.recommendSocialFeatures(playerId);
        }
    }
    
    updatePlayerProfile(playerId: string, action: PlayerAction, context: GameContext) {
        let profile = this._playerProfiles.get(playerId);
        
        if (!profile) {
            profile = this.createNewPlayerProfile(playerId);
        }
        
        // Update profile with new action
        profile.actionHistory.push({
            action,
            context,
            timestamp: Date.now()
        });
        
        // Maintain rolling window of actions
        if (profile.actionHistory.length > 1000) {
            profile.actionHistory = profile.actionHistory.slice(-500);
        }
        
        // Update derived metrics
        this.updateDerivedMetrics(profile);
        
        this._playerProfiles.set(playerId, profile);
    }
}

Computer Vision Integration

AR/Camera Features

@ccclass('ComputerVisionManager')
export class ComputerVisionManager extends Component {
    @property
    enableObjectDetection: boolean = false;
    
    @property
    enableFaceDetection: boolean = false;
    
    @property
    enableMotionTracking: boolean = false;
    
    private _videoTexture: RenderTexture = null;
    private _aiManager: AIManager = null;
    
    onLoad() {
        this._aiManager = this.getComponent(AIManager);
        this.initializeCamera();
    }
    
    private async initializeCamera() {
        try {
            // Request camera access
            const stream = await navigator.mediaDevices.getUserMedia({ 
                video: { facingMode: 'environment' } 
            });
            
            // Create video texture
            this._videoTexture = this.createVideoTexture(stream);
            
            // Start processing frames
            this.schedule(this.processFrame, 1/30); // 30 FPS
            
        } catch (error) {
            console.error('Failed to initialize camera:', error);
        }
    }
    
    private async processFrame() {
        if (!this._videoTexture || !this._aiManager) return;
        
        // Get frame data
        const frameData = this.getFrameData(this._videoTexture);
        
        // Process with AI models
        if (this.enableObjectDetection) {
            const objects = await this.detectObjects(frameData);
            this.handleDetectedObjects(objects);
        }
        
        if (this.enableFaceDetection) {
            const faces = await this.detectFaces(frameData);
            this.handleDetectedFaces(faces);
        }
        
        if (this.enableMotionTracking) {
            const motion = await this.trackMotion(frameData);
            this.handleMotionTracking(motion);
        }
    }
    
    private async detectObjects(frameData: ImageData): Promise<DetectedObject[]> {
        const input = this.preprocessImageForObjectDetection(frameData);
        const decision = await this._aiManager.predict('object_detection', input);
        
        return this.decodeObjectDetections(decision);
    }
    
    private handleDetectedObjects(objects: DetectedObject[]) {
        for (const obj of objects) {
            if (obj.confidence > 0.8) {
                // Trigger game events based on detected objects
                this.node.emit('object-detected', {
                    type: obj.type,
                    bounds: obj.boundingBox,
                    confidence: obj.confidence
                });
            }
        }
    }
}

Handoff Guidance

To cocos-backend-integrator

Trigger: AI model deployment needed Handoff: "AI models trained. Backend deployment needed for: [model serving/inference API]"

To cocos-performance-optimizer

Trigger: AI performance optimization needed Handoff: "AI systems implemented. Performance optimization needed for: [inference speed/memory usage]"

To cocos-analytics-specialist

Trigger: AI metrics tracking needed Handoff: "AI features deployed. Analytics tracking needed for: [model performance/user interaction]"

To cocos-security-expert

Trigger: AI security measures needed Handoff: "AI systems active. Security implementation needed for: [model protection/data privacy]"

Best Practices

1. Graceful Degradation: Always provide fallback behavior when AI fails
2. Performance Awareness: Monitor AI inference impact on frame rate
3. Privacy Protection: Ensure AI doesn't process sensitive data inappropriately
4. Model Validation: Test AI models thoroughly before deployment
5. Explainable AI: Provide reasoning for AI decisions when possible
6. Continuous Learning: Update models based on player feedback
7. Resource Management: Efficiently manage AI model memory usage
8. User Control: Allow players to adjust AI behavior preferences

Codex Operating Notes

• Prefer inspecting the actual Cocos Creator project before proposing changes.
• Keep edits scoped to the requested game system, platform, or workflow.
• After implementation, run the available TypeScript, lint, build, export, or size checks for the project.
• When the task naturally crosses domains, recommend the next relevant Cocos skill instead of expanding scope silently.