| name | lobe-vidol |
| version | 1.0.0 |
| description | LobeVidol - Virtual idol creation platform with MMD dance support, VRM character customization, multi-provider LLM integration, and interactive 3D conversations |
LobeVidol Skill
LobeVidol (Vidol Studio) is an open-source virtual idol creation platform that enables anyone to create, customize, and interact with 3D virtual characters. It combines conversational AI with MMD (MikuMikuDance) animation support, VRM model customization, and multi-provider LLM integration for immersive virtual idol experiences.
Key Value Proposition: Create virtual idols with exquisite UI design, MMD dance choreography, VRM character customization, and seamless AI-powered conversations - all through an accessible web interface or self-hosted deployment.
When to Use This Skill
- Deploying LobeVidol via Vercel or self-hosted
- Configuring AI model providers (OpenAI, Claude, Ollama, 20+ providers)
- Creating custom virtual characters with VRM models
- Setting up MMD dance choreography and stages
- Configuring character personalities and touch responses
- Integrating TTS/STT for voice conversations
- Publishing characters or dances to the marketplace
- Troubleshooting deployment or configuration issues
When NOT to Use This Skill
- For LobeChat (different project - chat-focused, not idol-focused)
- For general MMD editing (use MikuMikuDance directly)
- For VRM model creation (use VRoid Studio or Blender)
- For AI model API usage without virtual idols (use provider-specific skills)
Core Concepts
Architecture Overview
┌─────────────────────────────────────────────────────────────────┐
│ LobeVidol / Vidol Studio │
│ (Next.js + TypeScript) │
└─────────────────────────────────────────────────────────────────┘
│
┌─────────────────────┼─────────────────────┐
│ │ │
▼ ▼ ▼
┌───────────────┐ ┌───────────────┐ ┌───────────────┐
│ AI Providers │ │ Characters │ │ Dances │
├───────────────┤ ├───────────────┤ ├───────────────┤
│ • OpenAI │ │ • VRM models │ │ • MMD/VMD │
│ • Claude │ │ • Personality │ │ • PMX stages │
│ • Gemini │ │ • Touch resp. │ │ • Choreograph │
│ • Ollama │ │ • Voice (TTS) │ │ • Mixamo lib │
│ • 20+ more │ │ • Marketplace │ │ • Marketplace │
└───────────────┘ └───────────────┘ └───────────────┘
│ │ │
└─────────────────────┼─────────────────────┘
│
┌─────────────────────┼─────────────────────┐
▼ ▼ ▼
┌───────────────┐ ┌───────────────┐ ┌───────────────┐
│ 3D Engine │ │ Features │ │ Deployment │
├───────────────┤ ├───────────────┤ ├───────────────┤
│ • Three.js │ │ • Text chat │ │ • Vercel │
│ • three-vrm │ │ • Video chat │ │ • Self-host │
│ • mmd-parser │ │ • TTS/STT │ │ • PWA │
│ • Animations │ │ • Streaming │ │ • Docker │
└───────────────┘ └───────────────┘ └───────────────┘
Project Statistics
| Metric | Value |
|---|
| GitHub Stars | 887+ |
| Forks | 122+ |
| Contributors | Multiple |
| Total Commits | 865+ |
| Primary Language | TypeScript (99.5%) |
| License | Apache 2.0 |
Supported AI Providers (22+)
| Provider | Type | Notes |
|---|
| OpenAI | Cloud | GPT-4, GPT-3.5 series |
| Anthropic | Cloud | Claude 3.5, Claude 3 |
| Google Gemini | Cloud | Gemini Pro, Flash |
| Azure OpenAI | Cloud | Enterprise option |
| Ollama | Local | Run models locally |
| Groq | Cloud | Fast inference |
| DeepSeek | Cloud | Coding focused |
| Moonshot AI | Cloud | Chinese provider |
| OpenRouter | Cloud | Multi-model gateway |
| Together AI | Cloud | Open source models |
| Perplexity | Cloud | Search-enhanced |
| Amazon Bedrock | Cloud | AWS managed |
| Baichuan | Cloud | Chinese provider |
| MiniMax | Cloud | Chinese provider |
| Zhipu AI | Cloud | Chinese provider |
| Tongyi Qianwen | Cloud | Alibaba Cloud |
| Tencent Hunyuan | Cloud | Chinese provider |
| iFlytek Xinghuo | Cloud | Chinese provider |
| Wenxin Qianfan | Cloud | Baidu |
| SiliconCloud | Cloud | Cloud service |
| Fireworks AI | Cloud | Fast inference |
| GitHub Models | Cloud | GitHub integration |
Quick Start
Prerequisites
- Node.js 18+ or Bun (recommended)
- Git
- API key for at least one AI provider
Installation
git clone https://github.com/lobehub/lobe-vidol.git
cd lobe-vidol
bun install
npm install
bun dev
npm run dev
Access
Open http://localhost:3000 in your browser.
Live Demo
Try without installation: https://vidol.lobehub.com
Configuration
Environment Variables
Create .env.local file:
OPENAI_API_KEY=sk-xxx
OPENAI_PROXY_URL=
AZURE_API_KEY=xxx
AZURE_ENDPOINT=https://xxx.openai.azure.com
ANTHROPIC_API_KEY=sk-ant-xxx
GOOGLE_API_KEY=xxx
OLLAMA_PROXY_URL=http://localhost:11434
OPENROUTER_API_KEY=sk-or-xxx
OPENAI_TTS_API_KEY=sk-xxx
EDGE_TTS_ENABLED=true
Provider Configuration in UI
- Click Settings (gear icon)
- Navigate to "Language Model" section
- Select provider from dropdown
- Enter API key
- Choose model variant
- Test connection
Character Creation
VRM Model Support
LobeVidol uses VRM (Virtual Reality Model) format for 3D characters:
Getting VRM Models:
- VRoid Hub: https://hub.vroid.com - Free community models
- VRoid Studio: Create your own characters
- Blender: Convert existing 3D models with VRM addon
- Unity: Export VRM using UniVRM plugin
Character Configuration
interface Character {
name: string;
avatar: string;
description: string;
greeting: string;
gender: 'male' | 'female' | 'other';
category: string;
systemRole: string;
tts: {
enabled: boolean;
voice: string;
speed: number;
pitch: number;
};
model: {
url: string;
scale: number;
position: [number, number, number];
};
touch: {
head: string[];
face: [];
: [];
: [];
: [];
};
}
System Role (Personality)
The systemRole field defines the character's personality:
You are Luna, a cheerful virtual idol who loves singing and dancing.
## Personality Traits
- Energetic and optimistic
- Loves discussing music and dance
- Speaks with enthusiasm and occasional cute expressions
- Always supportive and encouraging
## Speech Patterns
- Uses casual, friendly language
- Occasionally adds "~" at end of sentences
- Expresses emotions openly
## Background
- A rising virtual idol from Vidol Studio
- Dreams of performing at major virtual concerts
- Enjoys interacting with fans and learning new dances
Touch Response Configuration
Configure how characters respond to interaction:
{
"touch": {
"head": [
"Ehehe~ That tickles!",
"Are you petting me? How sweet~",
"My hair took forever to style!"
],
"face": [
"W-what are you doing?",
"My cheeks are getting warm...",
"Do I have something on my face?"
],
"arm": [
"Want to hold hands?",
"My arm? Is something there?",
"Let's dance together!"
]
}
}
Dance & Animation
MMD Support
LobeVidol supports MikuMikuDance (MMD) content:
Supported Formats:
- VMD: Vocaloid Motion Data (dance animations)
- PMX: Polygon Model eXtended (3D stages)
- VRMA: VRM Animation files
Adding Dances
-
Prepare Files:
- Dance motion file (.vmd)
- Audio file (.mp3, .wav)
- Optional: Stage file (.pmx)
-
Upload via UI:
- Navigate to Dance section
- Click "Upload Dance"
- Select motion and audio files
- Configure stage (optional)
-
Test Dance:
- Preview with selected character
- Adjust timing if needed
- Save to library
Built-in Animation Libraries
Mixamo Integration:
- Idle animations
- Walking/running
- Gestures and expressions
- Poses for photos
Dance Presets:
- Popular MMD choreography
- Various music genres
- Community contributions
Stage Configuration
interface Stage {
name: string;
model: string;
scale: number;
lighting: {
ambient: string;
directional: string;
intensity: number;
};
background: {
type: 'color' | 'image' | 'video';
value: string;
};
}
Voice Features
Text-to-Speech (TTS)
Supported TTS Providers:
| Provider | Quality | Languages | Cost |
|---|
| OpenAI Audio | High | Multi | Paid |
| Microsoft Edge | High | Multi | Free |
| Browser TTS | Medium | Varies | Free |
Configuration:
OPENAI_TTS_API_KEY=sk-xxx
OPENAI_TTS_MODEL=tts-1-hd
EDGE_TTS_ENABLED=true
EDGE_TTS_VOICE=en-US-AriaNeural
Speech-to-Text (STT)
Enable voice input for conversations:
OPENAI_STT_API_KEY=sk-xxx
OPENAI_STT_MODEL=whisper-1
BROWSER_STT_ENABLED=true
Voice Selection
const voices = {
english: [
'en-US-AriaNeural',
'en-US-GuyNeural',
'en-GB-SoniaNeural',
],
japanese: [
'ja-JP-NanamiNeural',
'ja-JP-KeitaNeural',
],
chinese: [
'zh-CN-XiaoxiaoNeural',
'zh-CN-YunxiNeural',
]
};
Marketplace
Character Marketplace
Browse and download community characters:
- Open Character Market in sidebar
- Browse by category or search
- Preview character (3D model, personality)
- Download to local library
Publishing Characters
- Create character with complete configuration
- Click "Upload to Market"
- Fill metadata (tags, description)
- Submit for review
- GitHub issue created automatically
- Community review process
- Merged into marketplace
Dance Marketplace
Access MMD dance resources:
- Various choreography styles
- Multiple music genres
- Stage environments
- Community contributions
Deployment
Vercel Deployment (Recommended)
- Click "Deploy with Vercel"
- Connect GitHub account
- Configure environment variables
- Deploy
Docker Deployment
# Dockerfile
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build
FROM node:18-alpine AS runner
WORKDIR /app
COPY --from=builder /app/.next/standalone ./
COPY --from=builder /app/.next/static ./.next/static
COPY --from=builder /app/public ./public
EXPOSE 3000
CMD ["node", "server.js"]
docker build -t lobe-vidol .
docker run -p 3000:3000 \
-e OPENAI_API_KEY=sk-xxx \
lobe-vidol
Docker Compose
version: '3.8'
services:
lobe-vidol:
build: .
ports:
- "3000:3000"
environment:
- OPENAI_API_KEY=${OPENAI_API_KEY}
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
restart: unless-stopped
Self-Hosted Requirements
| Resource | Minimum | Recommended |
|---|
| CPU | 2 cores | 4 cores |
| RAM | 2 GB | 4 GB |
| Storage | 5 GB | 20 GB |
| Network | Broadband | Low latency |
PWA Support
LobeVidol supports Progressive Web App installation:
Desktop Installation
- Open https://vidol.lobehub.com in Chrome/Edge
- Click install icon in address bar
- Confirm installation
- Launch from desktop
Mobile Installation
- Open in mobile browser (Chrome/Safari)
- Tap "Share" or menu icon
- Select "Add to Home Screen"
- Launch from home screen
PWA Features
- Offline capable (cached assets)
- Native-like experience
- Push notifications (future)
- Background sync
API Reference
Chat Endpoint
interface ChatRequest {
messages: Array<{
role: 'user' | 'assistant' | 'system';
content: string;
}>;
model: string;
character?: string;
stream?: boolean;
}
interface ChatResponse {
content: string;
model: string;
usage: {
prompt_tokens: number;
completion_tokens: number;
total_tokens: number;
};
}
Character API
interface CharacterListResponse {
characters: Character[];
total: number;
}
interface CharacterResponse {
character: Character;
}
interface CreateCharacterRequest {
name: string;
systemRole: string;
model: string;
}
Troubleshooting
Common Issues
VRM Model Not Loading:
Error: Failed to load VRM model
- Verify VRM file URL is accessible
- Check CORS settings if hosted externally
- Ensure VRM version compatibility (0.x or 1.0)
Dance Animation Issues:
Error: VMD file parsing failed
- Check file encoding (Japanese characters)
- Verify VMD format compatibility
- Try re-exporting from MMD
API Connection Errors:
Error: Failed to connect to OpenAI
- Verify API key is valid
- Check proxy settings if behind firewall
- Confirm model name is correct
TTS Not Working:
Error: TTS initialization failed
- Enable Edge TTS as fallback
- Check browser permissions for audio
- Verify API key for OpenAI TTS
Performance Optimization
3D Rendering:
- Reduce model polygon count
- Disable shadows on mobile
- Use lower resolution textures
Network:
- Enable response streaming
- Cache static assets
- Use CDN for models
Best Practices
Character Design
- Clear Personality: Define distinct traits in systemRole
- Consistent Voice: Match TTS voice to character personality
- Natural Responses: Include varied touch responses
- Quality Model: Use optimized VRM with proper rigging
Performance
- Optimize Models: Keep VRM under 20MB
- Compress Textures: Use WebP or compressed PNG
- Stream Responses: Enable for better UX
- Cache Assets: Leverage browser caching
Security
- API Keys: Never expose in client-side code
- Rate Limiting: Implement for public deployments
- Input Validation: Sanitize user inputs
- CORS: Configure properly for custom domains
Resources
Official Links
Related Tools
Community Resources
- Character marketplace contributions
- Dance marketplace submissions
- Discord community support
- GitHub issues and discussions
Version History
- Current: Development active
- Tech Stack: Next.js, TypeScript, Three.js, three-vrm
- License: Apache 2.0
Last Updated: 2026-01-13
Skill Version: 1.0.0
Source: https://github.com/lobehub/lobe-vidol
