openclaw-windows-node

# OpenClaw Windows Node > Skill by [ara.so](https://ara.so) — Hermes Skills collection. Windows companion suite for OpenClaw AI assistant. Provides a WinUI 3 system tray app (Molty), shared gateway client library, and CLI utilities. The tray app connects to the OpenClaw gateway via WebSocket, displays status/sessions/usage/channels/nodes, provides Quick Send, embedded Web Chat, toast notifications, and optional Node Mode for agent control of Windows. ## What It Does - **System Tray App** - WinUI 3 app with flyout menu, gateway status, sessions, channels, usage, nodes, Command Center diagnostics, auto-updates - **Node Capabilities** - When enabled, exposes Windows as a controllable node: `system.notify`, `system.run`, `canvas.present`, `screen.snapshot`, `camera.snap`, `stt.transcribe`, `tts.speak`, etc. - **Gateway Client** - Shared C# library for WebSocket connection, pairing, pub/sub, RPC - **CLI Validator** - Command-line tool to test gateway connectivity and `chat.send` - **Auto-Updates** - GitHub Releases integration with channel selection (stable/alpha/dev) - **Deep Links** - `openclaw://` URL scheme with named pipe IPC for external activation ## Installation ### Prerequisites ```powershell # Windows 10 20H2+ or Windows 11 # .NET 10.0 SDK winget install Microsoft.DotNet.SDK.10 # Windows 10 SDK (for WinUI build) - via Visual Studio or standalone # WebView2 Runtime (pre-installed on modern Windows) ``` ### Build ```powershell # Check prerequisites .\build.ps1 -CheckOnly # Build all projects .\build.ps1 # Build specific project .\build.ps1 -Project WinUI # Build for ARM64 dotnet build src/OpenClaw.Tray.WinUI/OpenClaw.Tray.WinUI.csproj -r win-arm64 # Build MSIX package (for camera/mic consent prompts) dotnet build src/OpenClaw.Tray.WinUI -r win-x64 -p:PackageMsix=true ``` ### Run Tray App ```powershell # Build and launch the tray app .\run-app-local.ps1 # Run without rebuilding .\run-app-local.ps1 -NoBuild # Run isolated (separate settings file for testing) .\run-app-local.ps1 -Isolated # Test alpha updates from Release build .\run-app-local.ps1 -Configuration Release -Isolated -UpdateChannel alpha ``` ### Run CLI ```powershell # Show help dotnet run --project src/OpenClaw.Cli -- --help # Send message using tray settings dotnet run --project src/OpenClaw.Cli -- --message "test message" # Loop sends with probing dotnet run --project src/OpenClaw.Cli -- --repeat 5 --delay-ms 1000 --probe-read --verbose # Override gateway URL/token dotnet run --project src/OpenClaw.Cli -- --url ws://127.0.0.1:18789 --token "$env:OPENCLAW_TOKEN" --message "test" ``` ## Project Structure ``` openclaw-windows-node/ ├── src/ │ ├── OpenClaw.Tray.WinUI/ # System tray app (WinUI 3) │ │ ├── Services/ # Gateway, Notification, Update, Node │ │ ├── ViewModels/ # MVVM view models │ │ ├── Views/ # XAML pages and windows │ │ ├── Capabilities/ # Node capability implementations │ │ └── Models/ # Data models │ ├── OpenClaw.Shared/ # Gateway client library │ │ ├── GatewayClient.cs # WebSocket client │ │ ├── Models/ # DTOs for gateway API │ │ └── Extensions/ # Helper methods │ └── OpenClaw.Cli/ # CLI validator └── docs/ # Documentation ``` ## Key Components ### GatewayClient (Shared Library) The core WebSocket client for OpenClaw gateway integration. ```csharp using OpenClaw.Shared; using OpenClaw.Shared.Models; // Create client var client = new GatewayClient("ws://127.0.0.1:18789", "your-token-here"); // Handle events client.MessageReceived += (s, msg) => { Console.WriteLine($"Topic: {msg.Topic}, Data: {msg.Data}"); }; client.ConnectionStateChanged += (s, state) => { Console.WriteLine($"Connection: {state}"); }; // Connect await client.ConnectAsync(); // Subscribe to topics await client.SubscribeAsync(new[] { "gateway.sessions", "gateway.usage", "gateway.channels" }); // Send RPC request var response = await client.SendRequestAsync<ChatSendResponse>( "chat.send", new { message = "Hello from Windows!", sessionId = "session-123" } ); // Unsubscribe await client.UnsubscribeAsync(new[] { "gateway.sessions" }); // Disconnect await client.DisconnectAsync(); ``` ### Node Registration and Capabilities When Node Mode is enabled, the tray app registers as a node and handles capability requests. ```csharp // In NodeService.cs public class NodeService { private readonly GatewayClient _gatewayClient; private readonly Dictionary<string, INodeCapability> _capabilities; public async Task RegisterNodeAsync() { // Register capabilities var capabilities = new[] { "system.notify", "system.run", "canvas.present", "screen.snapshot", "camera.snap", "stt.transcribe", "tts.speak", "location.get", "device.info" }; var request = new { deviceId = GetDeviceId(), capabilities = capabilities, metadata = new { platform = "windows", version = GetAppVersion(), hostname = Environment.MachineName } }; await _gatewayClient.SendRequestAsync<object>("node.register", request); } public async Task HandleNodeRequestAsync(NodeRequest request) { if (!_capabilities.TryGetValue(request.Method, out var capability)) { throw new NotSupportedException($"Capability not supported: {request.Method}"); } var result = await capability.ExecuteAsync(request.Params); await _gatewayClient.SendRequestAsync<object>("node.response", new { requestId = request.Id, result = result }); } } ``` ### Implementing a Node Capability ```csharp // In Capabilities/SystemNotifyCapability.cs using Windows.UI.Notifications; using Microsoft.Toolkit.Uwp.Notifications; public class SystemNotifyCapability : INodeCapability { public string Method => "system.notify"; public async Task<object> ExecuteAsync(JsonElement? parameters) { var title = parameters?.GetProperty("title").GetString() ?? "Notification"; var message = parameters?.GetProperty("message").GetString() ?? ""; var priority = parameters?.GetProperty("priority").GetString() ?? "normal"; // Create toast notification new ToastContentBuilder() .AddText(title) .AddText(message) .AddArgument("action", "viewConversation") .AddArgument("conversationId", Guid.NewGuid().ToString()) .Show(); return new { success = true }; } } ``` ### Screen Snapshot Capability ```csharp // In Capabilities/ScreenSnapshotCapability.cs using System.Drawing; using System.Drawing.Imaging; using Windows.Graphics.Capture; public class ScreenSnapshotCapability : INodeCapability { public string Method => "screen.snapshot"; public async Task<object> ExecuteAsync(JsonElement? parameters) { var displayIndex = parameters?.GetProperty("display").GetInt32() ?? 0; var format = parameters?.GetProperty("format").GetString() ?? "png"; // Capture screen using Windows.Graphics.Capture var bounds = Screen.AllScreens[displayIndex].Bounds; using var bitmap = new Bitmap(bounds.Width, bounds.Height); using var graphics = Graphics.FromImage(bitmap); graphics.CopyFromScreen( bounds.Location, Point.Empty, bounds.Size ); // Convert to base64 using var ms = new MemoryStream(); var imageFormat = format.ToLower() switch { "jpg" => ImageFormat.Jpeg, "jpeg" => ImageFormat.Jpeg, _ => ImageFormat.Png }; bitmap.Save(ms, imageFormat); var base64 = Convert.ToBase64String(ms.ToArray()); return new { data = base64, format = format, width = bounds.Width, height = bounds.Height }; } } ``` ### Camera Capability ```csharp // In Capabilities/CameraCapability.cs using Windows.Media.Capture; using Windows.Media.MediaProperties; using Windows.Storage; public class CameraSnapCapability : INodeCapability { public string Method => "camera.snap"; public async Task<object> ExecuteAsync(JsonElement? parameters) { var deviceId = parameters?.GetProperty("deviceId").GetString(); var format = parameters?.GetProperty("format").GetString() ?? "jpg"; // Initialize MediaCapture var capture = new MediaCapture(); var settings = new MediaCaptureInitializationSettings(); if (!string.IsNullOrEmpty(deviceId)) { settings.VideoDeviceId = deviceId; } await capture.InitializeAsync(settings); // Capture photo var file = await ApplicationData.Current.TemporaryFolder .CreateFileAsync($"snap_{Guid.NewGuid()}.{format}"); var imageEncoding = format.ToLower() == "png" ? ImageEncodingProperties.CreatePng() : ImageEncodingProperties.CreateJpeg(); await capture.CapturePhotoToStorageFileAsync(imageEncoding, file); // Read and encode var buffer = await FileIO.ReadBufferAsync(file); var bytes = buffer.ToArray(); var base64 = Convert.ToBase64String(bytes); // Cleanup await file.DeleteAsync(); capture.Dispose(); return new { data = base64, format = format }; } } ``` ### STT (Speech-to-Text) Capability ```csharp // In Capabilities/SttTranscribeCapability.cs using Windows.Media.Capture; using Windows.Media.SpeechRecognition; public class SttTranscribeCapability : INodeCapability { public string Method => "stt.transcribe"; public async Task<object> ExecuteAsync(JsonElement? parameters) { var durationSec = parameters?.GetProperty("durationSec").GetDouble() ?? 5.0; var language = parameters?.GetProperty("language").GetString() ?? "en-US"; // Check if STT is enabled in settings if (!SettingsService.Current.IsSttEnabled) { throw new InvalidOperationException("STT is disabled in settings"); } // Initialize speech recognizer var recognizer = new SpeechRecognizer(new Windows.Globalization.Language(language)); await recognizer.CompileConstraintsAsync(); // Start continuous recognition var tcs = new TaskCompletionSource<string>(); var recognizedText = new StringBuilder(); recognizer.ContinuousRecognitionSession.ResultGenerated += (s, e) => { if (e.Result.Status == SpeechRecognitionResultStatus.Success) { recognizedText.AppendLine(e.Result.Text); } }; await recognizer.ContinuousRecognitionSession.StartAsync(); await Task.Delay(TimeSpan.FromSeconds(durationSec)); await recognizer.ContinuousRecognitionSession.StopAsync(); recognizer.Dispose(); return new { text = recognizedText.ToString().Trim(), durationSec = durationSec, language = language }; } } ``` ### TTS (Text-to-Speech) Capability ```csharp // In Capabilities/TtsSpeakCapability.cs using Windows.Media.SpeechSynthesis; using Windows.Media.Playback; public class TtsSpeakCapability : INodeCapability { public string Method => "tts.speak"; public async Task<object> ExecuteAsync(JsonElement? parameters) { var text = parameters?.GetProperty("text").GetString() ?? throw new ArgumentException("text is required"); var voice = parameters?.GetProperty("voice").GetString(); var provider = parameters?.GetProperty("provider").GetString() ?? "system"; if (provider == "elevenlabs") { return await SpeakWithElevenLabsAsync(text, voice); } // Use Windows speech synthesis var synthesizer = new SpeechSynthesizer(); if (!string.IsNullOrEmpty(voice)) { var selectedVoice = SpeechSynthesizer.AllVoices .FirstOrDefault(v => v.DisplayName == voice); if (selectedVoice != null) { synthesizer.Voice = selectedVoice; } } var stream = await synthesizer.SynthesizeTextToStreamAsync(text); var player = new MediaPlayer(); player.Source = Windows.Media.Core.MediaSource.CreateFromStream( stream, stream.ContentType ); player.Play(); // Wait for playback to complete var tcs = new TaskCompletionSource<bool>(); player.MediaEnded += (s, e) => tcs.SetResult(true); await tcs.Task; return new { success = true }; } private async Task<object> SpeakWithElevenLabsAsync(string text, string voice) { var apiKey = Environment.GetEnvironmentVariable("ELEVENLABS_API_KEY") ?? throw new InvalidOperationException("ELEVENLABS_API_KEY not set"); var voiceId = voice ?? "21m00Tcm4TlvDq8ikWAM"; // Default voice using var httpClient = new HttpClient(); httpClient.DefaultRequestHeaders.Add("xi-api-key", apiKey); var request = new { text = text, model_id = "eleven_monolingual_v1", voice_settings = new { stability = 0.5, similarity_boost = 0.75 } }; var response = await httpClient.PostAsJsonAsync(