The "Jarvis" Era for Smart Glasses has Arrived via Open Source
VisionClaw is an open-source iOS application designed to transform Meta Ray-Ban smart glasses into a fully agentic, real-time AI assistant. Developed by sseanliu, this project bridges the gap between wearable hardware and advanced multimodal AI models.
Key Technical Components:
🔹Visual & Audio Intelligence: It utilizes the Gemini Live API to process video frames and audio streams in real-time. This allows the AI to "see" what the user is looking at via the glasses' camera and converse naturally.
🔹Action Execution (Agentic): Unlike standard voice assistants, VisionClaw integrates with OpenClaw. This local gateway enables the AI to execute real-world tasks across 56+ connected tools, such as managing shopping lists, sending messages (WhatsApp/Telegram), controlling smart home devices, and performing web searches.
🔹Architecture: The app acts as a bridge, streaming low-latency video (approx. 1fps) and audio from the glasses to Gemini. When a command requires action, Gemini triggers a tool call which is routed back through the iOS app to the OpenClaw gateway running on the user's local network.
🔹Accessibility: The project includes an "iPhone Mode," allowing developers to test the full vision-to-action pipeline using the phone's camera and microphone without needing the specific smart glasses hardware.