We’re sharing the next major milestone in our non-invasive brain-to-text decoder research: Brain2Qwerty v2.
Building on v1, which was published today in @Nature, Brain2Qwerty v2 is the highest-performing end-to-end pipeline capable of real-time sentence decoding from raw brain signals. It advances beyond character-level performance to decoding words and semantics, enabling accuracy for overall communication.
We believe this research has the potential to make a real difference for the millions of people who suffer from brain lesions or disorders that prevent them from communicating.
🧵👇
Worked with physicist colleagues +@claudeai for a few weeks on this:
A transformer with no positional encodings and no Q/K/V attention — yet word order is built in for free.
Matches standard baselines on text, and early generation looks great.
Paper soon. 🚀
- <1B params
- supports 91 languages
- 5 pages/s on RTX 5090
- runs on CPU, GPU, MPS
- 83.3% olmocr bench score (top under 3B)
Surya OCR is a state-of-the-art model for document intelligence.
100% open-source.
Forget lidar.
One single camera.
Runs in real time & is open source:
A streaming 3D model that reconstructs scenes live, at ~20 FPS, over long sequences.
End-to-end.
Optimization tricks, cleanup steps?
Nope.
And it beats both streaming and even some offline methods.
Perception is becoming software-first.
Closer to machines that see and understand the world as it unfolds.
Thanks for sharing, @YinghaoXu1
📍Models: https://t.co/gnSDy919eX
Project page: https://t.co/zgpkgBvcik
Code: https://t.co/Js0MzHE387
Paper: https://t.co/FrzMojMMZC
——
Weekly robotics and AI insights.
Subscribe free: https://t.co/9Nm01QUcw3
NVIDIA Metropolis Blueprint for video search and summarization (VSS) 3 is here.
Now your coding agent can analyze massive live streams and libraries of videos with a simple natural language prompt. Here's what's new:
- 16 new agent skills: Search, summarize, alert, report, review clips. All from natural language prompts.
- One unified open source repo: Source code, Docker and Helm deployment profiles for fast, easy deployment.
- Multi-video reports and Nemotron 3 Nano Omni: Insights across video and audio at scale.
- 3D multi-camera tracking: Production ready + #1 SOTA for smarter scene understanding.
Try VSS skills 👉 https://t.co/XvKJ0Kb8VV
This is a new paradigm for interacting with Claude that is significantly more "inline" with all the other human activity org-wide. Once you do all of the under the hood engineering work to make this "just work" (e.g. across tools, integrations, compute environments, memory, security, etc.), Claude basically joins the team in a seamless way - you can talk to it as you would talk to a person and it can help with a very large variety of workloads.
Imo this is the 3rd major redesign of LLM UIUX. The first paradigm was that the LLM is a website you go to, the second was that it is an app you download to your computer. This third one is that it is a self-contained, persistent, asynchronous entity with org-wide tools and context, working alongside teams of humans. It really takes a while to wrap your head around it, but it works and it is awesome.
Object detection on 3D Gaussian Splats.
Accurate bounding boxes per detection, runs locally. @nigelhartmanxr really cooked here
@SensAIHackademy is hosting a lecture with @XRarchitect from @theworldlabs on the 8th on this:
https://t.co/wOWO17Pw5G
Anime.js 4.5 is out and it's a fun one:
Introducing the @threejs adapter 🎉
- Up to 50% less code for 3D animations
- CSS transform-like API for 3D objects (rotate, skew…)
- Simpler material color animations
- Easy instanced mesh animations
- Stagger 3D
And so much more! ⬇︎
Arbor: Control 3D Generation with Explicit Geometry
Text-to-3D makes plausible assets, but spatial control is still hard. Arbor adds typed 3D constraint: hull, avoidance, and touch.
Paper w/ @markb_boss@AjlEngelhardt @Simondonne @CG_Tuebingen Intern @StabilityAI
More 👇
This TTS model generates speech 167x faster than you can hear it.
Supertonic is an on-device TTS engine that runs via ONNX for cross-platform inference.
- no GPU
- 31 languages
- captures every emotion
- beats ElevenLabs on speed
- runs even on a Raspberry Pi
100% open-source.
[1/4]
Current 3D human pose reconstruction models are very impressive, but which model should you pick for your application? Introducing the Caltech Tennis Dataset (CalTennis), a large-scale video benchmark for evaluating monocular-to-3D pose estimation in the wild. CalTennis comprises over 11 million frames (51 hours) of tennis practice and match play. It is 10× larger than existing in-the-wild human motion video datasets and 3× larger than existing MOCAPground-truthed datasets, and it is the first large-scale benchmark to provide synchronized multi-view recordings of expert athletic motion.