At @kubocreate, we were working on learnable novelty and epiplexity using spectral objectives and we thought, why not apply this in an optimiser? We ran some experiments and then realized that it’s basically Muon. I went down the rabbit hole anyway. Read below:
We’re exploring how local video understanding could help wearable interfaces respond to what someone is doing.
We’ve shared the demo, measurements, and engineering details here. It's not perfect, but we're actively working to improve it!
https://t.co/TAOcKDJaCr
We ran V-JEPA 2.1’s full encoder and predictor locally on an iPhone 15 Pro.
That’s about 1.9 billion parameters, with no server involved in inference.
Here’s how we made it work. 🧵
The loaded models took about 108 ms per run with a fixed input.
In a separate live camera test, the app averaged about five updates per second for over five minutes.
The phone ran hot. Sustaining this performance with less heat is our next target.
@immad@VarunMayya And you could also use AI to launch your product quicker, right now. You can launch super quick, test and evolve as AI evolves.
Something that we’re doing. :”)