Wow, this tweet went very viral!
I wanted share a possibly slightly improved version of the tweet in an "idea file". The idea of the idea file is that in this era of LLM agents, there is less of a point/need of sharing the specific code/app, you just share the idea, then the other person's agent customizes & builds it for your specific needs.
So here's the idea in a gist format: https://t.co/NlAfEJjtJV
You can give this to your agent and it can build you your own LLM wiki and guide you on how to use it etc. It's intentionally kept a little bit abstract/vague because there are so many directions to take this in. And ofc, people can adjust the idea or contribute their own in the Discussion which is cool.
Baidu just released a new VLM : PaddleOCR-VL-1.5🔥 @PaddlePaddle
https://t.co/AE4jqVadfV
✨ 0.9B - Apache 2.0
✨ 94.5% OmniDocBench v1.5
✨ Multilingual OCR: strong on rare characters & ancient texts
AI phones - large models or small models? We recently open-sourced OpenPhone📱— a 3B parameter mobile agent foundation model! After a year of trial and error, here's what we learned about AI phones ✨
Open-Sourced AI Phone Agents: https://t.co/7qF3qItBvC
🤔 How do AI phones actually work?
Simple: AI helps you operate your phone. But how does AI communicate with different apps?
Option 1: API Calls 🔌
Ideally, we'd just call app APIs directly. Reality check — there are basically none! Big tech won't open their APIs because apps ARE their traffic moat. Building individual MCPs for each app? Engineering nightmare 💥
Option 2: GUI Interaction 🖱️
Since no APIs, let's do what humans do — look at screens and tap stuff. This approach is super generalizable, should work with any app. That's why most AI phones go the GUI Agent route now.
GUI Agents are basically multi-modal models:
- Input: screenshot + task description
- Output: coordinates for next tap
- Capability: screen understanding + task reasoning
📱 Three technical approaches for Phone Agents
- Pure cloud ☁️
What most AI phones do currently — heavily rely on cloud-based large models. Performance is definitely better than small models, but privacy🔒 and cost💰 concerns are real.
- Pure on-device models 📱
This is the direction OpenPhone is exploring. 3B parameters strikes a good balance — runs on phones, fast, private, and cost-effective. The trade-off is limited performance on complex tasks, given it's only 3B parameters.
- Hybrid edge-cloud 🤝
Probably the most practical route. Simple stuff and anything privacy-sensitive stays on-device, complex reasoning hits the cloud. The trick is the routing strategy — when to make the switch? Interesting part is teaching the on-device model to recognize its own capability boundaries.
🔮 Some Random thoughts
1. GUI Agents still have plenty of issues: slow, error-prone, multi-app accuracy sucks. Rich MCP ecosystem would make life easier, but don't hold your breath.
2. Right now everyone's just collecting data, then SFT+RL to optimize models. Basically throwing data at the problem — hopefully we get smarter ways to do this.
3. AI phone ceiling isn't just tech — it's ecosystem. Future apps might go dual mode: APIs for agents, GUI for humans🚀
4. Computer-Use Agents are shifting toward coding — writing code instead of just clicking around💻, because code execution is way more accurate and efficient. Works great on desktop, mobile's still challenging.
5. Future Digital Agents might need to pack everything into one model: coding + multimodal + tool-use.