PathOnAI.org

discord:
501(c)(3) open-source research nonprofit | 🏆 SF LeRobot Hackathon '25 | 6 papers (KDD/CIKM/WWW/NAACL/ECML-PKDD/NeurIPS)

[PathOnAI.org @PathOnAIOrg]

Our collaboration with @PathOn_Robotics

[Rex Zhang @RexDQZhang]

🤖 @PathOn_Robotics has been open source since day one. Today we're releasing something we never shipped.
From January to May, PathOn Robotics built an end-to-end "bring your own robot" pipeline — text → CAD → URDF/MJCF → gripper attach → ROS 2, every stage driven by coding agents. Describe an arm in a sentence, get a simulation-ready model and a working MoveIt stack.

Then we pivoted.

Rather than let five months of work sit in a private repo, we're releasing it to the DIY maker community. Out today: the architecture, the validated cases — what worked and what broke — and the design decisions we'd still defend. The load-bearing one: gripper attachment happens at the URDF/MJCF layer, not CAD, so you can bolt a custom hand onto an off-the-shelf arm without ever opening CAD.

One stage is already runnable today: end-effector attachment, as a Claude Code skill.

Coming soon: the rest of the agent skills, plus walkthroughs of each stage.
Generative robotics needs more shared plumbing. Hope this is useful to someone building.

https://t.co/xGxVUb6PHX

[PathOnAI.org @PathOnAIOrg]

Our collaboration with @PathOn_Robotics

[Rex Zhang @RexDQZhang]

@PathOn_Robotics has been open source since day one. Newest drop: an isomorphic 3D-printed leader arm for teleoperating the @AgilexRobotics PiPER arm .

The usual way to teleoperate a PiPER is to buy a second PiPER and drive it in teach mode. $3,000 for something to hold.
Ours prints for about $100.
Isomorphic means the leader shares the follower's joint structure — angles map straight across. No IK, no retargeting. You move the leader, the PiPER goes exactly there.
- 6 DOF on Feetech SCS215 serial bus servos
- Squeeze trigger at the wrist for the gripper
- Largest part is 220 mm — fits a 256 mm bed. No printer upgrade
- Full STEP assembly + 7 printable STLs
This is what open source since day one buys: when we need a tool, the catalog is already ours. We print it Friday and run it Monday.
👉 https://t.co/QeLpBByX6J
#robotics #opensource #teleoperation #embodiedAI

[Rex Zhang @RexDQZhang]

Looked back at my old posts today.

In 2012, bored with school, I threw myself into business development/consulting internships (BCG, EY-Parthenon), having no idea what was coming. By summer 2013, I realized a hard truth: I needed to build deep technical skills early in my career, and handle the business side later. I connected the dots and found Geospatial Analytics & Spatial Intelligence.
So, I made a hard pivot to teach myself AI/ML from absolute zero.

I didn't even have access to Google—I spent days just figuring out how to set up a VPN. No ChatGPT, no shortcuts. Somehow, I stumbled onto Kaggle and deep learning papers when I barely knew how to write a script. Just raw curiosity and grinding until 3 AM every single night trying to understand Python, ML, and algorithms.

Today, I'm a deeply technical builder—an AI Researcher (20+ papers at top conferences, 1k+ citations) and a Full-Stack Engineer spanning ML, Infra, and Robotics.

The pivot was brutal, but it's the best decision I ever made. It's not about where you start; it's what you're willing to learn in the dark.

Whenever I start something new, I remind myself: if I could figure it out back then with absolutely nothing, I can figure out anything now. 🛠️🤖

[Rex Zhang @RexDQZhang]

Testing our custom navigation stack right out of the basement. 🤖🐕🗺️

Turns out you don't need a multi-million dollar corporate robotics lab to test edge cases. A basement and a retaining wall work just fine.

Hyper capital-efficient engineering. We keep shipping. 🛠️🤖

[Rex Zhang @RexDQZhang]

Generative robotics needs an integration layer — this is my robot-assets-skills. 🤖🛠️

AI makes 3D design much faster now. Industrial-grade arms run real algorithms. The combo is the unlock — but mounting the two is the slow, boring part everyone hates.

It's a Claude Code bundle for robot URDF/MJCF integration. The flagship skill — attach-end-effector — mounts any hand/gripper on any arm in MuJoCo with zero errors. Extensively tested on Piper, SO-101 6DoF (ours), Elephant Robotics arms — paired with Allegro, Barrett, and our Symmetric Gripper.

[Rex Zhang @RexDQZhang]

Bought a Unitree Go2. It doesn't ship with 3D LiDAR.

Designed a bracket. Printed it in the basement. Mounted a Livox MID-360 on top. Same bracket we use in our internal stack.

What sensor should I add next? 🤖🛠️

For anyone wanting to repro — open-sourced the mount: STL, hardware list, install notes 👇

[Rex Zhang @RexDQZhang]

Welcome to the official PathOn Robotics 'Testing Facility' (aka the robot gym). Also known as the corner of my basement right next to my gym.

We've been running our custom sensor/software on the Go2 for our initial GTM. Bypassing the factory brain and injecting an enterprise-grade OS requires zero fancy office space—just a clear floor, a bright light, and absolute focus.

Hyper capital-efficient build mode. Let's keep shipping.🤖🛠️

[Rex Zhang @RexDQZhang]

Stock SO-101: 5 DoF + non-symmetric gripper.
SOTA grasp models: 6D poses + symmetric grippers.

So we upgraded the SO-101 to 6DoF and designed a symmetric parallel-jaw gripper — so any SOTA grasp model drops in. 🦾

Here's the full arc 🧵👇

[Elon Musk @elonmusk]

Congrats to the @Tesla_AI chip design team on taping out AI5!

AI6, Dojo3 & other exciting chips in work.

[PathOnAI.org @PathOnAIOrg]

🎉 LiteWebAgent just hit 100 stars on GitHub! ⭐
Thank you to everyone who has supported this project. Excited to keep building! 🚀
Check it out: https://t.co/4c1TJ0Yzx1
#OpenSource #WebAutomation

[Rex Zhang @RexDQZhang]

I took a day off but wrapped up some scripts on teleoperating a SO100 arm in a MuJoCo environment. I need to learn ManiSkill and Isaac for the simulation—I feel this is still a bit raw.

[Rex Zhang @RexDQZhang]

🤖 Expensive Lessons in Robotics: When a $20 Servo Teaches You About Safety Checks
I unfortunately burnt the gripper servo of my so-arm100 follower arm last weekend because I recalibrated my leader arm and then went straight ahead with teleoperation. A $20 lesson since I need to buy a new servo!
But here's the thing about failures in robotics – they force you to add proper safety checks.
Instead of just replacing the part, I took the time to:
✅ Build a new follower arm from scratch
✅ Write comprehensive testing scripts for calibration
✅ Add crucial safety checks before running teleoperation
✅ Create real-time positioning controls for the leader arm
The result? A system that safely moves to zero position and resets to rest position with proper validation at each step – no more burnt servos!

script: https://t.co/H8UxCObEUV

[Rex Zhang @RexDQZhang]

🎉 Exciting News! We're thrilled to announce that our paper "Visual Tree Search of Web Agent" has been accepted to ECML-PKDD 2025, one of the premier European conferences in machine learning and data science!
This breakthrough work comes from our talented https://t.co/hX1PBlxf2l community members, with another web agent paper in the pipeline.
What is VisualTreeSearch?It's a fully-deployed system that makes web agent decision-making transparent and interpretable. For the first time, researchers and practitioners can observe how AI agents navigate and make decisions on the web in real-time.
Key innovations include:
⚡ Ultra-fast API-based state reset (50s → 2s)

☁️ Scalable cloud infrastructure with WebSocket + ECS
🌳 Interactive tree visualization with live browser execution
🧠 Support for advanced algorithms including LATS (Language Agent Tree Search)
This open-source system bridges the critical gap between research and real-world deployment, providing essential infrastructure for debugging web agents, analyzing search strategies, and prototyping new planning algorithms.

Explore the project:
🔗 Project details: https://t.co/NgUUeZUCvd
🎮 Live demo: https://t.co/44f57hps1T💻 GitHub: https://t.co/cI8nGBWirF
#MachineLearning #AI #WebAgents #OpenSource #Research #ECMLPKDD2025

[Rex Zhang @RexDQZhang]

@LeRobotHF #LeRobotWorldwideHackathon

[Rex Zhang @RexDQZhang]

🤖 Excited to share my latest open-source project: a voice-controlled agentic robot, right before @huggingface @LeRobotHF Hackathon!
I've released the complete codebase for a system that seamlessly bridges natural language and robotic action:
🎙️ Deepgram ASR converts voice commands to text
🧠 OpenAI function calling translates text into precise robot instructions
⚡ Fine-tuned Pi0 model executes tasks via remote inference server
This demo showcases the potential of multimodal AI systems where voice becomes the primary interface for robotic control.
Check out the full implementation: https://t.co/N6986P8TQL
#AI #Robotics #OpenSource #VoiceAI #LeRobot

[Rex Zhang @RexDQZhang]

🤖 很兴奋分享我的最新开源项目：语音控制智能机器人，正值 @huggingface @LeRobotHF

我发布了一个完整的代码库，实现了自然语言与机器人动作的无缝连接：

🎙️ Deepgram ASR 将语音指令转换为文本 🧠 OpenAI 函数调用将文本转译为精确的机器人指令 ⚡ 微调的 Pi0 模型通过远程推理服务器执行任务

这个演示展示了多模态AI系统的潜力，语音成为机器人控制的主要交互界面。

查看完整实现：https://t.co/N6986P8TQL

#人工智能 #机器人技术 #开源 #语音AI #lerobot

[PathOnAI.org @PathOnAIOrg]

🎉 We just hit 100 members on the PathOnAI Discord!
🛠️ Building open-source tools. 🧠 Exploring AGI.
👾 Want to shape the future with us? Join here → https://t.co/TbLEeZI8bG
#AGI #OpenSource #AICommunity #Discord

[PathOnAI.org @PathOnAIOrg]

https://t.co/vhfhZOr8TS is officially registered as a 501(c) non-profit organization today🚀.

[Rex Zhang @RexDQZhang]

🚀 We've just launched a major website update at https://t.co/XNsv5r48ys! Connect with us across platforms:
▪️ LinkedIn: https://t.co/AGNnBcDaYa
▪️ GitHub: https://t.co/dFNwQ2jFZL
▪️ Twitter: https://t.co/6UGSMZ0QRD
Follow us to stay updated on our latest developments! #PathOnAI

[PathOnAI.org @PathOnAIOrg]

🚀 LiteWebAgent presented at NAACL 2025 this week! If you're interested in exploring AI research together, join our Discord: https://t.co/2rbP6EV9XW

[Rex Zhang @RexDQZhang]

Excited to introduce LiteWebAgent, an open-source suite for VLM-based web agents showcased at NAACL 2025 this week! This project offers a range of key features:

- Extensible agent framework incorporating planning, memory & tree search capabilities
- FastAPI integration enabling serverless deployment
- Interactive UI featuring voice control & real-time visualization
- Flexible deployment options including web app & Chrome extension

LiteWebAgent serves as a bridge between research and practical application, equipping users with all the essentials for real-world web agent implementations.

📄 Paper: Read more about the project here: https://t.co/g8HeYv06Bs
🔗 Project: Explore LiteWebAgent on https://t.co/hX1PBlxf2l: https://t.co/TAf9JoQBcK
⭐ GitHub: Access the project repository on GitHub: https://t.co/CwL5jI4OKV

Additionally, we have recently wrapped up VisualTreeSearch, our subsequent endeavor focusing on interpretable tree search within web agents. Learn more about this follow-up project here: https://t.co/DLTRenUyMG

#WebAgents #NAACL2025 #PathOnAI