I went researching on DR just like @RWenwenwen.
I realized my definition wasn't specific, and my image was wrong. ✖️❌
Yes, Domain Randomization(DR) typically involves manipulating physical conditions of the environment and objects during replay.
But it's not that simple. 🙃
Here's how it works:
1. Teleoperation + Data Storage:
After teleoperating the robots to complete a task, data on the robot's joint positions, end-effector(gripper) states,.. etc during tasks are stored in an Axis Unified Trajectory format.📦
Saving it in this format allows it to be easily replayed within Axis AI while scaling the training data.
2. Replay & Domain Randomization:
This saved trajectory data contains the robot's joint coordinates, gripper actions,. etc. at specific times and has to be replayed in a more photorealistic environment for training. 🏕️🏞️🎥
While replaying trajectories (regenerating demonstrations) in a more photorealistic environment (NVIDIA Isaac Sim), alterations are made to the demonstrations, in:
• Object mass, textures, slipperiness, friction.✳️
• Environment lighting. 🔅
(These changes in physics, objects & environmental features is Domain Randomization).
Due to domain randomization + the replay (cross-simulator replay), the replayed demonstrations are more varied and photorealistic, such that problems like:
"A robot unfamiliar with lifting heavy objects/materials drops them due to insufficient force generation." are avoided, because it is familiar with heavy mass already...
So, these more photorealistic demonstrations are what are presented to the Vision-Language-Action (VLA) models to train them.🦾
Introducing Fluton.
- Fluton is a cross-chain execution layer that makes all onchain interactions private by default. 🗽
- It obscures transactional data, helping to protect user onchain activity, balances and strategies at every part of an onchain interaction. 👀
- Fluton achieves this by use of FHE (Fully Homomorphic Encryption).
- FHE enables this: encrypt transactions, and process on encrypted data without a need for decrypting.
Fluton's privacy infrastructure is beneficial to financial institutions seeking private execution onchain, onchain users seeking private payments, large investors and more. 🏛️⭕
See → @FlutonIO
GAxis (✱,✱)☀️
New week. New challenges. New access.
We’re giving away 10 BitRobot access codes over the next 72 hours.
Winners will get access to SN/04 and start earning rewards from both Axis and BitRobot.
To join:
1. Follow @axisrobotics & @BitRobotNetwork
2. Like + repost this post
3. Comment with a screenshot or photo of where you’re training right now
Grinding tasks? Climbing the leaderboard?
Show us your journey.
B U R N
We walked 'round the fire, time after time. ⏳
Thinking we would get burnt. 🔥
But Instead, we were formed 🪨
Through the Agneepath, we walked; and got ritualized. 🐦🔥
gRitual. 🕯️
Have a great week, Community 🍃
Cross-simulator replay on @axisrobotics. 🗺️
The axis robotics web-based simulations use MuJoCo, a lightweight simulator engine. 🕸️
The MuJoCo simulation used is appreciated for ease of access and running through a browser, but there exists a flaw:
'It doesn't accurately reflect real-world photorealistic environments good for VLA model training...' 🗺️
So, robotic trajectories generated on MuJoCo have to be replayed on a more photorealistic simulator engine (likely NVIDIA Isaac Sim), where objects & environment would appear more photorealistic just like they are in real life. 🏕️
Why?
VLA models which process, learn and perform from what they see cannot generalize on unrealistic data; they could get stuck operating in the real world.
However, native MuJoCo trajectories do not just replay in Isaac Sim.🙎🏼
So, Axis employs its Unified Abstraction Layer, and Unified Trajectory format.
Following this layer, Trajectory data is stored in a unified trajectory format. 💾
Using this format, the downstream replay simulator can replay stored trajectory data without getting confused by the different naming language/conventions of the simulator where the trajectory data was generated at. 🙂
This data storage format makes it possible for:
- The NVIDIA Isaac Simulator to replay MuJoCo trajectories (not perfectly though).🎬
- Downstream VLA models to understand the data during training. 👍🏼
During this cross-simulator replay, domain randomization (see quoted tweet) is applied to scale up data quality.
However, cross-simulator replay doesn't promise to reproduce perfectly same execution from MuJoCo.
It promises replay of the same seQueNce of events in completing task from MuJoCo; in a more photorealistic scene (due to DR).
Please, confirm from https://t.co/fBNukx4RyN if any doubts.
I went researching on DR just like @RWenwenwen.
I realized my definition wasn't specific, and my image was wrong. ✖️❌
Yes, Domain Randomization(DR) typically involves manipulating physical conditions of the environment and objects during replay.
But it's not that simple. 🙃
Here's how it works:
1. Teleoperation + Data Storage:
After teleoperating the robots to complete a task, data on the robot's joint positions, end-effector(gripper) states,.. etc during tasks are stored in an Axis Unified Trajectory format.📦
Saving it in this format allows it to be easily replayed within Axis AI while scaling the training data.
2. Replay & Domain Randomization:
This saved trajectory data contains the robot's joint coordinates, gripper actions,. etc. at specific times and has to be replayed in a more photorealistic environment for training. 🏕️🏞️🎥
While replaying trajectories (regenerating demonstrations) in a more photorealistic environment (NVIDIA Isaac Sim), alterations are made to the demonstrations, in:
• Object mass, textures, slipperiness, friction.✳️
• Environment lighting. 🔅
(These changes in physics, objects & environmental features is Domain Randomization).
Due to domain randomization + the replay (cross-simulator replay), the replayed demonstrations are more varied and photorealistic, such that problems like:
"A robot unfamiliar with lifting heavy objects/materials drops them due to insufficient force generation." are avoided, because it is familiar with heavy mass already...
So, these more photorealistic demonstrations are what are presented to the Vision-Language-Action (VLA) models to train them.🦾