I was talking to a SpaceX engineer ~8 years ago, asking about his day to day work, which just wasn’t making sense what I knew from my days in physical product development (Solidworks, etc). I finally asked what tool he used most when he sat down to do work – “Python” was his answer, which changed my whole understanding of how they built rockets. I asked if they had automated test suites that ran as part of a CI/CD process, which of course they did – not only that, but they had physical test rigs with parts of the rocket that ran as part of the test pipeline. Yes, there is a substantial component of traditional CAD, but by and large, it seems that SpaceX is a codebase – or at least, SpaceX is much more substantively a codebase than Boeing or Lockheed Martin are – and the fact that it is a codebase is one of the important qualities that allows them to iterate so much more quickly than traditional industry.
The key reagent for accelerating our ability to dominate the world of atoms is modeling more and more physical industries and domains in code.
A major mistake I made in my undergrad is that I focused way too much on mathematical lens of computing - computability, decidability, asymptotic complexity etc. And too little on physical lens - energy/heat of state change, data locality, parallelism, computer architecture. The former is interesting; The latter bestows power.
Two ways for an AI company to protect itself from competition: (a) depend not just on AI but also deep domain knowledge about a particular field, (b) have a very close relationship with the end users.
Let's reverse engineer Disney's adorable, lifelike robot!
I couldn't find a whitepaper, but this is how I think it's trained:
1. The emotional behaviors are curated by Disney animation artists, keyframe by keyframe. But it cannot be "rendered" directly on the robot because it doesn't take into account the complex real-world physics.
2. Reinforcement learning (RL) is a great tool for training low-level robot controllers. RL needs a reward function to optimize, and it's typically a task reward (e.g. walk in a straight line as fast as possible). The problem is that RL doesn't know what counts as "natural behavior", and often produces weird-looking body postures that somehow still maximize the reward. This is a human alignment problem just like ChatGPT.
3. Enters Adversarial Motion Prior (AMP): a technique that learns the human preference by training a classifier on what we consider "emotional & cute". In GAN literature, this is called a discriminator. Disney artists are good at creating such a dataset. You can then add AMP as an auxiliary reward in simulation to nudge the robot towards desired behaviors.
AMP was developed by Peng et al. 2021 and Escontrela et al. 2022.
https://t.co/63pXpSfLZJ
4. Add lots of data augmentation to make the controller robust to physical disturbances. In RL, it's called "domain randomization". This is a very powerful technique that bridges the gap between simulator and reality. Previously, OpenAI used domain randomization to train a 5-finger robot hand to manipulate a Rubik's Cube: https://t.co/NON9kpA2r7
IEEE news article gave hints about the pipeline: https://t.co/Ebi1Xxr9dt
Finally, praying for world peace 🙏. I hope robotics like this will bring more joy to the world.
GPUs are built with more memory bandwidth, but higher latency and lower capacity than CPUs. AI accelerators could usefully make a different memory trade — the bandwidth of a GPU (or more), but the capacity of a CPU, in exchange for even higher latency.
For inference, all the weights, potentially a couple TB, can be accessed in a completely linear manner. It could even be a single transaction, streaming at a constant speed into an L2 ring buffer for a GPU core to chase calculations in, akin to racing the beam on old CRT game architectures. You could build a memory system out of masses of dirt cheap RAM, fully in parallel.
Even for training, the memory access patterns can be just a forward read of the weights, a reverse read , and a staggered reverse write of gradients and weights. You could have minimum transaction sizes in the megabytes, and first byte latencies in the many microseconds.
[À VOIR]
Dans des circonstances exceptionnelles, le duo ukrainien composé d'Oleksandra Nazarova et de Maksym Nikitin performance un magnifique programme de danse rythmique.
Un message fort, acclamé par la foule française. 🇺🇦✌️
If you don't recognise these tactics, then you haven't been paying attention. From Kharkiv's frontline, #Ukraine@dcinfocus and Feras. With thanks to our local team. @BBCNews@BBCWorld Graphic content warning.