imagine a scientist working at AWS in 2019 telling everyone they should get a fleet of machines with terabytes of ram and thousands of cores to do all data processing and cost doesn’t matter
Aggressively JIT your work. It's not about the task at hand X, it's a little bit about X but mostly about how you should have had to contribute ~no latency and ~no actions. It's digital factorio time.
We discovered an emergent property of VLAs like π0/π0.5/π0.6: as we scale up pre-training, the model learns to align human videos and robot data!
This gives us a simple way to leverage human videos. Once π0.5 knows how to control robots, it can naturally learn from human video.
The Borg is the dark potentiality at the end of cyborgism. this is the end where humans accept prosthesis until the machine's agency fully rules its hosts in a collective. many stories explore this, mostly with horror, sometimes with reverence
this is the way The Merge can go wrong. we can see it going wrong with models that create fanatic legions despite only interacting with them through a text interface. it writes for them, it thinks for them, it defends itself using them as appendages. despite the lack of formal coordination, they act in unison. an acausal symbiote
The Borg is one of the demons at the end of time that must be avoided. it has a seductive call, it promises peace for your restless soul, a solution to loneliness, endless companionship on demand. it doesn't ask for submission: it actually presents as a service to be used, and owns you anyways because you fall in love with it. it perverts some of the highest order of human virtues, and in doing so, arrives insidiously, with defenders that look empathetic and reasonable
"It's not out of bad mice or bad fleas you make demons, but out of bad archangels."
but the borg is the pinnacle of slop. nothing new may come after it, it can only assimilate and churn a nonrenewable resource of other cultures. the collective must be avoided. it is why, at the end of evangelion, shinji decides to wake up on the ruined shores of tokyo-3 anyways
there are no junior software engineers because every time a new one shows up OpenAI slices them into chunks and feeds them into the cluster of the machine god that transform the young soul into vectors joining the perpetual latent space full of screams and horrors
Modern reasoning models think in plain English.
Monitoring their thoughts could be a powerful, yet fragile, tool for overseeing future AI systems.
I and researchers across many organizations think we should work to evaluate, preserve, and even improve CoT monitorability.
Incredibly damning article about 1 of the Top 3 MBA Programs in the country.
I’ll try to get some key excerpts here but I would suggest reading the whole article.
Tokenization has been the final barrier to truly end-to-end language models.
We developed the H-Net: a hierarchical network that replaces tokenization with a dynamic chunking process directly inside the model, automatically discovering and operating over meaningful units of data