this is wild 🤯🤯🤯
just paired openai operator with replit agent to build an app
watch as two ai agents team up, exchange credentials, and start testing
ai agent 🤝 ai agent
the future is here and it’s insane!
It's only been 1 day of CES 2025, and the announcements have already been incredible.
The 10 most impressive reveals of CES 2025 so far:
1. A 360° AI-powered body scanning health mirror that can scan your heart, weight, and metabolic health
Introducing Gemini 2.0 Flash Thinking, an experimental model that explicitly shows its thoughts.
Built on 2.0 Flash’s speed and performance, this model is trained to use thoughts to strengthen its reasoning.
And we see promising results when we increase inference time computation!
Introducing Willow, our new state-of-the-art quantum computing chip with a breakthrough that can reduce errors exponentially as we scale up using more qubits, cracking a 30-year challenge in the field. In benchmark tests, Willow solved a standard computation in <5 mins that would take a leading supercomputer over 10^25 years, far beyond the age of the universe(!).
Interested in 3D object detection and egocentric vision? We open sourced a small yet challenging dataset called Aria Everyday Objects (AEO). We use it as one of the tasks for benchmarking a new class of Egocentric 3D Foundation Models we are working on: https://t.co/g2V4EtJe1L
🔥llm.c update: Our single file of 2,000 ~clean lines of C/CUDA code now trains GPT-2 (124M) on GPU at speeds ~matching PyTorch (fp32, no flash attention)
https://t.co/39G8UWusVp
On my A100 I'm seeing 78ms/iter for llm.c and 80ms/iter for PyTorch. Keeping in mind this is fp32, with no flash attention yet, and slightly stale PyTorch (2.1.0).
- It is a direct implementation of the training loop and backpropagation in C/CUDA.
- It compiles and runs instantly. No more "hit run then wait for tens of seconds for unknown reasons", for mountains of inscrutable abstractions to build a Universe.
- It deletes the need for the Python interpreter and a deep learning library.
- It allocates all the memory a single time at the start.
- It's pretty cool.
How:
Getting this to work required us to write a lot of custom CUDA kernels, and doing this manually (instead of using Tensor ops of aten/PyTorch and torch.compile etc.) is a bit like programming in assembly. And you spend quality time looking at more assembly (CUDA PTX/SASS). But this also means we get to hyperoptimize the code and possibly explore optimizations that torch.compile might find difficult to, which is awesome. Examples of optimizations that went in over the last few days:
- we're being clever with our memory consumption in the backward pass, only using a few buffers we need to propagate the gradients, saving memory capacity.
- one fused classifier kernel does the last layer forward pass, the loss, and kicks off the backward pass.
- many improvements to all the kernels involved, including e.g. gains from carefully constraining execution within the autoregressive mask in attention
- cuBLAS(Lt) calls for all heavy lifting matmuls, and fused bias accumulation
Big credits to two CUDA experts who appeared from somewhere on the internet to help this open source project, ngc92 and ademeure. We're hanging out of Github and Discords of CUDAMODE and my NN Zero to Hero.
Next steps:
- more optimizing of our (fp32) kernels, and especially switch to flash attention.
- mixed precision training (fp16 to start).
- multi-gpu training (DDP to start).
- data & evals to set up a proper GPT-2 training runs
- 🚀 repro GPT-2 (1.6B) training run.
- more modern architectures etc. (Llama 3?)
- writing, videos, exercises on building all of this from scratch.
Figure 1: eye candy: timing profile of the kernels (one layer). NVIDIA cutlass kernels with solid compute throughput taking up a lot of the running time => nice.
The best linear algebra course out there. PERIOD.
For Free!
MIT's Professor Gilbert Strang.
Go through these videos, and you'll never ever have a problem with linear algebra again!
https://t.co/NtWWwl9c85
Very impressive---indeed, awe-inspiring---AI demos this last week, e.g., from OpenAI (image generation) and Google (text generation).
These demos seem to convince many people that current AI is getting closer and closer to human-level intelligence. 🧵
(1/8)
📢📢📢 Happy to introduce that "Kubric: a scalable dataset generator" was accepted at #CVPR2022.
https://t.co/FRFlZHglP7
– Do you work on video and/or 3D vision from images?
– Having a hard time applying the "scientific method" to your research?
Look no further 🧵👇
The ongoing consolidation in AI is incredible. Thread: ➡️ When I started ~decade ago vision, speech, natural language, reinforcement learning, etc. were completely separate; You couldn't read papers across areas - the approaches were completely different, often not even ML based.
Reconstructing objects in 3D is an important #computervision problem with many AR/VR applications. We're releasing Common Objects in 3D (CO3D), a new data set of nearly 19,000 videos capturing objects along with highly accurate 3D reconstructions. More: https://t.co/hMDLRdx79R
I am pleased to announce that the camera ready version of my new textbook, "Probabilistic Machine Learning: An Introduction", is finally available from https://t.co/dSlKkwYpLr. Hardcopies will be available from MIT Press in Feb 2022.