Wow! Such a cool paper on fixing softmax and cross-entropy!
I enjoyed reading this paper today, so I made an explainer to make the ideas more accessible to a broader audience.
Check it out: https://t.co/cQXPuQ2bd3
Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:
We're hiring research interns in machine learning, generative modeling, and their applications!
Our research scope is broad, so if you're interested, feel free to reach out and tell us a bit about your research interests!
Apply here: https://t.co/rhDPO8ezE6
🇨🇭 We're hiring a research intern at Apple Zurich! Come work on multimodal pretraining and scaling laws with us (9+ months, could start asap).
📩 Send your CV and representative work to [email protected]
Please help spread the word!
We’re hiring Research Scientists and Research Interns at NVIDIA Research to work on some of the most exciting problems in generative AI:
🌎 World Models & Video Diffusion
🧠 Diffusion Language Models
🧬 Biomolecule Design
If your research aligns with any of these areas, please apply directly through the links below.
Open positions + application links:
https://t.co/MUAt4mcewx
My PhD advisor read everything I wrote line-by-line. Re-derived all my proofs himself on principle.
I recheck and rewrite everything AI hands to me: regardless of how correct-looking or lean-certified it is. I can answer for anything with my name on it. On the same principle.
Mighty can now build 3D occupancy maps from a single camera! 😁
Mighty runs VIO realtime on-device, giving us accurate camera motion. Using that + the camera feed, the SDK estimates depth and builds a 3D map of obstacles around it.
This means a robot or drone can use Mighty for things like:
- Collision avoidance
- Motion planning
- Autonomous navigation
No stereo camera or depth sensor needed. Just Mighty’s global shutter camera + IMU.
dude this ICML tutorial by Prof Mark Schmidt on optimization was so good I’m gonna download the presentation and rewatch it like a billion times. completely reworked my intuition on optimizers and neural net training in general (not that my intuition was good in the first place)
The video of my Stanford CS25 guest lecture, From Language Models to Native Multimodal Intelligence, is now online.
I discussed how the core ideas behind LLMs has shaped multimodal AI, from architectures to training paradigms and scaling, and where the next challenges may lie. 🧠🌐
🎥: https://t.co/sUVxSEjrVf