Lane McIntosh

machine learning at @tesla autopilot

[Tesla @Tesla]

FSD Supervised helps save lives

[Phil Duan @pduan]

Along for the ride in unsupervised FSD testing

[Andrej Karpathy @karpathy]

I took delivery of a beautiful new shiny HW4 Tesla Model X today, so I immediately took it out for an FSD test drive, a bit like I used to do almost daily for 5 years. Basically... I'm amazed - it drives really, really well, smooth, confident, noticeably better than what I'm used to on HW3 (my previous car) and eons ahead of the version I remember driving up highway 280 on my first day at Tesla ~9 years ago, where I had to intervene every time the road mildly curved or sloped. (note this is v13, my car hasn't been offered the latest v14 yet)

On the highway, I felt like a passenger in some super high tech Maglev train pod - the car is locked in the center of the lane while I'm looking out from Model X's higher vantage point and its panoramic front window, listening to the (incredible) sound system, or chatting with Grok. On city streets, the car casually handled a number of tricky scenarios that I remember losing sleep over just a few years ago. It negotiated incoming cars in tight lanes, it gracefully went around construction and temporarily in-lane stationary cars, it correctly timed tricky left turns with incoming traffic from both sides, it gracefully gave way to the car that went out of order in the 4-way stop sign, it found a way to squeeze into a bumper to bumper traffic to make its turn, it overtook the bus that was loading passengers but still stopped for the stop sign that was blocked by the bus, and at the end of the route it circled around a parking lot, found a spot and... parked. Basically a flawless drive.

For context, I'm used to going out for a brief test drive around the neighborhood to return with 20 clips of things that could be improved. It's new for me to do just that and exactly like I used to, but come back with nothing. Perfect drive, no notes. I expect there's still more work for the team in the long march of 9s, but it's just so cool to see that we're beyond finding issues on any individual ~1 hour drive around the neighborhood, you actually have to go to the fleet and mine them. Back then, I processed the incredible promise of vehicle autonomy at scale (in the fully scaleable, vision only, end-to-end Tesla way) only intellectually, but now it is possible to feel it intuitively too if you just go out for a drive. Wait, of course surround video stream at 60Hz processed by a fully dedicated "driving brain" neural net will work, and it will be so much better and safer than a human driver. Did anyone else think otherwise?

I also watched @aelluswamy 's new ICCV25 talk last week (https://t.co/RdaM23kvez) that hints at some of the recent under the hood technical components driving this progress. Sensor streams (videos, maps, kinematics, audio, ...) over long contexts (e.g. ~30 seconds) go into a big neural net, steering/acceleration comes out, optionally with visualization auxiliary data. This is the dream of the complete Software 1.0 -> Software 2.0 re-write that scales fully with data streaming from millions of cars in the fleet and the compute capacity of your chip, not some engineer's clever new DoubleParkedCarHandler C++ abstraction with undefined test-time characteristics of memory and runtime. There's a lot more hints in the video on where things are going with the emerging "robotics+AI at scale stack". World reconstructors, world simulators "dreaming" dynamics, RL, all of these components general, foundational, neural net based, how the car is really just one kind of robot... are people getting this yet?

Huge congrats to the team - you're building magic objects of the future, you rock! And I love my car <3.

[Ashok Elluswamy @aelluswamy]

https://t.co/Jr4i7tHUCr

[Lane McIntosh @lmcintosh]

@karpathy Thank you for all your hard work to help build Autopilot into what it is today!

[TESLA saves lives @Teslasaveslives]

#FSDBeta stops for pedestrian in the road on a rainy night

[IIHS @IIHS_autosafety]

IIHS has completed tests of the 2021 Tesla Model 3's new camera-based front crash prevention system, which rates superior for vehicle-to-vehicle interactions and advanced for pedestrian interactions. Details: https://t.co/y3vGCv0P49

[Andrej Karpathy @karpathy]

Help revolutionize the world with full self-driving by joining us at Tesla Autopilot: https://t.co/ekekjKDOZF It is very hard to find other places where AI expertise makes as much of a difference on as big of a problem.

[Hidenori Tanaka @Hidenori8Tanaka]

New paper out on #NeurIPS2019: “From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction”

with fantastic collaborators @aran_nayebi, @niru_m, @lmcintosh, Stephen Baccus, @SuryaGanguli.

https://t.co/hBncn88vNM

[Surya Ganguli @SuryaGanguli]

Officially got tenure in Applied Physics @Stanford! This feat is really due to a brilliant, creative, and fun group of students and postdocs I have had the great fortune of working with over the years. And thanks to amazing colleagues and mentors at Stanford and beyond!

[Andrej Karpathy @karpathy]

We see more significant improvements from training data distribution search (data splits + oversampling factor ratios) than neural architecture search. The latter is so overrated :)

[Tesla @Tesla]

🥁🥁🥁 🥁🥁🥁 🥁🥁🥁 please:

We are excited to announce that Model 3 just earned a 5-star safety rating from @EuroNCAP ⭐️⭐️⭐️⭐️⭐️ https://t.co/2oSQz4yaBB

[Paul Graham @paulg]

The old car companies think what separates them from Tesla is that Teslas are electric. But they're also separated by the fact that Teslas are software, and that, they will discover, is an unbridgeable gulf.

[Andrej Karpathy @karpathy]

New blog post: "A Recipe for Training Neural Networks" https://t.co/5lBy4J77aS a collection of attempted advice for training neural nets with a focus on how to structure that process over time

[Andrej Karpathy @karpathy]

So proud of the Tesla Autopilot team and very excited to see the the veil lifted on some of their work earlier today 👏 A number of people got to experience this:

[Jeff Dean @JeffDean]

Please read @elasri_layla's post about the undesirable behaviors she has experienced in the field of ML &amp; CS (the anecdote about her CS professor showing another female student how to use a mouse was pretty egregious).

Let's all strive to create a welcoming environment for all.

[Andrej Karpathy @karpathy]

"Tesla's Navigate on Autopilot Hands-On: Road trip to CES 2019!" https://t.co/ax9x1WCuyv with NoA enabled and a nav path set, Autopilot automagically suggests &amp; negotiates the right lane changes, takes the right forks, and overtakes slow vehicles while respecting passing lanes ✨

[Shivon Zilis @shivon]

Cards Against Machine Learning

[Surya Ganguli @SuryaGanguli]

My new post for the @Stanford Human Centered AI Initiative: a personal vision of how #neuroscience #psychology #ai #physics #mathematics and other fields can work together to both understand biological intelligence and create artificial intelligence! https://t.co/ySswh9mDUX

[Lane McIntosh @lmcintosh]

Wish more ML papers gave statistics on the success of their methods! "We observe that without Orthogonal Regularization, only 16% of models are amenable to truncation, compared to 60% when trained with Orthogonal
Regularization." https://t.co/UAh4XKPQ0T