Stephan Häuslschmid

IT guy at ROS.
Responsible for and others.
Fan of IoT and smart stuff. Trying to reduce my carbon footprint.

[Stephan Häuslschmid @SteveH80]

My son is having fun at the @Tesla destination charger at the Chiemsee beach bar. Thanks @elonmusk for the free energy, the free parking and for making this possible by bringing this awesome car to Germany. #teslamodel3

[Farzad 🇺🇸 🇮🇷 @farzyness]

Tesla's FSD v12 is a REALLY big deal.

Last night @elonmusk livestreamed a ~45-minute video of a Model S driving itself around using Tesla's latest self-driving software, FSD v12.

Self-driving is something the company has been trying to solve since Tesla Autopilot was released in 2015. Since then, Tesla has made steady progress improving the code to handle all kinds of road situations, has added cameras around the car, and removed sensors, all the while rewriting the code multiple times to solve for things the car couldn’t handle.

And even though Tesla cars can drive themselves in many situations, the biggest challenge with self-driving cars is the thousands (or millions) of situations drivers face on a daily basis that are completely unexpected or difficult to solve for, like other drivers acting irrationally, inclement weather, debris on the road, weird (or lack of) lane markings, etc.

Up to this point, Tesla and other companies have to spend a large amount of time running through simulated scenarios to generate code that would teach the system how to handle these situations. This code is oftentimes written by a human and needs to account for every variation of something happening on the road. And even then, there are thousands (or millions) of situations that a simulation won’t come with, since real life is so damn complicated and complex.

The approach many have taken to try and solve this problem is by overfitting their self-driving cars with a ton of different sensors like LIDAR, radar, ultrasonics, cameras, and other sensors in addition to generating High-Definition maps of the areas where the cars are meant to be driven in. They’ve done this in hopes that there’s a combination of sensors and map data that would allow them to solve for the most unexpected situations. Here’s a picture of a GM Cruise vehicle that uses this approach.

However, yesterday’s video has demonstrated a breakthrough in how self-driving cars can operate.

Instead of using many sensors and hardware on their cars to process the world, Tesla is using 8 cameras and a computer that’s specifically built to process video data. That’s 9 total parts vs everyone else’s 30, 40, 50+ parts to process the world.

Using only vision to process the world and not needing things like LIDAR, radar, and other sensors to interpret the physical objects around the car is impressive enough, but HOW the car learns to do this is what the real breakthrough is.

With FSD v12, Tesla takes the video data that is collected by its fleet of ~4 million cars and runs it through an AI that is built using Neural-Nets (like ChatGPT but for the real world), and then the AI figures out what the car should do based on what it sees.

What’s important to highlight here is that Tesla has done 0 work in telling the car how it should interpret the world. That means that the AI doesn’t explicitly know what a lane is, what a traffic light is, what a stop sign is, what a cone is, what a pothole is, what rain is, etc.

What Tesla does is it shows the AI a metric-ton of video of a car driving around using the 8 cameras that are outfitted around the car, and the AI learns how to do the same. The more video Tesla feeds it, the better the system gets. The more unique situations that are collected from the fleet of Teslas, the better the system gets at accounting for those situations.

This “AI code” will then be beamed to every Tesla in the world, and the on-board computer will be able to process its surroundings without the need to connect to Tesla’s AI server. It’s no different than you or I getting into a car and driving it. Instead of our 2 eyes collecting video around us and using our brain to process the info, the 8 camera system will collect the video and Tesla’s on-board computer will process it using the “AI code”.

In other words, Tesla has moved away from humans figuring out how to write code that tells the car what to do, and instead feeds video to an AI, and the AI figures out the best way to account for every situation.

The only thing Tesla needs moving forward is more video and more compute power (chips) to process videos.

That’s it.

This is profound. Everything that is captured on video with the 8 camera system is something the AI will be able to figure out how to navigate through. Snow. Potholes. Deer. Cyclists. Aliens. You name it. And this “code” that is generated by the AI will get better and better as Tesla’s compute capabilities grow with their purchases of NVIDIA’s H100 chips and the build-out of their in-house DOJO compute system. Not to mention the growing fleet of Teslas that will capture more and more video data around the world. Every Model S, 3, X, and Y that is sold today is a video-capturing robot that feeds the AI. And every Cybertruck and Compact Car will be the same.

And believe it or not, it gets even crazier.

Now that Tesla has come up with a real-world data collection and processing system with its cameras and on-board computer, this same system can be used on other physical products that can learn how to move around its surroundings.

This is where Tesla’s Optimus Bot comes into play. Tesla will be able to use its 8 camera system on the Tesla Bot to collect video of its surroundings, beam it back to the AI, the AI figures out how to best do that thing that it’s collecting video for, and then beam back the “AI code” for the Bot to process with its on-board compute.

We are not far away from a world where a humanoid robot watches you do the dishes, sends back the data to the mothership to process it, and then the next morning the Bot has learned how to do the dishes - not only using the video it gathered from you doing it, but from every other person in the world washing dishes.

Do this process with literally anything.

ChatGPT showed the masses what the potential of AI can be. NVIDIA showed the masses just how much demand there is for hardware that is used by AI systems.

And now, Tesla just showed the masses what AI means for real-world physical applications.

The world has changed - again.

[dcbel @dcbel_energy]

Our flagship product, dcbel r16, has just received @Intertek certification! This remarkable achievement means the r16 is the first certified residential bidirectional DC electric vehicle charger in the US - and a new era of home energy is about to unfold.
https://t.co/zzsYtZLCWn

[Stephan Häuslschmid @SteveH80]

Es ist über einen Monat her, dass ich Strom kaufen musste. Und das obwohl die letzten Wochen kaum Sonne zu sehen war und wir neben dem normalen Stromverbrauch zusätzlich mit Strom heizen (Wärmepumpe) und fahren (2 Elektroautos). Die #Energiewende ist möglich!

[Tim Urban @waitbutwhy]

The three exceptions are part of the little tree.

[image: Minna Sundberg]

[Edward Snowden @Snowden]

please tell me the white house did not spend the month of february scrambling jets to fire $400,000 missiles at the local hobby club's TWELVE DOLLAR BALLOON

lord have mercy

https://t.co/4D7DaTncvT

[Mimi🌷 @Mim_Tant]

Die IT droht mir damit, jederzeit meinen Verlauf checken zu können, ich drohe der IT damit jederzeit auf den Link des afrikanischen Prinzen, der mir meine Millionen auszahlen möchte, klicken zu können.
Es herrscht nun sowas wie Waffenstillstand.

[ian bremmer @ianbremmer]

awesome visualization of europe’s changing powers, 1500-2022
https://t.co/Sgq0M5Xxa0

[Captain Futura (Mehr auf Mastod.) @captainfutura]

#Habeck musste mal eben leere Gasspeicher aus den Klauen des russischen Staates befreien & befüllen, eine Raffinerie übernehmen, #Windkraft- und #Solarausbau beschleunigen, #Blackout vorbeugen, #Uniper retten - und alles in unter einem Jahr. Und als Belohnung? Verleumdung & Hass.

[Stephan Häuslschmid @SteveH80]

Ich feiere das so! Danke @BMWK und alle Beteiligten!

[Sven Giegold @sven_giegold]

Die Bundesregierung hat heute weitreichende steuerliche Entlastungen und Entbürokratisierungen für die dezentrale Nutzung von Photovoltaikanlagen beschlossen. Damit setzen wir den Kurs des konsequenten Bürokratieabbaus und des Ausbaus der Erneuerbaren fort. Ein Thread.

[Maxi Ferreira @charca]

Experimenting with 🚀 @astrodotbuild and the Shared Element Transition API.

It might *look* like an SPA, but it's a server-side rendered MPA with near-zero JavaScript ⚡️

Blog post coming soon to a theater near you!

[Leonhard Gandhi @energy_charts_d]

Beim #ARD Deutschlandtrend war
"Schnellerer Ausbau der Windenergie"
die wichtigste energiepolitische Maßnahme.
In den Tagesthemen wurde diese wichtigste Maßnahme einfach weggelassen.

[Stephan Häuslschmid @SteveH80]

@SBF_FTX Easy SEPA deposits without uploading any proof of transfers like other exchanges do it. Would enable recurring deposits by standing order.

[Stephan Häuslschmid @SteveH80]

@kyriii Deine Wallbox kann doch 32A, oder? Das ist natürlich Voraussetzung.

[Thomas Wilde @tdoubleyoo]

"Writing is a tool for thinking" - well put.

[Stephan Häuslschmid @SteveH80]

@kyriii Vielleicht über die Max ladeleistung? Kann die Zoe nicht mit 32A laden? Das wäre ja ein signifikanter Unterschied zum Tesla.

[Stephan Häuslschmid @SteveH80]

Geht’s nur mir so oder steigt die Anzahl an Leuten die versuchen Dinge zu reparieren die nicht kaputt sind? So eine Energieverschwendung…

[Picard Tips @PicardTips]

Picard management tip: Take your leisure time seriously. A relaxed captain is a sane captain.