Jason Devitt

Co-founder of Vindigo, Mr. Number, and Compound Eye.

[Jason Devitt @devitt]

@scottmessinger Follow me so I can DM you his email.

[Jason Devitt @devitt]

@j___lambert @fdellaert Part of the genius of that paper is that it doesn’t require the NeRF geometry to be accurate to train stereo; it only requires accurate view synthesis.

[Jason Devitt @devitt]

I'm at CES giving demos of
@compoundeyeinc
real-time 3D perception tech. We leverage automotive-grade cameras for on road and off road perception tasks. Find me at booth #61401 in Tech West at the Venetian Hotel Expo January 7th and January 8th. #CES2023 #AutonomousVehicles

[Jason Devitt @devitt]

@jon_barron @cut_pow Yes, but the diffusion model alone is used to inpaint the occlusion zones that are created when the frame is warped according to the depth map.

[Jason Devitt @devitt]

This beautiful work by @cut_pow suggests diffusion models have at minimum learned to reproduce static monocular depth cues. Not clear if that's the same as learning 3D geometry. But if not, MiDaS has no 3D knowledge either; it uses the same cues. FYI @jon_barron

[Dmitrii Tochilkin @cut_pow]

1.2 It heavily relies on the quality of depth maps, and uses the assumption that SD has implicit knowledge of the scene geometry in an image. So therefore it can plausibly inpaint missing parts without explicitly knowing underneath 3D meshes of the scene

[Jason Devitt @devitt]

@jon_barron @poolio @BenMildenhall @ajayj_ OK. I am really surprised that diffusion models usually produce images that follow rules of perspective. In some hand-wavey sense StableDiffusion has learned more about 3D geometry than it has about … hands.

[Jason Devitt @devitt]

#stablediffusion "a swarm of robots in a meadow, cyberpunk"

[Jason Devitt @devitt]

#stablediffusion is remarkable. The failure cases are as interesting as the successes. Prompt "photograph of a young woman looking through a telescope, professional" @karpathy

[Jason Devitt @devitt]

@kvogt @Cruise Congratulations @kvogt !

[Jason Devitt @devitt]

The term "self-driving" is problematic but this is a really good summary of what's available today.

[Compound Eye @compoundeyeinc]

Great to see @ConsumerReports' Kelly Funkhouser on @CBSMornings giving an accurate overview of the current state of driver assistance technology. https://t.co/qZzQPkZW0l

[Jason Devitt @devitt]

@imispgh @TheAutonocast @TaraniDuncan @compoundeyeinc You appear to be claiming that (say) highway-only L2 is impossible without lidar and "imaging" radar, or that it is useless. Many people thought so 10 years ago but it is a very lonely claim now.

[Jason Devitt @devitt]

@M_Parabellum @TheAutonocast @TaraniDuncan @compoundeyeinc Agreed, but snow-covered is very different than “whiteout”. @comma_ai does well on snow-covered streets with a single camera (forward facing)

[Jason Devitt @devitt]

@M_Parabellum @TheAutonocast @TaraniDuncan @compoundeyeinc The state of the art for commercially deployed systems I should say. It is possible to drive in a wide range of degraded conditions with passive sensors, but you specifically asked about a whiteout.

[Jason Devitt @devitt]

@M_Parabellum @TheAutonocast @TaraniDuncan @compoundeyeinc If you have a solution I am sure they and we would be interested. The state of the art for fully autonomous vehicles with all available sensors is driving in light rain.

[Jason Devitt @devitt]

@M_Parabellum @TheAutonocast @TaraniDuncan @compoundeyeinc You cannot drive safely on conventional roads in a true whiteout with any combination of sensors.

[Jason Devitt @devitt]

@M_Parabellum @TheAutonocast @TaraniDuncan @compoundeyeinc I don't see how you can get that out of the interview.

[Jason Devitt @devitt]

@imispgh @TheAutonocast @TaraniDuncan @compoundeyeinc The podcast was about primary sensors for ADAS, so I’m assuming you didn’t listen to it.

[Jason Devitt @devitt]

@imispgh @TheAutonocast @TaraniDuncan @compoundeyeinc There are lots of arguments for adding radar to a car, but since radar cannot read signs or see lane markings it cannot be the primary sensor on a vehicle. We just didn’t get to it.

[Jason Devitt @devitt]

Thanks for having us! We really enjoyed the conversation. @TheAutonocast

[The Autonocast @TheAutonocast]

New episode now live: @devitt and @TaraniDuncan of @compoundeyeinc came by to talk camera-based depth perception with the gang, and in the process blew up all the stale, contrived "camera or lidar" debates. You want to listen to this one! https://t.co/z5zY1ZKmCq