Karl Stelzner

Building systems that learn about the world through observation and interaction @TUDarmstadt

[Kristian Kersting @kerstingAIML]

Super proud of @karlstel 💪🏻he just defended his PhD. 🎉 great talk, great thesis 🙏 for all the stuff you have taught me.

[Karl Stelzner @karlstel]

@kerstingAIML Thank you for the kind words, and all your support and advice over the years!

[Karl Stelzner @karlstel]

@mortenlysgaard @kerstingAIML @arkosiorek Sorry about that! For me it has always worked reliably as long as I was logged into a Google account, but on headless servers even that can be a hassle. Thanks for the suggested alternatives, I'll look into them.

[Thomas Kipf @tkipf]

So excited to share Object Scene Representation Transformer (OSRT):

OSRT learns about complex 3D scenes & decomposes them into objects w/o supervision, while rendering novel views up to 3000x faster than prior methods!

🖥️ https://t.co/ppLAiordIA
📜 https://t.co/VnssN4QUu9

1/7

[Karl Stelzner @karlstel]

The code also includes the NeRF decoder variant (V-SRT). So it should be a good starting point if you're looking to get into 3D representation learning! 2/2

[Karl Stelzner @karlstel]

I've open sourced my independent PyTorch implementation of Scene Representation Transformer by Sajjadi et al.: https://t.co/GFXGfGdBKE
It comes with checkpoints which (roughly) reproduce the paper's main results.
Thanks to Mehdi for helping me clarify some details. 1/2

[Karl Stelzner @karlstel]

Credits for obtaining the above results on the DexYCB dataset via a variant of ObSuRF go to my student @IdrisNobel

[Karl Stelzner @karlstel]

Tomorrow I'm joining the #ICLR2022 workshop on Objects, Structure & Causality for a panel discussion and an invited talk on the role of 3D geometry for unsupervised scene understanding.
Come say hi: https://t.co/CDFHAfxb1E

[Thomas Kipf @tkipf]

Excited to announce the panelists for our panel discussion at the ICLR workshop on Objects, Structure & Causality: @KriegeskorteLab @karlstel @saramagliacane @rosemary_ke & @vansteenkiste_s!

Moderated by @cogscikid & @sindy_loewe.

Join us this Friday!
🖥️ https://t.co/pjiS1rWCar

[Jannik Kossen @janundnik]

Check out

👾Modelling Videos of Physically Interacting Objects👾
https://t.co/eQx4S3pV6g

my 2020 MSc Thesis building on our ICLR paper STOVE
"Structured Object-Aware Physics Prediction for Video Modeling and Planning" with @karlstel @c_voelcker @marcel_hussing @kerstingAIML

[Jannik Kossen @janundnik]

Extremely happy (albeit late) to announce that

🧪🏃‍♀️Active Testing – Sample-Efficient Model Evaluation🏃‍♀️🧪

is accepted to ICML2021!

📚🖨 https://t.co/YDttV8wjJn

Huge thanks to the fantastic co-authors
@seb_far, @yaringal, and @tom_rainforth of my first paper at @OATML_Oxford!

[Karl Stelzner @karlstel]

The compute required for this project seemed daunting, until we realized that you can train NeRF using only 2 scene function evaluations per ray when depths are known.
That may also be a good idea for training NeRF on real world data - depth cameras are a lot cheaper than GPUs!

[Adam R. Kosiorek @arkosiorek]

We developed ObSuRF: a NeRF autoencoder for unsupervised volumetric segmentation + we show how to use depth for efficient NeRF training https://t.co/LpYbQEVdVl with the amazing @karlstel (graduating this year, you should hire him!) and @kerstingAIML

[Karl Stelzner @karlstel]

@CianEastwood @arkosiorek @zdanielz @DaniloJRezende @kerstingAIML We'll have to try it I suppose :) My thinking is that the view encoder can learn to roughly place the image features in 3D space, since it has access to the camera pose. Once they are in that coordinate system, they should paint a pretty consistent picture.

[Karl Stelzner @karlstel]

@CianEastwood @arkosiorek @zdanielz @DaniloJRezende @kerstingAIML Essentially, that would be the same trick used here to avoid matching objects between frames: https://t.co/dC9baZXpep

[Karl Stelzner @karlstel]

@CianEastwood @arkosiorek @zdanielz @DaniloJRezende @kerstingAIML That's another axis to consider, yes. But I actually think S>1 should be straightforward for ObSuRF, since we can just encode views independently, and dump all features into the slot attention. Should work, as everything after is permutation equivariant. But haven't tried yet.

[Karl Stelzner @karlstel]

@CianEastwood @arkosiorek @zdanielz @DaniloJRezende @kerstingAIML Thanks for the pointer, I totally missed your paper, but it's obviously related. At first glance, it roughly looks like ObSuRF relates to MulMON as NerfVAE does to GQN.

[Karl Stelzner @karlstel]

@arkosiorek @kerstingAIML Thanks for the fun colab! Also check out the project website for more videos: https://t.co/zm8sMXqgqu
Code and data coming soon.

[Adam R. Kosiorek @arkosiorek]

We developed ObSuRF: a NeRF autoencoder for unsupervised volumetric segmentation + we show how to use depth for efficient NeRF training https://t.co/LpYbQEVdVl with the amazing @karlstel (graduating this year, you should hire him!) and @kerstingAIML

[Anirudh Goyal @anirudhg9119]

@GafniGuy someone is probably going to combine NeRFs with slot based models ;)

[Cian Eastwood @CianEastwood]

@arkosiorek @zdanielz @DaniloJRezende Look forward to seeing it :) If more interpretability/"manipulatibility" is desirable, it would be interesting to think about object-based z (i.e. multiple "slots") in the future, e.g. by combining with our multi-object-multi-view approach