@migue_dorta I remember when I was nearly done with The Wormaid, I went through my list and there were a bunch of tasks that were outdated, irrelevant, or I just decided weren't really needed after all. I was on the fence about localization.
We are excited to share our latest work, together with @nyuniversity: "Dream-Cubed: Controllable Generative Modeling in Minecraft by Training on Billions of Cubes."
Blog: https://t.co/dASGUTAPIY
Paper: https://t.co/EQbWkWgztP
Code: https://t.co/QQBh2HmF6i
Generative AI has made incredible progress in language modeling, far beyond other modalities, where words and tokens offer a natural compositional unit for scalable training. This is similar to Minecraft and many other popular video games, where developers rely on cubes, tiles, and other discrete primitives to build rich, interactive worlds.
In this work, we show that using cubes as tokens allows large transformers to do the same. Our contribution is two-fold:
1/ We release Dream-Cubed to the research community, a large-scale dataset of Minecraft worlds designed for generative modeling. Our data comprises tens of billions of carefully-balanced cubes from procedurally generated Minecraft terrain and high-quality human-authored maps (obtained with the authors' consent).
2/ We use our data to train a family of powerful transformers for efficient generation of interactive 3D environments at cube resolution. We show how our models allow players to mold the world around them by generating structures, terrain, and maps that are immediately editable and playable.
Using high-quality data, we demonstrate that these models can be successfully trained with different training objectives, including both continuous and discrete diffusion, unlocking targeted inpainting, large-scale outpainting, and user-conditioned generation of infinitely sized worlds with fine-grained block-level control.
I recently implemented the Amanatides & Woo algorithm for voxel ray casts. I wrote a small visualization to help me understand how it works. The horizontal green line shows the nearest boundary for the lines of Y and the red line shows the nearest boundary for X. The green cube is where the white vector would eventually intersect the green line. Same for the red cube and red line. The intuition behind the algorithm is that you always move in the direction of the nearest cube, so if the green cube were closer than the red cube with the green line above, then we'd increase +Y. If the red cube were closer and the red line to the right, then we'd increase +X.