Andrii Maksai

[Hanxiao Liu @Hanxiao_6]

Happy to share our recent work!

[Andrii Maksai @AMaksai]

@giffmana I let Claude peek into my tmux pane for that purpose sometimes

[Andrii Maksai @AMaksai]

@WenjieYang00 Agreed, though I think at least part of the issue is aggressive post-training for tool use which masks some vision capabilities.

Thanks for reading:)

[Andrii Maksai @AMaksai]

For a while now, I wanted to check how well frontier models can look at an image and handwrite on top of it. So I did a thing:

InkSlop: Vibe-coded benchmark for Spatial Reasoning with Digital Ink

Link and key points below👇

[Andrii Maksai @AMaksai]

@giffmana @helloiamleonie @JinaAI_ Not in the infographics, but it's actually quite pronounced in the paper

[Andrii Maksai @AMaksai]

The bonus: Results with Nano Banana Pro & notes on vibe-coding, vibe-analyzing, vibe-debugging, and vibe-reporting the whole thing included.

[Andrii Maksai @AMaksai]

The finding: Models really prefer tool use to actually looking. And when the tools are taken away, performance drops for most task & model combinations (ex. 0% on mazes)

[Andrii Maksai @AMaksai]

@ibab Plot twist: Did YOU write it? :P

[Andrii Maksai @AMaksai]

@giffmana I didn't but of course, GPT is not a savage! If you squint really hard, you can also see that the editor is in vim mode :P

[Andrii Maksai @AMaksai]

@giffmana Mostly Fig. 4, but sometimes Fig. 6 for a bunch of trivial parallel changes (because switching in tmux >> switching sessions in CC extension)

[Andrii Maksai @AMaksai]

@NandoDF Depends on the performance on the task, whether this is the largest model, and how easy it is to synthesize the prompts, I guess?
SFT on most if the model has not seen a lot of this type of data before, distill a larger model, maybe with synthetic prompts if doable, RL otherwise?

[Andrii Maksai @AMaksai]

@ericjang11 Not sure anything beyond https://t.co/zl8xicVEgo was ever publicized

[Andrii Maksai @AMaksai]

@giffmana Nice! Nit: GitHub links all go to 404, although maybe that's just temporary?

[Alex Wiltschko @awiltschko]

Well, we actually did it. We digitized scent. A fresh summer plum was the first fruit and scent to be fully digitized and reprinted with no human intervention. It smells great.

Holy moly, I’m still processing the magnitude of what we’ve done. And yet, it feels like as we cross this finish line we are instantly at a new starting line. I’ll have more to share about what’s in store that we’re building on top of this.

A huge HUGE congrats to the entire team across scientific, engineering, operational, and creative disciplines. It takes a village named Osmo to do this.

I don’t know if this is embarrassing, but I carry the plum scent with me a lot of places and smell it constantly. It makes me smile.

I’m curious, if y’all want to smell it? If we made a limited release fragrance of the first teleported scent and dedicated the proceeds to science, would you want it?

[Google AI @GoogleAI]

Today we describe a model taught to read and write so it can extract and digitize the strokes of handwriting without the need for specialized equipment. It then outputs realistic looking digital handwriting that can be handled like standard digital text. https://t.co/y3hj54ONkP

[Andrii Maksai @AMaksai]

@francoisfleuret Isn't this somewhat identical to having one layer do f(x)=-x, and the second one writing down some completely new content there? Is the hypothesis that having zeroes in the mask makes it easier from the optimization perspective somehow?