Jakob Uszkoreit

[Jakob Uszkoreit @kyosu]

Learned to synthesize RNA in our lab in Palo Alto, last week. The batch includes the first mRNAs designed by neural networks at Inceptive. It’s getting real!

[Jakob Uszkoreit @kyosu]

@NandoDF “There is a way to do it better. Find it!”
— Maybe Edison, though it doesn’t matter

[TEDAI San Francisco @TEDAISF]

Can AI unlock discoveries beyond human cognition?
@kyosu, Co-Founder and CEO of Inceptive, explores how #AI can solve problems beyond human reasoning in his TEDAI Talk, recorded in SF.
https://t.co/dPUFj0zKw7

Early Bird Tickets for TEDAI 2025: https://t.co/rFRrzLMJVG

[Jakob Uszkoreit @kyosu]

@karpathy ARR generated through products is the best value proxy humanity created, so far. The community keeps forgetting that test sets are just (usually meh) approximations of samples from some true input, label distribution - namely products.

[Jakob Uszkoreit @kyosu]

Taking @dbahdanau-style attention mechanisms to refine a single set of vectors using information from that same set, aka self-attention, in https://t.co/8QY2JUWnVr was a key step in removing other "backbone" mechanisms like recurrence or convolution.
A year later, the Transformer then used a way more efficient formulation and implementation for transduction/generation.

[Jakob Uszkoreit @kyosu]

@karpathy /me loves it.

[Jakob Uszkoreit @kyosu]

@sea_snell Awesome direction!

[Jakob Uszkoreit @kyosu]

@sedielem 🍿 indeed.

[Jakob Uszkoreit @kyosu]

@_akhaliq Super neat! Unconditional residual connections caused strong information locality across layers, resulting in ~uniform compute allocation across representations. This seems like the right fix.

[Jakob Uszkoreit @kyosu]

@j_foerst Hahahahaha

[Jakob Uszkoreit @kyosu]

@srush_nlp Why not a Gathering On FOundation MOdels: GO FOMO?

[Jakob Uszkoreit @kyosu]

@ameenaelbibany @InceptiveCom @KHeyries @__dskim Super excited to have you on board in building this!

[Jakob Uszkoreit @kyosu]

@vijaypande @nvidia @a16zBioHealth @a16z Thank you, Vijay, and all the @a16z team for supporting us in so many ways.

Hard to express how excited all of us at @InceptiveCom are about what we're building with you.

[Jakob Uszkoreit @kyosu]

@anthonycosta Thank you, Anthony!
We, too, are incredibly excited about working closely with you and the team towards the incredible opportunities ahead.

[Jakob Uszkoreit @kyosu]

@NandoDF Isn’t that exploration what we’re seeing with diffusion models? As learned, iterative optimizers they’re a strict generalization of autoregressive models, and with the appropriate parameterizations allow error correction over time. Also cool hacks like Khan’s tutor’s refinements.

[Jakob Uszkoreit @kyosu]

@ShriramKMurthi I was so inspired dwhen I saw him hacking on code in a terminal window during a talk. Well into his 60s. He gave the talk.

[Jakob Uszkoreit @kyosu]

@a_kazemnejad Awesome! We had an inkling something like this could work and quipped we’d title the paper “Position Paper” - but then startup.

… just in case you need a(nother) title.

[Jakob Uszkoreit @kyosu]

@kchonyc Same here. It also affects Google Photos, less crucial, still annoying. There must be a longer story around API changes and policies that I don’t have the time to figure out…

[Jakob Uszkoreit @kyosu]

@ELLISforEurope @mcaron31 @TacoCohen Congratulations Mathilde and Taco! I, for one, have learned so much from both of you.

[Jakob Uszkoreit @kyosu]

@_arohan_

[Jakob Uszkoreit @kyosu]

@egrefen @CohereAI Very cool!
I was hoping to find the code of the Obsidian integration, somewhere. Did I miss it?