Justin Jung

Research Scientist @ Biohub, formerly @SpringtailAI | Hobby Blogger

[Justin Jung @jsjung00]

A really interesting phenomenon of diffusion:

It's L2 denoising training objective has a closed form solution, the conditional mean, which reduces to a gaussian kernel weighted average of the training points.

Sampling from this would only return training data, yet diffusion models generalize.

In this blog post, we highlight how properties of euclidean geometry (the concentration of gaussians, the L2 spacing of high dimensional points in R^d) of the diffusion objective and the training dataset can explain diffusion generalization: https://t.co/aSmBOYmHNB

[Marco Mascorro @Mascobot]

After coding is solved, the next frontier is computer use. Today, we are launching Use Computer, the infra for evaluating and training models to use all kinds of computers 👇

[Justin Jung @jsjung00]

@jdeschena @caglarml @Jaeyeon_Kim_0 @JonathanGeuter @LucaAmb @ssahoo_ @zhihanyang_ @dvruette @yjelid @LiDavid2002 @LucaEyring @AndrewC_ML @ValentinDeBort1 @ArnaudDoucet1 @chandavidlee @WeiGuo01 @modal very interesting work, and love the interactive blog format/style!

[Justin Jung @jsjung00]

@jm_alexia @MehdiEsmaexl Very exciting! Looking forward to the paper

[Justin Jung @jsjung00]

@ayaanzhaque this is awesome congrats!

[Justin Jung @jsjung00]

@hla_michael really well written! thanks for sharing

[Justin Jung @jsjung00]

@celestepoasts Congrats!

[Justin Jung @jsjung00]

@elliotarledge Do you have any thoughts on why the drift model has worse generation quality here? The paper seems to report good metrics

[Justin Jung @jsjung00]

@jm_alexia Agreed—had my LLMs are sycophantic and bad for brainstorming moment today

[Justin Jung @jsjung00]

🧵(6/6) And thanks to the authors of ESM3 and the open source model release for making this investigation possible

@THayes427 @proteinrosh @halilakin @sofroniewn @denizzokt @ebetica @robert_verkuil Vincent Tran @deaton_jon @MariusWiggert @rohilbadkundri @irhumshafkat Jun Gong @awfderry @rsm_ai @countablyfinite @TheYousufKhan Chetan Mishra Carolyn Kim @BartieLiam @mnemeth101 @pdhsu @TomSercu @salcandido @alexrives

[Justin Jung @jsjung00]

🧵 Can a small (1.4B parameter) protein language model solve challenging protein scaffold design tasks by scaling inference compute?

Yes—but not simply through scaling the number of samples generated

[Justin Jung @jsjung00]

🧵(5/6) 📔Full details: https://t.co/EiDBbgGvAl

💻Repo to reproduce: https://t.co/FU3zNZgpbw

[Justin Jung @jsjung00]

@_onionesque congrats!

[Justin Jung @jsjung00]

@giffmana do you use multiple models and compare responses?

[Justin Jung @jsjung00]

@JCJesseLai @NeurIPSConf love your work, thanks for writing

[Justin Jung @jsjung00]

@sedielem would love to join in SD!