Katherine Tian

cs/stat @harvard, working on calibration & factuality of LLMs, prev @GoogleAI tensorflow, golden state @warriors fan

[Katherine Tian @kattian_]

Thanks Paige! Great working on the @TensorFlow team

[👩‍💻 Paige Bailey @DynamicWebPaige]

Announcing a cool new feature in @TensorFlow: differentiable map ops! 🙌✨

If you've ever wanted to store and train embeddings in an embedding model, this should make the process *much* simpler. 😄

https://t.co/0ybR7BB6QN

Amazing work from our TF intern, @kattian_ (@Harvard)!

[Katherine Tian @kattian_]

@frank_liquid is cooking, as usual

[Liquid @liquidtrading]

Introducing Co-Invest Extension:

The first way to trade everything, everywhere

Get it here: https://t.co/cy3ml736eI

[Katherine Tian @kattian_]

Want to learn about SAE feature sensitivity, and our new method for measuring it efficiently? Catch Claire’s spotlight talk and poster at the NeurIPS Mech Interp workshop tomorrow, 11-12:30 in Upper Level Room 30A-E. Super fun project by @cla_tian, mentored by @NathanHu12 and me!

[Nathan Hu @NathanHu12]

Activating examples of SAE features are often interpretable, but does a feature reliably activate on all inputs of a given concept? We built an automated evaluation to study feature sensitivity: whether features activate on text similar to their top examples.

[Katherine Tian @kattian_]

@tonychenxyz @PrincetonCS excited for you!!

[Katherine Tian @kattian_]

@JulianL093 let’s goo! bro is locked in

[Katherine Tian @kattian_]

@leonardtang_ ergonomic keyboard

[Katherine Tian @kattian_]

@rach_it_ @Harvard @elmelis @wattenberg @viegasf @KempnerInst 🙌

[Katherine Tian @kattian_]

@ShunyuYao12 @karthik_r_n @princeton_nlp @PrincetonPLI Congratulations!!!

[Katherine Tian @kattian_]

@LuxiHeLucy congrats!

[Jack Morris @jxmnop]

one of the most important things I know about deep learning I learned from this paper: "Pretraining Without Attention"

this what I found so surprising:
these people developed an architecture very different from Transformers called BiGS, spent months and months optimizing it and training different configurations, only to discover that at the same parameter count, a wildly different architecture produces identical performance to transformers

this may imply that as long as there are enough parameters, and things are reasonably well-conditioned (i.e. a decent number of nonlinearities and and connections between the pieces) then it really doesn't matter how you arrange them, i.e. any sufficiently good architecture works just fine

i feel there's something really deep here, and we may be already very close to the upper bound of how well we can approximate a given function given a certain amount of compute. so we should spend more time thinking about other questions, such as what that function should actually look like (what data? which objective function?) and how to make it more efficient

[Katherine Tian @kattian_]

@aakaran31 @PDSoros congrats!!!

[Katherine Tian @kattian_]

@xiamengzhou congrats! 😊

[Katherine Tian @kattian_]

@walden_yan congrats on the launch!!

[Chelsea Finn @chelseabfinn]

I’m really excited to be starting a new adventure with multiple amazing friends & colleagues.

Our company is called Physical Intelligence (Pi or π, like the policy).

A short thread 🧵

[Katherine Tian @kattian_]

@aryaman2020 👏👏

[Katherine Tian @kattian_]

@chenxcynthia @Harvard 🤩

[Katherine Tian @kattian_]

@nibnalin @amirbolous that’s a reasonable hypothesis but it doesn’t seem like that’s the full story - evan chen reviewed a subset of proofs and verified they’re not all just coordinate bashing

[Katherine Tian @kattian_]

@jxmnop maybe relative representations can help? your question reminded me of this paper: https://t.co/1D77c2y8nx

[Katherine Tian @kattian_]

@ericmitchellai @PreetumNakkiran 👀

[Katherine Tian @kattian_]

@srush_nlp here's my video explanation (20min) with some math overview @ maybe ~5: https://t.co/FBZCjFrWAn

[Raj Movva @rajivmovva]

We wrote a position piece on how LMs have expanded the toolkit for social equity researchers, especially in health - check it out, and feel free to share thoughts! https://t.co/1k4FG4BszE.

(and feeling very lucky to have worked with so many cool co-authors who I look up to!)