David Andrzejewski

Software! Systems! Data! @SFMachineLearn

Elsewhere:
@[email protected]
@davidandrzej.bsky.social

[David Andrzejewski @davidandrzej]

Proposal to re-brand "deep learning" as "stochastic regularized estimation of compositionally structured nonlinear functions".

[Jason Alan Fries @jasonafries]

🚀 Checkout our ICLR 2024 Spotlight Poster #165 today (Session 3)
🩺 "MOTOR: A Time-to-Event Foundation Model For Structured Medical Records"

✨ Highlights:

- The first TTE foundation model for structured EHRs
- Improves SOTA TTE by 4.6% & boosts label efficiency up to 95%
- TTE pretraining scales to 16k tasks & reduces GPU memory usage by ~35%
- Model weights available for research use!

🔍 Tutorial: https://t.co/1zKF05JS6A
🤖 Model: https://t.co/Ka9YUHNJNG
#ICLR2024 #AI #Healthcare

[Korl @Korl_co]

Korl is available for public beta!

We’ve built a platform that auto-generates consumable product presentations in seconds, each optimized for a common use case and audience.

Start with a 2-week free trial from our site: https://t.co/zQNRwa3Spr

[David Andrzejewski @davidandrzej]

Fun stuff - "In-Context Learning Creates Task Vectors" (via @_akhaliq) isolates something (more or less) resembling the "program key" described below as an internal attention state, demonstrating with state substitution/patching experiments https://t.co/cWBc33ql7v

[François Chollet @fchollet]

My interpretation of prompt engineering is this:

1. A LLM is a repository of many (millions) of vector programs mined from human-generated data, learned implicitly as a by-product of language compression. A "vector program" is just a very non-linear function that maps part of the latent space unto itself.

2. When you're prompting, you're fetching one of these programs and running it on an input -- part of your prompt serves as a kind of "program key" (as in database key) and part serves as program argument(s). Like, in "write this paragraph in the style of Shakespeare: {my paragraph}", the part "write this paragraph in the stye of X: Y" is a program key, with arguments X=Shakespeare and Y={my paragraph}.

3. The program fetched by your key may or may not work well for the task at hand. There's no reason why it should be optimal. There are lots of related programs to choose from.

4. Prompt engineering represents a search over many keys in order a find a program that is empirically more accurate for what you're trying to do. It's no different than trying different keywords when searching for a Python library.

5. Everything else is unnecessary anthropomorphism on the part of the prompter. You're not talking to a human who understands language the way you do. Stop pretending you are.

[David Andrzejewski @davidandrzej]

Frequent SF (Nob Hill) sighting recently: astonished & delighted tourists taking photos/videos of the self-driving cars.

[David Andrzejewski @davidandrzej]

Nice @NetflixEng blog on Causal ML for title card art (people love a single expressive face) https://t.co/aCQLhZevQT

The science is finally catching up to @joeveix "Your Pretty Face is Going to Sell" https://t.co/44ZUAFdfs3

[David Andrzejewski @davidandrzej]

@NicoWolf16 @predict_addict @ml_angelopoulos @stats_stephen Ah yes, fair point and they do cover/mention it in the paper - would be more accurate to have said "w/o assumptions (beyond exchangeability)"

[David Andrzejewski @davidandrzej]

Can we get rigorous guarantees on predictions w/o assumptions on the model OR the data distribution? Incredibly, yes - excellent intro to Conformal Prediction by @ml_angelopoulos & @stats_stephen: https://t.co/sh5xeFjjhA

[Valeriy M., PhD, MBA, CQF @predict_addict]

@davidandrzej @ml_angelopoulos @stats_stephen https://t.co/lxwSiHuIZn

[David Andrzejewski @davidandrzej]

Paper itself is an absolute treat: great diagrams, abundant code samples, and as a nice bonus some historical context around the origins and development of the research ideas.

Credit: saw it in @deliprao's "AI Research & Strategy" newsletter https://t.co/uz2PU219nb

[David Andrzejewski @davidandrzej]

...furthermore: outperforms XGBoost, does Lasso in one-pass, seems not to rely on nearest-neighbor.

Future work: "...an intriguing possibility where we might be able to reverse engineer the Transformer to obtain better learning algorithms." (!)

[David Andrzejewski @davidandrzej]

Easy to get "breakthrough fatigue" in ML recently, but in this work the NN learns *how to learn* linear regression, decision trees, 2-layer ReLU nets 😲

"...we train Transformer models to discover algorithms for different learning problems."

https://t.co/xuY1SecOrP

[David Andrzejewski @davidandrzej]

Day 1 of #Illuminate22 starts tomorrow! Check out some of the talks that will kick off at 1:00 am PT/4:00 am ET including @SumoLogic's general session on #reliability management and #security in one platform as well as speakers from @eSentire and @Zyston. https://t.co/d0j1fBzNPY

[David Andrzejewski @davidandrzej]

That said, it worked! Big thanks to all.

[David Andrzejewski @davidandrzej]

ML on the M1 has magically transported me back to my youth: a hellscape of Python toolchain chaos with insane fixes like “find a random .py file in your /lib and swap the order of two import statements” https://t.co/vZ4VlYSfDF

[David Andrzejewski @davidandrzej]

@hkarthik @Carnage4Life is there an innocuous technical / data reason why (anecdotally) users overwhelmingly experience asymmetric “slippage” (ie, longer waits) in the predictions?

[David Andrzejewski @davidandrzej]

lmao coinbase: “I understand”

[David Andrzejewski @davidandrzej]

@davidcarlton https://t.co/5u0qqkT4Wf

[David Andrzejewski @davidandrzej]

Pondering the TBs of data crunched in an ultramodern cloud ML platform in order to power cutting-edge causal models targeting tightly business-aligned KPIs in one of the world’s most esteemed tech firms, all leading up to this email.

[David Andrzejewski @davidandrzej]

I’m hiring for an ML Platform Engineer role in the SF Bay Area - we’ve got some interesting problems, please let me know if you’d want to learn more! https://t.co/e5hF1pk7X8

[David Andrzejewski @davidandrzej]

Very nice ad placement here https://t.co/mXlYbnDdQf