Richard Diehl Martinez

Thinking about llm efficiency, pre-training,

[Richard Diehl Martinez @richarddm1]

Introducing Pico: a lightweight framework for language model learning dynamics

[Richard Diehl Martinez @richarddm1]

People feel strongly about Suzhou

[Richard Diehl Martinez @richarddm1]

Maybe something like 50% of the posters were also from Chinese institutions - would be interesting to do a meta-analysis re research trends between countries.

[Richard Diehl Martinez @richarddm1]

This was a conference for Chinese researchers. 80% or so of sponsors were looking to hire in china.

[Richard Diehl Martinez @richarddm1]

Also/ kind of surprised by the lack of interpretability research. Could probably count on one hand mentions of SAEs

[Richard Diehl Martinez @richarddm1]

Not many linguisticsy or information theoretic papers. It’s almost retro if someone has ‘NER’ on their poster

[Richard Diehl Martinez @richarddm1]

Conference was light on theory, heavy on engineering. In part because of booming interest in post-training/agents, papers skewed more towards engineering efforts than theoretical science

[Richard Diehl Martinez @richarddm1]

That said, the amount of work required to publish a demo paper is 5x a normal paper. It’s a bad deal if all you care about quantity over quality.

[Richard Diehl Martinez @richarddm1]

This was my first demo paper. The fact that demo papers get to be presented with a poster and a monitor/display is ideal. It’s so useful to have some medium other than a poster to present your work. We should at least have a whiteboard for every poster.

[Richard Diehl Martinez @richarddm1]

First, if you’re interested in developing language models more scientifically, check out my paper: https://t.co/rdcmYQ3CFy (or the website https://t.co/bwL9a20JyD)

[Richard Diehl Martinez @richarddm1]

Just got back from EMNLP. Here are some quick thoughts about the conference: 1/n

[Richard Diehl Martinez @richarddm1]

well that was your daily reminder that we're all screwed if cloudflare goes down womp womp

[Richard Diehl Martinez @richarddm1]

@uktaxcalculator @runpod 🥴

[Richard Diehl Martinez @richarddm1]

@runpod dashboard down again for anyone?

[Richard Diehl Martinez @richarddm1]

Damn these guys have some good takes 👀 if you want more checkout https://t.co/No41QplUPk @PretrainedPod

[Richard Diehl Martinez @richarddm1]

@djseo @purepathwill @roozm this dude has some vibey design demos on his website too - https://t.co/0hGftDLsnR

[Richard Diehl Martinez @richarddm1]

@watchdogmo @piercefreeman @PretrainedPod appreciate the feedback! feel free to share us with ur friends if you like the content. 🙏

[Richard Diehl Martinez @richarddm1]

so i started an ai podcast - https://t.co/oFiDLjUt6G (@PretrainedPod) - with my bud @piercefreeman .

Pierce is the most cracked engineer I know - I do AI research. We go more technical than most 'techie' science pods. Most recent ep is on Kimi. Lmk what you think.

[Accelerate_Science @AccelerateSci]

📢 New blog! Find out how @richarddm1 from @Cambridge_CL used funding from @AccelerateSci to develop @Pico_LM - an open-source framework with less than 500 lines of Python code that democratises AI model training for researchers.

Find out more: https://t.co/XN9dajPEbM

[Pierce Freeman @piercefreeman]

Text diffusion models might be the most unintuitive architecture around

Like: let's start randomly filling in words in a paragraph and iterate enough times to get something sensible

But now that google's gemini diffusion is near sota, I think we need to take them seriously