Dana

[Dana @thedanes]

back on twitter only for the coolest stuff 👀

[Gabe Pereyra @gabepereyra]

Update on @harvey’s model training effort.

We post-trained a model we are calling Tenet:
- Achieves SOTA on LAB
- Generalizes 3rd party legal benchmarks
- Uses sub-agents for domain specific capabilities

Tenet uses Kimi K3 as base and was post-trained in collaboration with @FireworksAI_HQ:
- Rank-64 LoRA over the full network
- GSPO with importance-ratio masking
- 134 B300 GPUs for 2 months

Despite not being trained on 3rd party legal datasets we found improvements on:
- @mercor’s Apex Agents - Corporate Law
- @crosbylegal’s Redline Bench
- LegalBench

Tenet also learned how to use domain-specific subagents (separate post-trained models) for complex tasks:
- M&A Diligence: training in an RLM harness for long-horizon tasks (with @baseten)
- Review Table: specialist models for high-volume structured data extraction (with @appliedcompute)
- Firm Knowledge: parametric memory and structured notes for more efficient enterprise search (with @engram)

These results suggest we can significantly scale training and we plan to:
- Scale both human and synthetic data significantly and scale training to 1K and then 10K GPUs
- This scale will let us move to full parameter fine-tuning and larger models
- We are now starting to post train models in our production harnesses
- Post-train other open-source base models to provide customers with model choice

If these problems sound interesting we are hiring for our post-training team

[Dana @thedanes]

@HawaiianAir Thanks, I DM'd on Instagram

[Dana @thedanes]

@HawaiianAir Or can you perhaps send me a DM first to open the conversation because I am now following you

[Dana @thedanes]

@HawaiianAir I'm not allowed to send a DM unless you guys follow me -- are you able to do that briefly while i send a message?

[Brian Chesky @bchesky]

Airbnb 2023 Winter Release

(sound on🔈)

[Carrie Ann @barelycarrie]

I love when I compliment a woman’s outfit and it turns into a 20 minute conversation about the UX/UI design failures of Zara’s e-commerce platform

[Dana @thedanes]

@Planner5D Is the site down right now?

[Big Doc @doctuh_p]

I don’t know about y’all, but I don’t have any more passwords left in me.

[Hot Little Mongoose @HLMongoose]

This is the best thing I’ve read in awhile.

[Alyssa Limperis @alyssalimp]

Usually I’m like noooo I hate daylight savings now we have one less hour of light and this year I’m like yeahhhh let’s go ahead and wrap it up early good call

[Dana @thedanes]

@nrattazzi California burrito from Saguaros or Mexico Viejo. Literally taking Joe there for his birthday next weekend solely to eat burritos (he grew up there, trust him!!)

[Dana @thedanes]

@imaliwaller My friends and I made bets....

[Rep. Eric Swalwell @RepSwalwell]

Wishing you well, Louie! And thank you for standing with millions of women who don’t want you coming between them and decisions they make with their doctor. How about this: you stay out of their doctor’s office and we stay out of yours.

[kylee @stillsobIue]

beautiful moments during protests that the media won’t show; a thread 🖤

[Sean O'Donnell @TheSeanODonnell]

Been sending this to a lot of people lately instead of arguing. My mental health is sky rocketing.

[Dana @thedanes]

@JennineMatthias 💯

[Elon Musk @elonmusk]

Coachella should postpone itself until it stops sucking

[Dana @thedanes]

@dconrad 🤣🤣🤣

[Dana @thedanes]

@michelleswift so young 😭

[Dana @thedanes]

One time I heard my dad tell one of the MANY people who ask him if he wished he had a son: "the only reason I wish I had a son is because when he got married, I'd have another daughter" #girldads #girlsdad