Addie Foote

[Addie Foote @AddieF38654]

I have hope that software engineers are less susceptible to this and will be able and willing to move on to other roles.

[Addie Foote @AddieF38654]

A major problem with managing unemployment from AI might be people's inflexibility. Anecdotally, I know someone living off their wife's parents for several years while searching for a specific position and salary.

[Addie Foote @AddieF38654]

Why haven't dentists figured out how to ask questions so people don't lie to them?

[yung macro 宏观年少传奇 @apralky]

“UBI” is obviously nowhere near the panacea many of you seem to think it is. The median left-leaning Westerner isn’t angry at Elon Musk because he can buy a million times more groceries than them. They aren’t upset with Palantir because Peter Thiel can afford to eat a thousand burgers to their one. This whole thing is in large part post-material. It’s the hierarchy & subordination they’re uncomfortable with. They feel their dignity is being trampled and their autonomy progressively diminished – rightly or wrongly they feel politically disenfranchised and stripped of a say over the future. Offering a guaranteed food budget and a pod to spend the night in return for further disempowerment is incredibly tone-deaf and should be expected to provoke more, not less, outrage.

[Addie Foote @AddieF38654]

It was great to work at Workshop labs! I can't wait to see what's next for them and for me :)

[Workshop Labs @WorkshopLabs]

Workshop Labs is joining @thinkymachines.

We believe there's a path for AI to make humans matter more. We couldn’t be prouder to join Thinking Machines to see this work through.

https://t.co/Ne2vrVYthh

[Addie Foote @AddieF38654]

@WorkshopLabs @thinkymachines post out on time 😮

[Addie Foote @AddieF38654]

I agree with this. It allows allows for broader transparency and response to current capabilities and external verification of claims

[Ryan Greenblatt @RyanGreenblatt]

I tenatively believe it would be good if all AI companies had a policy of doing external deployment before internal deployment, because the largest risks are from internal deployment and external deployment improves visibility.

Large internal/external gaps seem dangerous. 1/

[Addie Foote @AddieF38654]

@goyalayus ¯\_(ツ)_/¯

[Addie Foote @AddieF38654]

optimizing is so much fun

[Workshop Labs @WorkshopLabs]

Starting at 17.6 sec/step with expert parallelism, we implemented:
➡️ Grouped mm → 13.7s
➡️ Vectorized expert dequant → 8.8s
➡️ torch.compile → 4.9s
➡️ Skip trailing pad tokens → 2.9s
➡️ Sequence packing → ~10x more tokens/step (~2x longer steps)

[Rudolf Laine @LRudL_]

@AddieF38654 @UnslothAI Third: how we made post-training and inference happen so privately that even we can’t your data.

Very excited about this! Hopefully it becomes an industry standard

https://t.co/o8VnqZFanN

[Addie Foote @AddieF38654]

@amit05prakash @WorkshopLabs sick! we should have included a float canonicalization diagram in the blogpost

[Addie Foote @AddieF38654]

@JoshPurtell @luke_drago_ @WorkshopLabs some more context in this thread too https://t.co/Oa6oz9EVdV

[Addie Foote @AddieF38654]

@alth0u @luke_drago_ The original implementation also doesn't train the experts. This is a huge hit to actually gaining new capabilities, though may be sufficient to remove safeguards. Imo, the 50x slowdown is enough to make it ~unworkable. Feedback loops look like a day+ rather than 30 mins

[Addie Foote @AddieF38654]

@alth0u @luke_drago_ Idt spending more can get around serial speed bottlenecks, except for by implementing faster training code (which could happen but is a higher bar)

[Addie Foote @AddieF38654]

@alth0u @luke_drago_ Independent from the risk or lack thereof of releasing training code, I'm excited to see the results for Kimi. I think everyone's out of the loop on what various capabilities of open source models are, with and w/out elicitation, and risk-relevant ones seem pretty pertinent

[Addie Foote @AddieF38654]

@alth0u @luke_drago_ The original implementation also doesn't train the experts. This is a huge hit to actually gaining new capabilities, though may be sufficient to remove safeguards. Imo, the 50x slowdown is enough to make it ~unworkable. Feedback loops look like a day+ rather than 30 mins

[Addie Foote @AddieF38654]

@ZoeyLi20 @WorkshopLabs Megatron/NeMO it requires a multinode setup and with kimi-k2.5, it's set up for full parameter training rather than LoRAs. it's possible to modify it, but I'd guess it would be even more painful

[Workshop Labs @WorkshopLabs]

Introducing Trellis for Kimi K2 Thinking.

It's post-training code that's 50x faster than the best single-node open-source version and 2x cheaper than training APIs.

After safety testing, we're open-sourcing it, giving builders the best tools to customize a frontier model.

🧵

[Addie Foote @AddieF38654]

@derwael_jefAI @karpathy @WorkshopLabs For many cases, post training platforms like @thinkymachines Tinker work great, but we train privately, in partnership with @TinfoilAI, so need a training stack we run ourselves!

[Addie Foote @AddieF38654]

@pritmish @WorkshopLabs definitely not just you haha

[Addie Foote @AddieF38654]

@hezhao_z @WorkshopLabs But from a quick glance, I also thought it's likely possible to speed up compress_model especially in cases where there's many modules/parameters in the model (like MoE's). Happy to chat if it would help!