Sam Foreman

models / pre-training @argonne ; training big models for science on supercomputers : views are my own

[PyTorch @PyTorch]

DeepSpeed now supports the Muon Optimizer.

Optimized specifically for internal 2D weights within neural networks, Muon is gaining traction for its significant memory savings and strong convergence metrics during LLM training.

In our latest blog post, the DeepSpeed team shares a deep dive into their integration setup, implementation of hybrid optimizer strategies, and early benchmark results. @PKUWZP

Read the full technical breakdown here 👉 https://t.co/t7JxOqkM6S

[Sam Foreman @saforem2]

@DavidSHolz yeah there are a at least few teams that i’m aware of!

i use it in pretty much all of my ongoing projects, and we’ve been using it @argonne_lcf for some efforts around portable benchmarking and federated / resilient training

[tetsuo.cpp (no slop) @tetsuo_cpp]

Oh, you're writing CUDA kernels? Everyone's on Triton now. Just kidding, we're all on Mojo. We're using cuTile. We're using ROCm. We have an in-house DSL compiler targeting the NVGPU MLIR dialect but wait, Tile IR just dropped so we're going to target that instead. Our PM is on TileLang. The team lead was on CuTe but now she's back to handwriting PTX. If you're not on Pallas, you're ngmi. Our intern is building on TT-Metalium for our Wormholes. Our CFO approved an order for some big chungus wafer-scale chips so now we're porting our kernels to CSL. Our CTO is working on a kernel-less graph compiler so we won't need to write kernels anymore. Our CEO thinks we're talking about the Linux kernel. We're building Claude for dogs.

[Sam Foreman @saforem2]

@marcsh @DavidSHolz it’s not easy but we managed to train aeris on 110k gpus

https://t.co/5rW2qVbmSO

[PyTorch @PyTorch]

Don't miss @DeepSpeedAI virtual office hours on May 26 at 12:00 PM America/New_York to ask questions of @toh_tana member of DeepSpeed TSC & get the latest recent key updates, including AutoSP (sequence parallel), AutoEP (expert parallel), and AutoTP (tensor parallel).

[DeepSpeed @DeepSpeedAI]

Great News! Thanks to DeepSpeed AutoSP, efficient long context LLM training is now easily accessible.

[Kyle Cranmer @KyleCranmer]

The shot heard around the (US scientific research) world

[Sam Foreman @saforem2]

@landon20s @argonne @Fermilab @MorningstarInc @TempusAI @CMEGroup @MotorolaUS @AbbottNews wow shouts out! excellent first pick

[Bryce, the CUDA Colonel @blelbach]

Today, NVIDIA is launching the next paradigm shift in GPU programming: cuTile BASIC

Write perf portable BASIC kernels and deploy them at any scale from edge inference devices like your calculator to entire GPU clusters

We're going back to BASIC

https://t.co/meF2T0jUSc

[Sam Foreman @saforem2]

feel free to reach out if you or your team is interested in submitting a proposal !!

[U.S. Department of Energy @ENERGY]

https://t.co/KwC3AUorN3

[James Ferguson @Psy_Fer_]

Yea your plotting library is cool, but can it do this?

(yes, that's plotting in the terminal, with just ascii, utf-8, and ansi)

[baby keem @babykeem]

how do u fix openclaw internal reasoning leaking

[Sam Foreman @saforem2]

@RhysSullivan @opencode that clock is sick

[Jason Sheltzer @JSheltzer]

Amazing paper from @PhilHolliger

https://t.co/fzpL8Qj5Fh

[Yad @YadKonrad]

i call this dev quality of life improvements,
few others:
alias npm='pnpm'
alias claude='claude --dangerously-skip-permissions --teammate-mode tmux'
alias code='zed'
alias cursor='zed'

[Sam Foreman @saforem2]

alias uvi='uv pip install'

[Sam Foreman @saforem2]

anyone and everyone!

free compute
+ funded postdoc for good ideas!

[venktesh @vvenki22]

@saforem2 Hi is this open to everyone or the PI should be based in US ?

[Sam Foreman @saforem2]

@vvenki22 open to everyone!

[Sam Foreman @saforem2]

🚀 ALCF APEX (AI Program for EXploration) is open!

Are you a domain scientist or an AI researcher interested in large-scale agentic AI for science?

APEX teams get leadership compute + close collaboration with ALCF experts + fully-funded postdoc

𝗗𝗲𝗮𝗱𝗹𝗶𝗻𝗲: 𝗙𝗲𝗯 𝟮𝟳, 𝟮𝟬𝟮𝟲
@argonne_lcf