Ish

Machine Learner

[Thariq @trq212]

What is effort really? When do you change it it and why not just use max effort for everything?

I dove deep into this problem, looking into evals and doing my own tests and I was quite surprised by the results. https://t.co/KO2D51j34H

[Ish @ishgirwan]

@anshnanda @RayFernando1337 Why not use TDD. Write failing tests first and then the implementation.

[Ish @ishgirwan]

@trq212 I no longer feel the need to use it but used to be a significant part of my workflow.

[Ish @ishgirwan]

@lihanc02 Try the concise mode

[Ish @ishgirwan]

@trq212 @noahzweben How do you decide when to use the app or the terminal?

[Ish @ishgirwan]

This is super cool 👌

[Fuli Luo @_LuoFuli]

Nearly half a year of silence. We spent it studying one problem: how far RL can scale.

MiMo-V2.6 is in the middle of its RL run right now. Three things we scaled: compute (~2B tokens per step, 1568 prompts × 16 rollouts, fully async), environments and harnesses (multi-task agentic RL, mixed across multiple harnesses in one run), and grader compute (agentic in-group credit assignment, with test-case and rubric-based rewards). We'll open-source the details piece by piece over the coming weeks.

Streaming the run: https://t.co/ZSxahzJRju

[Ish @ishgirwan]

@a16z How is agency measured? Pedigree is much easier to measure and that’s why becomes the default choice for hiring 🤔

[Ish @ishgirwan]

Well said 👍

[lauren @poteto]

@neogoose_btw someone is learning something new, no need to be a bully about it.

[Ish @ishgirwan]

@lydiahallie Thanks that helps 😊. Is thinking higher in Fable 5.1 compared to 5 , which is why it burns more tokens ?

[Ish @ishgirwan]

@batuhan Wow, I can barely afford one 😟

[Ish @ishgirwan]

@bentlegen Will it be possible to be on Apple Podcasts too?

[Ish @ishgirwan]

@poteto @jamonholmgren Wow pstack has so many skills, do you have a list of must haves that we should have in our dynamic workflows and that have worked really well for you

[Ish @ishgirwan]

@bcherny @mattpocockuk It feels these levels are more relative than absolute values depending upon a user’s skill levels on a certain axis.

[Ish @ishgirwan]

@claudeai Found it very useful 👍

[Claude @claudeai]

Claude Academy is now live.

Whether you're figuring out what AI is or already using Claude every day, there's a path that meets you where you are. The courses and tutorials are free and open to anyone at https://t.co/WRiSRAvK6l

[Ish @ishgirwan]

@rajpalleti314 @askalphaxiv Thanks. Closing and opening the app a few times worked 👍

[Ish @ishgirwan]

@hkproj Weekend sorted 😅

[Umar Jamil @hkproj]

In this (extremely long) video, I'll build a distributed training framework from first principles using PyTorch.

We'll derive everything from first principles, explaining the mathematics behind distributed training as well as every auxiliary component we build along the way, including Multi-head Latent Attention (MLA), Rotary Position Embeddings (RoPE) and YaRN.

I'll explain process groups, ranks, collective communication operations, distributed autograd, device meshes and distributed tensors, and then use these foundations to implement the major forms of parallelism used to train modern language models.

We'll derive and code pipeline parallelism, data parallelism, FSDP, HSDP, tensor parallelism and context parallelism, including pipeline schedules, block matrix multiplication, model sharding and ring attention. We'll then combine these techniques and examine how tensor layouts and gradients must be tracked as data moves between devices.

Most importantly, we won't study each parallelism technique in isolation. We'll combine pipeline, data, tensor, context and expert parallelism into a single working framework and follow the tensors, communication operations and gradients as they move across GPUs. By the end, you'll understand not only how these techniques work individually, but how modern distributed training systems compose them to train large dense and mixture-of-experts models.

We'll also build the important components of a modern transformer, including rotary embeddings, long-context extensions, multi-head latent attention and a mixture-of-experts architecture. This leads into expert parallelism, expert tensor parallelism, token dispatching, all-to-all communication and combining different parallel dimensions through dense and sparse device meshes.

We'll also build the supporting training infrastructure, including the training loop, optimizer, learning-rate schedule, checkpointing and metrics.

The goal is to give you the foundations required to understand how distributed training frameworks work internally and how to design and implement new parallelization strategies yourself. No prior knowledge of distributed training is required.

Link to the video: https://t.co/ioSYostMYS

[Ish @ishgirwan]

@mattpocockuk How about using @semgrep