Felipe Cruz-Salinas

Pre-training @cohere

[Felipe Cruz-Salinas @fffffelipec]

@kroscoo you're welcome

[Sam Bowyer @sambowyer__]

My @cohere internship project w/ @kroscoo and @acyr_l is on arXiv! We show that efficient benchmarking (predicting scores from a subset of questions) can be greatly improved using standard feature-selection & regression techniques (mRMR and kernel ridge)! https://t.co/ZVX3lK8N68

[Nick Frosst @nickfrosst]

New leaderboard for audio transcription just launched and our apache 2.0 Cohere-Transcribe is at the top. This eval didn't exist when we trained the model, so its nice to see us do so well on it.

https://t.co/HVEtH9QS7Q

[Felipe Cruz-Salinas @fffffelipec]

See more here: https://t.co/QiqbydgyNN

[Felipe Cruz-Salinas @fffffelipec]

We released a new agentic coding model today :) I'm very excited for this new family of models!

[Cohere @cohere]

Small: 30 billion parameters, 3B active.

Efficient: Benchmarks to 33.4 on the Artificial Analysis Coding Index, competitive among similar sized models.

Open Source: Apache 2.0 license so developers can experiment, test, and build their way.

Learn more: https://t.co/2xIxJR3xcr

[Felipe Cruz-Salinas @fffffelipec]

Cohere's first MoE and Apache 2 open-weights model :) . Hope you like it, we're just getting started 🚀

[Cohere @cohere]

Introducing: Cohere Command A+

We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.

[Dwarak @DwaraknathG]

Extremely excited to present Command A+, our first sparse model!

I am very proud of the work we did to enable this model. We built our sparse training stack from the ground up over the past year with a lot of custom kernels, performance engineering to enable us to train large sparse models with a very small compute footprint.

We presented our work on a fully dropless FP8 kernel stack for sparse models at Nvidia GTC earlier this year. We are extremely ambitious and are marching towards a state-of-the-art model training stack, come join us if you are interested in pushing the frontier!!

Apply here - https://t.co/BhV5ffuarA
Our work here - https://t.co/o86oK5iD2E

[Cohere @cohere]

Introducing: Cohere Command A+

We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.

[Aidan Gomez @aidangomez]

Cohear 👂

Cohere's first audio model. Apache 2.0. #1 on the Open ASR leaderboard. Multilingual transcription across 14 languages.

[Felipe Cruz-Salinas @fffffelipec]

tfw you realize you’re a fossil now

[tender @tenderizzation]

fossils remember that a fancier residual stream++ architecture was the top signal for convnets

will it be the same for attention?

[rohan anil @_arohan_]

A bit polarizing comment: its too late but I kind of think whoever named TP DP EP combinations probably slowed down progress by inventing terminology thats borderline absurd to describe basic sharding

[Kris Cao @kroscoo]

I'm recruiting at Cohere for someone to work on pretraining model evaluations! If you're interested in really understanding how to evaluate base models, please apply at the following link (and DM me as well so I know to look for your application).

https://t.co/waggFaXYlr

[Felipe Cruz-Salinas @fffffelipec]

Maybe it’s time to have a new COLM-style conference just for large-scale DL systems. Even if these papers were accepted, I think the chance they would win a best-paper award is very low, and that is very sad. MLSys exists, though

[Johannes Hagemann @johannes_hage]

The PipelineRL paper getting rejected at NeurIPS reminds me of when the Megatron-LM paper got rejected from every conference back in 2020

scientific reviewers still don’t recognize a good systems paper when they see one

https://t.co/KMPnx5HlnX

[Felipe Cruz-Salinas @fffffelipec]

@WHinthorn Congrats Will! :)

[Felipe Cruz-Salinas @fffffelipec]

Future paper ideas: romanized-comic-sans-ligature is all you need for tokenizer-free LLMs

[theseriousadult @gallabytes]

this is still a tokenizer

[Felipe Cruz-Salinas @fffffelipec]

If you're interested into making large pre-training (and training overall) as fast/efficient as it can be, this is a very nice opportunity. Come work with us :)

[Dwarak @DwaraknathG]

I am hiring highly skilled performance engineers for my team! You will be working on optimising pretraining for models >100B params on O(1000s) of GPUs, and hardware-aligned architecture design. We are cooking a lot of very exciting projects and I can safely say you will have a lot of fun! Link in thread. <3

[Dwarak @DwaraknathG]

I am hiring highly skilled performance engineers for my team! You will be working on optimising pretraining for models >100B params on O(1000s) of GPUs, and hardware-aligned architecture design. We are cooking a lot of very exciting projects and I can safely say you will have a lot of fun! Link in thread. <3

[Felipe Cruz-Salinas @fffffelipec]

@jxbz a nice one to add: what if your input/output embeddings are tied? :)

[Bowen Yang @RobertY970316]

Excited to share that our paper Rope to Nope and Back Again: A New Hybrid Attention Strategy (https://t.co/siCUzQnsLJ) has been accepted at NeurIPS 2025! Thanks to my amazing co-authors @bharatvenki, @DwaraknathG, John Lin, @davidcairuz, Phil_Blunsom &amp; @acyr_l for their hardwork.

[Felipe Cruz-Salinas @fffffelipec]

~1y ago I was pretty mindblown using reasoning models for easy but non-trivial codeforces problems. Now we're really living through our own AlphaGo moment for competitive programming.

[Mostafa Rohaninejad @MostafaRohani]

1/n
I’m really excited to share that our @OpenAI reasoning system got a perfect score of 12/12 during the 2025 ICPC World Finals, the premier collegiate programming competition where top university teams from around the world solve complex algorithmic problems. This would have placed it first among all human participants. 🥇🥇