combin8

[combin8 @combin8or]

@anemll @Apple Hell yeah. I wonder if that PCIe block on the M5 Pro die shot we were looking at was actually 6.0 or if they're going with different versions.

Anyway, now we just need the industry to embrace domain specific experts so they can be easily streamed

[combin8 @combin8or]

@ronaldmannak @anemll They can each run their own copy of macOS either way, though I think something lighter weight would be better suited. But yeah, we just need Apple to get on board 😅

[combin8 @combin8or]

@ronaldmannak @anemll There's no making a server like what you're talking about without updating software. Also updating software isn't hard and we do it all the time as a matter of course

[combin8 @combin8or]

@anemll @ronaldmannak mcio ftw

[combin8 @combin8or]

@evadne @ronaldmannak Yep, PCIe switches w/ nontransparent bridging would be the most straightforward since they already exist, though it would also make sense for Apple to develop TB5 switches for their non-internal products

[combin8 @combin8or]

@mgaruccio @ronaldmannak You’d want weights on both the SSD and distributed so as to make maximal use of available PCIe lanes. In the case of the M5, that’s ~1/3 of the weights on the SSD and ~2/3 distributed

[combin8 @combin8or]

@ronaldmannak About the interconnect, I’m not seeing any TB PHY chips. Maybe they’re using a PCIe switch on the backplane? I really want to see more pics

[combin8 @combin8or]

@anemll I hope Apple is an early adopter of HBF. Not sure how they’ll handle the consumer presentation since it is a wear part. Probably makes sense to make it part of the SSD logical volume as cache and automatically promote things like LLMs

[combin8 @combin8or]

@ronaldmannak I’d expect the SSD would contain weights. Even the base M5 has a fast SSD (~4x PCIe 4.0), and when I/O matters (e.g. streaming weights) you’d want to use all of it. The base M5 also has 3x TB4 ports at 40Gbps, so ~8x PCIe 4.0 for distributed&comms. I’d love to see the comms board

[combin8 @combin8or]

@ivanfioravanti 👀 that’s soon!

[combin8 @combin8or]

@teortaxesTex Right, all the evidence points to Zai, but still people are guessing elsewhere like there’s a big twist

[combin8 @combin8or]

@ivanfioravanti Nope, it’s good

[Percy Liang @percyliang]

🚢 Marin 535B-A23B started training this week! As usual, the whole process is open.
Voyage plan: pretraining (80%) + midtraining (20%) on 18.75T tokens on 11 x GB200 NVL72 for ~3 months (2.7e24 FLOPs). Post-training will follow.
Before kicking off the run, we trained a 4-rung scaling ladder from 1.6B-A61M (48B tokens) to 27.7B-A1.2B (926B tokens) to debug issues, and to make a forecast of our hero run. This is by far our biggest run, so definitely expecting the unexpected.

[combin8 @combin8or]

@andimarafioti Big open model from google would be awesome

[prefix.dev @prefix_dev]

Conda packages can have extras now.

Optional dependency groups you declare and then consume, via v3 repodata. Plus when conditions and variant flags.

This should also be available on conda-forge in the near future!

📖 : https://t.co/ZwoEhsYTwL

[combin8 @combin8or]

@onthexitter69 I’d like to compare on my M5 Max and M4 Pro. Which model specifically? HF link?

[Jayden Teoh @jayden_teoh_]

Interesting work by @mc_mozer and @savvyRL! Been looking forward to its release.

The fact that this method works training-free still continues to astonish me. I have been thinking a lot about why would transformers learn internals that resemble recurrent processing. The fact that transformers hidden representations perform temporal straightening naturally (Hosseini et al., 2023) suggests that there exists a Markovian kernel that can take one hidden state to the next via a delta. Similarly, works that distill transformer layers into SSMs like MOHAWK (Bick et al., 2024) further corroborates this claim.

I suspect that recirculation would only get more accurate and stable as model size increase. There is evidence of emergent capability in large language models which is the ability to introspect on their activations (Lindsey et al., 2026).

Is fixed-depth really a limitation of transformers? Perhaps fixed-depth parallel computation is a way to amortize the sequential overhead of learning recurrent models. Further use of harnesses such as RLM and this recirculation method would unlock the recurrent depth that the transformer is implicitly trying to learn, but never given the inference tools to leverage.

[combin8 @combin8or]

@rosinality No code? 😕