Rozzai

hazardous® materials™ engineer

(happy to DM)

[driss guessous @drisspg]

The perfetto gods have shined upon us: https://t.co/ECcRbcBHST

No longer need my postproc passes: https://t.co/UgXn02nuQO

[Rozzai @_rozzai]

@zetalyrae Don't you feel like internet/social media can be described in exactly the same way? While it may feel sad, this comes with the territory and could be net gain.

[Rozzai @_rozzai]

@henrylhtsang @cis_female https://t.co/bAEhNkAY6x

https://t.co/fXrbVBttgm

try stream-k :^)

[Rozzai @_rozzai]

@JohnJumperSci @demishassabis

[Rozzai @_rozzai]

@Benthamsbulldog @AlexBores Not publicly, sure, but candidates have enough leverage/tools to let PACs know that this is messaging they are okay with, or in worse case support.

[Rozzai @_rozzai]

@Benthamsbulldog @AlexBores https://t.co/2ggR4a6R0r

Sure. This is still very bad. And to some limits PACs are able to coordinate with candidates, it would be pretty easy to clarify that this is over the top messaging (Redboxing, b-rolls, references, etc) without FEC rules violation. This is very common.

[Rozzai @_rozzai]

@Benthamsbulldog @AlexBores https://t.co/k0LdWviWfr

https://t.co/hrnCHJiLS4

Those two ads. While I initially had very good impression of him, those were really over the top.

[Rozzai @_rozzai]

@lost_nomad__ This is was pretty much overdone in late 2000s with Knausgård tbh, even in movies with stuff like modern remake of «Scenes from the Marriage».

People don’t actually really like it, those things remain pretty niche and elitist (not esoteric/hard to understand, though)

[Rozzai @_rozzai]

@devarbol what living in dubai does to a person

[Rozzai @_rozzai]

@devarbol feels like starve the beast strategy (retarded)

[Rozzai @_rozzai]

@neocartesian @typedfemale typed's magical realism adventure

[Rozzai @_rozzai]

@neocartesian @typedfemale

[Rozzai @_rozzai]

@henrylhtsang or it was volta don't remember

[Rozzai @_rozzai]

@henrylhtsang i think ampere had some warp divergence guarantees that were later removed and it was possible to exploit them on very specific shapes

[Rozzai @_rozzai]

@c20 If you are going to visit Armenia, you should probably talk to @ArmenAgha

[Rozzai @_rozzai]

@neocartesian @allTheYud that would make FDT academic and hence bad so...

[Hao AI Lab @haoailab]

(5/5) On a B200, Blackwell's tensor cores are so fast that the softmax now becomes a bottleneck in addition to the GEMMs. Quantizing PV adds scale-factor overhead that piles onto the softmax warps. So we run NVFP4 QK + BF16 PV, with a careful TMEM overlap schedule to fit scale factors into an already-full pipeline.

Result: 1801 TFLOPS and up to 1.39x over FlashAttention-4.

2x/4x faster exp on B300/Rubin should push this further, and end-to-end FP4 serving, once blocked by attention quality, is now within reach.

[Rozzai @_rozzai]

@real_redp @difficultyang Sorry, could you elaborate a bit?

[Rozzai @_rozzai]

@Simon_Vt https://t.co/e6ZYmsUlPU

This sounds like an approach described in Alex Aiken's paper and the previous work on superoptimizers -- this is really cool, but I was always unsure about scalability of this as solvers take a ton of time, especially for JIT compilation.