Drew

gpu, compiler, etc

[Drew @weirdcuttlefish]

@ChampRDS like father like son

[Drew @weirdcuttlefish]

i shouldve done a PhD or joined a research lab 😀

[Drew @weirdcuttlefish]

@DreCooln @clipcloud we already tried that with UFC 😞

[Drew @weirdcuttlefish]

@snowclipsed @joefioti

[Drew @weirdcuttlefish]

@MillieMarconnni Dijkstras algorithm was known by the Greeks.

[Drew @weirdcuttlefish]

@Porter_lsland @d33rLMAO fish and chips are good 😢

[Drew @weirdcuttlefish]

@athe_red @reiaposting you have not visited the bass pro pyramids and it shows.

[Drew @weirdcuttlefish]

@IshaanGoyal05 @adcock_brett nah what do you mean by that 😭

[Drew @weirdcuttlefish]

@elonmusk is xai bottlenecked by CPU as opposed to system level issues?

[Drew @weirdcuttlefish]

@AnthonyPatch15 @maharshii scheduling distributed system hard!

[Drew @weirdcuttlefish]

@carllerche @Leik0w0 makes sense, it is very nice and i used a lot of the blogs when i was implementing a toy asyncio in C++.

[Drew @weirdcuttlefish]

@carllerche @Leik0w0 fwiw (wrt the original post), i do not think tokio is meant to be “fast.” it does optimize a bit (thread local queues for example), but it assumes your workload is supposed to be IO-bound. it is very nice to program with.

[Drew @weirdcuttlefish]

@Leik0w0 @carllerche not to say a framework for this wouldn’t be interesting, just that tokio may not apply well. i have definitely had fun thinking about what that would look like (and not doing anything about it) 😛😛😛

[Drew @weirdcuttlefish]

@Leik0w0 @carllerche iirc for latency, hazy uses a static inst schedule and for throughput they use a dynamic queue of inst. for the latter each block/grouping polls from the inst queue (on what is ready) to distribute work. this is much different from dynamically handling coroutines of diff shapes.

[Drew @weirdcuttlefish]

@Leik0w0 @carllerche tokio would be more useful to define/handle cuda stream.

[Drew @weirdcuttlefish]

@Leik0w0 @carllerche tokio is designed for io-bound workloads. i think rayon/intel tbb/etc. is probably more relevant to the GPU.

the hazy MKs are great but there are a lot more considerations for memory management that CPU async libraries don’t handle well.

[Drew @weirdcuttlefish]

@SheriefFYI C++ coroutine API was a mistake.

[Drew @weirdcuttlefish]

@jsuarez maybe codex xhigh made a bad assumption the buffer would be small.