Arthur Reynolds

Occasional shitposter. Like AI, astronomy, software, math, the usual nerdy stuff essentially.

[Arthur Reynolds @ArthurReyn]

@zeeg Yeah, like general baseline capabilities of say a llama 3B model + being fable level good at coding.

[Arthur Reynolds @ArthurReyn]

@dirkgodshalk @paulg make no mistakes

[Arthur Reynolds @ArthurReyn]

@max_spero_ @ctjlewis "Being good at leetcode" is a sufficient condition for "understanding the code which agents are writing", but it is not a necessary one.

[Arthur Reynolds @ArthurReyn]

@max_spero_ @ainthusiast Leetcode preparation takes a lot more time than making a resume (which can now be made faster with claude code)

[Arthur Reynolds @ArthurReyn]

@Marceli__P Ah sorry, I wasn't clear enough, I meant the tool on which these several icons are created.

But I asked chatgpt about this, and learnt that it's probably Make, a no-code workflow automation platform.

[Arthur Reynolds @ArthurReyn]

@sdianahu https://t.co/3kKgmnmJBf

[John Carmack @ID_AA_Carmack]

Memory cost and capacity are significant issues for AI accelerators.

Unlike game rendering, model inference can have a deterministic memory access pattern. You don’t need “random access memory” at all for model weights, and you could tolerate cold-start latencies in the multiple milliseconds, as long as continuous reads were delivered at the necessary bandwidth.

NAND flash is over 100 times cheaper per GB than HBM, so there should be opportunity there, even after giving a flash controller a 1024 bit interface with HBM bandwidth.

You could make a specialized pin protocol that just supported pipelined transfer of full 16KB+ pages from the flash to program-managed accelerator scratchpad memory and improve per-pin performance over HBM, but it might be more convenient to make it still look like a true random access memory with very fragile performance characteristics, where anything but sequential reads falls off a 1000x+ performance cliff.

That has the advantage of automatically using existing cache hierarchies, and providing a natural path to update the flash memory with new model weights. With the stream-to-scratch interface, code has to be completely rewritten before it works at all, while the ram-emulation interface will start off just extremely slow, and you can incrementally sort out the changes for full performance.

There may be cases where there isn’t enough scratchpad SRAM to hold the weights for a layer, which might force you to deploy the old optical drive optimization technique of duplicating data in multiple places on a sequential read to avoid seeking, but there would be capacity to burn.

It might be possible to do something like cuda graph capture to record a memory access trace and have everything magically remapped to a linear sequence, but deploying programmer / agent elbow grease to manage transfers and access in a scratch ram ring buffer would be lower risk.

A split memory system consisting of some channels of flash and some channels of HBM will probably be suboptimal compared to a uniform memory, but it could be much cheaper, and allow much larger models to be run.

I think th case is strong for inference, but you have to stretch more for training. You can still linearize all the weight memory accesses, both reads and writes, but flash memory would quickly wear out from the writes, even if they were all perfectly page aligned. Replacing low-latency HBM with massively parallel cheap(er) DRAM at high latency might still be a worthwhile cost savings.

[Arthur Reynolds @ArthurReyn]

@bryan_johnson I wish we could predict which gene edit to the cells in the stomach preserves stomach function while becoming invisible to the autoimmune response without creating new risks. In silico screening would be the dream here.

[Arthur Reynolds @ArthurReyn]

@adamghaida @TheIshanGoswami Ah ok, yes, the set of complex numbers has exactly the same cardinality as the set of real numbers.

"but if it’s not in R what is it in!!!" If it's not in R, it could be in C.

[Arthur Reynolds @ArthurReyn]

@CSamarinas @TheIshanGoswami exactly

[Arthur Reynolds @ArthurReyn]

@adamghaida @TheIshanGoswami C is bigger than R

[Arthur Reynolds @ArthurReyn]

@Spshulem You just have to try a bit more, and with time, you can be 30 and broke

[Arthur Reynolds @ArthurReyn]

@BlutoTweets so the best bio hack is gene therapy? bryan johnson tried some of it

[Arthur Reynolds @ArthurReyn]

@rougaroux_ @PoplarKulak @puppyganker @Templarpilled Why is that weird?

[Arthur Reynolds @ArthurReyn]

@sahill_og start with understanding backprop

https://t.co/81xrHw9JnL

[Arthur Reynolds @ArthurReyn]

@bryan_johnson > sleepmaxxing before it was cool
> collecting biomarkers like Pokémon cards
> treating health like a startup
> somehow made bedtime a competitive sport
> the final boss of annual checkups

[Arthur Reynolds @ArthurReyn]

@bryan_johnson wish i could help you in cooking

[Arthur Reynolds @ArthurReyn]

@NptexasNancy @OrwellNGoode reverse pension

[Arthur Reynolds @ArthurReyn]

@juanftl @OrwellNGoode you get reverse pension starting at 18

[Arthur Reynolds @ArthurReyn]

@davep Even one pizza is too much for a team