/ext/mind

[/ext/mind @michalsuch]

@KinasRemek Zdaje sie ze google idzie w przeciwna strone, optymalizujac intelligence-per-parameter w swoich modelach, np. typu Gemma.

[John Carmack @ID_AA_Carmack]

Memory cost and capacity are significant issues for AI accelerators.

Unlike game rendering, model inference can have a deterministic memory access pattern. You don’t need “random access memory” at all for model weights, and you could tolerate cold-start latencies in the multiple milliseconds, as long as continuous reads were delivered at the necessary bandwidth.

NAND flash is over 100 times cheaper per GB than HBM, so there should be opportunity there, even after giving a flash controller a 1024 bit interface with HBM bandwidth.

You could make a specialized pin protocol that just supported pipelined transfer of full 16KB+ pages from the flash to program-managed accelerator scratchpad memory and improve per-pin performance over HBM, but it might be more convenient to make it still look like a true random access memory with very fragile performance characteristics, where anything but sequential reads falls off a 1000x+ performance cliff.

That has the advantage of automatically using existing cache hierarchies, and providing a natural path to update the flash memory with new model weights. With the stream-to-scratch interface, code has to be completely rewritten before it works at all, while the ram-emulation interface will start off just extremely slow, and you can incrementally sort out the changes for full performance.

There may be cases where there isn’t enough scratchpad SRAM to hold the weights for a layer, which might force you to deploy the old optical drive optimization technique of duplicating data in multiple places on a sequential read to avoid seeking, but there would be capacity to burn.

It might be possible to do something like cuda graph capture to record a memory access trace and have everything magically remapped to a linear sequence, but deploying programmer / agent elbow grease to manage transfers and access in a scratch ram ring buffer would be lower risk.

A split memory system consisting of some channels of flash and some channels of HBM will probably be suboptimal compared to a uniform memory, but it could be much cheaper, and allow much larger models to be run.

I think th case is strong for inference, but you have to stretch more for training. You can still linearize all the weight memory accesses, both reads and writes, but flash memory would quickly wear out from the writes, even if they were all perfectly page aligned. Replacing low-latency HBM with massively parallel cheap(er) DRAM at high latency might still be a worthwhile cost savings.

[/ext/mind @michalsuch]

@FinansowyUmysl przyczyna jest pewnie bardziej prozaiczna, okazało się nagle, że każdy z subskrypcją może się włamać do systemów NSA i Pentagonu, więc pośpiesznie je łatają

[/ext/mind @michalsuch]

@bcherny That would be... interesting to hire an exorcist for AI being possessed by angel/demon.

[Andrej Karpathy @karpathy]

Software horror: litellm PyPI supply chain attack.

Simple `pip install litellm` was enough to exfiltrate SSH keys, AWS/GCP/Azure creds, Kubernetes configs, git credentials, env vars (all your API keys), shell history, crypto wallets, SSL private keys, CI/CD secrets, database passwords.

LiteLLM itself has 97 million downloads per month which is already terrible, but much worse, the contagion spreads to any project that depends on litellm. For example, if you did `pip install dspy` (which depended on litellm>=1.64.0), you'd also be pwnd. Same for any other large project that depended on litellm.

Afaict the poisoned version was up for only less than ~1 hour. The attack had a bug which led to its discovery - Callum McMahon was using an MCP plugin inside Cursor that pulled in litellm as a transitive dependency. When litellm 1.82.8 installed, their machine ran out of RAM and crashed. So if the attacker didn't vibe code this attack it could have been undetected for many days or weeks.

Supply chain attacks like this are basically the scariest thing imaginable in modern software. Every time you install any depedency you could be pulling in a poisoned package anywhere deep inside its entire depedency tree. This is especially risky with large projects that might have lots and lots of dependencies. The credentials that do get stolen in each attack can then be used to take over more accounts and compromise more packages.

Classical software engineering would have you believe that dependencies are good (we're building pyramids from bricks), but imo this has to be re-evaluated, and it's why I've been so growingly averse to them, preferring to use LLMs to "yoink" functionality when it's simple enough and possible.

[/ext/mind @michalsuch]

@bryan_johnson should I keep it as a main llm prompt?

[/ext/mind @michalsuch]

@Stasiu_Botowski @grok sprawdz oryginalne badanie (https://t.co/bRs4KYfaMt) i ocen metodologie i wnioski

[/ext/mind @michalsuch]

Every day with Claude code be like

[/ext/mind @michalsuch]

@r0ck3t23 llms are great because they showed how to scale based on language. now to get the embodied idea of intelligence we need data from robots and more human sensors

[/ext/mind @michalsuch]

@elonmusk @AnthropicAI Then Why Ai?

[/ext/mind @michalsuch]

@cezarykrysztopa @grok chce wydrukowac cos co zrobi efekt w podobnym stylu, jak to zrobic?

[/ext/mind @michalsuch]

@FinansowyUmysl spokojnie, znerfowali GPT zeby bylo taniej, ale da sie go ustawic system promptem zeby dobrze odpowiadal. Musza to dostosowac, bo malo jakiemu userowi chce sie pracowac nad lepszym promptem.

[/ext/mind @michalsuch]

@elonmusk isn't this exactly following your "Idiocracy is here" claim?

[/ext/mind @michalsuch]

@saasmakermac can I ask it to build RalphBlaster?

[/ext/mind @michalsuch]

@PrzemekShura Spokojnie, potem zdejmą wizy, Amerykanie wyślą tam ludzi i ups, USA musi chronić swoich obywateli, bo przecież historycznie Czukotka zawsze była blisko Alaski.

[/ext/mind @michalsuch]

@NotFarLeftAtAll @grok explain manipulation techniques used

[/ext/mind @michalsuch]

@bfialek ok. 10% kofeiny metabolizuje się do teobrominy w watrobie. byc moze to sie doklada na korzystny wplyw kawy. Yerba mate czy guarana ma tez sporo teobrominy i zawsze kojarzyla mi sie z tym innym rodzajem pobudzenia i skupienia od kawy.

[/ext/mind @michalsuch]

@histories_arch This is why cultural relativism matters, "morality is culturally conditioned". Easy to jugde now, but it was a different world.

[/ext/mind @michalsuch]

@Piechocinski zostaje ip box. ale przycięcie b2b również uderzy np. w lekarzy.

[/ext/mind @michalsuch]

simply search for mouse jiggler video, run on mobile and put your mouse on it

[Shukriya Karam Nawazish @typicalnawazish]

Dear team at cognizant,

To whom so ever it might concern.