Gach

Data & AI Architecture | Fulbrighter 🇬🇪🇪🇺 |

[Zhengyao Jiang @zhengyaojiang]

The first experimental evidence of recursive self-improvement (RSI).

Autoresearching the autoresearch agent for eight days.

The result beats the harness we hand-tuned for two years, on held-out benchmarks: 🧵(1/7)

[Gach @georgegach]

Everyone is building loops for agentic orgs; almost none are building the (1) thing that decides what the agents should do, (2) what they are allowed to touch, (3) and what actually happened after. The loop is commodity, the ledger is the moat. Harnesses will churn every six month; run history and state model compound forever.

[Gach @georgegach]

Excellent! https://t.co/3s3z6JJTzd https://t.co/YngdXDO4oV

[John Carmack @ID_AA_Carmack]

Memory cost and capacity are significant issues for AI accelerators.

Unlike game rendering, model inference can have a deterministic memory access pattern. You don’t need “random access memory” at all for model weights, and you could tolerate cold-start latencies in the multiple milliseconds, as long as continuous reads were delivered at the necessary bandwidth.

NAND flash is over 100 times cheaper per GB than HBM, so there should be opportunity there, even after giving a flash controller a 1024 bit interface with HBM bandwidth.

You could make a specialized pin protocol that just supported pipelined transfer of full 16KB+ pages from the flash to program-managed accelerator scratchpad memory and improve per-pin performance over HBM, but it might be more convenient to make it still look like a true random access memory with very fragile performance characteristics, where anything but sequential reads falls off a 1000x+ performance cliff.

That has the advantage of automatically using existing cache hierarchies, and providing a natural path to update the flash memory with new model weights. With the stream-to-scratch interface, code has to be completely rewritten before it works at all, while the ram-emulation interface will start off just extremely slow, and you can incrementally sort out the changes for full performance.

There may be cases where there isn’t enough scratchpad SRAM to hold the weights for a layer, which might force you to deploy the old optical drive optimization technique of duplicating data in multiple places on a sequential read to avoid seeking, but there would be capacity to burn.

It might be possible to do something like cuda graph capture to record a memory access trace and have everything magically remapped to a linear sequence, but deploying programmer / agent elbow grease to manage transfers and access in a scratch ram ring buffer would be lower risk.

A split memory system consisting of some channels of flash and some channels of HBM will probably be suboptimal compared to a uniform memory, but it could be much cheaper, and allow much larger models to be run.

I think th case is strong for inference, but you have to stretch more for training. You can still linearize all the weight memory accesses, both reads and writes, but flash memory would quickly wear out from the writes, even if they were all perfectly page aligned. Replacing low-latency HBM with massively parallel cheap(er) DRAM at high latency might still be a worthwhile cost savings.

[John Carmack @ID_AA_Carmack]

Paper review: LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
https://t.co/TpFFnwPWkc

Nice clean github: https://t.co/HOuqEf0HaF

This is the application of the LeJEPA results to world models, trained offline on experience from three different robotics style tests with one to two million steps in each dataset. Re-states the benefits of the SigReg loss relative to prior world model approaches.

Uses ImageNet standard 224x224 RGB pixel input images with an unmodified ViT-Tiny vision transformer from HuggingFace to generate latents. One extra post-projection step is needed to give SigReg the necessary freedom to perturb the latents into independent gaussians, since ViT ends with a layernorm’d layer. Also tested with ResNet-18, which still performed well, but slightly worse.

Uses a 192 dimensional latent. Performance slightly dropped when doubling the latent size to 384; it would be nice to know if it was stable there, or if it continued worsening with excessive latents. There is a relationship between batch size and SIGReg, the larger latent may have improved performance if the batch size was increased.

The predictor is implemented as a ViT-S backbone – Why a vision transformer when the latent is flat? Uses a history of 3 sets of latents for two of the benchmarks and 1 for the other. Performance was markedly better with the “small” ViT model than the “tiny”, but the larger “base” model degraded notably, which is interesting.

Dropout of 0.1 on the predictor significantly improved performance. 0.2 was still better than 0.0, but 0.5 was worse.

Trained with a batch of 128 x 4 trajectories. I wish their training loss graphs were more zoomed in with grid lines.

Performs planning at test time instead of building a policy by training in imagination like Dreamer / Diamond. Rolls out 300 initially random sets of actions up to a planning horizon H of 5 (at frame-skip 5). Iterates up to 30 times using the Cross Entropy Method (CEM). The main paper body mentions using Model Predictive Control (MPC) strategy, where only the first K planned actions are executed before replanning, but appendix D says they execute all 5 planned actions.

After training, they probe the latent space to demonstrate that it does capture and represent physically meaningful quantities. They also implement a decoder from the latent space back to pixels – not used by the algorithms, but helpful to see what things the latent space is actually representing. They tested incorporating the reconstruction loss into training, but it hurt performance somewhat.

They wound up with a 0.1 lambda for SigReg, as opposed to 0.05 in the LeJEPA paper. 1024 sigreg projections, but observe the number has negligible impact

I like the JEPA framework, but so far my attempts to use it on Atari games with value functions have not matched my other efforts.

[John Carmack @ID_AA_Carmack]

I always lost performance when I tried to use silu/gelu activations in my RL value networks, and I finally understand why.

If the pre-activation values are small, the smooth curve through zero is basically a linear activation, destroying the representation power of the network. You need a batch/layer/rms norm on the preactivations to put them in the range the smooth activations are designed for.

Internal norms generally hurt performance on our RL tasks, but combining them with a smooth activation at least works basically as well as a raw relu (but slower). So, not actually a win, but the lightbulb of understanding was good!

[John Carmack @ID_AA_Carmack]

256 Tb/s data rates over 200 km distance have been demonstrated on single mode fiber optic, which works out to 32 GB of data in flight, “stored” in the fiber, with 32 TB/s bandwidth. Neural network inference and training can have deterministic weight reference patterns, so it is amusing to consider a system with no DRAM, and weights continuously streamed into an L2 cache by a recycling fiber loop. The modern equivalent of the ancient mercury echo tube memories. You would need to pipeline a bunch of them to implement modern trillion parameter models, but fiber transmission may have a better growth trajectory than DRAM does today, so it might someday become viable.

Much more practically, you should be able to gang cheap flash memory together to provide almost any read bandwidth you require, as long as it is done a page at a time and pipelined well ahead. That should be viable for inference serving today if flash and accelerator vendors could agree on a high speed interface.

[Gach @georgegach]

Simulations use double buffering to guarantee a synchronous state transition. It solves the problem of simultaneously reading from and writing to the same memory by using one buffer for reading and a second for writing. I've looked up alternatives like Two-pass and state bits but they're essentially micro-optimizations for memory-constrained small systems, trading CPU cycles for RAM. So far Flipper Zero's 256KB seemed plenty enough but I wonder how far can it stretch in the lifecycle of the game down the line. 512x256 canvas x 2 = 262KB already on unsigned char. Maybe bit-packing to the rescue. I missed thinking low-level in C...

[Gach @georgegach]

Just another Conway's cellular automaton simulation on Thompson's B3678/S34678 rulset on @flipper_net ... so far

[Gach @georgegach]

@maguitoeth @MKBHD Gotta refill the R&D budget and this really is the best AR/VR headset around, for the time being at least.

[Gach @georgegach]

@Angel0Martinez3 @MKBHD I think it has the easiest answer. Magnetic external battery with minutes to change the power bank to another. It's quite elegant.

[Gach @georgegach]

🌱 Tskaltubo, Georgia 💚

[AK @_akhaliq]

Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
project page: https://t.co/6nzZPACkzV

sota FID(7.27 on COCO), without ever training on COCO, human raters find Imagen samples to be on par with the COCO data itself in image-text alignment

[Jordi A. @jordiae]

AlphaStar (2019) vs. Gato (2022) architectures:

[OpenAI @OpenAI]

GPT-3 can now make changes to existing content, not just predict what comes next. Released in the API today: https://t.co/cDOWJwQeZI