procgenerated

Computers were a mistake.

[procgenerated @procgenerated]

@CliffordSosin Having spoken to Fable (and Sol) at length, neither are presently super-intelligent. Almighty search engines yes, but often careless or dim.

There is a large gap between "having access to encyclopedic knowledge" and "being able to reason carefully about it".

[procgenerated @procgenerated]

I spent years in an industry where people would regularly discuss heavily technical tweets by @ID_AA_Carmack .

I later left, doing many different things, but somehow I've ended up right back in an industry where people regularly discuss Carmack's heavily technical tweets.

Help

[procgenerated @procgenerated]

if i zoom all the way out, i'm instancing each pixel individually

this seems like an excellent way to spend half my gpu's capacity

[procgenerated @procgenerated]

i got instanced rendering working and immediately went mad with power

as one does

[procgenerated @procgenerated]

@HazyResearch Tuple structuring lets you tightly control task scope *and* analyze the trained model's weights according to the semantics they refer to.

Once your task is made of tuples, adding support to a toy model is easy: `hidden = embed(toks).flatten(-2, -1)`.

3/3

[procgenerated @procgenerated]

I wanted to pick my model's dimensions based on something other than vibes, so I started with reverse-engineering retrieval in Transformers.

Turns out retrieval is solved geometrically with an emergent spherical code, and head dimension doesn't matter. Go figure.

1/5

[procgenerated @procgenerated]

@HazyResearch TSAR trains quickly, has no grokking phase, and requires only a single layer with a single attention head: You can't ask for much simpler of a setup.

Not only that, but the semantics of the tuple structure is imposed on the hidden state and propagates through the model.

2/3

[procgenerated @procgenerated]

My paper is finished, or at least somewhat presentable. Exists on zenodo, and has a one-line repro on GitHub.

Caught in the weird "well, what now?" phase. Feels like I oughta be doing some kinda ad blitz for it, research isn't useful unless it's used!

[procgenerated @procgenerated]

zenodo blocked me from accessing their site the day after I uploaded a new version of my paper

apparently that counted as "unusual traffic"

incredible

[procgenerated @procgenerated]

Pardon?

[procgenerated @procgenerated]

i think i found an empirical scaling law

[procgenerated @procgenerated]

@EskilSteenberg i can only aspire to such minimal side-effects

is your entire state copied for each frame, or is there some sort of 'commit' that occurs at the end of each?

[procgenerated @procgenerated]

the real dystopia is that they're writing python

[procgenerated @procgenerated]

the year is 2055
AI coding agents still use typing.List, typing.Dict, typing.Optional...