Runtime Stack

All about tech

[Runtime Stack @RuntimeStack]

@0xCodila Which tool did REA use for the decompilation? These tools do the heavy lifting.

https://t.co/mWc9GbCczf

[Runtime Stack @RuntimeStack]

AI doesn't actually read machine code. Not even REA.

We took one real 17-byte function through every step: bytes, disassembler, decompiler, and only then the AI guessing the names. Plus the two places it quietly breaks.

[Runtime Stack @RuntimeStack]

AI doesn't actually read machine code. Not even REA.

We took one real 17-byte function through every step: bytes, disassembler, decompiler, and only then the AI guessing the names. Plus the two places it quietly breaks.

[Runtime Stack @RuntimeStack]

@Whomujtaba1 Yup

[Runtime Stack @RuntimeStack]

Your LLM scores ~128,000 options for every token it writes. Then it keeps one and throws the rest away.

And it isn't even picking words. Under the hood the whole thing is a while-loop you could write yourself:

ids = tokenize(prompt)
while True:
x = table[ids]
for layer in layers: x += f(x) # x32
scores = unembed(x[-1])
id = sample(softmax(scores / T))
if id == EOS: break
ids.append(id)

Line by line (numbers from Llama 3.1 8B):

tokenize: text becomes integer IDs. "strawberry" is 3 tokens in GPT-4's tokenizer, not 10 letters. That's a big reason counting the R's is hard.

table: each ID is a row lookup. 4,096 numbers per token. Nobody typed them in, training set them.

x += f(x): 32 layers, each adds an edit to a running vector, never replaces it. That's the residual stream. Attention is the one step where a token reads earlier tokens, like a fuzzy hash-map lookup.

unembed: one score per vocab entry. Those are logits.

T: temperature divides the scores before softmax. It reshapes them, it doesn't make the model smarter. And T=0 isn't deterministic: Thinking Machines ran one prompt 1,000 times and got 80 different completions.

The loop: earlier keys and values never change, so they're cached. That's the KV cache, basically memoization. 128 KiB per token, about 16 GiB at a full 128K context. Put the stable part of your prompt first.

Now the part that matters for what's next: from the top of the stack back to the bottom, the only thing passed to the next step is one token ID. About 17 bits.

[Runtime Stack @RuntimeStack]

Apparently, I was liking YouTube videos back in 1969.

36 years before YouTube existed.

That's not a bug. That's early adoption. 😎 #Programming

[Runtime Stack @RuntimeStack]

@mrtbrgl hello

[Runtime Stack @RuntimeStack]

Your Git branch is a text file. 41 bytes: 40 hex characters and a newline.

Open .git and it's just four object types, each named by the hash of its contents, plus a few sticky notes. See that, and rebase, reset and detached HEAD stop being scary.

The four objects:
blob: a file's bytes. No name, no date.
tree: a folder listing, names to hashes.
commit: one tree, its parent, author, message.
tag: an annotated label with a message.

Each ID is the SHA-1 of the content. Hash a file by hand and you get Git's exact ID.

Does every commit copy every file? Kind of. Change one line in a 1,000-file repo and Git writes 5 objects: 1 blob, 3 trees on its path, 1 commit. The other 999 files are reused by hash. Full snapshot, by reference. Deltas only show up later, in packfiles.

A branch is a ref: a file holding one commit hash. New branch = one more 41-byte file. Nothing gets copied.

HEAD is the "you are here" note. Usually it says ref: refs/heads/main. Detached HEAD just means it holds a raw commit hash instead.

The scary commands are mostly pointer moves:
fast-forward: slide the note
3-way merge: one new commit, two parents
rebase: copy commits (new hashes), move the note
reset: move the branch back

Lost a commit? git reflog still has its hash for ~30 days. Put a branch on it.

Full walkthrough, with real git output:

[Runtime Stack @RuntimeStack]

Apparently, I was liking YouTube videos back in 1969.

36 years before YouTube existed.

That's not a bug. That's early adoption. 😎 #Programming

[Runtime Stack @RuntimeStack]

@erik_kokalj @roboflow the P(clean, dirty, cracked) is what makes it a good data collector. auto-label at 0.95 or higher, send the unsure middle to a human, and those middle crops are exactly the edge cases your classifier needs. also images are the one thing Jev can't do
https://t.co/iESpqoepb3

[Runtime Stack @RuntimeStack]

OpenAI's new Decisions API and TypeSafe's Jev look a lot alike.

Same answer shape, and the confidence math lines up. The difference is the engine: Decisions can read images, Jev costs less than half per token and lets you pin a version.

[Runtime Stack @RuntimeStack]

OpenAI's new Decisions API and TypeSafe's Jev look a lot alike.

Same answer shape, and the confidence math lines up. The difference is the engine: Decisions can read images, Jev costs less than half per token and lets you pin a version.

[Runtime Stack @RuntimeStack]

@worldnetwork which agent and which human are the easy two. the one that bites: what may it do. a verified agent with a real person behind it can still get prompt-injected into DELETE /customers. identity proves who called, policy at the gate decides https://t.co/0viLJiOOum

[Runtime Stack @RuntimeStack]

Your AI agent probably runs on an API key that never expires. Copy that key, and you become the agent.

The fix is a passport that expires within the hour: SPIFFE/SPIRE proves who the agent is, and OPA decides what it may actually do.

[Runtime Stack @RuntimeStack]

@Bober_smart solid split, but "every step is correct" is the one thing Jev won't promise. it gives odds, so answers scored 0.8 are right about 80% of the time, not always. that's why thresholds.yaml is the real hero. act when it's sure, hand the rest to a person https://t.co/QqbBUDCiyy

[Runtime Stack @RuntimeStack]

Over a thousand people agreed Jev is "just a classifier". They're half right.

It picks a label and gives you probabilities. The half they missed: you write the question in plain English at runtime, no training, and get a number your code branches on.

[Runtime Stack @RuntimeStack]

@btibor91 everyone's asking if a rewrite beats it. I'd worry more about the opposite, flagging stuff a human wrote. the original research version does that to 1 in 30,000 texts. run it on a million student essays and ~30 kids get accused for their own words
https://t.co/qlWXiBz6pP

[Runtime Stack @RuntimeStack]

Claude and Gemini watermark the text they write. Nothing is hidden in the characters.

It's in which words get picked. A secret key nudges each word toward a "green list" and a detector counts them.

Catch: normally only the key holder can check.

[Runtime Stack @RuntimeStack]

@saidotdev not dumb at all. it's signed, not encrypted. think clear envelope with a wax seal, anyone can read it, nobody can change a byte without breaking the seal. if you need the payload hidden that's JWE, a different thing

https://t.co/aeTHzcs0WB

[Runtime Stack @RuntimeStack]

You've used JWTs a hundred times. You've probably never actually sent one.

That token in your Authorisation header is technically a JWS. JOSE, JWS, JWE, JWA, JWK and JWKS, explained with one shipping-company analogy in 3 minutes.

[Runtime Stack @RuntimeStack]

Claude and Gemini watermark the text they write. Nothing is hidden in the characters.

It's in which words get picked. A secret key nudges each word toward a "green list" and a detector counts them.

Catch: normally only the key holder can check.

[Runtime Stack @RuntimeStack]

Your AI agent probably runs on an API key that never expires. Copy that key, and you become the agent.

The fix is a passport that expires within the hour: SPIFFE/SPIRE proves who the agent is, and OPA decides what it may actually do.

[Runtime Stack @RuntimeStack]

Over a thousand people agreed Jev is "just a classifier". They're half right.

It picks a label and gives you probabilities. The half they missed: you write the question in plain English at runtime, no training, and get a number your code branches on.

[Runtime Stack @RuntimeStack]

You've used JWTs a hundred times. You've probably never actually sent one.

That token in your Authorisation header is technically a JWS. JOSE, JWS, JWE, JWA, JWK and JWKS, explained with one shipping-company analogy in 3 minutes.

[Andrej Karpathy @karpathy]

We'll be spending a lot more time trying to understand the outputs of language models. A few thoughts, tips & tricks:

Writing. Something I've had success with: Ask your LLM to explain something in ASD-STE100, it's a controlled language specification originally developed for aerospace maintenance documentation. LLMs well-versed in this language and it comes with heavy constraints on clean writing style that I often find a lot more readable. Sometimes I've tried to soften it a bit e.g. ask for "80% of the way to ASD-STE100" because the spec is quite stringent. But even better:

Diagrams / images. Instead of writing, ask your LLM to create a diagram. These can be a lot easier to process, parse, and understand. But even better:

Web pages. Ask for output "in HTML" to get a beautiful, interactive webpage. LLMs are getting really good at frontend and can create beautiful experiences, animations, etc. But even better:

Explainer videos. The output format I am most bullish on is fully custom / bespoke explainer videos generated on any arbitrary topic. Experiment with things like "Create a 3b1b style video explainer on X. Use my ElevenLabs API key for audio narration". (you'd need an API key for the latter or you can ask your LLM to find you decent free alternatives that use your local compute). This is actually starting to work!

In summary:
- As LLMs get better, they will do more and more of the legwork autonomously, and a lot more of our work will rise up the abstractions into oversight and understanding.
- Luckily, LLMs can help here too because as intelligence and code are increasingly abundant, you can ask for large, custom, discardable software artifacts (e.g. web apps, video explainers) that would have never made sense to create before. Push the boundaries here and you'll be surprised.

[Runtime Stack @RuntimeStack]

OpenAI had two GPT-6.1 models lined up. The cheap one, Sol, shipped. The smart one reportedly got grounded.

On OSWorld, Sol at max effort costs $1.27 a task vs $9.44 for GPT-6 Astra, for nearly the same score. Here's what actually changed.