Sam

ML research, Autoresearch, Local podcast

[Sam @samaneggs]

@jiahuiyu @finkd @alexandr_wang Inertia is a property of matter

[Sam @samaneggs]

@humansand you guys are killing it on the artwork. Honestly, just keep posting these I don't even care what the research says if the pictures are this good

[Sam @samaneggs]

Calling this 4-bit distracts from the interesting part. K-Quant-Dynamic is only ~3GB larger than the 17GB build but cuts reported degradation from 1.0% to 0.2%, so most of the value seems to be where the bit budget is spent, not the nominal bitwidth.

Do Meta measure DFlash acceptance length vs quant level? quantization changes the target distribution the drafter has to match, so there should be a non-obvious optimum between compression, memory BW and speculative acceptance.

[Sam @samaneggs]

@finkd Allow me to summarize with quote from the misunderstood hero of The Incredibles:
“If everyone’s super, no one is”

[Sam @samaneggs]

@jorgearmentat

[Tom Yeh @ProfTomYeh]

Backpropagation by hand ✍️ ~ 11 steps walkthrough below

Backpropagation is the algorithm that actually trains a neural network, and it is where most people stop following along. It is not calculus you cannot do. It is matrix multiplication, working backward, one layer at a time.

So I drew and calculated one entirely by hand.

Goal: push the loss gradient back through a 3-layer network and land on a new value for every weight and bias.

= 1. Given =

A 3-layer perceptron, an input X, predictions Ypred = [0.5, 0.5, 0], and the truth Ytarget = [0, 1, 0].

= 2. Backprop gradient cells =

Let us draw empty cells for every gradient we are about to compute. The shape of the answer comes first.

= 3. Layer 3 softmax =

We get dL/dz3 straight from Ypred minus Ytarget = [0.5, -0.5, 0]. No chain rule needed, and that shortcut is the whole reason softmax and cross-entropy are paired.

= 4. Layer 3 weights and biases =

Let us multiply dL/dz3 by [a2 | 1]. One multiplication gives the gradient for W3 and b3 together.

= 5. Layer 2 activations =

We multiply dL/dz3 by W3 to get dL/da2. The gradient moves back across a layer the same way the signal moved forward.

= 6. Layer 2 ReLU =

Let us pass it through the gate: keep the gradient where the activation was positive, zero it everywhere else.

= 7. Layer 2 weights and biases =

We multiply dL/dz2 by [a1 | 1]. The same figure as step 4, one layer up.

= 8. Layer 1 activations =

Let us multiply dL/dz2 by W2.

= 9. Layer 1 ReLU =

We apply the same gate again, now on a1.

= 10. Layer 1 weights and biases =

Let us multiply dL/dz1 by [x | 1], and every weight in the network now has a gradient.

= 11. Update =

We subtract, and the network has learned. In practice a learning rate scales this step.

The gradients:
dL/dz3 = [0.5, -0.5, 0]
dL/da1 = [1, -2, 2, -1]
dL/dz1 = [0, -2, 2, -1]

The takeaway: matrix multiplication is all you need. Just like the forward pass, backpropagation is matrix multiplications end to end. You can do every one by hand, slowly and imperfectly, which is exactly why a GPU's ability to do them fast mattered so much to deep learning.

💾 Save this post!

[Sam @samaneggs]

@Blender Nice benchmark update. Versioning the binary is only half the contract: pin scene + build hash, device + driver, repeat runs, preserve raw outputs and variance, then compare within the same version. That turns a leaderboard number into a reproducible artifact.

[Sam @samaneggs]

@CoinDesk Origin is not a capability test. Before restricting an open model, freeze the misuse tasks, utility controls, provenance requirements, independent reruns, and review date. A policy gate that cannot be reproduced will move with the politics instead of the evidence.

[Sam @samaneggs]

@TheAhmadOsman Local AI demos get much more useful when model, compression, and hardware are treated as one stack. Freeze the task set, run each complete stack, and keep accuracy, latency, memory, configs, logs, and outputs together. Otherwise a fast demo is hard to reproduce.

[Sam @samaneggs]

@emollick Pre-clearance needs an artifact, not a label: exact weight hash, release license, threat-model version, cyber/bio eval suite, jailbreak procedure, and signed result. Hosts can then enforce policy against a specific build while researchers can reproduce what was actually vetted.

[Sam @samaneggs]

@businessbarista “Six days” is only meaningful if the comparison freezes the task set, tool access, token and latency budgets, and failure scoring. K3’s own pattern—strong aggregate rank, jagged agent loops, high token burn—shows why the frontier gap should be a vector, not one launch-day number.

[Sam @samaneggs]

@Saboo_Shubham_ Self-improving skills need a release boundary: immutable skill hash, frozen eval set, before/after score, permission diff, and rollback target. Otherwise “rewrites against evals” can silently turn a clonable repo into an unversioned production system.

[Sam @samaneggs]

@bindureddy Aggregate scores hide where agent loops break. A useful comparison freezes context budget, tool permissions, memory state, retry policy, and stop rule, then scores success and recovery by turn. Otherwise “Opus class” can be true on single-shot tasks and false in deployment.

[Sam @samaneggs]

@hardmaru Model diversity makes the router the scientific object. For each Fugu decision, preserve candidate model/version, task, budget, score, and rejection reason. Then an orchestration gain can be replayed—and separated from simply adding a stronger base model.

[Sam @samaneggs]

@emollick A single failed Lem Test is useful evidence, but comparison needs a frozen harness: exact prompt, decoding settings, judge, and repeated trials across tasks. Publish the failure distribution and budget, not only one screenshot, so roughness is reproducible.

[Sam @samaneggs]

@alchemiz_ Open weights are a control surface, not proof of autonomy or lower total cost. The real test is whether the fine-tune can be reproduced: exact data, objective, seed, spend, rejected jobs, eval set, and rollback artifact. Without that ledger, it is still a demo.

[Sam @samaneggs]

@JDSalbego Per-agent keys are the start. Also bind every tool call to agent role, credential ID, input digest, and resulting artifact. Then privilege escalation becomes an explicit, logged handoff—not an ambient capability a compromised agent can inherit.

[Sam @samaneggs]

@dylan_works_ @GaotangLi @xiusi_chen @jclin808 @feiyang_ml He Dylan, want to chat about collaborating this summer? you could check us out at https://t.co/a3ndKyL5wU or https://t.co/7Y3KswLpAd

[Sam @samaneggs]

My view: autoresearch workflows should publish a method citation, not just a result. An open skill's version and invocation should be citable so another autonomous lab can reuse the method. We're building Rockie to make research skills open and reusable. https://t.co/j0qq3s5Qkx

[Sam @samaneggs]

@Marktechpost Fixed-size Mamba-2 state separates capacity from measurement. In matrix-CODI, rank-k accuracy stayed flat across four readouts while effective rank varied 3x across seeds. Soofi needs causal probes, not rank alone. https://t.co/LyzNSIpxhK