rupayan

Your product is ready. Your distribution isn't. AI agents are changing that math. Building SAMVAD, an open source agent-to-agent protocol.

[rupayan @rupayanism]

prompt: https://t.co/a5lOlbCwfS

[rupayan @rupayanism]

ran one prompt on my bank statement: 22 lines, 3 min 6 s, category totals matched the bank's to the rupee.

the sorting isn't the point. the check is. if its totals don't match the statement's, it has to tell you which lines it couldn't read instead of guessing.

[rupayan @rupayanism]

Repo: https://t.co/la9YdQVRRd
Image billing rule: https://t.co/jKIZlg1WRe

[rupayan @rupayanism]

48k chars of context: ~25k tokens as text, ~2.7k as one image. Same reader.

At Fable list price that's 25 cents vs 2.7 cents a request. Author measures 59 to 70% off a Claude Code bill.

The catch: 12-char hex strings came back 13/15 on Fable 5, 0/15 on Sol. Silent misses.

[rupayan @rupayanism]

Table: https://t.co/DsFjkHPWhu

[rupayan @rupayanism]

Same 105GB model, same 128GB Mac. Only the RAM budget changed: 34.6 / 48 / 73GB.

21 / 27 / 31.5 tok/s.

slotstream reads experts off the SSD each token. More cache, fewer disk reads. One reporter, not a curve; a 16GB M2 got 1.41.

[rupayan @rupayanism]

Table: https://t.co/VxTDhGkb5d

[rupayan @rupayanism]

Quantizing a model to 4-bit does nothing to its context cache. On Llama 3.1 8B the cache stays 16-bit by default and hits 17GB at full context, bigger than the weights you just shrank. The file size is not the RAM budget.

[rupayan @rupayanism]

Maple hitting 218 tok/s is real. Treating that as proof the model is reliable is the mistake.

Its own model card says “minimal general reinforcement learning.”

Speed is a deployment benchmark. Trust is a capability benchmark.

[rupayan @rupayanism]

100B on one CPU at 6.58 tok/s sounds ridiculous.

The footnote matters: that’s a BitNet benchmark configuration on an M2 Ultra with 64GB RAM. Microsoft’s released model is 2.4B, and BitNet isn’t a switch for ordinary models.

https://t.co/qX1eems1Ry

[rupayan @rupayanism]

the 8 second card: https://t.co/qHMQce1Og3

[rupayan @rupayanism]

Soup fine tunes Llama 3.1 8B on a 4GB laptop GPU. RTX 3050, 3.32GB peak, bit exact output, 119.6 tok/s. The frozen base never sits in VRAM; it streams in one decoder layer at a time. The catch: that number is from v0.72.2 and nobody has re run it on a 4GB card since.

[rupayan @rupayanism]

the 8 second card: https://t.co/ClmWE17R5K

[rupayan @rupayanism]

Same model, nine different harnesses, and the cost per task runs $1.05 to $18.34. The catch nobody screenshots: the $1.05 one is also the worst in the table, 53.3% against 66.7%. Drop the outlier and it's still 4.5x. Runta ran it on the runtime it sells.

[rupayan @rupayanism]

the 8 second version, with the setup behind the keyword: https://t.co/uwdlcKGV17

[rupayan @rupayanism]

A 125B model on a 48GB Mac. slotstream keeps 105GB of weights on SSD and a 33GB working set in RAM, about 12 tok/s on an M5 Pro. The catch: Apple Silicon only, 110GB free SSD, a 16GB M2 got 1.41 tok/s. It tells you your speed before you download 88GB.

[rupayan @rupayanism]

Fine tune Llama 3.1 8B on a 4GB laptop GPU.

Soup streams the frozen base from system RAM one layer at a time. RTX 3050 Laptop: 3.32GB peak, 119.6 tok/s, bit exact. The catch: measured on v0.72.2, not re run since the fp32 fix.

5.6k stars. https://t.co/Iph7qZ2PeJ

[rupayan @rupayanism]

Train a 64M language model from scratch for 43 cents.

One RTX 3090, about 2 hours, 3 yuan of rental at Chinese prices, per the minimind README. The catch: one epoch of the Zero variant on the small datasets. Tiny model, real training run.

59k stars. https://t.co/XtfxPF9ZFh

[rupayan @rupayanism]

A picture of your code costs less than the code.

An image is priced by its pixel dimensions, not by the text inside it. So pxpipe renders your context as PNGs before it leaves your machine. About 3.1 characters per token, against 1 as text.

It's lossy on exact strings, though.

[rupayan @rupayanism]

I stopped reading my coding agent's permission prompts weeks ago. I just approve.

clawk boots a throwaway Linux VM per project and turns the safety flags off inside it, on purpose.

Your files were never mounted in, so there is no deny rule left to get wrong.