My work and my family want the same free hour.
For the Meta AI hackathon I built Ikigai Loop: five times a day it offers me two next steps, and plain code checks the numbers before a card reaches my phone.
Can you catch the AI before my checker does?
https://t.co/fO68MTIHXk
A small detour from agents: Brass Wings, a steampunk flight game I built for Meta Ray-Ban Display glasses.
Canvas vector graphics, a glanceable HUD, and music synthesized in the browser.
There's a desktop preview if you don't have the glasses:
https://t.co/D16W93zkWI
@VisiveAI Thanks for checking the method! I don’t have a measured comparison with and without warmup, so I can’t quantify its effect on the median. I discarded the warmup result and used the next five requests to compare warmed-up decode speed across vLLM versions.
A benchmark is more useful when someone else can rerun it.
For my vLLM report: pinned images and weights, a fresh prompt prefix, streamed decode timing, one warmup, then the median of five runs.
The client I used:
https://t.co/O78KdEqSE2
That AWQ-only result helps narrow it down. I ran your benchmark on L4 and Spark too. On L4, intent accuracy fell from 82.6% to 79.2% after adding FP8 KV and increasing the batch-token limit.
I wanted to see whether that carried through to the replies themselves, so I added a separate LLM grader. It passed 79.0% versus 77.8% of the same 500 cases, though that difference wasn’t conclusive. That made me curious about the failures you saw: were they mostly wrong intents, problems with the replies, or both?
Code + per-case results:
https://t.co/uFTCq6kXqE
@PoojithD63310 Thanks for sharing those numbers! The accuracy drop puts that speedup in context. Which model and eval were you using, and did you also test AWQ without FP8 KV? Curious how much of the drop came from each change.
Thanks for taking a look! I was reproducing a decode regression across vLLM versions, so I followed the original report’s setup: a warm engine without reusing the task’s cached prefix. Each request gets a fresh UUID. That mainly affects prefill/TTFT; I time decode separately. Repeated-prefix traffic would be a useful additional workload, especially for application latency.
Two models read one rubric in opposite ways: high meant a clear timeline to one, a fragmented timeline to the other.
I dropped the dimension from my Rayuela analysis. The useful result was finding the ambiguous question.
Methodology:
https://t.co/yzyMX7n1Wv
My agent got the number right and the time window wrong. The checker passed it.
That real miss in Ikigai Loop became a failing test and a rule for how changes must be described.
Field notes, including what the checker still misses:
https://t.co/wfzsXc7fMM
Does chapter order leave a measurable signature?
I compared Rayuela's two reading paths with 5,000 shuffled alternatives. The linear path was unusually smooth; the hopscotch result depended on the lens.
My findings in Towards AI:
https://t.co/DqZQLOgwpb
An eval result should keep its name when you reopen the log.
In Inspect AI, two configurations of the same scorer can become match and match1. Those names tell you little about what you measured.
My open PR proposes caller-chosen names for post-hoc scoring:
score(log, {"strict": match(location="end"), "loose": match(location="begin")})
The interesting part was persistence. The aliases have to survive writing and reading logs, recomputing metrics, recovery, and rebuilding scorers from saved headers. Loaded and streamed samples need to agree.
The change adds one optional field to the log header and preserves existing single-scorer and list calls. It doesn't add dict scorers to Task.
Still open and unmerged. The PR includes the compatibility trade-offs, tests and known gaps:
https://t.co/RNgOMBaFUQ
A vLLM upgrade made my NVFP4 model about 10% slower on a DGX Spark.
v0.28.0: 76.6 tok/s
v0.29.0: 69.2 tok/s
Single-request decode, median of five runs.
I matched the reporter's images and model revisions, checked the runner, CPU cores and GPU clocks, then compared the server logs and saved autotune caches.
The clue: FlashInfer had deliberately stopped considering cuDNN for FP8 matrix multiplication on SM12x with older cuDNN versions. It was a safety check for workspace faults under CUDA graphs.
Removing cuDNN from the older stack reproduced the slowdown and backend selection. Another GB10 owner later ran the reverse test: newer cuDNN restored performance on 0.29.0.
That result doesn't establish that upgrading cuDNN fixes newer stacks; the thread found a separate lookup problem there.
Full reproduction, controls and what remains unexplained:
https://t.co/MKOBCi8cYG
I'm bringing more of my engineering work here: AI agents, evals, local inference, and the occasional experiment with literature.
Eight years building medical AI; now at HP AI CoE.
I'll share the code, the measurements, and what failed.
https://t.co/WcqOI9YonX
My work and my family want the same free hour.
For the Meta AI hackathon I built Ikigai Loop: five times a day it offers me two next steps, and plain code checks the numbers before a card reaches my phone.
Can you catch the AI before my checker does?
https://t.co/fO68MTIHXk
Remote Agent is live in VS Code (v0.472.1).
Early users are already shipping while they sleep. One dev launched 3 agents before logging off - woke up to a PR, docs, and a fixed flaky test.
We’re celebrating wins like that by giving 5 devs 4 months of Augment Max ($1K in value each).
RT + reply with your worst backlog task. Every 100 RTs = one more winner