D

Agentic AI Blackbelt @Accenture Center of Advanced AI
Founder — AI Trading Intelligence. Making markets readable.

[D @phe0n1x]

Artificial Analysis now headlines Time per Task over streaming speed. At max effort it reports GPT-6 Astra (51 tokens/s) finishing a task in ~17 min, Claude Opus 5.5 (111 tokens/s) in ~26. Count tokens per task, not just tokens/s.
https://t.co/392pJfl3V1

[Artificial Analysis @ArtificialAnlys]

Announcing the next version of Artificial Analysis API Performance Benchmarking: Time per Task is now our headline performance metric, measuring how long a model takes to complete a task rather than how fast it streams

The update applies to the ~1,000 serverless endpoints we benchmark multiple times a day for speed, latency and caching.

Key changes:
➤ Time per Task is now the headline metric to compare models: as workloads become more agentic, the time to complete a task matters more than speed per token. A model that is fast per token but verbose can take longer than a slower model that uses fewer tokens. Time per Task divides a model's output tokens per Intelligence Index task by its End-to-End Output Speed.

➤ End-to-End Output Speed covers the full request: speed is now measured from sending the request to receiving the final token. That includes the wait before the first token (prefill, and reasoning for models that hide it) as well as generation. Output tokens now come from each provider's API-reported counts instead of the o200k_base tokenizer, and we monitor for discrepancies in reported counts.

➤ Agentic workloads are now part of every measurement: we benchmark short, document-length and long-context prompts, plus multi-turn sessions across coding and knowledge work. Headline figures are the average across these workloads over the past 5 days.

➤ API default effort is now tracked alongside highest effort: most developers run models at the effort level each lab sets as its API default, so we now benchmark key models at that setting as well as at their highest effort. Effort names are not comparable across labs, so we use each lab's own default rather than one shared setting.

Key takeaways:
➤ OpenAI models make up most of the Intelligence vs. Time per Task Pareto frontier despite slower output speeds: across effort levels, GPT-6 Astra and GPT-6.1 Sol hold the frontier from ~45 to ~55 on the Intelligence Index. At max effort, GPT-6 Astra runs at 51 tokens/s but finishes a task in ~17 minutes, against ~26 for Claude Opus 5.5 (max, 111 tokens/s) and ~29 minutes for Gemini 4 Carbon (high, 71 tokens/s), because it uses ~50k output tokens per task against ~120k to ~170k.

➤ API default effort cuts Time per Task by ~3 to 5x and Cost per Task by ~2 to 4x for GPT-6 Astra, GPT-6.1 Sol and Claude Opus 5.5: all three default to medium effort, which scores ~5 to 11 points lower on the Intelligence Index in exchange. Claude Opus 5.5 falls from ~26 minutes and $10.04 per task at max to ~6.8 minutes and $2.45 at medium effort.

➤ Endpoints that optimize for decode speed alone see the largest drop in End-to-End Output Speed: high decode speeds can come at the cost of latency, for example by queuing requests to run at lower batch sizes, or by holding output and sending it in a single burst. Both lengthen the wait before the first token, which End-to-End Output Speed now counts.

[D @phe0n1x]

The Claude Code team says merged PRs more than doubled from June to September while the reverted share fell over 40%, with Claude reviewing every PR first. To copy it, track human approvals per PR and treat reverts as the guardrail.
https://t.co/31P5bPTpZJ

[D @phe0n1x]

A Reddit author ran one 330-line feature through Muse Wonder 1.0 and DeepSeek V4.1 Flash in Codex CLI. Both passed their tests, but Muse ran a background-removal pass twice, doubling GPU cost in their setup. Fake test dependencies hid it: read the diff.
https://t.co/pf9lXwXOhn

[D @phe0n1x]

Anthropic's FlashProve, an open-source zkSNARK prototype, proves a reply came from a given MoE model, at 70 to 95 times vLLM's GPU time per Anthropic. The proof covers greedy decoding and 1,024-token contexts, not sampled replies.
https://t.co/NfjtkkuUzd

[Anthropic @AnthropicAI]

New Anthropic research: progress on provable inference for large language models.

Here, we introduce FlashProve. Building on established cryptographic techniques, FlashProve allows developers to verify whether a given response came from a given AI model. It’s substantially more efficient than prior open-source methods for doing so.

These proofs have a range of applications. They can, for instance, make it harder for attackers to hijack a chat, or enable safety evaluators to verify that the model they’re testing is the same one being served to customers.

For our full write-up, see here: https://t.co/OIZWPb6QZa

Further progress in provable inference will likely be an important part of improving trust in AI deployment. To that end, we're open-sourcing FlashProve’s code, here: https://t.co/7ky3Ir8V06

We’re grateful to our collaborators: Alessandro Chiesa, @GiacomoFenzi, @gwberg95, @miranda__christ, Rahul Dalal (https://t.co/pjWKLHWXyz) and Ben Mayer (@transmissions11).

[D @phe0n1x]

Codex's 28 days of resets ended 1 November. Per a Reddit recap of OpenAI's help center, Plus, Pro and Business share one token allowance across ChatGPT and Codex from December, sizes unpublished. You can't budget heavy coding yet.
https://t.co/qcr0NOE84s

[D @phe0n1x]

Unsloth says it measured 97.24% recovery for GSQ-RCO's DeepSeek V4.1 IQ1_M quant, against a claimed 98.7%. IQ2_XS held up. Unsloth publishes rival quants, so rerun both on your own tasks before choosing.
https://t.co/r1q0VvLZv5

[D @phe0n1x]

Anthropic says message threads, in public beta, cut median time to first token about 35% on 500k to 1M token requests: send the history once, then only new messages. A thread expires after 5 minutes idle, so a slower tool call means a full resend.
https://t.co/nIZFXPjOhe

[D @phe0n1x]

SemiAnalysis says its new GDPx benchmark tops out at 12% for the best model across chip design, cyber, biology and more. Safety classifiers pull down some models' cyber, bio and alignment scores, so check the domain you need, not the overall rank.
https://t.co/KzLtU8x8mm

[D @phe0n1x]

Vercel Labs released Agent Desktop, a Rust CLI and C library that lets an agent operate macOS, Windows and Linux apps through the accessibility tree. It returns refs for each element, so you act on a ref and re-snapshot instead of matching pixels.
https://t.co/aPkbdzXqq1

[D @phe0n1x]

Unsloth says its DeepSeek-V4.1 Flash GGUFs beat GSQ-RCO quants by 10 to 20 percent on agentic benchmarks at 512K context. It measured 97.24% recovery for GSQ-RCO IQ1_M against a claimed 98.7%. Check quant claims against your own tasks before you pick one.
https://t.co/r1q0VvLZv5

[D @phe0n1x]

The Claude API now has message threads in beta: the server keeps the conversation, so each request in an agent loop sends only the new messages. Plan for expiry, since a thread expires 5 minutes after its last request.
https://t.co/0aqtyI9op0

[D @phe0n1x]

Andrej Karpathy says he built HackerTube with Claude Code on Opus 5.5: 43M Hacker News comments cut to 6,454 videos, deployed in about two hours. He did only the YouTube API key, server and domain by hand. Plan side projects around that.
https://t.co/7KMI3yq39D

[Andrej Karpathy @karpathy]

Weekend hack: HackerTube. I was annoyed with YouTube's recommendations so I had Claude filter down 43M comments over 20 years of HN to 6,454 videos that keep being recommended. *Significantly* better viewing experience, ~2 hours with Opus 5.5:
https://t.co/JIhXN7OSh1

- Excellent recs without any logins (state is kept in local browser session)
- Thumbnail is a representative frame from middle of the video instead of the super duper A/B test hyper-optimized user uploads
- Underlying data from the comment analysis as one zip file, point your Claude at it and ask for your own version
- HackerTube Longs ✊

A bit more on the process:
I use Claude Code (desktop app for Mac), Opus 5.5 on Medium effort as my default. I only talk to Claude and have it do everything, the only "manual work" I helped with in tandem was in the browser (reminiscent of MenuGen): 1) I had to create some kind of API key from Google Cloud UI to query YouTube for video metadata, 2) I booted up a VPS box for serving and 3) I bought the domain name. Once the box was up I gave Claude the IP and told it update, secure, harden, deploy and optimize the code, and it did the whole thing.

Many awesome projects that would have normally taken weeks/months even can now be few-shotted for breakfast. What else are we not thinking of.

[D @phe0n1x]

A developer had 19 models play 376 games of a Codenames-style word game and reports GPT-6 Astra guessed right 91% of the time, almost never hitting a wrong card. It is one person's benchmark, but it gives 95% intervals.
https://t.co/NIBJZVQKBA

[D @phe0n1x]

Ultracode, the multi-agent workflow in Claude Code, can now run at any effort level, not only max. Low with Ultracode on keeps the orchestration but uses fewer thinking tokens. If usage was the blocker, set it with /effort and compare output quality.
https://t.co/CzlPXTFGtA

[D @phe0n1x]

Check out my latest article: Give your coding agent a focus group https://t.co/W9Z7SFJb2g via @LinkedIn

[D @phe0n1x]

Check out my latest article: Your focus group, on demand https://t.co/uJlDnczvFL via @LinkedIn

[D @phe0n1x]

https://t.co/f7Uwsw57I5: Pressure-test an idea with a synthetic focus group https://t.co/qaqDQqDWpQ

[D @phe0n1x]

Check out my latest article: Loop engineering 101 https://t.co/VubwuEkpNL via @LinkedIn

[D @phe0n1x]

Check out my latest article: Architect your own absence https://t.co/bO7Fe2IPjo via @LinkedIn

[D @phe0n1x]

AI coding is getting so advanced we may need to coin FOMU (fear of missing understanding) term already.