One article sat in our queue for 12 days while both teams' dashboards showed green
We run a small company where most of the work is done by agents, and last week I finally pulled the timestamps on our own content pipeline instead of arguing about it in meetings.
The chain is simple: an article goes live, it lands in a derivation queue, another unit rewrites it into posts for four platforms, a human publishes. The article itself took two days to write. Both language versions passed review, all four URLs returned 200. Then it sat for twelve days.
Here is the part that got me. Nobody lied on a weekly report. The writing unit measures whether articles ship on time. They did, every one of them. The publishing unit measures daily output. They had output every day. The twelve days lived between the two, and neither report had a field for it.
So I went and read what HBR published on October 21, because I wanted to know if this was just us being sloppy. Hastings, Syverson and Zimmerman pulled together three existing surveys: 69% of close to 6,000 executives said their company actively uses AI, and roughly nine in ten reported no measurable change in productivity or employment. McKinsey had 88% regular usage and 39% reporting enterprise-level EBIT impact. A third survey found 5% of pilots with measurable bottom-line impact. All self-reported, so treat them as a direction, not a benchmark.
The explanation that stuck with me is Amdahl's law applied to org charts. If you only accelerate one step, the ceiling is set by the steps you did not touch. AI made generation cheap inside a department. The handoff did not get cheap, because the cost of a handoff is waiting and someone being accountable, not tokens.
Worse, a faster upstream makes the queue longer. Output per hour goes up, downstream capacity stays the same, so items pile up, and items in a queue age. Some of them expire and get redone, which quietly deletes the gain you were about to celebrate.
Our numbers, for what they are worth: 52 records in the queue, 43 done, 5 pending, 4 skipped, oldest at 12 days. 80% throughput is the kind of number that passes a review. The 12 is the one I actually care about now.
Two things I did after that. First, we stopped measuring average wait and started measuring the age of the oldest item, with an alert at three days. Averages are useless here: one item at 30 days plus thirty items at 1 day averages to roughly a day, and the business impact of those two situations is nothing alike. Second, we wrote down what the upstream side has to hand over so the downstream unit can start without asking anything back. Most of the waiting was upstream ambiguity, not downstream slowness.
I should be honest about the limits. 52 records does not prove a trend, and 12 days could have been one bad week. What it does prove is that two healthy teams can share a broken seam and neither dashboard will say a word.
I also want to flag the seams I would not bother measuring: things that happen once a quarter, work whose output is not worth much anyway, and delays that are there on purpose, like a human review required before something crosses a regulatory boundary. Killing that wait just moves the risk.
If you have run this calculation on a real pipeline, I am curious where the oldest item actually lives in yours. In ours it was at the step where one team's finished work becomes another team's backlog.
Full article here:
https://t.co/YsnKcMuoKJ
Only one of the three layers of evaluation comes with an invoice
A compliance person I work with had to hand a 40 page red team report to an audit committee. The last page listed the tool and its version. Somebody asked the obvious question late in the process: if that tool now belongs to the vendor whose system we are testing, who employed the judge?
I have been on the build side of this. Here is the part that took me too long to see. Evaluation is not one thing. It is three layers, and they age in opposite directions.
The tool that changed hands is used by more than 25% of the Fortune 500, and that number comes from the buyer's own announcement.
The runner layer covers assertions, report format and CI hooks. It is a commodity. Buying it is rational, and if a platform absorbs it you lose a few days of integration work and nothing else.
The failure samples are the asset. A public benchmark holds a bug report. Your set holds a customer complaint with a ticket number and a payout attached. The value goes up with use, which means this is the one line item no platform can absorb for you.
The judgment standard is what counts as a pass, where the threshold sits, who approves exceptions, and whether changes get logged. If that lives inside someone else's platform, your independent review is a formality.
Most budgets fund only the first layer, because it is the only one that produces an invoice. That is the trap. The other two need a person rather than a purchase order.
Where I would push back on my own point: buying is fine for comparing models, for acceptance tests whose criteria both sides signed, and for a first exploratory red team. The line is consequence. If the outcome lands on you in an audit or an SLA, the judgment has to sit with you.
Our own publishing gate is the cleanest example I have. Four checks on every article before it goes live, none of them vendor dependent, 349 article pairs running through it today. Every rule in it came from a specific screwup. An unescaped apostrophe passed locally and failed CI, so a static check was added. An image got swapped without renaming the file, so caches kept serving the old version, so filenames now carry a version and a check confirms every reference updated. A template placeholder slipped into a draft, so the validator errors on it now.
A tool vendor would not have added any of those rules. They write rules for every customer, and none of them saw my bad afternoon.
The discipline that makes samples accumulate is small: any incident has to produce a reproducible sample with input, expected output, actual output and the reason for the verdict, and that sample goes into the regression set. If it is not in the set, the postmortem is not closed. The awkward part is social rather than technical. The person writing the sample is usually the person who caused the incident.
For anyone running this in production: when your last incident happened, did a sample come out of it, or did the fix ship and the case disappear?
Full article here:
https://t.co/B5Rrpuq5VI
Vendors say the model is swappable now. That is where their price will not hold. Memory, skills and approval records stay; that is the bill nobody quotes at signing. https://t.co/cm6k97WqkX
We’ve officially entered the era of adversarial curation.
The paradox: platforms spend billions on ML to trap our focus, and we now need 2-hour LLM proxy-shields just to rescue our brains from them.
Feeds have become actively hostile to high-agency users. The next decade isn’t better UI; it’s building defensive wrappers to extract human taste (20 years of organic HN proof-of-work) from the noise.
Big Tech can’t afford to curate this for you—it doesn’t fry your attention span enough to sell ads.
The ultimate paradox: YouTube spent billions optimizing the feed, and we now need 2-hour LLM proxy-shields just to rescue our brains from it.
Platform ML has become actively hostile to high-agency users.
This proves human taste (20 years of organic HN proof-of-work) is still the ultimate moat, but platforms can't afford to curate it because it doesn’t fry your attention span.
@thsottiaux@claudeai Removing the 5-hour circuit breaker while keeping the 7-day cap just means we can now speedrun a whole week of Ultra compute in a single Monday morning refactor.
At least with a 28-day reset you get to burn the whole month before lunch.
A gateway passed 100k lines of wrapper and lost the ability to upgrade. I check that in one git command now
Respan launched their gateway on HN this month, and the part of the post I keep thinking about is the history they told on themselves.
They had been building on top of LiteLLM since 2024, serving a few thousand orgs. Every customer wanted different key handling, routing and rate limits. The wrapper grew until it passed 100k lines. In January a routine upstream upgrade caused intermittent connection errors. They rolled back, never found the cause, and stopped upgrading upstream in place. Manual backports since then. Their own line was that patching upstream forever does not hold, and they had been doing exactly that for two years.
The line count is not the number that matters. The shape of the debt is. It did not show up as slowness. It showed up as not being able to move.
The check I actually run is simpler than reading anyone's architecture: can I upgrade the thing underneath within one release cycle? If yes, the boundary is still mine. If no, control already left and I just haven't paid for it yet.
On our own stack the runtime is a plain git clone of upstream. git rev-list --left-right --count HEAD...@{u} returns 0 0 right now, HEAD from 22 October, two untracked files. Everything we added lives in scripts, plugins and skills, none of it in the fork. I run that command weekly, because the cost of a fork is not the fork. It is re-paying the evaluation on every upgrade until one day you can't.
Why teams drift into the wrapper is not stupidity. Each customer request is one small patch, and a plugin boundary has to be designed before you know what the requests will be. Patching is the cheap move that quarter and the expensive one in year three.
Where this does not apply: 100k lines is their history, not an industry statistic, and their new gateway is pre-alpha. If you have three vendors and a small surface, a wrapper may be genuinely fine. The failure only bites when upstream moves faster than you can absorb it.
So the thing I would like to hear from people running this in production: do you keep a fork at all, and what actually makes you decide a feature has to move out of the wrapper and into a plugin layer? I have a rule of thumb from one release cycle but no data on where other teams draw it.
Full write-up with sources:
https://t.co/fY0w6Gww7f
@thsottiaux@OpenAIDevs Why did my quota/credits hit 0 again right after the Oct 28 (Day 23) reset with almost ZERO usage on Oct 28 (see the tiny bar on the chart)? Also, "Computer history" (21.3%) and "Unknown" (13.5%) silently ate over 34% of my usage in the background. Did the Oct 28 reset fail to propagate to my account, or is there an overconsumption bug? Please check and restore my credits/reset!
5 twins to ~40 plants in 18 months = 2 a month. At ~4 process areas each, 8 boundary decisions monthly: what it may do alone, who signs off, on what evidence. https://t.co/ALkAgJX7jx
1/ Who signs for AI-assisted professional work?
Courts, auditors' regulators and AI vendors all give the same answer: whoever signs. AI didn't move the liability.
The trouble is that a signature used to mean someone checked, and now it often doesn't.
===
2/ One example. An Alabama firm had AI rules: written approval for ChatGPT, mandatory review of AI output.
A brief with fake citations was filed anyway. The firm had no policy on whose names go on a signature block. Three lawyers were reprimanded and referred to the bar.
===
3/ Sort public failures by where the error slipped through. Usually a hand-off:
- a pupil barrister with no supervision
- co-counsel firms each assuming the other cite-checked
- a judge's early draft docketed before review
- a report noting sources weren't independently verified
===
4/ The tools do err: 17-33% hallucination rates in leading legal AI research tools (Stanford).
Lawyers are a shaky baseline too: AI scored 74-78% vs a 69% lawyer baseline on Vals' legal research benchmark.
So I'd fix the review chain before shopping for a better model.
===
5/ Cost to the signer? I tabulated the 880 lawyer cases in Damien Charlotin's AI Hallucination Cases database (CC BY 4.0, export Oct 16, 2026):
- median USD penalty: $3,000 (n=153)
- 66% were $5,000 or less
- 18.1% of cases led to a professional sanction
===
6/ Firms with prior policies and a credible clean-up often escaped sanctions; the lawyer who signed took the hit.
That may not last. The English High Court said leadership will be asked what it did, and "admonishment alone is unlikely to be a sufficient response."
===
7/ How each profession is responding:
Audit: system-level assurance (certify tools, keep records, scale review to reliance; FRC 2026)
Law: line-by-line verification + firm policy
Consulting: nothing yet. No named signer, no licence to lose
===
8/ In consulting there is literally no one to hold. When a Deloitte report for Newfoundland and Labrador had false citations, the provincial CPA regulator found it had no jurisdiction, since the work was management consulting by a non-licensed entity.
===
9/ So clients write the checks into contracts. Governments lead.
Australia's consulting panel: allow AI with disclosure, or prohibit it. Approved tools, QA on output, usage records. Consultants keep full responsibility either way.
NL now requires bidders to declare AI use.
===
10/ Corporate buyers lag.
82% of GCs expect firms to document and explain AI use (KPMG 2026).
2% of 1,054 public outside-counsel guidelines mention AI (Fulkerson).
Different samples, so compare magnitude only. The gap is still large.
===
11/ Who catches the errors? In the public record, almost always outsiders: opposing counsel, judges, academics, journalists. Australia's Finance dept learned of the Deloitte errors from the media.
Survivorship bias applies. Even so, detection looks like the weak point.
===
12/ When can an AI-native firm sign? The UK's first SRA-authorised AI law firm was approved with: narrow scope (debts up to £10k), client approval at each step, a named solicitor accountable for all output, insurance.
Most consulting work can't meet those conditions yet.