A completed contract or render is only the final answer.
Labs have millions of final answers.
What post-training teams are buying is the evolution: the red lines, the revisions between 2D and 3D, and the context that explains why a change happened.
If you want to know what AI will actually be good at next, track what data labs are buying right now.
In their S-1, Anthropic's vendor spend on data and evals jumped $250M in six months. Reddit's licensing backlog dropped by a third with zero renewals.
The era of scraped forum posts and random conversational text is over. Researchers are paying for private evals, traces w/ tool calls, and tasks playing out in deterministic environments.
The fastest way to make a data buyer pass is fuzzy consent.
Does not matter how clean your dataset is. If your terms of service did not explicitly clear commercial AI training, no serious lab will touch the intake.
RL environments starve on clean web text.
Models do not learn to reason from the final answer. They need the failed attempts, rollbacks, and console logs from real production runs.
Messy operational data is the entire moat.
Labs are drowning in static Q&A pairs.
Every post-training intake call right now is asking for the messy middle: failed tool calls, retries, and state rollbacks inside real enterprise workflows.
We just licensed 24 TB of production Revit and CAD data. Digging into the pipeline makes one thing clear about physical AI.
Most teams treat this as a visual scale problem, scraping meshes or dumping synthetic Unreal scenes into S3 buckets. The catch is that an agent training on hollow surfaces only learns to render pretty pictures without picking up real physics. Reinforcement learning falls apart when walls have zero density and doors are just painted textures.
Production CAD is completely different because every file carries decades of frozen engineering logic. Columns track the exact load they hold, doors have real swing clearances, and every wall is tied to building codes. When you move one beam, the entire structural model pushes back on you.
That kind of deterministic ground truth is the unlock post-training researchers care about, because the agent actually has real physical constraints to fail against.
@DataLabelBanker Probably right on scarcity. The bottleneck is consent and payment rails for speakers, more than finding the speakers. Whoever makes that clean wins the supply.
@rtheoryxyz Which is why the people setting the prices should be the ones who've eaten the costly mistakes. Most rubrics are written by people who've never had to pay for one.
@BesherH Good angle. Beneficiary stories have a consent problem that goes beyond the website owner. The people in them never agreed to training use, so opting out may be the conservative default until that's sorted.
@torenceli Agreed that it's progress, not a business model. A training opt-out tells a crawler what not to do. It doesn't tell a buyer what they're allowed to do. That's still a contract.
@burkov Feels like this argues for paying more per label, from fewer, better-qualified raters, rather than scaling the crowd. Have any of the studies compared expert-only subsets head to head?
@raven_protocol Curious how they handle data rights on the open recipes. If the SFT and preference sets are released too, provenance documentation becomes as important as the code.
Synthetic data works until an agent hits a production edge case.
Generated text only reproduces what models already know. It cannot hallucinate the human judgment calls made during a live operational breakdown.
Real execution exhaust is irreplaceable.
@maishimadamd@LiudasPanavas The patient-merge example says a lot. Retrieval worked, orientation failed. In healthcare that's the error that gets caught in chart review, not in a benchmark. Are clinicians writing your rubrics?
@IlirAliu_@aliansarinik@micro1_ai The bit on serving competing labs without leaking is what buyers will care about most. Curious what the actual controls are: separate teams, separate data rooms, or both.
@courtneyr The draft itself says private agreements may override the signal, and Microsoft says it sets up no licensing or payment arrangements. So the token records who said no. It says nothing about who gets paid for yes.
@neilkale Congrats on shipping. Graders are where I'd expect most customers to struggle. Are teams bringing their own rubrics, or does the platform help turn domain experts' edits into reward signal?