Praxis

AI, Info, Education & Tech

[Praxis @praxis2001]

@kimmonismus The bigger play is product adoption imo. Teach teams to actually use Claude across workflows, and Academy becomes a distribution layer—not just an education portal.

[Praxis @praxis2001]

@ClaudeDevs @CopilotKit The interesting part is the abstraction: Managed Agents handle the agent runtime, AG-UI handles the interaction layer. That makes the UI replaceable without rebuilding the agent itself. That’s a pretty important primitive for agentic apps.

[Praxis @praxis2001]

AI Sales Operating System — Part 3/3

Parts 1–2 covered architecture, database, and the research→verification pipeline. This is where AI stops being trusted blindly: human approval, hard send gates, safety checks, and how the whole thing gets tested and monitored in production. (3/3)

—

STEP 17 — HUMAN APPROVAL

[see image 4]

Slack message includes: company, ICP score, intent score, buyer, why-now, evidence/sources, verifier result, confidence, subject, message body, plus lead ID, message ID, and workflow execution ID for debugging. Then: APPROVE / EDIT / REJECT.

Approval state machine:

DRAFT → VERIFIED → PENDING_APPROVAL → APPROVED or REJECTED → SEND_GATE

Editing never skips re-verification:

PENDING_APPROVAL → EDITED → RE-VERIFY → PENDING_APPROVAL

Never: EDIT → SEND. An edited message is an unverified message until the verifier checks it again.

—

STEP 18 — CRM SYNC

Map only what you need — lead_id → external ID, company_name → Company, icp_score → ICP Score, intent_score → Intent Score, why_now → Buying Signal, approval_status → Approval, verification_status → Verification.

Sync rule: PostgreSQL is your system state. The CRM is for sales visibility. Don't let the CRM become the source of truth for workflow logic — it will drift.

—

STEP 19 — SUPPRESSION + THE FINAL SEND GATE

Immediately before sending, re-check suppression — don't rely on a check done earlier in the run:

SELECT 1 FROM suppressions WHERE LOWER(email) = LOWER($1) OR LOWER(domain) = LOWER($2) LIMIT 1;
Also re-check the message thread for a new inbound reply. That prevents an automated send going out after a prospect already replied.
Full send path: APPROVAL → SUPPRESSION CHECK → DUPLICATE CHECK → VALIDATION → RATE LIMIT → SEND
The send node itself should be deliberately boring — no AI reasoning after the approval gate, just hard checks:
const approved = json.approval_status === "APPROVED"; const verified = json.verification_status === "PASS"; const suppressed = https://t.co/cjAmuEBOvU_suppressed === true;

if (!approved) throw new Error("BLOCKED: not approved"); if (!verified) throw new Error("BLOCKED: not verified"); if (suppressed) throw new Error("BLOCKED: suppressed");

return { json };
—

STEP 20 — RATE LIMITS + RETRIES

Configure MAX_MESSAGES_PER_RUN, MAX_MESSAGES_PER_HOUR, MAX_MESSAGES_PER_DAY. Don't rely on an in-memory counter — it resets on restart. Query actual send events instead:
SELECT COUNT(*) FROM lead_events WHERE event_type = 'EMAIL_SENT' AND created_at >= NOW() - INTERVAL '1 hour';
Retry logic — but only for the right failures: timeout / temporary failure → retry (wait 5s, then 15s, then fail to human review) rate limit → wait per the provider's response authentication failure → stop + alert (don't retry bad credentials) invalid request → stop + inspect permission failure → stop + alert

—
STEP 21 — REPLY HANDLING
Webhook receives lead_id, thread_id, from, body → normalize → find lead → classify → route.

Categories: INTERESTED, QUESTION, NOT_NOW, NOT_INTERESTED, REFERRAL, OUT_OF_OFFICE, UNSUBSCRIBE, LEGAL, SECURITY, OTHER.

INTERESTED → sales. QUESTION/LEGAL/SECURITY → human, always. NOT_NOW → nurture. NOT_INTERESTED → close. UNSUBSCRIBE → suppress.

Never let AI autonomously respond to LEGAL, SECURITY, COMPLAINT, THREAT, DATA REQUEST, or PRIVACY categories — route to a human, no exceptions.

The moment a meaningful reply lands: status → REPLIED, cancel or invalidate any pending automated follow-ups. Automation stops the instant a human is in the loop.

—
STEP 22 — AUDIT EVERYTHING
Every important transition writes an event: RESEARCH_COMPLETED, ICP_SCORED, INTENT_DETECTED, BUYER_IDENTIFIED, MESSAGE_GENERATED, MESSAGE_VERIFIED, HUMAN_APPROVED, EMAIL_SENT, REPLY_RECEIVED, REPLY_CLASSIFIED — each with lead_id, agent, input, output, confidence, timestamp, and the n8n execution_id. That last field is what lets you trace lead → event → workflow execution → the exact node that failed.
—

STEP 23 — TEST BEFORE YOU TRUST IT
Run 10 seeded leads through and check actual vs. expected:
01 perfect ICP → PRIORITY 02 wrong industry → STOP 03 missing info → HUMAN 04 duplicate → DUPLICATE 05 no buying signal → NO_SIGNAL 06 unsupported personalization → VERIFIER_FAIL 07 API failure → RETRY → HUMAN 08 interested reply → SALES 09 unsubscribe → SUPPRESS 10 legal/security question → HUMAN
Then chaos-test the failure paths deliberately: malformed JSON, empty API response, timeout, duplicate webhook, missing email/domain, invalid score, provider auth failure, database unavailable, Slack approval timeout. The failure path is part of the product, not an afterthought.
Track an evaluation dataset over time — expected vs. actual, failure_type, severity — and measure more than reply rate: factual accuracy, unsupported-claim rate, classification accuracy, duplicate rate, human correction rate, verification failure rate.
—

STEP 24 — MONITOR + ADD A KILL SWITCH

Track: workflow success/failure rate, parse failure rate, research failure rate, verification failure rate, human approval rate, avg processing time, cost per lead, reply rate, positive reply rate.

Automatic alerts: verification_failure_rate > 20% → pause outbound duplicate_rate > 5% → alert API failure rate > 10% → alert unsubscribe rate spikes → pause campaign

Add one config value, checked at the final send gate: OUTBOUND_ENABLED. If false, block all sends. One switch, one emergency stop.

—

STEP 25 — COST

cost per lead = total AI cost ÷ leads processed cost per opportunity = total AI cost ÷ qualified opportunities (usually the more useful number)

Break it down by stage too — research cost, scoring cost, personalization cost, verification cost — so you know exactly which stage is expensive.

—

STEP 26 — DEPLOYMENT

Build → unit test → 10 leads → failure test → human review → 50 leads → 100 leads → 500 leads → production. If quality drops at any stage, stop scaling before you go further. Use separate dev/staging/production environments and separate credentials for each — never test a new prompt directly against production outbound.

Before flipping OUTBOUND_ENABLED to true, verify: database connection, LLM credential, enrichment credential, CRM credential, email credential, Slack approval, webhook, error workflow, suppression check, and that the send gate actually blocks an unapproved message. Test that last one on purpose — try to send something unapproved and confirm it fails.

—

ONE LEAD, START TO FINISH
Input: Example Inc, [email protected], VP Sales.
Webhook creates lead_id, status NEW → normalization cleans the domain/email → duplicate check clears → enrichment returns 180 employees, B2B SaaS, US → research returns evidence-backed signals at 0.92 confidence → ICP scores 92/100 → routes to PRIORITY → intent detects a verified signal at 88/100 → buyer mapping recommends VP Sales at 0.89 confidence → all of it merges into one context packet → the writer drafts a subject + body referencing only verified facts → the verifier checks every claim and returns PASS at 0.94 → Slack shows the full packet to a human → APPROVE → the send gate checks OUTBOUND_ENABLED, verification, approval, suppression, and rate limit — all pass → email sends → CRM updates → prospect replies "Thanks, I'd like to learn more" → classified INTERESTED → status REPLIED → automated follow-ups stop → sales takes over.
Every one of those transitions is logged with a timestamp, agent, input, output, confidence, and execution ID. That's the full lifecycle, and it's fully reconstructable after the fact.
—
THE BIGGEST LESSON
Don't ask "what's the best AI prompt?"
Ask, for every stage: What decision are we making? What information does it need? What evidence supports it? What should the output be? How do we validate it? What happens when validation fails? What is AI allowed to do — and when must a human step in? How do we measure the result?
Repeat that for every stage and you move from AI chatbot → AI workflow → AI system → AI operating system.
The difference isn't a better prompt. It's the engineering around the model.

[end of thread — full build across 3 parts]

[Praxis @praxis2001]

I built an AI Sales Operating System that finds leads, writes outreach, and follows up — while I sleep.

Most "AI sales automation" posts show:

Lead → GPT → Email

That's not an operating system. That's a demo.

A real system has to:

→ ingest leads → normalize data → remove duplicates → enrich accounts → research companies → collect evidence → score ICP fit → detect buying signals → identify the right buyer → generate personalization → verify every claim → get human approval → update the CRM → send safely → classify replies → stop when necessary → log every decision → recover from failures → monitor performance
Here's the full build. (1/3)
—
STEP 1 — DEFINE THE SYSTEM
The objective: turn a raw lead into a verified, human-approved sales action.
The system should be able to answer: Who is this company? Are they a fit? Why might they care now? Who owns the problem? What evidence supports that? What should we say? Is it factually safe? Should a human approve it? What happened afterward?
Build it as a state machine. Every lead has a state — and the workflow decides the state, never the AI.
Success path: NEW → ENRICHING → RESEARCHING → SCORED → PERSONALIZING → VERIFYING → PENDING_APPROVAL → APPROVED → SENT → REPLIED
Failure states: DUPLICATE, INVALID, RESEARCH_FAILED, VERIFICATION_FAILED, HUMAN_REVIEW, SUPPRESSED, SEND_FAILED
—
STEP 2 — THE ARCHITECTURE
[see image 1]
Lead Source → Normalization → Deduplication → Enrichment → Research → ICP + Intent + Buyer → Context Packet → Personalization → Fact Verifier → Human Approval → Send Gate → Email + CRM → Reply Classifier → Sales / Nurture / Suppress / Human
Build this as 8 separate n8n workflows, not one giant 100-node canvas:
01_lead_intake 02_research_and_scoring 03_personalization_and_verification 04_human_approval 05_send_gate 06_reply_handler 07_error_handler 08_monitoring
—
STEP 3 — THE STACK
n8n → orchestration PostgreSQL → state + audit history LLM → reasoning and structured generation Search/enrichment provider → external company/contact data CRM → system of record Email provider → outbound/inbound Slack → human approval
Credential rule: never put secrets inside code nodes, prompts, database records, Slack messages, or lead JSON. Use n8n's credential store / environment config only. The AI sees business data. It never sees credentials.

—

STEP 4 — THE CORE PRINCIPLE

The AI recommends. Rules validate. Humans authorize sensitive actions.

Never: AI → SEND

Always: AI → VALIDATE → VERIFY → HUMAN → SEND
The final send node needs a hard gate. Conceptually:

if (json.approval_status !== "APPROVED") { throw new Error("SEND_BLOCKED: approval required"); }

A prompt failure can never accidentally become an email.
—

STEP 5 — THE DATABASE
[see image 2]

CREATE EXTENSION IF NOT EXISTS pgcrypto;

CREATE TABLE leads ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), company_name TEXT NOT NULL, domain TEXT, contact_name TEXT, contact_title TEXT, email TEXT, source TEXT, status TEXT DEFAULT 'NEW', icp_score NUMERIC, intent_score NUMERIC, confidence NUMERIC, created_at TIMESTAMP DEFAULT NOW(), updated_at TIMESTAMP DEFAULT NOW() );

CREATE INDEX idx_leads_email ON leads (LOWER(email)); CREATE INDEX idx_leads_domain ON leads (LOWER(domain)); CREATE INDEX idx_leads_status ON leads (status); CREATE INDEX idx_leads_icp_score ON leads (icp_score);

Without indexes, duplicate checks and queue queries get expensive as the table grows.

Five more tables round out the schema:

research — raw structured research + evidence + sources (never store just the summary, you need the evidence later)
messages — subject/body, personalization_reason, verification_status, approval_status, human_edited_body
lead_events — the audit log: lead_id, event_type, agent, input, output, confidence, timestamp
suppressions — email/domain/reason, checked before every send
lead_state_history — previous_state, new_state, reason, timestamp for every transition
Each gets its own indexes on lead_id and the fields you'll filter on (approval_status, verification_status, email, domain).
—
STEP 6 — DEFINE THE ICP BEFORE TOUCHING AI

Don't tell the model "find good leads." Give it measurable criteria as one canonical config object, reused across every agent:

{ "industries": ["B2B SaaS", "FinTech", "Professional Services"], "employee_min": 50, "employee_max": 500, "regions": ["United States", "Canada", "United Kingdom"], "target_roles": ["CEO", "Founder", "VP Sales", "Head of Growth"], "minimum_score": 70 }

Pass this same object into the ICP scorer, the buyer mapper, the research agent, and the personalization agent. Change the ICP once — not four prompts.
—
STEP 7 — THE SCORING MODEL
Industry fit — 20 Company size — 20 Role fit — 20 Problem fit — 20 Geography — 10 Business signal — 10 TOTAL — 100
Each category must return a structured object, never a bare number:
{ "score": 0, "max": 20, "reason": "", "evidence": [] }
Never let the model just return "87." You need to know why it returned 87 — and the total should be calculated deterministically in code, not by the LLM.

Routing (numeric, never natural language like "good lead"):

85–100 → PRIORITY 70–84 → DEEPER RESEARCH 50–69 → NURTURE 0–49 → STOP

That's the foundation: architecture, stack, database, and scoring model.

Part 2 covers lead intake, enrichment, research, the verification loop, and how the writer and the fact-checker are forced to see the exact same evidence. →

[Praxis @praxis2001]

AI Sales Operating System — Part 2/3

Part 1 covered the architecture, database, and scoring model. Now: how a raw lead actually becomes verified, evidence-backed research — and why the writer and the fact-checker must see the exact same data. (2/3)

—

STEP 8 — LEAD INTAKE

n8n Webhook, POST /sales/lead, respond immediately.

Test payload:

{ "company_name": "Example Inc", "website": "https://t.co/ALNMupgMwp", "contact_name": "Jane Doe", "contact_title": "VP Sales", "email": "[email protected]", "source": "manual_test" }
Don't connect expensive enrichment nodes until the raw webhook payload is stable.
—
STEP 9 — NORMALIZE BEFORE ANYTHING ELSE
Domain: strip protocol, strip "www.", lowercase. https://t.co/bU31yw7Cm3 → https://t.co/urMSOrwWa1
Email: trim + lowercase, then validate format. [email protected] → [email protected]
const valid = /^[^\s@]+@[^\s@]+.[^\s@]+$/.test(email);
If invalid → status INVALID → stop. Don't let a malformed lead enter the pipeline.

—

STEP 10 — DUPLICATE CHECK

This is the query that's usually wrong in copy-pasted tutorials — make sure the parameters are properly separated:

SELECT id, status FROM leads WHERE LOWER(email) = LOWER($1) OR LOWER(domain) = LOWER($2) LIMIT 1;
Params: $1 = normalized_email, $2 = normalized_domain.

Match found → DUPLICATE, log event, stop. No match → create the lead with status 'NEW', save the returned UUID, and carry lead_id through every downstream node.
Debugging duplicates that slip through? Check in this order: email normalization, domain normalization, SQL parameter binding, database connection, then the actual duplicate record. Also check case sensitivity, trailing spaces, the "www." prefix, protocol, and subdomain handling — that's where most false negatives hide.

—
STEP 11 — PROVIDER ABSTRACTION (do this before calling any external API)
Every provider — enrichment, CRM, email — gets normalized to your own internal schema immediately after the response comes back. Example, enrichment:
External response: { "company": { "name": "...", "employees": 180, "industry": "B2B SaaS" } }
Converted immediately to: { "company_name": "...", "employee_count": 180, "industry": "B2B SaaS" }

const company = https://t.co/sY4ulovjbI || {}; return { json: { company_name: https://t.co/GOVjMSPq1j ?? null, employee_count: company.employees ?? null, industry: company.industry ?? null } };

If the provider changes its response format, only this adapter changes — nothing downstream. This is the difference between a system you can swap providers on in an afternoon, and one you have to rebuild.

Required enrichment fields before continuing: company_name, domain, industry, company_size, country. Missing any of them → RETRY once → still missing → HUMAN_REVIEW. Never silently continue with incomplete data.

—

STEP 12 — RESEARCH: FACTS, NOT A SALES PITCH
System prompt:
"You are a B2B company research analyst. Your ONLY responsibility is factual research. Do not write sales messages. Do not recommend products. Do not invent information."
Required output:
{ "company_name": "", "industry": "", "business_model": "", "employee_range": "", "core_product": "", "target_customer": "", "growth_signals": [], "hiring_signals": [], "expansion_signals": [], "relevant_business_problem": "", "evidence": [], "sources": [], "confidence": 0.0 }

Hard rules: every material claim requires evidence. If evidence is unavailable, return UNKNOWN. Never turn an assumption into a fact. Never manufacture a buying signal. Never fabricate a source. Never write outreach. JSON only.

Each evidence item is structured, not a loose sentence:
{ "claim": "...", "source": "...", "evidence": "...", "confidence": 0.0 }

That structure is what makes verification possible later. A claim with no evidence cannot be used for personalization — full stop.

Validate the model's JSON before trusting it: is it parseable, are required keys present, are arrays actually arrays, is confidence numeric, does evidence actually contain sources? Malformed → retry once → retry again → fail → human review. Log the raw model response separately from the parsed one; never overwrite the original failure.

—

STEP 13 — SCORE, DETECT INTENT, MAP THE BUYER

[see image 3]
ICP Scorer input: company + contact + research + your ICP config. Output mirrors the scoring model from Part 1, with every sub-score capped (industry_fit ≤ 20, size_fit ≤ 20, etc.) and the total calculated deterministically in code — never by the LLM.

Intent detection looks for: new hiring, leadership change, expansion, new product, new market, tech change, strategic or operational initiatives. A verified announcement outweighs a vague inference. If nothing credible exists, the output is explicitly NO_CURRENT_SIGNAL — never manufacture urgency.
Buyer mapping identifies the role most likely to own the problem, prioritizing: problem ownership → decision influence → budget authority → operational responsibility. The system can infer "VP Sales" from verified org context. It must never invent "Jane Doe is definitely the buyer" unless that identity is actually verified — if not, return ROLE_ONLY.

—
STEP 14 — ONE CONTEXT PACKET, NOT THREE VERSIONS OF THE TRUTH
{ "lead": {}, "company": {}, "contact": {}, "icp": {}, "intent": {}, "buyer": {}, "evidence": [], "unknowns": [], "confidence": 0.0 }
Store it in its own table, keyed by lead_id. Without a controlled packet, the research agent, the writer, and the verifier can each end up working off a different version of "what's true." With one packet, the writer and the verifier are guaranteed to see identical facts — which is the only way verification means anything.
—
STEP 15 — PERSONALIZATION, WITH HARD LIMITS
Structure: observed change → why it may create a relevant problem → specific value → simple next step. Never generic praise ("I was impressed by your amazing company...").
Writer rules: use only facts in the context packet. Never invent familiarity, company activity, pain points, or results. Never make unapproved promises. If confidence is below 0.80, return HUMAN_REVIEW_REQUIRED instead of a message.
Output: { "subject": "", "body": "", "personalization_reason": "", "evidence_used": [], "confidence": 0.0 }

Set explicit limits on subject length, body length, number of claims, number of personalization facts, and CTA count — otherwise the model will happily turn a two-line email into an essay.

—

STEP 16 — THE FACT VERIFIER LOOP

This is the gate that actually earns the word "verified" in your marketing copy.

Verifier checks the generated message against the research evidence for: company facts, person facts, timing, personalization claims, promises, and unsupported assumptions.

{ "decision": "PASS", "errors": [], "required_changes": [], "verified_claims": [], "unsupported_claims": [], "confidence": 0.0 }

PASS only when every material factual claim is supported by supplied evidence. The verifier never infers missing evidence to fill a gap.

Routing: PASS → approval. FAIL → rewrite → re-verify. Cap it at 2 attempts total — after that, HUMAN_REVIEW_REQUIRED. Never build an unlimited AI retry loop; increment verification_attempts on every pass through.
Confidence gates, layered on top (not instead of) hard validation: 0.90+ → normal continuation 0.75–0.89 → continue, but flag <0.75 → human review
Confidence is a signal, not permission. The workflow still enforces the hard checks regardless of what the score says.
That's the reasoning pipeline — research, scoring, intent, buyer mapping, personalization, and verification, all forced through one shared, evidence-backed context.
Part 3: the human approval gate, the send gate that blocks anything unapproved, suppression and rate limiting, reply handling, and how one lead moves start-to-finish through the whole system. →

[Praxis @praxis2001]

Unitree Robotics just delivered one of the most dramatic market debuts in recent Chinese tech history.

The Hangzhou-based company priced its Shanghai STAR Market IPO at roughly $9.1 billion. On the first day of trading, the stock closed 460% above the offer price (having touched as high as 629% intraday), pushing its market value close to $50 billion.
A few facts that make the move notable:

Unitree is the first major pure-play humanoid robot maker to list on a mainland Chinese exchange.
It shipped more than 5,500 humanoid units in 2025 — currently the highest volume in the world.

2025 revenue reached approximately $252 million with net profit of about $41 million, making it one of the few companies in the sector that is already profitable.

Chinese manufacturers accounted for the overwhelming majority of global humanoid shipments in the first half of 2026; Unitree alone held roughly 31% share according to industry data.

The valuation multiple is extreme by any conventional measure — roughly 200 times 2025 sales at the closing price. Investors are clearly underwriting a future in which humanoid robots move from viral demos and research labs into meaningful commercial deployment.

Whether the current price anticipates that future correctly, or simply reflects the scarcity of public pure-play exposure to the theme, will be tested over the next several years.

What is already clear is the signal: capital markets in China are treating humanoid robotics as a strategic growth industry rather than a speculative science project.

Unitree’s debut sets the first real public benchmark for the entire category.

[Praxis @praxis2001]

@StockMKTNewz $9.1B IPO → ~$50B in one day. The market is pricing humanoid scale years ahead of actual

[Praxis @praxis2001]

@elonmusk 6.76% while the older platforms are crawling. The momentum is real.

[Praxis @praxis2001]

@arakharazian The interesting part isn’t the 82 vs 76. It’s that data-retention rules and price are now showing up as growth drag, not just principle.

[Praxis @praxis2001]

@elonmusk @wudijo The highest-status move is still taking credit in public while the actual work stays invisible. Until that flips, team play stays optional.

[Praxis @praxis2001]

@yacineMTB Maybe “writing code” is already the wrong mental model.

The interesting skill now is directing the agent, reviewing what it produces, and catching it when it's confidently doing something stupid.

[Praxis @praxis2001]

@leerob @bot The Bot harness is probably the more interesting signal than the model benchmarks.

If Grok is genuinely outperforming Cursor there, that suggests xAI is starting to compete on the actual coding loop — not just the underlying model.

[Praxis @praxis2001]

@unusual_whales The interesting part is the product-design liability angle.

If engagement mechanics themselves become evidence of harm, this could get much bigger than a Meta case. Every platform optimizing for retention is watching this.

[Praxis @praxis2001]

@KobeissiLetter The wealth effect matters more than the headline number.

+$150k in median market wealth since January can boost spending fast — but it also makes the economy much more sensitive to an equity drawdown.

[Praxis @praxis2001]

@JasonL_Capital The real Nvidia moat isn't just the GPU anymore.

Once the stack is built around CUDA, networking and systems, switching costs get ugly. That's what makes the “toll booth” analogy interesting.

[Praxis @praxis2001]

What happens when the medical device you approved isn't the same medical device six months later?
That sounds obvious.
It isn't.

The FDA has already authorized more than 1,000 AI-enabled medical devices through established premarket pathways. But the traditional regulatory model was built around a relatively stable proposition:

Define the intended use.
Establish safety and effectiveness.
Validate performance.
Authorize the device.
Monitor it.

Generative AI unsettles that model.
A system can accept open-ended inputs, produce variable outputs, perform multiple subtasks and change as its underlying model, prompts, retrieval strategies, guardrails or orchestration change.

Some systems also rely on foundation models developed by third parties, making it harder to separate the behavior of the device from the model underneath it.

The difficult question is no longer simply:
“Does the system work?”
It is:
“Does the evidence still hold when the system changes?”

That is the bigger regulatory problem.
The FDA has already started adapting to it.

Its August 18, 2025 final guidance on Predetermined Change Control Plans provides recommendations for describing planned AI-enabled device modifications and the methodology used to develop, validate and implement them. The goal is to allow iterative improvement while maintaining reasonable assurance of safety and effectiveness.

And FDA is now thinking more broadly about how AI performance should be evaluated as clinical environments change.

Patients change.
Data changes.
Clinical workflows change.
Users change.
And the software can change.

That means a system that performs well during premarket evaluation may not necessarily produce the same evidence of safety and effectiveness months later.

So what should the AI actually be measured against?
A physician?
Another AI system?
A clinical reference standard?
The current standard of care?

For some systems, the relevant comparison may even be the performance of the human-AI team rather than the AI operating in isolation.

That distinction matters because medical AI doesn't operate inside a benchmark.

It operates inside a clinical environment.
Then agentic AI makes the problem even harder.
A conventional model can fail through one incorrect prediction.

An agent can fail through a sequence of individually reasonable actions.

It may retrieve information, interpret it, call a tool, make an intermediate decision, observe the result and continue.

The final failure may not belong to any single step.
It can emerge from the interaction between the steps.
So eventually, the unit of evaluation may have to move beyond the answer itself.

What did the system access?
What did it do?
Where did uncertainty appear?
When should it have stopped?
When should it have escalated?

And did its behavior remain reliable once it entered the real clinical environment?

This is where the larger technology story gets interesting.

The most valuable layer in healthcare AI may not sit inside the model.

It may sit around it.
Infrastructure for managing change, validating performance, monitoring behavior, maintaining auditability and establishing evidence across the device lifecycle.

Because once AI becomes dynamic, validation cannot remain static.

The FDA is still working through what the eventual framework should look like.
But the direction is becoming clearer.

Healthcare AI is moving from a world where you prove that a device works to one where you may have to continuously establish that a changing system still works.

That is a fundamentally different regulatory problem.
And potentially a fundamentally different technology category.

The next healthcare AI moat may not be the model with the best benchmark on day one.

It may be the infrastructure that can continuously establish that the system remains safe, effective and appropriate as the model, data and clinical environment change.

In medical AI, continuous assurance may become as important as intelligence itself

[Praxis @praxis2001]

@Im_IrushiK Revenue and profit are lagging indicators. The timeline is a leading one. Right now they’re pointing in opposite directions.

[Praxis @praxis2001]

@elonmusk If “everyone knows” it was a flagrant injustice, the fact that most people still feel the need to add a long “yes, but…” is itself the interesting data point.

[Praxis @praxis2001]

The bigger problem isn't the $200 price, it's having to think about the limit every time you use the product.

If Codex is good enough to become an all-day workflow, burning the entire allowance in a day turns the product from “my AI” into “a resource I have to ration.”

Usage headroom is becoming a feature.

[Praxis @praxis2001]

@nikitabier This is one of the underrated parts of working with really good algorithm engineers.

You give them “this feels wrong” and they somehow turn it into a measurable objective, find the failure mode, then realize the solution requires inventing half a new field.