The trick that I think makes AI writing feel human
After every scoring round, I ask the model one question:
"What would an experienced engineer disagree with here?"
The counter arguments get fed into the next rewrite. That's where the tradeoffs and failure cases come from.
Design question I'm stuck on:
How do you stop an AI revision loop from running forever?
My answer: stop when ANY of these hits:
• score ≥ 90
• improvement < 2% (plateau)
• 5 iterations (cost ceiling)
Is there a rule I'm missing?
The weights were the hardest decision. Depth has to matter more than keywords.
Approval rule:
score ≥ 90 AND every claim has an inline source.
If a claim has no source, the article fails, no matter how well it reads.
Today I'm deciding who gets to judge the article.
Six reviewers, each scoring a different thing:
Technical depth: 30%
Persona/voice match: 25%
Research coverage: 20%
Citation audit: 10%
SEO: 10%
Reader simulation: 5%
Working name: BlogTwin.
Not a prompt wrapper. Not a text generator.
A pipeline that researches → drafts → gets scored → revises → and only ships if it clears 90/100.
Building it in public. Follow along if you want to see where it breaks.
The mistake isn't the model.
It's single-pass generation: prompt → text → publish
No human writes like that. We draft, get torn apart, and rewrite.
So I'm giving my AI an editor, a skeptic, and an SEO nerd. And making it loop.
@cneuralnetwork I've worked on these. Can help with details.
1. Permission-aware RAG: retrieval that respects access
2. Ticket → resolution workflow: state, branching, approval
3. Document extraction: LLM extracts, code validates
4. Data onboarding: validate, flag errors, user fixes exceptions
Every AI-written engineering post I read has the same problem.
It sounds right. It's shallow, it has no tradeoffs, and it has no scars.
So I'm building something that doesn't get to publish until it survives 6 critics.
Starting this weekend. 🧵
Instead of writing code, you edit the model itself.
Fine-tune a model, subtract the base weights, and you get a "task vector." Add it to another model to transfer a skill. Subtract it to remove a behavior. Merge, patch, and swap LoRAs like libraries.
Bottom Line: your agent makes the decisions. Your environment is the real world teavhing it what those decision mean.
If that world is broken, no amount of algorithm tuning saves you.
What's the metric you wish you'd started tracking sooner?
I'm curious.
Your RL agent's reward curve is going up !!
Genuine Question: how do you know the agent got smarter, and not the environment quietly getting easier, slower, or just wrong ?
Most people never check. Here's why that's a problem.
The real discipline: version everything that can change your result (sim code, physics, reward fn, policy, deps). Repeat every benchmark run. Track fallback usage separately from real agent performance.
A number you can't reproduce isn't a number.