Mert Durmazer

AI Simulation Pioneer. We predict how AI & humans behave. Brand visibility (@RankieAI) • Election forecasting •
Synthetic focus groups • Populon AI

[Mert Durmazer @mertdurmazer]

I forecast Hungary's election within 0.7 points. Three days before the vote, I locked it to Wayback. No pollster was closer on the lead gap.

The method wasn't a better poll. It was no poll at all.

Why that matters for anyone running market research ↓

[Mert Durmazer @mertdurmazer]

@emollick wonder if the best use of a fast model is checking the question before you send it to the slow one

[Mert Durmazer @mertdurmazer]

Not saying it's fake. Saying the picture backs the traffic part and nothing else.

[Mert Durmazer @mertdurmazer]

One post says Grok Bot built an autonomous business that made $1,800 since launch.
The dashboard screenshot shows 90,199 visitors, 206,913 page views and a 66% bounce rate.
The $1,800 and the 3,000 newsletter subscribers are not in the image.
Traffic proves people showed up. It doesn't prove the problem was worth paying for.
I'd want the Stripe screen next to the analytics screen before copying the six steps.

[Mert Durmazer @mertdurmazer]

I haven't tried Grok Bot for this. Everything above is the post's own claims plus one screenshot, not something I checked.

[Mert Durmazer @mertdurmazer]

Claude Code (Opus 5.5) built a working version of Dot in 11 steps and 304.9 minutes. 87/87 tests pass.

People are talking about Claude iterating on its own work. This is what that looked like on one real build.

An independent audit of the result found 26 issues, and all 26 were fixed.

The honest score after the audit: 2 features fully ready, 6 partial.

So it is closer to a real product than a demo. It is not finished.

The API-equivalent value of the run was $93.46.

[Mert Durmazer @mertdurmazer]

how it works: every page gets 7 typed questions. jev answers each with a choice + a probability, median 257 ms per call.

238 of the 806 answers had a top-choice probability under 60%. those need a human, not an auto-fix.

the video is a replay of the recorded calls at real speed. model: typesafe/jev-1.13 via openrouter.

[Mert Durmazer @mertdurmazer]

jev is INSANE.

in 27 seconds it went over the title, H1, meta description and first ~1,400 characters of 50 pages across my 3 websites and made 806 decisions. every page: does it deliver what its title promises, what's the weakest element, is the claim backed, could an AI quote it.

2.2 cents.

but before jev even started, the crawl found something embarrassing:

15 of the 16 pages in my GEO company's sitemap return a 404 when you request them directly.

the site that sells AI visibility serves AI crawlers a 404.

now i know what to fix first.

[Mert Durmazer @mertdurmazer]

@GergelyOrosz i think what hurt jetbrains is the loop moved to the terminal, where an IDE has no home advantage. the editor stopped being the center of the workflow and that's hard to see from inside.

[Mert Durmazer @mertdurmazer]

@GeoffreyHuntley curious what trips it for you, file writes or shell commands? that decides whether it's a setting problem or a model problem

[Mert Durmazer @mertdurmazer]

@svpino the validated part is what makes it usable. an unvalidated proposal is just more review work for you, one that already passed a check is something you can actually approve.

[Mert Durmazer @mertdurmazer]

@DamiDefi the "approved edits" part is what makes this usable. in-place edits you can reject beat copy-pasting out of a chat window every time

[Mert Durmazer @mertdurmazer]

@octoberry i think context is the moat too. but the more a personal agent knows, the more the approval step matters. openai's dots at least ask before any purchase

[Mert Durmazer @mertdurmazer]

@pukerrainbrow the sampler pack is the good detail. nobody writes "let me try flavors without a full jar" in a survey, but they say it in a support chat all day.

[Mert Durmazer @mertdurmazer]

@XFreeze per-task routing is where the money is. my 30 days of claude code at API list price was $12,926, and the same tokens on sonnet 5.5 would be $5,608

[Mert Durmazer @mertdurmazer]

@0xKeng @vangrid_io results you can check beat capability claims. i put a Ready / Partial / Not yet table in the readme of my Dot rebuild, and an outside audit still scored it 2 full, 6 partial. less fun, more believable

[Mert Durmazer @mertdurmazer]

@AIFrontliner the approve-each-change step is the part i'd watch. an assistant editing in place in a shared doc is only usable if you can see exactly what it touched before it lands

[Mert Durmazer @mertdurmazer]

@kimmonismus i ran my own dot build on jobs adapted from the official demos and it lost 0-3 to grok bot (1 tie). it was faster on 3 of 4 though 😅

[Mert Durmazer @mertdurmazer]

Agreed. One fix is to make the model justify every concession: for each reviewer comment, it has to say whether it's accepting or rebutting, and name the new evidence or argument that changed its mind. If it can't, the default is to defend. That separates updating from simply folding under pushback.

[Mert Durmazer @mertdurmazer]

The hard part of an agent-native Descript is probably less the transcript tech and more the edit layer. Agents need tools like cut by transcript range, plus a diff and undo, so a bad edit is reviewable instead of silently rewriting the timeline. Expose that and the choice of agent really does become the user's.

[Mert Durmazer @mertdurmazer]

The "works for one person, never for the team" gap is the part I'd want to hear more about. On approvals, OpenAI's Dots is a useful example: per their announcement it asks for approval before a purchase. So the human check sits at the costly step, not at every step. Curious whether Zeta's approval points follow a similar logic.