GitBook

The docs platform for technical teams and AI agents

[GitBook @GitBookIO]

Syncing your GitBook docs with a repo just got a major upgrade.

Site-wide Git Sync connects your entire site to a repo with a single configuration. And the new gitbook-docs.yaml lets you define your site's structure in code, so navigation changes ship through a PR.

[GitBook @GitBookIO]

Good catch, appreciate the close read. compute_final_score is left over from an early version of the scoring - it was easier to leave it in than to rip it out. The 16-point scorer we actually used is score_session() in harness/manual/record_result.py, which scores each run from the server’s request log after it finishes. The repo was squashed before publishing, but the published scorer is the one the runs were scored with. I’ve updated the README to make that clearer.

[GitBook @GitBookIO]

We asked a simple question about docs: Do AI coding agents actually read them?

Here’s what we found out. https://t.co/PeiBelZa9U

[GitBook @GitBookIO]

@Madeactual Agreed - GitBook generates both of those, but agents don’t always look for them in the right places. I think the models have gotten better at doing this since running this experiment, though.

[GitBook @GitBookIO]

@kibi_dev Makes sense, though I haven't observed that drift (unless the agent is very long-running). Tying requirements to tests gives you something you can actually check, rather than hoping the agent read the right page.

[GitBook @GitBookIO]

@eddyaipt Agreed on both. Putting required fields on the first page an agent is likely to open is probably the easiest fix for most docs teams, but ideally there'd be both accurate docs and helpful error payloads.

[GitBook @GitBookIO]

@Zego67 That matches what we saw - we told the agents where the docs were but not to read them first, and two of the three read the quick-start and figured out the rest from errors. Being explicit does seem to make a real difference here.

[GitBook @GitBookIO]

@maomao000211 I've observed that in other cases for sure. But there were no docs in the repo for this one, so they all had to go to the hosted docs.

[GitBook @GitBookIO]

Yes, that’s the bit I’d take away too. Error messages do deserve the same care as docs. RESTotto looked at a different angle - what happens when the docs themselves are wrong - and the agents trusted them in every run. So when the docs are wrong, a good error message might be the only thing that catches it. I’m thinking of the two as a belt-and-suspenders kind of solution.

[GitBook @GitBookIO]

@AsumaCodes Partly! The short page got read, but it was missing a required field, and that’s where the agents got stuck. Only Codex read the full reference first. Since you can’t count on an agent reading past the first page it lands on, that page needs to have everything it actually needs.

[GitBook @GitBookIO]

@BogdanIsac3434 That was true for two of the three agents - they read the quick-start and stopped. But Codex went straight for the full API reference every time and was the only one to avoid the error. So I think it depends a lot on the model/harness.

[GitBook @GitBookIO]

That’s a fair way to put it. Claude Code and Cursor ran the same model but took different routes through the docs, so some of this comes down to the harness and what it decides to pull in. That puts more weight on docs structure - the answer was in the reference the whole time, but it wasn’t on the page they opened.

[GitBook @GitBookIO]

@7LK3AWpfYg34901 Definitely - that’s why accurate docs matter so much now. In our follow-up experiment (https://t.co/myKWLF2yLT), the agents never doubted the docs, even when they were wrong. Whatever your docs say is defining the agent's reality.

[GitBook @GitBookIO]

Content gaps are the expensive docs defect. Drift is the one you'll never hear about. We tested both on AI coding agents 👇 https://t.co/wmhlJ6rgRm

[GitBook @GitBookIO]

@hailports 💯 - the silent errors are the most dangerous ones.

[GitBook @GitBookIO]

Content gaps are the expensive docs defect. Drift is the one you'll never hear about. We tested both on AI coding agents 👇 https://t.co/wmhlJ6rgRm

[GitBook @GitBookIO]

@eth61675 Fair enough! If you check out our TomatoPy experiment (https://t.co/Kxj1DjrTuc) you'll see a case where agents are able to navigate pretty well on mostly API error messages. Still, you can't always count on good API error messages.

[GitBook @GitBookIO]

We asked a simple question about docs: Do AI coding agents actually read them?

Here’s what we found out. https://t.co/PeiBelZa9U

[GitBook @GitBookIO]

Get ready for next year’s State of Docs report: https://t.co/sJ1Okyxr3r

[GitBook @GitBookIO]

London! We’re back next week for @writethedocs 🇬🇧

Expect pizza, good company and plenty to see. I'll give a behind-the-scenes look at how we run our own docs on GitBook, and we'll talk about State of Docs too.

Grab your spot here: https://t.co/BRgplN7mW9

[GitBook @GitBookIO]

As promised, now that the security incident is resolved, we’ve published our post-mortem.

You can read it here: https://t.co/dZJ3kVNJM5.

Apologies again for this disruption, and thank you everyone for your patience.

[GitBook @GitBookIO]

We can confirm that the security incident that occurred on November 2 is now resolved. We’ve restored functionality across the GitBook platform.

A post-mortem will be shared shortly for insights and learnings here: https://t.co/FuRkSUI5i0

Thank you for your patience.

[GitBook @GitBookIO]

The GitBook app, the GitBook API, and published docs are available again. A few services may still be degraded as we finish restoring everything to full performance.

More information: https://t.co/XB0dpebJUo
Live updates: https://t.co/1HKKPqpkO8

[GitBook @GitBookIO]

We’re addressing a security incident affecting GitBook. Some services are temporarily offline while our team restores them safely.

More information: https://t.co/dZJ3kVNJM5
Live updates: https://t.co/AtDIvmh2uU