James Smith

Product @ Cloudflare, CEO Human Native, ex-Google/DeepMind, Dad, Product Management practitioner

[James Smith @jws7]

Agents Week at Cloudflare starts today. We're shipping the infrastructure for what comes after the browser -- the tools, protocols, and payment rails that let AI agents interact with the web as first-class citizens. Not just crawling it. Transacting on it. https://t.co/871HxBQbhE

[kathyl @kathyyliao]

we're not joking when we say our /crawl does not bypass Cloudflare's bot detection or Captchas

/crawl identifies as a bot + respects robots.txt, which is that standard used by websites since (checks notes) 1994 to indicate to crawlers/bots which parts of a website they are allowed to visit.

see the blocking in action below

[ᴅᴀɴɪᴇʟ ᴍɪᴇssʟᴇʀ 🛡️ @DanielMiessler]

People are completely missing the point of this feature. Most accidentally and probably some on purpose.

They stated very clearly that this obeys all blocking and content rules for your site. It’s literally the opposite of bypassing those controls.

The purpose of the system is to get rid of all the halfassed AI crawlers all over the Internet that are doing a crappy job of pulling your content.

They are loud, rude, and wasteful. And they’re filling the internet and all our logs with massive amounts background noise.

Cloudflare knows they are going to crawl no matter what.

This feature is simply giving them a legit way to do it efficiently so that they don’t clog up the entire Internet doing it in a way that it’s 1000x less efficient.

And it’s still uses all your rules for what can and cannot be crawled. And all of your other Cloudflare controls around crawling are still enforced.

If this is adopted to any significant degree, Cloudflare will be absolute heroes for reducing terabytes of crawler-slop background noise.

[James Smith @jws7]

@richterich_ @harshil1712 @craigsdennis @CloudflareDev Website owners decide the rules. Cloudflare just respects their wishes.

[Harshil @harshil1712]

Saw a lot of misinformation around the new /crawl endpoint @CloudflareDev announced yesterday.

Go try scraping https://t.co/dvC9KZiY6k with the /crawl endpoint. We said we are going to protect your site, and we stand by it 🧡

[James Smith @jws7]

The web was built for humans reading HTML. AI agents are drowning in error pages meant for browsers. @Cloudflare just shipped RFC 9457-compliant structured error responses: Markdown and JSON instead of heavyweight HTML. 98% fewer tokens wasted.

https://t.co/J4glEMhvs3

[James Smith @jws7]

At @Cloudflare we will continue to build for AI Developers and Content Creators. We need to get the future economy of the internet right, and that means building ways to help AI *responsibly* ingest content in accordance with site owner's wishes!

[Cloudflare Developers @CloudflareDev]

We want to clarify something: one of the motivations for building this feature is that building a crawler that respects crawling best practices is hard. Too many crawlers use this as an excuse not to abide by site owners' preferences, and respect directives like robots.txt.

/crawl does this out of the box.

Additionally, the /crawl endpoint cannot bypass Cloudflare's bot detection or Captchas, and self-identifies as a bot. It respects all the same protections available to operators today: AI Crawl Control, robots.txt, Content Signals and Pay-Per-Crawl.

So, site owners can and should still choose how their content is consumed, and our /crawl endpoint will respect it.

[James Smith @jws7]

If AI agents consume 50% of web traffic but pay 0% of the cost to create the content they consume, who keeps making content? This is the question the entire internet needs to answer in 2026.

[James Smith @jws7]

The 2026 #F1 regulations are the biggest shake-up ever.

New active aero, new power units, completely rewritten rules.

If you're not watching the season opener this weekend, you're missing the best engineering competition on the planet.

https://t.co/RbwimBJLHd

[James Smith @jws7]

@Cloudflare Excited to be onboard! Can't wait to work with @eastdakota @dok2001 @sam @LakeAustinBlvd and the rest of the team!

[Cloudflare @Cloudflare]

Cloudflare acquires Human Native, an AI data marketplace specializing in transforming content into searchable and useful data, to accelerate work building new economic models for the Internet. https://t.co/yvwoA6lwbK

[James Smith @jws7]

Human Native 🤝Cloudflare

We are thrilled to announce that @humannativeai has been acquired by @Cloudflare !

Check out the full story: https://t.co/8UtVv4VLED

[James Smith @jws7]

@OfficialFPL Trying to setup my team and it's asking me to update my password. When I type in a new one it gives an error: "New password cannot be the same as your current password" - this is a new randomly generated password it's definitely not one I've used before. Help please!

[Allie K. Miller @alliekmiller]

Google just dropped a 100% free learning path on Generative AI with 9 Courses 👇

Intro to Gen AI
Intro to LLMs
Intro to Responsible AI
Intro to Image Generation
Encoder-Decoder
Attention Mechanism
Transformers and BERT
Image Captioning
Gen AI Studio

https://t.co/cyojw6P4Tm

[James Smith @jws7]

@GergelyOrosz You should point people to @infracost - great tool developed by @AliKhajeh and others to solve exactly this problem!

[Packy McCormick @packyM]

ok this is wild

[Thomas H. Ptacek @tqbf]

I’m sorry, I simply cannot be cynical about a technology that can accomplish this.

[Raphaël Millière @ ICML 2026 @raphaelmilliere]

I asked ChatGPT to rewrite Bohemian Rhapsody to be about the life of a postdoc, and the output was flawless:

[David Perell @david_perell]

28 short pieces of life advice:

1. Block off 90 minutes in your calendar every morning to work on the most important thing. Wake up early if you need to. Don’t compromise.

[Dave Burke @davey_burke]

That time when we showed the Queen how to get driving directions to Windsor Castle on an early version of Google Maps for phones. The Queen was super curious and engaging. My first & last royal focus group! Circa 2008.