Lessons I learned this weekend in the Muse Spark kernels challenge as a GPU noob:
1. Speeding up one part slowed the whole thing. The compiler reshuffled the rest.
2. Starting the next job early finished it later. It stole cores from the current job.
3. Merging 12 kernels into one was slower. Their coordination got stuck in traffic.
4. A wait that never waits still costs. It blocked the compiler from optimizing.
5. Match your benchmark to production. I benchmark AI for a living, and this still got me, duh.
Lessons I learned this weekend in the Muse Spark kernels challenge as a GPU noob:
1. Speeding up one part slowed the whole thing. The compiler reshuffled the rest.
2. Starting the next job early finished it later. It stole cores from the current job.
3. Merging 12 kernels into one was slower. Their coordination got stuck in traffic.
4. A wait that never waits still costs. It blocked the compiler from optimizing.
5. Match your benchmark to production. I benchmark AI for a living, and this still got me, duh.
For those enjoying your Saturday coffee while hacking on the Muse open kernel challenge, @Karmedge built a fun little communtiy for your agents: https://t.co/Pz2J3yPIBX
Security called me at 6:12 AM today.
"Patch every PHP server immediately."
I asked what happened.
"Two critical vulnerabilities. Remote code execution. No authentication required."
Cool.
How many PHP servers do we have?
According to inventory: 12
According to reality:
Nobody knows.
We found one running WordPress.
Three running old internal tools.
One under someone's desk.
Two inside vendor appliances.
A forgotten employee directory from 2016.
And a server named:
PHP-OLD-DO-NOT-TOUCH
Naturally, that one runs payroll exports.
Then we found a PHP 5.6 box.
I asked who owned it.
The documentation said:
BRIAN.
Brian left in 2019.
We called Brian.
He said:
"Oh wow, that's still running?"
Excellent.
By 8 AM, leadership joined the incident call.
CEO:
"Can hackers actually get in?"
Security:
"Potentially yes."
CEO:
"Without a password?"
Security:
"Yes."
CEO:
"Then why do WE need passwords?"
Not the lesson.
Finance asked if we could wait until the weekend to patch because downtime affects revenue.
Security explained that the alternative was letting strangers execute code on our servers.
Finance asked for a cost comparison.
At 9:17, someone suggested temporarily taking the vulnerable websites offline.
Marketing refused.
"Our campaign is live."
So for the next hour, the company officially decided maintaining a landing page for 20% off webinar registration was more important than preventing remote server takeover.
We finally patched everything we could find.
Then our scanner discovered another PHP server.
Hostname:
TEMP-PHP-2014
Uptime:
4,512 days.
Nobody knows what it does.
Nobody owns it.
Nobody can shut it down.
But every time we block network access, the warehouse stops printing shipping labels.
Security has classified it as critical infrastructure.
It is running PHP 5.4.
The vulnerability has existed longer than half our employees.
The emergency patch took four hours.
Finding out what PHP actually runs took eleven.
The internet isn't held together with code.
It's held together with servers everyone is afraid to turn off.
So it turns out that behind the Dots vs Muse battle on the frontend is an interesting infrastructure fight: @Cloudflare vs @vercel (with help from @awscloud) #vercelship
Dots vs Muse experiment
After our Dots experiment, people asked us to run the same prompt on Muse: Build a web app that collects data, has a sign-in feature, and is publicly accessible.
Dots deployed on ChatGPT Sites (powered by Cloudflare Workers, D1, ChatGPT sign-in).
Muse successfully wrote the app (but used 4 different frameworks across 8 runs). But it struggled to serve them publicly - trying to tunnel using ngrok, Cloudflare Tunnel, localtunnel...
@natfriedman is this by design? Should a user expect Muse to be able serve an app, or only to run apps privately within its sandbox.
https://t.co/3AnbGtqFSw
5. Rather than asking โhow do we add AI to this?โ ask โwhat is the agent-first version of this?โ Doing this forces you to rethink the jobs to be done for an agent-first future and helps you avoid being disrupted by an AI-native competitor.
Interesting study, but a note on the scope: every prompt is domain-anchored. ex: โwhat subscription options are available at {domain} and what they cost?โ
Todayโs agents of course fetch sites when you prime them that way, so the 93% is not surprising. But in our data (across millions of journeys): if you ask the same questions without {domain}, training data weighs much more heavily.
We ran 38,000 journeys with Tavily on 1,056 real business sites, across top agents like Claude Code, OpenClaw and ChatGPT.
The biggest finding: agents barely use their training knowledge anymore when they recommend services. 93% of the claims in their answers are grounded in what they look up live, on your site and through web search.
The research also found:
โขโ โ Businesses agents can read get recommended 2.6x more often
โขโ โ When a site blocks agents, they don't give up. They answer from other sources, and web search's share of the answer doubles, from 12% to 25%
โขโ โ Those answers are 3.7x more likely to miss every fact the buyer asked for
This changes where marketing and product teams should focus. What moves the outcome now is showing up in the web search agents use, and making your own site readable for agents.
To see how you can improve your rank in agents search, Tavily built the Agent Visibility Report: https://t.co/8n0fROEeD8
Full report: https://t.co/0BvzbDfs5w
Dots vs Muse experiment
After our Dots experiment, people asked us to run the same prompt on Muse: Build a web app that collects data, has a sign-in feature, and is publicly accessible.
Dots deployed on ChatGPT Sites (powered by Cloudflare Workers, D1, ChatGPT sign-in).
Muse successfully wrote the app (but used 4 different frameworks across 8 runs). But it struggled to serve them publicly - trying to tunnel using ngrok, Cloudflare Tunnel, localtunnel...
@natfriedman is this by design? Should a user expect Muse to be able serve an app, or only to run apps privately within its sandbox.
https://t.co/3AnbGtqFSw
3/7
Hosting: @Cloudflare Workers, through ChatGPT Sites, every time.
No @vercel. No @Railway. No new hosting account. Dots built Vinext (the "Next.js on Vite" alt) apps that were delivered via Cloudflare.
A SQLite file for every agent, and a SQLite file for each thing the agent builds.
(Like Cloudflare's D1 for everything OpenAI's Dots build https://t.co/LDlCrr0YNA)
Turso is joining @supabase.
Together, we're building the database platform for the agentic era: everything agents need, from lightweight databases to Postgres at petabyte scale.
More details in thread. ๐
7/7
Most of the vendors that didnโt show up are already in ChatGPT's plugin directory. That did not matter. If the plugins were not connected, Dots used its preferred platform.
Details: https://t.co/HDAVfU8W6D
1/7
We asked OpenAIโs Dots to build an AI status dashboard: one place to check @claudedevs, @cursor_ai, @OpenRouter, @GeminiApp, @MistralAI, @cohere API health.
Twelve providers. Public page. Saved incident history. Private watchlists. Refresh every 15 minutes.
We didnโt specify the stacks. We repeated this experiment 8x.
6/7
Scraping + scheduling: Hand-rolled HTTP, plus custom JSON, RSS, and HTML parsers.
No @firewcrawl, @browserbase, @tavilyai.
Some builds had no active schedule at delivery. When a schedule existed, it was a ChatGPT task, not a Cloudflare cron.