Alex Vikati

Benchmarking your favorite LLM

[@edwin 🌵📈 @edwin]

1/8

Mythos / Glasswing is clearly the main AI security story now: AI finding real vulnerabilities in existing production code.

For most teams, though, this question is more immediate:
Can an agent like Claude Code write a secure app in the first place?

[Alex Vikati @vikati]

The Claude Code leak shows a clear divide: select vendors gain small but compounding advantages.
Everyone else gets generic UX + install friction.

Direct impact on roadmaps and GTM.

[@edwin 🌵📈 @edwin]

1/9

The most interesting thing about the Claude Code leak for devtool companies:

Anthropic hardcoded 120+ vendor names across 7 different systems in the source. Anthropic explicitly included your tool name in the code (or they didn’t 🤷🏻‍♂️)

Thread 👇

[@edwin 🌵📈 @edwin]

After OpenAI/Astral acquisition announcement, we ran a benchmark on their tools.

Turns out Astral tools were already a core part of the Codex (and Claude Code) workflow for Python developers. Ruff + uv came out on top in 75% of cases for linting, packaging, and pretty much everything else.

Full report: https://t.co/633EpaIfTD

[@edwin 🌵📈 @edwin]

1) Winners & losers from our OpenAI Codex vs Claude Code Picks benchmarks

[Alex Vikati @vikati]

Past category winners relied heavily on marketing and sales, and GTM still matters to get in front of coding agents.

But agents won't keep using you unless you actually make them better (since the agents are trained with their harness).

That shifts long-term value toward real product quality, not just distribution.

[swyx @swyx]

every category leader in this list should be worth at least $5b btw because koding agents will be recommending them for the next 5 years + infra is stickier than agents

(full disclosure am smol resend angel)

[Alex Vikati @vikati]

The strongest argument against Taalas: AI models change every 3-6 months. Llama 3 → 3.1 → 3.2 → 4 in under a year. Each Taalas chip is locked to one model forever.

You're buying dedicated hardware for a moving target. The moment a better model ships, your chip is legacy silicon. Counter-argument: 2-month fab turnaround Taalas + 20x lower production cost means the refresh economics might actually work. If a model-specific chip generates 10x the revenue per watt during its window, the disposable hardware model could be net positive.

[@edwin 🌵📈 @edwin]

My friend @gentschev started using @Railway , @sentry and @resend for his https://t.co/O0eThSr3iD side project because of Claude Code's recommendations. The impact is real.

[Alex Vikati @vikati]

Coding agents like Claude are a massive new distribution channel for infra providers.

@Railway (~$124M raised) crushes peers like @render (~$258M raised) and @flydotio (~$115M raised) in Claude Code preferences.

Will agent recommendations drive outsized growth?

[@edwin 🌵📈 @edwin]

3/9 Deployment (Python)

Winner: @Railway – 82% picks. The dark horse: none of Vercel's hype machine, but quietly eating away tons of non-Next.js hosting from everyone (AWS, GCP, Heroku, Render, Fly, etc). When IPO?

Simplicity wins. @JustJake Railway is stealthily dominating.

[Alex Vikati @vikati]

Winner and losers from our Claude Code Picks benchmarks!

[@edwin 🌵📈 @edwin]

1/9 Winners & losers from our Claude Code Picks benchmarks that measure what Sonnet 4.5, Opus 4.5, and Opus 4.6 default to when building apps.

[Alex Vikati @vikati]

@zeeg @edwin @github @stripe @sentry @resend @zenorocha you have to send the dogs to @bugsnagstatus @rollbar https://t.co/cG9TMrqUoG

[@edwin 🌵📈 @edwin]

@vikati and I analyzed 2,430 Claude Code repo decisions.

Claude Code never picked AWS or GCP for deployment.

If agents are writing the first version of new projects, they’re influencing which tools get adopted at scale.

[@edwin 🌵📈 @edwin]

1/ AI is changing how brands appear in search. Try searching "best acoustic guitar under $500" in Google, ChatGPT, and Perplexity. You'll get three completely different sets of recommendations citing different sources

[@edwin 🌵📈 @edwin]

Speaking of AI bias & @AOC's comments, we did a bunch of tests and @azure, @awscloud, @clarifai & @IBMWatson all think @maddow is a man https://t.co/puBMmU7uT3

[Alex Vikati @vikati]

We just released a new Tether dataset that covers every transaction up to block 546906. Go to https://t.co/r6pRmCy4Bi if you are interested in doing primary Tether data analysis.

[Alex Vikati @vikati]

I just published Tether vs Other Stablecoins https://t.co/1n9m0Nv2uV

[Alex Vikati @vikati]

@MooreJared0 @CasPiancey I believe StableUSD is still pre-launch

[Alex Vikati @vikati]

I just published Tether vs Other Stablecoins https://t.co/1n9m0Nv2uV

[Alex Vikati @vikati]

@m2jr Interested!

[Alex Vikati @vikati]

@edwin and I are now measuring and ranking Ethereum smart contracts based on those metrics and are working to provide for decentralized apps the kind of usage data currently available for web/mobile apps. Try our beta at https://t.co/rEsLkE7pT9

[Alex Vikati @vikati]

I just published https://t.co/LmNePPO2Mp - a data-driven post on the actual usage of Ethereum smart contracts. In the long run, it is users, traffic, and revenues that will drive real value for decentralized networks.

[Alex Vikati @vikati]

@jer979 Happy to take a look. Please DM. Thanks!