Ed Shrager

Tweets about AI

[Ed Shrager @edshrag]

1/
The personal agent, that’s the big thing - Bill Gates

Introducing LLynx🐈, a building block to help enable action-oriented AI Agents. It’s fast, accurate & small (3B params)

🙏Built on resources from @langchain @hwchase17 @huggingface @_philschmid @chipro @lmoroney

Demo👇

[Ed Shrager @edshrag]

@boborado Remember what?

[Ed Shrager @edshrag]

@boborado Do you have any that aren't sold?

[Ed Shrager @edshrag]

@boborado Do you have more B300s?

[Arsh Shah Dilbagi @arshdilbagi]

The hard part about LLM failures is that their outputs rarely look like failures.

The demo “works.”
The output sounds coherent.
The user actively uses the product.
And your dashboard looks normal.

Meanwhile, the system can be wrong, unsafe, or quietly driving up token spend. And you won’t notice until the damage adds up.

Prompts often serve as business logic (policies, safety, and product context). But many teams ship them without the basics, such as versioning, reviewable changes, end-to-end traces, and eval gates.

In production, it doesn’t crash. It degrades via wrong answers, policy misses, and surprise spending.
No crash. No error. No alert.

I cover this exact issue in my @Stanford CS 224G guest lecture on AI Observability and Evaluations.

Here are the core ideas:
• If you only log the final output, you’re guessing. Full traces show where it broke.
• Evals are feedback loops. Use clear pass/fail criteria tied to outcomes.
• Run evals continuously on production traces and don’t wait for support tickets.

The moat isn’t prompt cleverness. It’s a measured improvement.

Full lecture + blog below 👇

[Tuhin Srivastava @tuhinone]

Baseten’s day 0 bet was that inference was the technology that would enable the best user experiences AI could deliver–fast, smart, reliable, secure. And that those experiences would rely not only on a handful of giant general intelligence models, but millions of specialized models built by companies for their specific customers and use cases.

Whether you’re a doctor, developer, lawyer, mechanic, researcher, construction worker, marketer, etc, you’re accelerated by specialized tools worthy of your craft. To me, this is one of the most meaningful promises AI can deliver on.

We’re starting to see it now. Many of the main-character AI companies on the application layer are built on highly-specialized models for highly-specialized workflows–Abridge, Clay, Cursor, OpenEvidence, Hebbia, Mercor, Notion–these businesses are booming because customers love specialized tools.

There are probably hundreds of custom models in production today. Soon, there will be thousands and then millions. All enabled by a high-performing inference layer.

Inference has emerged as one of the hardest problems in modern AI systems. Delivering reliable, low-latency experiences requires deep coordination across distributed infrastructure, kernel-level performance, and software ergonomics—even world-class teams struggle to do this well. As a result, as consumers and developers, we’ve grown to accept sluggish performance, frequent downtime, and inconsistent quality across both application companies and model providers.

Meanwhile, the demands on inference are accelerating: AI adoption is trending towards ubiquity with reasoning models that are orders of magnitude more compute-intensive. This will only increase as more companies catch on to the virtues of owning their end-to-end IP rather than relying on black-box model APIs on shared infrastructure. Whether we can realize the impact of this generational shift will depend on our ability to serve these models reliably at scale.

We knew we could make the technology work, but the biggest delight of it all has been seeing what our customers do with it. The (many-model) future is bright.

[Ed Shrager @edshrag]

@nrudder_ @get_sphere @a16z Congratulations, huge news. Fully deserved mate

[Ed Shrager @edshrag]

@theworldlabs Only the best teams can create products this good. Congrats to the entire @theworldlabs team for this breakthrough.

[Ed Shrager @edshrag]

LOL i love this. just amazing!!!!!

[Origin Financial @useorigin]

Bad financial advice has existed for ages. Origin’s AI Advisor changes that.

[okWOW @okwowAI]

For decades, technology was inching closer to being more personal, more intuitive, more human.

80s: point and click instead of cryptic commands
90s: the web remembered our preferences.
2000s: touchscreens let us touch our ideas.
2010s: devices predicted our needs...

🧵 1/7

[sarah guo @saranormous]

excited to be doubling down with team @baseten and welcoming friend and world-class investor and operator @jaysimons and the team @bondcap! more from @tuhinone on the road ahead, here: https://t.co/shhYWYOz9W

[sarah guo @saranormous]

https://t.co/SRuCfa2CiQ

[Tuhin Srivastava @tuhinone]

Today, we’re excited to announce our $150M Series D, led by BOND, with Jay Simons joining our Board. We’re also thrilled to welcome Conviction and CapitalG to the round, alongside support from 01 Advisors, IVP, Spark Capital, Greylock Partners, Scribble Ventures, and Premji Invest.

The last eighteen months have been a whirlwind; as the AI application layer has taken off, we've been proud to play a small part supporting world class companies run their production workloads. Thanks to all our customers including Abridge, Bland, Clay, Gamma, Mirage, OpenEvidence, Sourcegraph, WRITER, and Zed Industries.

We’re just getting started. If you’re building the next generation of AI products, we’d love to work with you.

[Dan Noskin @dnosk]

Captions is now Mirage: a new name, a fresh identity, and limitless stories ahead

[NVIDIA AI @NVIDIAAI]

📈 @baseten users are scaling smarter with us:
✅ 5× throughput on high-traffic endpoints
✅ 50% lower cost per token
✅ Up to 38% lower latency on the largest LLMs

Built on NVIDIA Blackwell + TensorRT-LLM + Dynamo on @googlecloud—driving efficiency, speed & adoption at scale.

Learn More: https://t.co/OX8dUL2GZN

[Ed Shrager @edshrag]

Partners are the key to success ❤️.

As AI adoption grows, the magic created by this technology deserves a rock solid infrastructure layer to support it.

At Baseten we're building the ecosystem to support the most reliable, fastest inference infrastructure in the world. Join us

[Baseten @baseten]

We're excited to announce Fall into Inference: a multi-month deep dive into our cloud ecosystem and how we use Multi-cloud Capacity Management (MCM) to power fast, reliable inference at scale.

Over the next few months, we’ll showcase how we use MCM to power real-world AI use cases with partners including Google Cloud, Amazon Web Services (AWS), OCI, CoreWeave, Nebius, Vultr, and NVIDIA.

Stay tuned for weekly technical blogs, case studies, and deep dives with our partners!

[Baseten @baseten]

Welcome to Baseten @DannieHerz!

We’re thrilled to announce that Dannie Herzberg has joined as our new President to lead Baseten’s GTM and operations.

As @tuhinone shared: "Dannie is biased towards action, dependable, and long-term in her thinking, and she knows that the customer experience is everything."

Here’s to building the next chapter of Baseten with you, Dannie!

Read more from Tuhin about Dannie here → https://t.co/lrBr9R8nS3

[dax @thdxr]

baseten was the worst perfoming on gosucoders qwen3 benchmarks last week

but they called me immediately to figure out how to fix they've been working the last 3 days

and now it's fixed - it's up there with the best options

and it's fixed for everyone, not just opencode users

[Ed Shrager @edshrag]

@thdxr 💪💪💪 it's how you react + the result that counts 🚀

[Ed Shrager @edshrag]

@amiruci fast, or last 🚀