Why hasn't AI automated more work? What's holding coding agents back? And are the labs' valuations built on the right expectations?
@CompleteSkeptic, co-founder and CEO of @typesafeai, was the perfect person to discuss this and more. Diogo coauthored OpenAI's InstructGPT paper, and a few weeks before we recorded, TypeSafe launched Jev, a "system one" model that returns structured decisions instead of text.
It was a ton of fun to discuss this with Diogo.
The conversation includes:
0:00 Intro
1:14 The Early Bet on RLCD
4:40 Built for Humans vs Computers
6:39 Roadmap & Model Limitations
11:04 Mental Models of Models
16:07 Do LLMs Have a Place in 10 Years?
17:32 New Model Shapes
21:24 Problems Still Unsolved
27:00 The Tyranny of the KV Cache
31:21 Rethinking Coding Agents
34:40 Multimodality
37:52 Images, Games & Computer Use
39:36 Could This Have Been Built 5 Years Ago?
42:01 Benchmarks & Copycat Releases
44:12 Fundraising
45:27 Quickfire
Listen here:
YouTube: https://t.co/K8kp8aFv3T
Spotify: https://t.co/GImUYej9It
Apple: https://t.co/KcuUJNkT3Q
When I’m on call, most of my work is a chart biopsy. What if a fellow was available around the clock to review the chart, present the patient, and help coordinate care?
At The Conversation last month, we showed what that looks like: the @AbridgeHQ Assistant turning my goals into a plan, working across systems, and coming back when it needs my judgment.
For me, the purpose is presence. Full attention on the patient, and confidence that the plan we make together gets carried through.
We’ve talked about the Learning Health System for twenty years. We’re building it now with our partners.
Full demo: https://t.co/pDSTc0QTkh
Congratulations to @goodwin_ml and the entire @fractile_ai team on raising a $720M Series C! We’re proud to co-lead the round alongside Thrive Capital, Founders Fund, and Lightspeed.
As agents take on longer tasks and reasoning models think longer, what limits the most capable AI systems is increasingly not what they know but how fast they can produce useful work. Fractile is rebuilding the hardware underneath frontier AI to change that. Its processors interleave memory and compute, so large models can generate output dramatically faster across very long contexts.
Walter made this bet in 2022, as a doctoral student at Oxford, well before most of the industry saw it coming. He has since built a team of semiconductor veterans spanning the full stack, from transistor-level circuit design to the software that serves models in the cloud. In every conversation, we were struck by how directly the company’s roadmap is tied to where frontier AI is heading.
Fractile is a chip-first frontier AI company, building the foundation for what AI systems will be able to do next. We couldn’t be more excited to back Walter and the team on this journey.
We have raised a $720M Series C, led by @ThriveCapital, @lightspeedvp, @Redpoint and @foundersfund, to allow us to run faster in shipping the world's fastest chips for AI.
We've seen over the past year that many of the hardest problems that humanity faces - in science, medicine, physics - may be solved in our lifetimes by AI.
One hallmark of AI applied to the toughest problems is that extreme scaling of inference compute becomes vital. In tackling a Millenium Prize problem, for instance, @OpenAI ran 10,000 parallel agents for 80 hours. Costing over $10M, this level of capability is far from democratised today.
When we study agentic scaling, there are two paths: parallel, and sequential. In just the same way that most hard problems solved by humans don't parallelise efficiently, we see the same with agents: fewer agents, thinking for longer, scales better in every way except for the time that you wait for a response.
Building radically faster inference hardware, which doesn't force a trade-off on throughput, model sizes or context lengths, is a key step in bringing down the cost and time we incur in solving our hardest problems.
We're thrilled to be backed to bring this to the world!
Thrilled to co-lead @fractile_ai's $720M Series C and get to work with @goodwin_ml and the incredible team.
Agents are running longer on bigger models and larger contexts. They’re bottlenecked by memory.
Fractile is the chip-first frontier AI company built to break that, stacking DRAM directly on logic for speed and scale. Over many whiteboards in Bristol, it became clear that Fractile’s full-stack approach is unlike anything else in silicon. You won’t find a more talented and fast-moving team. Couldn’t be more excited for everything ahead!
We’re delighted to announce that Fractile has raised $720M in a Series C led by @thrivecapital@foundersfund@lightspeedvp and @redpoint, with further participation by brilliant new and existing investors.
Why hasn't AI automated more work? What's holding coding agents back? And are the labs' valuations built on the right expectations?
@CompleteSkeptic, co-founder and CEO of @typesafeai, was the perfect person to discuss this and more. Diogo coauthored OpenAI's InstructGPT paper, and a few weeks before we recorded, TypeSafe launched Jev, a "system one" model that returns structured decisions instead of text.
It was a ton of fun to discuss this with Diogo.
The conversation includes:
0:00 Intro
1:14 The Early Bet on RLCD
4:40 Built for Humans vs Computers
6:39 Roadmap & Model Limitations
11:04 Mental Models of Models
16:07 Do LLMs Have a Place in 10 Years?
17:32 New Model Shapes
21:24 Problems Still Unsolved
27:00 The Tyranny of the KV Cache
31:21 Rethinking Coding Agents
34:40 Multimodality
37:52 Images, Games & Computer Use
39:36 Could This Have Been Built 5 Years Ago?
42:01 Benchmarks & Copycat Releases
44:12 Fundraising
45:27 Quickfire
Listen here:
YouTube: https://t.co/K8kp8aFv3T
Spotify: https://t.co/GImUYej9It
Apple: https://t.co/KcuUJNkT3Q
Why hasn't AI automated more work? What's holding coding agents back? And are the labs' valuations built on the right expectations?
@CompleteSkeptic, co-founder and CEO of @typesafeai, was the perfect person to discuss this and more. Diogo coauthored OpenAI's InstructGPT paper, and a few weeks before we recorded, TypeSafe launched Jev, a "system one" model that returns structured decisions instead of text.
It was a ton of fun to discuss this with Diogo.
The conversation includes:
0:00 Intro
1:14 The Early Bet on RLCD
4:40 Built for Humans vs Computers
6:39 Roadmap & Model Limitations
11:04 Mental Models of Models
16:07 Do LLMs Have a Place in 10 Years?
17:32 New Model Shapes
21:24 Problems Still Unsolved
27:00 The Tyranny of the KV Cache
31:21 Rethinking Coding Agents
34:40 Multimodality
37:52 Images, Games & Computer Use
39:36 Could This Have Been Built 5 Years Ago?
42:01 Benchmarks & Copycat Releases
44:12 Fundraising
45:27 Quickfire
Listen here:
YouTube: https://t.co/K8kp8aFv3T
Spotify: https://t.co/GImUYej9It
Apple: https://t.co/KcuUJNkT3Q
Why hasn't AI automated more work? What's holding coding agents back? And are the labs' valuations built on the right expectations?
@CompleteSkeptic, co-founder and CEO of @typesafeai, was the perfect person to discuss this and more. Diogo coauthored OpenAI's InstructGPT paper, and a few weeks before we recorded, TypeSafe launched Jev, a "system one" model that returns structured decisions instead of text.
It was a ton of fun to discuss this with Diogo.
The conversation includes:
0:00 Intro
1:14 The Early Bet on RLCD
4:40 Built for Humans vs Computers
6:39 Roadmap & Model Limitations
11:04 Mental Models of Models
16:07 Do LLMs Have a Place in 10 Years?
17:32 New Model Shapes
21:24 Problems Still Unsolved
27:00 The Tyranny of the KV Cache
31:21 Rethinking Coding Agents
34:40 Multimodality
37:52 Images, Games & Computer Use
39:36 Could This Have Been Built 5 Years Ago?
42:01 Benchmarks & Copycat Releases
44:12 Fundraising
45:27 Quickfire
Listen here:
YouTube: https://t.co/K8kp8aFv3T
Spotify: https://t.co/GImUYej9It
Apple: https://t.co/KcuUJNkT3Q
Danila Shtan, @nebiusai CTO, on the state of open-model inference:
"As an industry, we didn't optimize sh*t."
Run a new open model as published and the results are slow and expensive:
Danila Shtan is the CTO of @nebiusai, the AI cloud racing AWS, Google and Microsoft.
Danila has been with the company since the beginning. On November 1, 2023, Nebius launched its first public cloud with about 1,600 H100s. That same day, Danila told the engineering team to throw the entire stack away and start from scratch.
Today, Nebius is a Nasdaq-listed AI cloud with a $17.4 billion deal with Microsoft, a Meta agreement worth up to $27 billion, and a $2 billion investment from NVIDIA.
Danila is betting Nebius can grow fast enough to fight the hyperscalers as equals before they turn their full attention to AI cloud. He thinks the market is barely scratching the surface.
0:00 Intro
1:25 What Differentiates AI Clouds
6:16 Nebius’ Origin Story
10:15 Bringing New Nvidia Chips Online
14:55 What Different Customers Want
19:58 Latency vs. Cost
21:53 Disaggregated Prefill & Decode
24:59 Inference’s Low-Hanging Fruit
28:31 What Nebius Should Productize
30:11 The Tavily Acquisition
31:56 How Data Centers Differ
34:29 Bottlenecks in the AI Buildout
37:03 How New Workloads Shape Clusters
39:17 Why Hyperscalers Didn’t Win This Market
43:39 The Future of Neoclouds
45:42 What Gets Absorbed by AI Hyperscalers
49:43 OpenAI x Baseten
52:07 Quickfire
Listen here:
YouTube: https://t.co/HMU0Fs7NNd
Spotify: https://t.co/1NhkMDqCSd
Apple: https://t.co/HQg4UKWQ8s
CTO of @nebiusai, Danila Shtan, on what it’s like getting NVIDIA’s newest chips live:
An iPhone is planned three years out, with the design settled a year before launch. NVIDIA’s platforms keep changing while the chips are still being produced.
Danila Shtan is the CTO of @nebiusai, the AI cloud racing AWS, Google and Microsoft.
Danila has been with the company since the beginning. On November 1, 2023, Nebius launched its first public cloud with about 1,600 H100s. That same day, Danila told the engineering team to throw the entire stack away and start from scratch.
Today, Nebius is a Nasdaq-listed AI cloud with a $17.4 billion deal with Microsoft, a Meta agreement worth up to $27 billion, and a $2 billion investment from NVIDIA.
Danila is betting Nebius can grow fast enough to fight the hyperscalers as equals before they turn their full attention to AI cloud. He thinks the market is barely scratching the surface.
0:00 Intro
1:25 What Differentiates AI Clouds
6:16 Nebius’ Origin Story
10:15 Bringing New Nvidia Chips Online
14:55 What Different Customers Want
19:58 Latency vs. Cost
21:53 Disaggregated Prefill & Decode
24:59 Inference’s Low-Hanging Fruit
28:31 What Nebius Should Productize
30:11 The Tavily Acquisition
31:56 How Data Centers Differ
34:29 Bottlenecks in the AI Buildout
37:03 How New Workloads Shape Clusters
39:17 Why Hyperscalers Didn’t Win This Market
43:39 The Future of Neoclouds
45:42 What Gets Absorbed by AI Hyperscalers
49:43 OpenAI x Baseten
52:07 Quickfire
Listen here:
YouTube: https://t.co/HMU0Fs7NNd
Spotify: https://t.co/1NhkMDqCSd
Apple: https://t.co/HQg4UKWQ8s