Siddharth Kotwal

[Siddharth Kotwal @SiddKotwal]

https://t.co/aZcpich4ww

[Gokul Rajaram @gokulr]

Perpetual dissatisfaction

I've now closely worked with/for several founders of $25B+ companies, and one characteristic I've observed in all of them: PERPETUAL DISSATISFACTION WITH THE STATE OF THE COMPANY.

They are not satisfied, even as the company hits its goals, scales and grows, and even as most employees / investors feel happy with where the company is at.

This dissatisfaction manifests itself most strongly in reviews and 1:1s. In business and product reviews, they push teams to think bigger, take more risks, move faster. In 1:1s, they urge their people to show more urgency, hire better, be more customer-centric. However well a team or person is performing, they ALWAYS have a list of 5-6 things that team or person could be doing better.

Why is this the case? I've realized it's the case because they have such a huge and continuously expanding vision that they always see a big gap between the present and their vision for the future.

This dissatisfaction / urgency / push transfers to the broader team, who pushes harder and stays hungry.

I've also seen that as soon as a founder starts drinking their own kool aid and believes that the company has "made it", it's a leading indicator of the beginning of the end. Why? Because a founder's complacency immediately leads to the team becoming even more relaxed, and that torpedos the company.

Note: I'm not advocating being an ass. Far from it. Expressing dissatisfaction is absolutely possible to do respectfully and without being a jerk.

I've started using this as a tell for truly great founders. Never satisfied, never happy with the state of the company, never self-congratulatory. Obviously you can only see this after a few months of working with them.

Leaders: examine your own behavior. If you are projecting a satisfied and complacent demeanor to your team, it's probably magnified 10x in their behavior. "My boss thinks everything is hunky dory". Add an edge to your interactions with your team. Be dissatisfied.

[Jerry Liu @jerryjliu0]

Introducing ExtractBench, the most comprehensive benchmark for information extraction from complex enterprise documents.

The latest models are pushing the frontier of coding and knowledge work, but surprisingly they still struggle on complex doc extraction tasks in production. A well-tuned extractor must parse multi-page filings without dropping rows, emit exact spatial citations for auditability, and handle messy scans. Also they must do all of this at a viable per-page cost so that you can scale this to millions of docs in production (you can’t be paying upwards of $1 in tokens per page!)

Existing extraction benchmarks fall short: they are not large/diverse enough in document domain (finance, energy, gov, auto), elements (long records, scans, grounding), and schemas.

So our applied research team built ExtractBench.

We evaluated 14 systems: frontier VLMs, coding agents, and specialized extraction APIs, against 370 enterprise documents: 4,869 pages, 67 document types.

Our biggest finding 🧪: Short documents mask critical system flaws. On files past 50 pages, commercial VLMs collapse below 35% recall due to silent list truncation. They hold high precision, but lose output attention and drop most of the table rows.

ExtractBench evaluates value accuracy, long-record completeness, spatial grounding, and per-page cost with zero LLM judges. It is 100% deterministic and reproducible.

In tandem with ExtractBench, we’re also introducing 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗣𝗹𝘂𝘀, a new Extract tier in LlamaParse that debuts at #1 on the leaderboard: 95.6% value accuracy, at less than a third the cost of the closest peer.

Explore the findings, download the dataset, or run the harness:

Blog:
https://t.co/tsXfovKJaz
GitHub: https://t.co/auzmOGgUl3
HuggingFace: https://t.co/1eKisSBrE8

We will be actively evolving both our extraction benchmark as well as our extraction harness over time.

If you check out either ExtractBench or LlamaParse, let us know your feedback!

[Bessemer @BessemerVP]

Today we’re launching 𝐓𝐡𝐞 𝐀𝐠𝐞𝐧𝐭𝐢𝐜 𝐀𝐰𝐚𝐤𝐞𝐧𝐢𝐧𝐠, the deep-dive research and playbook on agentic development. ⭐

20+ AI-pilled engineering teams → 63-pages of insights and frameworks to guide a company’s transformation.

Developer teams want to know:
"How are we measuring up? 🧵

https://t.co/akAwYDVlLu

[Madhu Guru @realmadhuguru]

There’s going to be a ton of money in making open-weight models exceptional at boring, specific business domains.

Pick a size x domain combination and go deep. Eg mid-market legal, smb retail, enterprise logistics.

The hyperscalers have the primitives but will struggle with domain depth, scrappiness and sheer will to make a model exceptional for one particular kind of business.

And that is your opportunity.

[Lisan al Gaib @scaling01]

I think there are a few reasons for why LLMs have this very bland and non-committal writing style:
- next-token prediction tries to preserve optionality
- hallucination training makes LLMs non-committal and wary of having opinions
- the training data doesn't have any labels on what the text is trying to convey, so generated text can feel aimless
- the training data is probably contains little everyday talk between people, but probably a lot of blogs and articles, which makes conversation feel too curated
- whatever writing RL they are doing probably uses rubrics that judge more general things like quality, coherence, writing style, ...
- general assistant persona training, which is opposite of what makes people interesting. people are interesting because they are odd, because they have new perspectives on things, not because they are hyper agreeable and non-confrontational

human writing often feels more authentic because it's not trying to be fancy and it feels like there's more direction and decisiveness

I think these problems are 100% fixable today, and will be fixed with continual learning

[Gokul Rajaram @gokulr]

https://t.co/MbbiuxaPS1

[Siddharth Kotwal @SiddKotwal]

@dbreunig @pamelafox What’s the simplest place to understand what type of queries are better supported by a KG search over flat-search over markdowns?

[elune @elune0x]

10 agent evals every AI engineer should know

1) golden set

a frozen set of cases you run after every prompt model or tool change

use it as your baseline to see whether the system improved or quietly broke

OpenAI Evals helps you build repeatable benchmark sets and compare model changes

→ https://t.co/dr1GZlC75R

2) llm as judge

a second model scores open ended answers against a written rubric

use it when there is no exact output to compare against

OpenEvals provides ready made evaluators for LLM applications

→ https://t.co/S2yhnByFIP

3) rubric scoring

score correctness tone safety and cost separately

one quality number hides the problem

DeepEval helps you create custom metrics and score every dimension independently

→ https://t.co/q9Z6Xmixia

4) trajectory eval

grade the path the agent took

not only the final answer

AgentEvals checks agent actions decisions and tool calls across the full trajectory

→ https://t.co/0oziAl54az

5) tool unit tests

test every tool with fixed inputs and outputs

no model in the loop

MCP Inspector helps you inspect and test MCP servers tools and responses separately

→ https://t.co/IVmt5qpWIN

6) regression suite

replay previous runs against every new prompt model or toolset

then compare the results

Promptfoo helps you run repeatable eval suites catch regressions and add checks to CI

→ https://t.co/zxi2PuWuhe

7) a b in prod

split real traffic between two versions and compare actual outcomes

GrowthBook provides feature flags controlled experiments and product analytics

→ https://t.co/DGlE3JjDD3

8) human review

sample real runs and let a person grade them honestly

use it to calibrate your automated judge

Argilla helps teams collect human feedback review outputs and build better datasets

→ https://t.co/QHWb7skWjr

9) shadow run

let the candidate run on real traffic while its output is shown to nobody

use it before a risky rollout

Langfuse helps you trace production runs compare candidates and monitor eval results

→ https://t.co/IrhDf38tRn

10) red team

attack the system before somebody else does

jailbreaks prompt injection data leaks and tool abuse

Garak scans LLM systems for vulnerabilities and unsafe behavior

→ https://t.co/w8ObyW4ZKv

offline evals tell you it works

online evals tell you it still works

you probably do not need all ten today

start with the two that would have caught your last outage

bookmark this

[Siddharth Kotwal @SiddKotwal]

@alighodsi For someone that has only used Codex/Claude APIs, what is the easiest way to understand why ontologies or a KG is useful if coding agents can flat-retrieve everything?

[Nathan Lambert @natolambert]

Where Kimi K3 puts the balance of power — and bends the trajectory — of the AI ecosystem.
https://t.co/KuzZZdt8xo

[David Senra @FoundersPodcast]

New episode: Peter Thiel on How to Build a Creative Monopoly

Or what I learned from reading Zero to One for the 3rd (or 4th) time:

0:00 Every great entrepreneur is first and foremost a designer.

0:21 Long-term planning is often undervalued by our indefinite short-term world.

1:24 Founders only sell when they have no more concrete visions for the company. Definitive founders with robust plans don't sell.

2:03 A startup is the largest endeavor over which you can have definitive mastery.

3:44 Don’t do anything someone else can do.

4:29 Today's best practices lead to dead ends. The best paths are new and untried.

5:05 Technology is miraculous because it allows us to do more with less, ratcheting up our fundamental capabilities to a higher level. By creating new technologies, we rewrite the plan of the world.

6:25 Indeed, the single most powerful pattern I have noticed is that successful people find value in unexpected places, and they do this by thinking about business from first principles instead of formulas.

9:16 Brilliant thinking is rare, but courage is even in shorter supply than genius.

9:53 The present is the past biting into the future.

10:37 Small groups of people bound together by a sense of mission have changed the world for the better.

11:25 This is what startup has to do: question received ideas and rethink business from scratch.

13:36 The most contrarian thing of all is not to oppose the crowd, but to think for yourself.

15:57 Creative monopoly means new products that benefit everybody and sustainable profits for its creator.

18:50 If you're interested in making things, you'll be less afraid to pursue this activity single-mindedly and thereby become incredibly good at them.

22:03 If you focus on near-term growth above all else, you miss the most important question you should be asking: Will this business still be around a decade from now?

26:30 Every monopoly dominates a large share of its market. Therefore, every startup should start with a very small market. Always err on the side of starting too small. If you think your initial market might be too big, it almost certainly is.

28:53 Competition is for losers. Avoid competition as much as possible.

29:44 The definitive person determines the one best thing to do and then does it. The definitive person strives to be great at something substantial. They strive to be a monopoly of one.

31:15 What is luck? The ability to exploit accidents. — Napoleon

33:20 We do not live in a normal world. We live under a power law. And as a result, company outcomes follow a power law. A small handful of companies radically outperform all others.

34:00 If you do start your own company, you must remember the power law to operate it well.

34:05 The most important things are singular.

36:29 Making mistakes is the privilege of the active.

38:00 The best entrepreneurs know this: Every great business is built around a secret that is hidden from the outside. A great company is a conspiracy to change the world. When you share your secret, the recipient becomes a fellow conspirator.

40:54 A small company depends on great people much more than a big company does.

41:15 The founding of a company lasts as long as a company is creating new things, and it ends when creation stops.

41:25 No company has a culture. Every company is a culture.

41:48 Since time is your most valuable asset, it is odd to spend it working with people who you don't envision any long-term future together.

42:09 Recruiting is a core competency for any company. It should never be outsourced.

42:37 Your company should be a tribe of like-minded people fiercely devoted to the company mission.

44:02 Superior sales and distribution by itself can create a monopoly, even with no product differentiation. The converse is not true.

44:24 Advertising matters because it works. It works on nerds and it works on you.

44:49 Advertising doesn't exist to make you buy a product right away. It exists to embed subtle impressions that will drive sales later.

46:29 Poor sales rather than bad product is the most common cause of failure. If you can get just one distribution channel to work, you have a great business.

49:00 When Howard Hughes was awarded the Congressional Gold Medal in 1939 for his achievements in aviation, he didn't even show up to claim it.

52:00 The most important task in business, the creation of new value, cannot be reduced to a formula and applied by professionals.

[Ali Ghodsi @alighodsi]

Highly recommended interview with @matei_zaharia about open source @omnigent_ai . Meta-harnesses are the future!

[CG @cgtwts]

Moonshot AI CEO Yang Zhilin just gave one of the clearest masterclasses on how frontier AI models are actually built.

[SemiAnalysis @SemiAnalysis_]

A year ago, the big three was OpenAI, Anthropic, and Google. Things have changed.

Moonshot's Kimi K3 sits above Gemini on every composite benchmark, and it's open source in 10 days.

New episode: what K3 reveals about frontier margins, model sizes, and who's actually still in the game.

00:11 Is Kimi K3 the Third Best Model?
04:04 Why Delay the Weights?
05:30 2.8T Parameters and Serving Constraints
06:48 Frontier Margins and the 3x Price Hike
11:10 New Architecture, What Comes Next
14:09 Will Open Source Catch Closed?
19:51 Built for Chinese Accelerators
22:57 The Harness Is the Product
28:49 We're Still Early

[Gokul Rajaram @gokulr]

OWN THE LEARNING, RENT THE MODEL
@jainarvind, Founder & CEO, @glean, interviewed by @HarryStebbings Stebbings (@20VC)

Summary: The model layer is commoditizing fast. Arvind Jain says more than 90% of enterprise use cases can now run on many different models, including open source, and he expects the majority of enterprise workloads to run on open models within three years. The move that matters is owning your context and the institutional learning that accumulates inside your agents, because whoever owns that learning owns the compounding advantage. Jain also makes a contrarian bet: AI grows companies. He plans to take Glean from 1,000 to 5,000 people because per-person productivity climbs, but you have to do ten times the work to earn the same revenue.

1. Model-Layer Commoditization. More than 90% of enterprise use cases can now be handled by many different models, including open source. Glean turns that into a product: it picks the right model for each task and reaches for open source whenever the answer quality holds, which it sells to customers as cost control. The value has moved off the model and onto the orchestration around it. When any competent model can do the job, the layer that chooses and routes captures the margin.

2. The Open-Source Inflection. Open source has just reached within three months of frontier capability, and Jain expects most enterprise workloads to run on open models within three years. The trigger was concrete: a Chinese model, GLM, is the first his own team trusts to carry the majority of their workloads. The driver is cost, now that AI budgets blow past their annual number in a month or two. Once quality is close enough, an order-of-magnitude price gap decides where the work runs.

3. Own The Learning. The real danger with frontier providers is operational dependence. As an agent does a job over and over, the undocumented institutional learning accumulates inside that agent. If you do not run the agent or own what it learns, you have handed your operations to the AI company. Jain's rule for enterprises: use these models, but keep control of the compounding learning, because it belongs to you.

4. China Or Not. The open-source decision comes down to one question: are you comfortable running a Chinese model? Everyone will be fine with open source itself; the hesitation is whether a Chinese model hides a back door or could later be used against them. The models run inside your own contained environment with nothing sent back, so what holds people back is comfort. The early movers make the leap first, and then it becomes normal.

5. Labs As Asset. For any company not training frontier models, the labs are a huge asset. The vertical products Anthropic and others launch into design, legal, and finance are shallow, and Jain sees AI expanding the market: non-designers now use Claude for design while designers keep using Figma. Founders losing sleep over the labs should stop worrying and go solve problems. The frontier labs let Glean ship a product it could never have built alone.

6. Bundling Breaks. Microsoft is Glean's most significant competitor, and its play of shipping a product 70% as good but bundled into the enterprise suite genuinely works. What weakens it is consumption pricing: when you pay per unit of work, the bundle loses its built-in advantage, because users can pick the best tool for each job and you pay only where the work happens. Stebbings pushes back that enterprises still prefer one approved vendor over fifteen. Jain concedes the real killer is price, because it is hard to compete with free.

7. Investing Around The Model. Getting ROI from AI is mostly a throughput problem. Most enterprises bolt AI onto their systems through MCP and let the model brute-force its way to the context it needs, which is slow and expensive because most tokens get burned just assembling raw materials. The fix is to invest around the model and feed it the right context so it works faster and cheaper. Companies chasing "replace the worker" are aiming at the wrong target.

8. The Headcount Bet. While most CEOs shrink teams, Jain plans to grow Glean from 1,000 to 5,000 people. His logic: if you and your competitor both have the same AI and you shrink while they keep their people to build a 10x better product, they win. Per-person productivity will climb, but the bar climbs with it, since you now have to do ten times the work to earn the same revenue. The bigger company aimed at more ambitious output takes the market.

9. Don't Replace Yourself. Telling employees to replace themselves with AI is the wrong goal, because it gives the technology too much credit. AI can handle 90% of most roles but not the last intangible, the way an assistant cannot quite pick your spouse's birthday present. In competitive work a 90% solution loses to a rival who runs the same AI and keeps a sharp human on top. Aim to make each person the best at what they do.

10. The Million-Dollar Agent. Glean built a triage agent that resolves 95% of production issues for a 15-person on-call team, and it costs a million dollars a month. Even Jain calls it questionable whether that beats the humans it replaced, a reminder that today's AI can be absurdly expensive for what it delivers. His bet is that inferencing costs fall by orders of magnitude, and that the recent per-token price increases were labs proving they are good businesses before going public. If he is right, the economics of every AI agent look very different in two years.

11. The Token Power Law. Glean's approach to token budgeting was to do nothing and let people figure out what the technology could do. The result is a power law: some employees spend $10,000 to $15,000 a month in tokens while others spend $20. Everyone adopted the basic use case, question answering, which is the single largest application of AI in the world, but the advanced use cases sit with about 5% of the workforce. Adoption is universal but shallow, and the deep use stays rare.

12. Capital As A Failure Path. Too much available capital is quietly creating failure paths for startups. A seed-stage company will pay half a million dollars for an engineer while Google, which knows it does not need to buy talent that way, will not, and that structure does not last. At the same time this is a genuine land grab: every company wants a product like Glean's, and getting in today is ten times easier than getting in later. Jain even admits his own discipline may now be too conservative for the moment

[sarah guo @saranormous]

Hard to overstate what a big deal this is for @SierraPlatform. Horizon is an expansion into customizable and compounding outcomes-seeking agents.

Bret, Clay and team are expressing their worldview in their product. They are shipping the AI business model and products that are aligned with Sierra customers (pay for ROI) and deepen Sierra's customers’ moats (their customer relationships).

[Jerry Liu @jerryjliu0]

I'm glad people still understand the importance of building high-quality retrieval systems in 2026, especially as the outer models/harnesses are getting better every day.

Making agentic retrieval work in production doesn't necessarily require groundbreaking new techniques around retrieval or planning. This article shows that you do need to spend engineering time tuning chunking, synchronization, reranking, tool API design, permissioning, and more. Some interesting tidbits:
* Concatenating onto existing Slack threads as one contiguous chunk with heuristics for determining additional relevant context
* Real-time updates for Slack
* Separate chunking/updates for codebases
* The discovery that hybrid search works well (not like this was a super new realization, but always good to validate and understand the specific parameters they tweaked)
* Having natural guardrails on which data sources are relevant for which projects

Building simple retrieval is easy, building production retrieval is hard. We had to deal with a lot of these challenges when productionizing our own Index feature within LlamaParse.

[Ashpreet Bedi @ashpreetbedi]

I’m open-sourcing 75 code examples of how labs are using agents for training data.

Labeling, DPO juries, rejection sampling, step-level rewards, instruction generation, dataset curation, benchmark decontamination.

Learn these and go get that bag 💰

https://t.co/5PpxwPhPE5

[Harrison Chase @hwchase17]

there needs to be an OPEN standard for memory

OKF (Open Knowledge Format) is one such standard

excited to announce that we're now using it in OpenWiki

[Nikesh Arora @nikesharora]

The CIO mindset - CIOs are busy nowadays. CEOs want progress on AI, their C-Suite peers want AI transformation resources, and everyone wants to move fast.

On the other hand the AI landscape is far from static, coding is a real use case, but costs are rising and headcount savings are being overwhelmed by increased demand. Other POCs on AI are still being figured out and ROI isn't fully realized. CIOs need to determine long term architectures, choose partners and technologies which will likely change a few times before we stabilize.

On the day job, (non AI side), CIOs need to decide which SaaS contracts to commit to where to retain flexibility, how to deliver the promises already made with projects mid stream, and what to redirect to pay for the AI landslide.

They need to find budget dollars to work on all of the following:
Priorities
1. Figure out AI, evaluate, commit to clear ROI and keep your powder dry.
2. Deal with Mythos and prepare for a new world of Cyber, rethink point solves vs platforms.
3. Create capacity in budget for increasing AI usage.
4. Evaluate the vendor landscape and evaluate how the vendors will evolve and determine strategic redirection where needed.
5. Prioritize hardware purchases where compute and memory prices are going to be on the rise for the foreseeable future.

This is normal in a technology super cycle. The CIOs will have to power through these dynamics. We are definitely seeing this in the equity markets with everyone trying to pick winners and losers. I think it's too early to draw long term conclusions. Customers are still adjusting to the economic realities of AI and reallocating spend, this reset is being reflected in the equity markets.