I have to be the downer here... but I am not convinced the selection bias issues have been resolved.
I have been working on a very similar experiment and the observational results look *amazing* even after matching... but the treatment effects are much less persuasive, often because many of these businesses just don't engage with the AI that much.
Where else can you work on "AI and machine learning, agentic commerce, cryptography, security, distributed computing, hardware design, economics [...] market and mechanism design, foundations of computing, and pure mathematics"? @a16zfrontier
Apply to join us this summer, QED!
New WP w/ @orgRem & @TarunKhannaHBS: https://t.co/oaCQSLomrP
Often, when we discuss value of platforms, we talk about pooling data across users improving each user's prediction. But is there still an externality when the best model looks incredibly different across users? 1/N
@orgRem@TarunKhannaHBS@VC31415 Getting two AutoDML working papers applied to labor in one day! Glad to see this method taking off. https://t.co/S2SH4WnquI
@martin_casado Will watch! Have been working though this syllabus https://t.co/nH7trPYVjH
Instead of coding pipelines, the frontier startups I see are writing down excellent evals, building self-improving harnesses, and letting the models rip.
Organizations are going to radically change.
I think the idea that we will ask one AI "how should we allocate goods in the economy" is very unlikely and the wrong mental model. Knowledge and preferences are super heterogeneous and contextual. My bet is we end up with a world with billions of models, from intelligences we have do our shopping to firm-specific models for every firm that manages their hiring andHR needs. What's cool about a world of billions of models is that if we (policymakers) have access to the models people/firms use, we should be able to simulate out all sorts of policy alternatives super fast using the actual "response functions" of market participants. This isn't centralized Claude guessing at what I would do, but because the economy would be increasingly embedded in models, we can do "centralized" calculations over the distributed economy on all sorts of cool and interesting ways.
I have some new work coming out in next week getting towards this question in a labor market. We show that having access to hiring models from multiple firms generates insights that should allow us to make the market much better at allocating resources (in this case workers to firms). So the idea that AI can let us "central plan" isn't crazy, it's just that planning is still going to involve contextual, local models and then simulating out how they aggregate and impact prices, choices, ....It's Hayek in silico.
@evavivalt Cool stuff! Excited to dig in...I have the same question: are we running weird enough experiments? Maybe now that the AI can take all the boring effects we can do crazier stuff.
Even beyond signalling models, the slop letters have even less content than reference letters normally do!
Telling me a candidate "makes arguments that pay their debt to one another" is simply meaningless.
I think the problem with most AI generated content is there is very little content.
If you could use AI to generate short information rich ref letters, I would love it. Tech just isn't there yet.
@abhishekn It's so bad. One in 10 ten letters are substantial Claude slop...which isn't a great signal and the content is vacuous.
I would prefer one page with actual information rather than 8 pages of non-sense.
Good luck wading through the slop sea!
Launching what I’m calling “an agent a day keeps Phase 2 failures away”
It’s come to my attention that many people don’t see how AI can be helpful for biotech. So, every day for the next 30 days, I will be posting an agent that can be helpful for biotech and pharma companies. The agents will span all departments and phases of R&D.
Hopefully this will inspire some folks to use AI in novel ways
🚨 New research: AgriLLM for smallholder farmers.
We ran a field experiment to evaluate the impact of AI agricultural advice in the Philippines and India.
Preprint: https://t.co/jgKFyy2bE4
With wonderful @MAliAouad, Niyazi Taneri and @CanberkUcel
🧵
Half a billion smallholders produce a third of the world's food, yet they often lack timely agricultural information, a well-known and pressing challenge. The Gates Foundation and Google have already committed $100M to bring AI to smallholders.
We built AgriLLM, an AI agronomist chatbot, over years of iteration and in partnership with leading agri-firms. We randomly gave half the farmers AgriLLM and the other half an otherwise identical control chatbot that doesn't discuss agriculture. We tracked usage over one growing season, then measured key outcomes with a detailed end-of-season survey.
SO, what did we find?
1. Farmers found the tool useful, trusted it, and recommended it. Usage was nowhere near TikTok-scrolling, but the average farmer still consumed about 1.5 pages' worth of information.
2. Access to AgriLLM changed how farmers farmed. The clearest gains were in crop protection, where in-season information matters a lot and is relatively easy to act on. We also found notable improvements in broader farming practices.
3. We found encouraging but limited evidence of more sustainable farming. Self-reported crop-residue burning fell significantly, though it was the only one of seven sustainability measures that moved.
4. Finally, we saw no change in yields or costs within a single season. This doesn't mean AI can't improve farmers' productivity or support sustainable farming, but it suggests information alone may not be enough. Where improvements depend on complementary resources like finance or machinery, we found no gains. For example, AgriLLM increased farmers' interest in Direct Seeded Rice (DSR), a sustainable rice-farming method, but that interest didn't translate into adoption, possibly because DSR requires machinery or other resources.
Huge thanks to @WhartonAIAI, @CambridgeJBS , @essec, MIT Generative AI Impact Consortium (MGAIC), MIT Sloan School of Management's ASEAN fund, Wharton Global Initiative, and Wharton Tanoto Initiative for generously supporting our research, as well as numerous colleagues, the research team in the field, and practitioner partners!
🚨 The NBER I3 Fellowship is now taking proposals!
**Outstanding oppty for PhD students in economics and adjacent fields working on innovation studies.**
Pros: Funds + networking, mentorship, opportunities to attend & present your work at NBER confs.
Cons: None!
Main expectations are that (i) you develop a new public data product and (ii) you join us at the December NBER meeting to share your progress.
Proposals must be submitted by Nov 1, 2026 using this Google form: https://t.co/wHzYlepmjD
Please share w/ students and colleagues!
@krisgulati Cool work! Doesn't measure taste in the way people think about taste. It's about compute efficiency....which matters a lot but different construct.