It's been interesting and puzzling to witness the problems with accuracy in UK economic statistics over the past few years. (See the links in the next tweet for more.) It seems that the Office for National Statistics, ONS, now struggles to effectively measure basic figures such as employment, trade, and inflation. This resulted in a quite scathing government report published last summer, where Robert Devereux, a former permanent secretary, concluded that "most of the well-publicised problems with core economic statistics are the consequence of ONS’s own performance."
There's a lot of discussion about the travails facing the UK these days (including this big piece in The Atlantic a few weeks ago[1]), and the problems with the ONS feel like an unsettling microcosm of diffuse decline in broader institutional competence.
Anyhow: at Stripe, we became curious about the UK's published entrepreneurship data. While we observe a boom in many parts of the world, official figures don't show a similar increase in the UK. In the latest Stripe Economics post, we dug into the data, and, as far as we can tell, the official figures are probably misleading. The good and the bad news (mostly good, I think!) is that the UK is almost certainly witnessing an unmeasured boom in entrepreneurship: https://t.co/R7oTZNmxc6
UK-specific issues aside, I suspect that this measurement question is illustrative of forthcoming econometric challenges. Keeping the world's macro indicators up-to-date in response to the faster-than-usual changes wrought by AI will be both increasingly difficult and increasingly important in the coming years.
[1] https://t.co/OAnwRmpyON
Went to see Spain vs France with 17 yo. About 10 minutes in I asked who seemed to him to have the upper hand so far, because it wasn't who we'd been expecting. They kept it the whole match.
One of the many properties that code has that makes it highly amenable to agents is that you can more or less quickly test it. You can either go see if the application works manually, or you can actually run a test on what you built.
Most other areas of work don’t have this benefit. You only get the testing when the final product hits the real world in some capacity - a stock trade is executed, a contract is negotiated, a sales pitch is delivered, and so on.
There’s probably going to be a whole new set of opportunities for how we begin to test the rest of work in this way. Ultimately it will mean more agents being layered into workflows.
It also means we need much better evals on most of our workflows. Most work today in enterprises doesn’t have an associated eval to know if something broke or improved with a model, prompt, or system change.
The enterprises that are able to eval their knowledge work the best also stand to gain the most from AI. Will become a critical aspect of agent adoption over time.
We ran into Amit from the winter 2007 batch. We still remember him vividly, because after I told him in rehearsals that he had to speak louder, the first sentence of his Demo Day presentation was probably the most emphatic ever delivered.
Sustaining the health of the thymus and our immune system, preventing its involution, may be key to promoting healthspan @nature@EdwrdChen
https://t.co/LM7KzJ3zoH
This often happens with technology. Probably more often than not. It's as if the system as a whole has some kind of foresight that individual participants in it don't have.
Before vibe coding became a thing, programming was already evolving in that direction. It already increasingly consisted of installing and configuring stuff other people wrote, without reading the source.
It's interesting in this case how well AI fits into existing trends. Programming wasn't evolving in this direction because AI was coming. No one knew it was. And yet we end up with what looks like a smooth acceleration along much the same path.
What Ukraine is accomplishing against Russia in the Sea of Azov is incredible. I hope Taiwan is taking notes about how to apply lessons to the waters around it.
Here's a marvelous paper. It proposes that genetically programmed human language learning doesn't require any genetically specified language mechanism, just a tendency to pay attention to certain stimuli, and to try really hard to predict them. Then generic learning mechanisms can do the hard work. The stimuli to pay attention to are rapid ostensive rhythmical streams of actions. This covers language, and sign language, and also music.
"Noticing language: What echolocation tells us about language emergence" by Brett Reynolds. (Epistemic status: I had this idea a couple of years ago. But I didn't think hard about it or write a paper, so I deserve no credit.)
https://t.co/DYMJlOAv3G
A few thoughts on what we will see in AI structurally for the foreseeable future:
* Frontier intelligence continues unabated and pushes the industry forward continuously. The top labs will continue to buy the best and the most data, build the most compute, be at the forefront of improved training breakthroughs, and so on. A few different approaches stratify the market on pricing and capability, but overall competitive pressure brings down pricing on a per task basis. That said, we just ask more from the models over time - as one thing gets cheaper, we just use more - so frontier spend and use remains robust.
* Open weights rapidly absorbs frontier breakthroughs (and drives other breakthrough directions given the constraints), offering both lower cost intelligence and the ability to be post trained for specific workflows and domains. This creates a healthy counter balance to the frontier as you can run models “at cost” on a hyperscaler at any time, and tune models just for your tasks.
* The Applied AI layer has a huge opportunity to combine frontier intelligence with open or cheap closed models to orchestrate workflows in any given domain. Due to evals, deep domain context, being trusted with enterprise data and workflows, this layer can maximize performance and cost combination. The applied AI layer will also often have their own RLed models especially for high volume, predictable tasks in their systems.
* Individual enterprises will generally focus on their enterprise context, making sure they can get any AI system the right data and information to work with, in a continuously improving way. Some will go off and train their own models for specific areas of work (large banks, pharma, etc.) where they can get real alpha from doing so given the many tradeoffs, but most will spend energy on making sure they can get all of the gains from AI breakthroughs on their data and workflows.
Net net: even though some of this gets framed as zero sum, there’s just a ton of opportunity for all layers of the stack and approaches.
The rise and fall of wokeness: DEI commitments in corporate securities disclosures filed with the SEC. To me this seems a trailing indicator; most other measures of wokeness take off well before 2019 and peak in 2020 or 2021. But the shape! That's what a moral fashion looks like.
An initial startup idea can't usually be both grand and precise. In practice they're usually either grand and vague or precise and small. Precise and small is better. You know who your initial users are, and you expand outward. With grand and vague you can't even get started.
@heysamir_ There's a simple explanation. I form my opinions on issues individually instead of accepting the entire left- or right-wing set of opinions en bloc.
I wrote about it here: https://t.co/ogMaJHs63d