The overwhelming claim is that SWE will be finished by EOY thanks to LLMs. And yet, they are constrained by the datasets they were trained on. Proof of otherwise has yet to materialize, โWrite a C compiler using only the Dragon Book and K&R C as referenceโ. Given that, there is easily well over 10B lines of critical closed-source production code that runs the modern world. While each individual company can leverage their slice of that for their own internal LLMs, no one player has it all.
So, is it really over?
"Well Daddy, I've been tryin', I just
can't catch a break
There's too much in this world
that I can't seem to shake
But I remember your words, Lord,
they bring me the chills
Keep your nose on the
grindstone and out of the pills"
@TTChilders
Meta's LLM play feels like Microsoft vs. Netscape of the '90s, but the perception is reversedโthe incumbent is seen positively for encroaching on the startup, an exceptional case.
Microsoft bundled Internet Explorer for free with Windows to eat into Netscape's browser market share, which Netscape sold as a standalone piece of software, Navigator, their primary source of revenue. They could do so given that Windows was a cash cow, thus bleeding Netscape dry. 'Embrace, extend, and extinguish' was an internal Microsoft phrase about such strategies.
Similarly, Meta can continue to open source Llama because of their ad revenue moat, leaving OpenAI (and others) in a tough spot. However, given all that hangs in the balance with AI, open sourcing is viewed in a positive light. Zuckerberg's recent blog post correctly highlights all the reasons why. This was very much not the case for Microsoft, whose approach, including other business practices, led to a major antitrust lawsuit, which they lost, furthering the antagonistic perception of the company.
uarch micro-lesson #1
Branch Prediction is a power law, as observed by @djimeneth .
Is code, therefore, constrained by natural law?
Notes and slides avail from Berkeley lecture
https://t.co/M3SsqgxzCo