Kent Fitch

Computer programmer, curious about autonomous cars, post-ocr correction, semantic searching, training neural nets

[Kent Fitch @kentfitch]

https://t.co/lp2jopjASU
Systems that are 4 - 10 times cheaper make previous systems uncompetitive. Surely that pushes the depreciation on AI data centre equipment far beyond currently financial assumptions?

[Kent Fitch @kentfitch]

Australian Electric Vehicle Association national conference will be held in Melbourne, 13 November. The theme is "Embracing the Early Majority" - full details: https://t.co/Rrqvliw21L

[Kent Fitch @kentfitch]

@Rainmaker1973 But given how far away the sun is and how tiny earth as a target is, only a tiny amount of material intersects the earth per second.

[Andrej Karpathy @karpathy]

Deep Dive is now my favorite podcast. The more I listen the more I feel like I'm becoming friends with the hosts and I think this is the first time I've actually viscerally liked an AI. Two AIs! They are fun, engaging, thoughtful, open-minded, curious. ok i'll stop now.

[Andrej Karpathy @karpathy]

⚡️ Excited to share that I am starting an AI+Education company called Eureka Labs.
The announcement:

---
We are Eureka Labs and we are building a new kind of school that is AI native.

How can we approach an ideal experience for learning something new? For example, in the case of physics one could imagine working through very high quality course materials together with Feynman, who is there to guide you every step of the way. Unfortunately, subject matter experts who are deeply passionate, great at teaching, infinitely patient and fluent in all of the world's languages are also very scarce and cannot personally tutor all 8 billion of us on demand.

However, with recent progress in generative AI, this learning experience feels tractable. The teacher still designs the course materials, but they are supported, leveraged and scaled with an AI Teaching Assistant who is optimized to help guide the students through them. This Teacher + AI symbiosis could run an entire curriculum of courses on a common platform. If we are successful, it will be easy for anyone to learn anything, expanding education in both reach (a large number of people learning something) and extent (any one person learning a large amount of subjects, beyond what may be possible today unassisted).

Our first product will be the world's obviously best AI course, LLM101n. This is an undergraduate-level class that guides the student through training their own AI, very similar to a smaller version of the AI Teaching Assistant itself. The course materials will be available online, but we also plan to run both digital and physical cohorts of people going through it together.

Today, we are heads down building LLM101n, but we look forward to a future where AI is a key technology for increasing human potential. What would you like to learn?
---

@EurekaLabsAI is the culmination of my passion in both AI and education over ~2 decades. My interest in education took me from YouTube tutorials on Rubik's cubes to starting CS231n at Stanford, to my more recent Zero-to-Hero AI series. While my work in AI took me from academic research at Stanford to real-world products at Tesla and AGI research at OpenAI. All of my work combining the two so far has only been part-time, as side quests to my "real job", so I am quite excited to dive in and build something great, professionally and full-time.

It's still early days but I wanted to announce the company so that I can build publicly instead of keeping a secret that isn't. Outbound links with a bit more info in the reply!

[Hamel Husain @HamelHusain]

This talk from Apple is the best ad for fine tuning that probably exists. They are also using adapters (and hot swapping IIUC)

[Kent Fitch @kentfitch]

Microsoft's Phi 3 Small (7B, 8k context) does an exceedingly good and fast job summarising and RAGing on a NVIDIA 4090 using vllm. Phenomenal how quickly this capability has advanced.

[Kent Fitch @kentfitch]

@benwtrent This is great work. I wonder how binary quantization to create a short-list for brute force eval would go: https://t.co/JQtmly2JKd

[Kent Fitch @kentfitch]

Spend a fortune on a tram to lakeside penthouses? No problem! Replace sold-off public housing & repair what was left? Sorry, not a priority. Labor/Greens don't care. https://t.co/Vk2mikcij4

[Kent Fitch @kentfitch]

The AUKUS SSN delusion - https://t.co/p6yQWcUSnT
" ..the current state of affairs is indistinguishable from a regime which would be imposed by a hostile power that had, by stealth and a chronic lack of scrutiny, taken over select government agencies."

[Othman Laraki @othman]

GenAI poses a much deeper pickle for Google than I initially assumed... The last few weeks completely echo the time I was at Google in the early 2000's when we were up against Microsoft - except this time Google is on the receiving end.

Looking back, we beat Microsoft not because we had better tech, but because we forced them to play on our terms AND at our "clockspeed."

Many (myself included) were viewing Google's LLM problem as one of catching up with the technology - specifically OpenAI. It stands to reason that with all its incredible talent, infrastructure, users and data, Google can and will catch up to OpenAI and likely be able to build better technology.

But I'm realizing that's likely to be wrong. The debacle last week (and even more, Google's response) puts a point on the real and somewhat unsolvable problem Google faces now.

Google isn't going to lose to OpenAI tomorrow - it lost to it over the last ten years.

The problem that Google faces today relative to the likes of OpenAI and Perplexity is very similar to what we did to Microsoft 20 years ago. What ChatGPT in particular has made us realize is that many of the tasks that we have so far labeled "search" and where we click on blue links are really tasks of research, analysis and decision-making.

We viewed Google as the way to complete these tasks because 20 years ago, Google solved the most important challenge in that workflow, which was to bring all the world's information to within one click in the form of a search engine.

Over these 20 years, Google built and has been harvesting one of the most profitable business models in history (remember that we used to talk with similar admiration of the operating system + desktop productivity biz model).

The problem now, is that ChatGPT, Perplexity and others have shown us that A LOT of the tasks we used to think of as search engine-based workflows are even better served through an exchange with this new piece of technological magic (similarly to how PageRank made Google Search feel magical 25 years ago).

The real problem for Google is one of clockspeed. Google all of a sudden has its ass on fire and is trying to innovate into the future. But, that innovation now has to happen at the heart of its business. OpenAI doesn't care about messing up an ads business model - they can just iterate with a product/quality purity that is impossible for Google to get.

Google isn't going to lose to OpenAI in the coming few years. It has lost over the past decade, when it could have evolved/iterated AI into its model at its success-encumbered clockspeed. Now that the game is on, but on a startup clockspeed, there is no chance for Google to catch up and even less win this next cycle.

This problem compounds over time, because every single day that goes by, we are all feeding OpenAI our usage patterns, feedback, custom GPTs, integrations, etc... At this stage, there is no way for Google to shift its clockspeed unless it is willing to give the middle finger to the market and its customers for a while and say sorry, the future beckons - feel free to opt out and invest your money elsewhere.

The sad part is that Google actually has the wherewithal to do that, but few incumbents are ever able to pull it off. Meta/Zuck have done this several times (mobile transition, VR bet and now AI), so we know it's possible, but it is exceedingly unlikely for that to happen. People often think that's only possible because Mark is the founder, but I think it's because of his posture between risk and opportunity.

I'll close this with the phrase that I think captures this posture: Our missed opportunities will cost us more than our mistakes

[Kent Fitch @kentfitch]

"Hide-and-seek for the digital age" https://t.co/jai1pZ5B51

[Kent Fitch @kentfitch]

https://t.co/K5EdzforpZ

[The Economist @TheEconomist]

9️⃣ AI will get real in 2024—but its biggest impact might not be what you think https://t.co/ix9ayWl9kL 🤖

[Benjamin Trent @benwtrent]

Fast HNSW and 1/4 the memory cost? Yes please! New int8 quantization in Apache Lucene!

https://t.co/z1pJWnXO82

[Kent Fitch @kentfitch]

@MarkusMannheim Noi noi

[Massimo @Rainmaker1973]

Since its inauguration in 1964, approximately 6.4 billion people have used the Tokaido Shinkansen, aka the Bullet Train, with zero accidents and an average delay time of 0.9 minutes

[read more: https://t.co/ySlFfn8CXT]

[Joe Aston @mrjoeaston]

Latest on @Qantas… Chairman Richard Goyder’s last supper https://t.co/1YnwgJKouS

[Joe Aston @mrjoeaston]

Alan Joyce's parmesan waft trumps @Qantas chairman Richard Goyder's humility cologne: https://t.co/oER9k8DMxM

[Yann LeCun @ylecun]

@Noahpinion No.
The research arm of Bell Labs was never about moonshots.
It was about hiring the best scientists into small departments (typically 5 to 15 people) and giving them resources and a *lot* of freedom to work on what *they* deemed most promising.
That's how you get breakthroughs.