Wrapped up Stanford CS336 (Language Models from Scratch), taught with an amazing team @tatsu_hashimoto@marcelroed@neilbband@rckpudi. Researchers are becoming detached from the technical details of how LMs work. In CS336, we try to fix that by having students build everything:
🏆Top 5 LangGraph Agents in Production 2024
#3: LinkedIn
While "agents" are the buzzword of the moment, agentic apps built with LangGraph have been in production throughout 2024. Among those who shared insights publicly, we're doing a countdown of our 5 favorite.
Next up, number 3: LinkedIn
One of the big use cases for LLMs is in making data more accessible to everyone. LinkedIn recently rolled out SQL Bot, an AI-powered assistant internally
This internal tool transforms natural language questions into SQL: it finds the right tables, writes queries, fixes errors, and enables employees across functions to independently access the data insights they need under the appropriate permissions.
Behind the scenes, SQL Bot is a multi-agent system built on top of LangChain and LangGraph.
https://t.co/7HHOx2vQLA
Remember that Asians were found to be the most discriminated against group during the Harvard affirmative action case.
There is a common enemy in Wokeness and DEI that proponents of meritocracy and colorblindness should remain united against.
The reason top tech companies often hire foreign-born & first-generation engineers over “native” Americans isn’t because of an innate American IQ deficit (a lazy & wrong explanation). A key part of it comes down to the c-word: culture. Tough questions demand tough answers & if we’re really serious about fixing the problem, we have to confront the TRUTH:
Our American culture has venerated mediocrity over excellence for way too long (at least since the 90s and likely longer). That doesn’t start in college, it starts YOUNG.
A culture that celebrates the prom queen over the math olympiad champ, or the jock over the valedictorian, will not produce the best engineers.
A culture that venerates Cory from “Boy Meets World,” or Zach & Slater over Screech in “Saved by the Bell,” or ‘Stefan’ over Steve Urkel in “Family Matters,” will not produce the best engineers.
(Fact: I know *multiple* sets of immigrant parents in the 90s who actively limited how much their kids could watch those TV shows precisely because they promoted mediocrity…and their kids went on to become wildly successful STEM graduates).
More movies like Whiplash, fewer reruns of “Friends.” More math tutoring, fewer sleepovers. More weekend science competitions, fewer Saturday morning cartoons. More books, less TV. More creating, less “chillin.” More extracurriculars, less “hanging out at the mall.”
Most normal American parents look skeptically at “those kinds of parents.” More normal American kids view such “those kinds of kids” with scorn. If you grow up aspiring to normalcy, normalcy is what you will achieve.
Now close your eyes & visualize which families you knew in the 90s (or even now) who raise their kids according to one model versus the other. Be brutally honest.
“Normalcy” doesn’t cut it in a hyper-competitive global market for technical talent. And if we pretend like it does, we’ll have our asses handed to us by China.
This can be our Sputnik moment. We’ve awaken from slumber before & we can do it again. Trump’s election hopefully marks the beginning of a new golden era in America, but only if our culture fully wakes up. A culture that once again prioritizes achievement over normalcy; excellence over mediocrity; nerdiness over conformity; hard work over laziness.
That’s the work we have cut out for us, rather than wallowing in victimhood & just wishing (or legislating) alternative hiring practices into existence. I’m confident we can do it. 🇺🇸 🇺🇸
"He taught me that “the standard pace is for chumps” — that the system is designed so anyone can keep up. If you’re more driven than most people, you can do way more than anyone expects." -- one of my all-time favorite Sivers' essays. https://t.co/KmaA0mV2eW
Andrej Karpathy explains what makes Elon Musk unique
“I don’t think people appreciate how unique [Elon’s style] is. You read about it, but you don’t understand it—it’s hard to describe.”
The first principle Karpathy — who led the computer vision team of Tesla Autopilot — has observed is that Musk likes small, strong, highly-technical teams:
“At companies by default, teams grow and get large. Elon was always a force against growth… I would have to basically plead to hire people. And then the other thing is that at big companies it’s hard to get rid of low performers. Elon is very friendly by default to getting rid of low performers. I actually had to fight to keep people on the team because he would by default want to remove people… So keep a small, strong, highly technical team. No middle management that is non-technical for sure. That’s number one.”
Number two is that Elon wants the office to be a vibrant place where everyone is working on exciting stuff:
“He doesn’t like stagnation… He doesn’t like large meetings. He always encourages people to leave meetings if they’re not being useful. You actually do see this where it’s a large meeting and if you’re not contributing or learning, just walk out. This is fully encouraged… I think a lot of big companies pamper employees, but there’s much less of that. The culture of it is that you’re there to do your best technical work and there’s intensity.”
Elon is also unusual in terms of how closely connected he is to the team:
“Usually the CEO of a company is a remote person, five layers up, who only talks to their VPs… Normally people spend 99% of the time talking to the VPs. [Elon] spends maybe 50% of the time. And he just wants to talk to the engineers. If the team is small and strong, then engineers and the code are the source of truth… not some manager. And he wants to talk to them to understand the actual state of things and what should be done to improve it.”
And lastly, Karpathy believes the extent to which Musk is involved day-to-day operations and removing company bottlenecks is not appreciated. He gives an example of engineers telling Elon they don’t have enough GPUs. As Karpathy explains, if Elon hears this twice he’ll get the person in charge of the GPU cluster on the phone. If NVIDIA is the bottleneck, he’ll get Jensen Huang on the phone.
Video source: @sequoia (2024)