@engineering The code didn’t consider content diversity (except author diversity so we don’t only see from the same author). When I started to get interested in a topic, almost every feed is about that particular content.
DeepSeek (Chinese AI co) making it look easy today with an open weights release of a frontier-grade LLM trained on a joke of a budget (2048 GPUs for 2 months, $6M).
For reference, this level of capability is supposed to require clusters of closer to 16K GPUs, the ones being brought up today are more around 100K GPUs. E.g. Llama 3 405B used 30.8M GPU-hours, while DeepSeek-V3 looks to be a stronger model at only 2.8M GPU-hours (~11X less compute). If the model also passes vibe checks (e.g. LLM arena rankings are ongoing, my few quick tests went well so far) it will be a highly impressive display of research and engineering under resource constraints.
Does this mean you don't need large GPU clusters for frontier LLMs? No but you have to ensure that you're not wasteful with what you have, and this looks like a nice demonstration that there's still a lot to get through with both data and algorithms.
Very nice & detailed tech report too, reading through.
I was an exec at @mozilla when @Google decided that the browser was too critical to be run by a nonprofit they don’t control. Sounds a lot like @Microsoft and @OpenAI today. This is an unstable structure at its core. Hope all my friends over there are doing ok
I’m the only candidate with the guts to say it, but here’s the TRUTH: vote *NO* on the $106BN foreign aid warfest. We’re drowning in a $33 trillion national debt problem, caused in part by $6.5 trillion wasted in the Iraq and Afghanistan wars. The cost of funding the Ukraine war won’t be limited to the next $61 billion infusion that Biden proposes now but *serial* further involvement in a war against Russia that does not advance U.S. interests & increases the risk of major conflict with a nuclear power, at a time when we should instead focus on negotiating a reasonable path to peace in Ukraine while weakening the Russia-China alliance that represents a true threat to U.S. interests.
Meanwhile, Israel is barreling ahead to a potentially catastrophic ground invasion of Gaza without clear objectives. "Destroy Hamas” is not on its own a viable or coherent strategy. Instead, Israel is likely to invite attacks from Hezbollah in the north, which would almost surely draw the U.S. into a prolonged all-out regional war in the Middle East that won’t advance American interests. The U.S. should be clear with Israel that further U.S. support is contingent on Israel identifying clear objectives for success in Gaza and a coherent plan for what comes after toppling Hamas, even if Israel is successful in doing so. Right now these critical questions remain unanswered.
Biden’s pathetic gambit to conflate Ukraine, Israel, and U.S. border funding requests is a ploy to evade debate on the merits at a time when we need open debate more than ever. Republicans shouldn’t fall for it. I call on lawmakers in both parties to vote NO on this disastrous proposal.
StackOverflow is laying off 28% of its workforce.
This may be the first large layoff directly due to AI:
> people asking ChatGPT instead of StackOverflow
> usage & ad revenue declines
> having to lay people off to stay profitable / survive