Speedrunning GPT-2 is now routine thanks to @karpathy.
But can we speedrun GPT3-175B?
We attempted to match accuracy on a <$10K budget; while we didn't quite reach it, our first results show that quality data, engineering, and native FP4 can get close.
Details in ๐งต
๐จ AI is in legal hot water.
Lawsuits over copyrighted training data are mounting โ and content owners are pulling out fast.
Top opt-outs?
๐ฐ News & Media
๐ฌ Science & Tech
๐ฅ Health Info
But hereโs the thing:
How much do those datasets actually matter for model performance?
๐งต๐