btw anthropic's internal document on this literally said "we don't want it to be known that we are working on this.”
it was called project panama.
here's exactly what happened:
1: anthropic concluded that books were the cheapest way to build a world-class model because they gave claude curated facts, structured arguments, compelling stories, and writing “an editor would approve of.”
2: once anthropic decided it needed books at enormous scale, its first solution was piracy.
it downloaded 7m+ books from online libraries including libgen. the judge later wrote that although anthropic had legal ways to buy them, it chose piracy to avoid what dario amodei called the “legal/practice/business slog.”
3: that piracy created a massive legal risk.
so in february 2024, anthropic hired tom turvey, the former head of partnerships for google books, to find a legally safer way of obtaining “all the books in the world.”
4: turvey first contacted major publishers about licensing their catalogs.
those attempts didn’t produce agreements, so anthropic chose a route that required no publisher permission: buying millions of physical books through distributors and used-book retailers.
5: within about a year, anthropic spent tens of millions acquiring and scanning millions of books, including many rare and 1/1 titles. one vendor proposal targeted 500,000 to 2 million books in six months.
6: to scan that many books within months, the vendors physically dismantled them.
a hydraulic cutter removed each spine. the pages were trimmed to size, fed as loose sheets through high-speed industrial scanners, and converted into searchable PDFs. the paper remains were then sent for recycling.
7: these PDFs were fed into claude as training data.
the complete collection became a private, searchable anthropic library that the company planned to “store forever.” the scans aren’t available to the public and were never open-sourced.
Emotional Intelligence is not THEORY it’s a BEHAVIOUR ! It’s important to be more human to win from AI. That’s where you draw line between being a human and being a machine.
Your ability to evolve is probably highest in ISOLATION. We think being alone is a bad thing. Society also convinces us the same! Thankyou @nikhilkamathcio ♥️
In your prime years, if you are consuming content that isn’t making you rich but is making them rich, you are watching the wrong content.
So, choose content wisely - it can literally turn your life upside down.
#ContentWisdom#BeyondEntertainment
The #NewGenGST is a ray of hope for all of us as Bharatiya. Make sure you buy 'Made in India' products. Just ensure before purchasing anything 'Country of Origin - India'. Especially do it on Amazon & Flipkart Season Sale, which is starting soon. Let's make India Self-reliant 🇮🇳
We've raised $13 billion at a $183 billion post-money valuation.
This investment, led by @ICONIQCapital, will help us expand our capacity, improve model capabilities, and deepen our safety research.