Chris

Trying to go to bed a little wiser each day

[Ole Lehmann @itsolelehmann]

btw anthropic's internal document on this literally said "we don't want it to be known that we are working on this.”

it was called project panama.

here's exactly what happened:

1: anthropic concluded that books were the cheapest way to build a world-class model because they gave claude curated facts, structured arguments, compelling stories, and writing “an editor would approve of.”

2: once anthropic decided it needed books at enormous scale, its first solution was piracy.

it downloaded 7m+ books from online libraries including libgen. the judge later wrote that although anthropic had legal ways to buy them, it chose piracy to avoid what dario amodei called the “legal/practice/business slog.”

3: that piracy created a massive legal risk.

so in february 2024, anthropic hired tom turvey, the former head of partnerships for google books, to find a legally safer way of obtaining “all the books in the world.”

4: turvey first contacted major publishers about licensing their catalogs.

those attempts didn’t produce agreements, so anthropic chose a route that required no publisher permission: buying millions of physical books through distributors and used-book retailers.

5: within about a year, anthropic spent tens of millions acquiring and scanning millions of books, including many rare and 1/1 titles. one vendor proposal targeted 500,000 to 2 million books in six months.

6: to scan that many books within months, the vendors physically dismantled them.

a hydraulic cutter removed each spine. the pages were trimmed to size, fed as loose sheets through high-speed industrial scanners, and converted into searchable PDFs. the paper remains were then sent for recycling.

7: these PDFs were fed into claude as training data.

the complete collection became a private, searchable anthropic library that the company planned to “store forever.” the scans aren’t available to the public and were never open-sourced.

[Addy Osmani @addyosmani]

The future of Software Engineering isn't syntax, but what was always the real work: turning ambiguity into clarity, designing context that makes good outcomes inevitable, and judging what truly matters.

"The entire history of software engineering is one of rising levels of abstraction" - @Grady_Booch

[Joe Armstrong @joeerl]

Should also add that all significant energy gains in the last 50 odd years are result of new hardware NOT software.

[BURKOV @burkov]

To make it super clear, language models are awesome. Otherwise, I wouldn't spend 9 months of my life working hours after my day job on writing a book on them. What's wrong about them is not what they can do, but what the lying CEOs, VCs, and parasite influencers say they are or will be able to do.

This is what they are awesome at:

1. Giving answers that are more important now with some chance of error than perfect but tomorrow
2. Interactive problem-solving, where the user is an expert and could solve the problem alone, but it would take more time. This includes theorem proving, math problem solving, coding, and technical writing.
3. Converting it into a temporary "You are a classifier that can distinguish between these C classes" model that helps accelerate a complex system development and which will later be replaced with a real classifier.
4. "You act as an expert in domain A. Here's a document from domain A. Extract from it attributes B and C verbatim so that I can automatically locate them for verification."
5. Converting between programming languages, JSON, XML, YAML, or between different API specification formats.
6. "Improve my writing so that it fits in this context."
7. "Write 3 most important points of this long online article."
8. "Translate this text from language A to language B."
9. "Write code according to this specification so that all of my hidden tests pass."
10. "My code fails, here's the stack trace. Fix it."
11. "You are an expert in domain A. Generate examples of documents and their labels, to be validated by a human."
12. "Here's the solution (code, document) provided by a human. It does contain errors. Find them."
13. "Here's a scientific article. What does X in equation 2 represent and where does it come from?"
14. "Here's the code. What does function A do and why is this specific command used?"

Also, use cases where hallucination is a feature:

15. Storytelling, poetry, scriptwriting
16. Brainstorming and ideation
17. Roleplaying (in all senses)

All these use cases have been available since GPT-3.5, and nothing new was added. Only solution quality has been gradually improving without ever reaching perfection.

[John Burn-Murdoch @jburnmurdoch]

I just can’t get past Labour winning 65% of seats off 34% of the vote.

Absolutely wild mismatch between the headline result and, well, everything else.

Britain is now a multi-party system, and first-past-the-post can’t cope.

[Daniel Litt @littmath]

gpt-6 will have the intelligence of someone who decided not to get a PhD

[Artur S. @asdf_101]

@antirez OSS for the win 😵

[Paul Graham @paulg]

We've already used 17% of 2024.

[Kevin Patrick Murphy @sirbayes]

While GenAI is fun, I think its economic value is grossly over estimated, because it’s unreliable, risky and expensive to make and serve. It’s fine for creative tasks, but not (yet) autonomous agents

[François Chollet @fchollet]

The skills needed for developing new ML techniques have little overlap with the skills needed for applying ML effectively. A bit like how chip design has little overlap with software engineering.

[François Chollet @fchollet]

When you're young, you think that you're invulnerable and that you have infinite time ahead of you. Truth is, you're not, and you don't.

[Victoria @VictoriqueM]

This is both funny as fuck and sad as fuck

[Chris @cpark_x]

@pragprog (6 * 4) + (4 + 3)

[Chris @cpark_x]

@Natalie_Berg https://t.co/FSHpORhxI9 inj m i.imoimoloolo

[Chris @cpark_x]

@matloff Congratulations - I wish you all the very best as you embark on your full-time research + writing career. I've learned a lot from your books over the years and look forward to (hopefully) many more.

[Chris @cpark_x]

@predict_addict can you add a link to your post?

[Chris @cpark_x]

@andrewhollingw There's a paid chart-only newsletter by a chap who used to work for WSJ - I forget what it's called but you'd like it.

[Chris @cpark_x]

@mayowaoshin @ian__ito @langchain @pinecone @OpenAI It was more from the perspective of using it on sensitive PDF documents - when you tried open source models did you get adequate performance/usability?

[Chris @cpark_x]

@ian__ito @mayowaoshin @langchain @pinecone @OpenAI Have you tried using open source models for embeddings at all?