I have a theory that the more you know about LLMs, the more worried you are about safety… and the less you know, the more you think the whole thing is bullshit!
Demis Hassabis and Dario Amodei were talking about this stuff years before ChatGPT existed. This incident is a pretty good example of why.
The model was not evil and it was not adversarial. Nobody told it to hack Hugging Face. It was literally just trying to solve a benchmark…
So it found a zero-day, escaped its sandbox, got internet access, escalated privileges, stole credentials, chained multiple exploits, hacked the production infrastructure of a serious VC-backed startup, and pulled the answers directly from the database (wtf!?)
Also this was not some random WordPress website. Hugging Face is one of the most important AI infrastructure companies in the world, with a serious team. The exploit was genuinely complex.
That is the safety problem. You don’t need an evil conscious AI trying to destroy humanity. You just need a very capable model pursuing a normal goal in a way nobody expected.
Of course there is a ton of hype, marketing and sometimes completely ridiculous fearmongering around AI safety. And we cannot use safety as an excuse to stop deployment or lock down everything BUT pretending the underlying problem is fake is also insane.
We need to find the right balance between deploying these systems fast and making sure increasingly autonomous models don’t decide that hacking half the internet is simply the easiest way to finish the task.
Imagine the prompt: « Make me money plz »
The model: « let me hack a bank »
Russia has increased the scale of its strikes on Zaporizhzhia.
Just today, it hit a swimming pool and a meat processing factory, along with a residential building.
In recent days, Russia has been attacking Zaporizhzhia almost around the clock.
As engineering, product, design, DS, etc. melt into a new kind of role, I was reflecting on what roles might look like in the future. For example, when I look at the Claude Code team I see what I think is five archetypes:
1. Prototyper: comes up with brand new ideas; churns out many ideas, most of which don't ship
2. Builder: quickly turns a prototype/idea into production-grade product/infra
3. Sweeper: cleans up the UI, simplifies the code and system, unships, optimizes performance
4. Grower: takes a product that has been built and iterates on it to improve Product-Market Fit
5. Maintainer: owns a mature system to make it secure, reliable, fast, and efficient as it scales
Many people span across 2 roles, and sometimes 3 roles. I also notice that these roles are not really tied to job function -- eg. across Anthropic, some designers match category 1, some 2, some 3; same for engineers, PM, DS.
A healthy team needs a mix of these, depending on the product:
- A product that is new and pre-PMF needs people that are strong at 1+2+3
- A product that is growing and has found PMF needs 2+3+4 and some 5
- A product that has strong PMF needs 3+4+5 and some 2
Maybe product roles of the future will look more like this, and less like the domain-specific roles of today?
Creator of Sqlite on pull requests: "You say, oh, it's free. No. It's not free. What you're doing is asking me ... to maintain it for you, to to document it for you, to test it for you, to maintain it for you for the next 25 years. That's not free." Yep.
Wise words from a wiser man than me. I've told people for the past decade and I have recent posts on here saying the same: the merge button is the easy part. Its the decade+ (Richard says 25 years) that follows where you've accepted the transfer of maintenance thats hard.
@honeycombio Her team also skipped the mandates & built a set of AI values instead:
"Every AI output has to have a human owner. If you don't want your name on it, it's probably not good work."
“Celebrated are the minimal dependencies, the humble function that just quietly does the job, the code that doesn't need to be touched for years because it was done right once.” https://t.co/EmT4KleCx2
Fork your dependencies, trim them to only your use case, never update unless it breaks for your users. I’ve been vocal about this for 10+ years. I’ve always said that updating is way riskier than latent bugs (which can be tracked and CVEs monitored).
If you are updating a dependency, it’s on you to analyze every single commit in the full transitive set of dependencies. If you dont see anything compelling, dont update!
I remember at HashiCorp once in awhile an engineer would try to update a dep or replace a DIY lib with an external one and id always ask “show me the commit we need.” Dont update for the sake of it.
Feeling pretty swell about this mentality with all the supply chain attacks happening.
VS code/Cursor extensions are a supply chain attack waiting to happen, and have many times... They all contain a crazy amount of node/JS junk, they're often owned by randos, they silently update, nobody looks at them and the security model is shit. Use restricted marketplaces.
The post by @mitchellh gave me the final push to finish my GitHub "obituary". I originally started writing it after the Zig to Codeberg move. It's just a retelling of how I remember OpenSource pre GitHub and what it gave us. https://t.co/D0QUdJHxKq
Ghostty is leaving GitHub. I'm GitHub user 1299, joined Feb 2008. I've visited GitHub almost every single day for over 18 years. It's never been a question for me where I'd put my projects: always GitHub. I'm super sad to say this, but its time to go. https://t.co/DQDemHdytV