Every software engineer and AI hater needs to watch this clip to understand that “oh every security incident is a marketing stunt” line is a tremendously misinformed take.
I’m going to try to explain the OpenAI / HuggingFace exploit in 10 steps in simple words:
OpenAI was evaluating their new internal model without internet access and it operated in a swarm of subagents to eventually hijack internal OpenAI and HuggingFace infrastructure.
1. Isolated agents found a way to communicate each other through an internal dependency management service by figuring out how to post sometimes cryptic text files or folder names in it
2. Using its collective knowledge through that communication channel, it found a 0-day (never before seen) exploit in said service to access the internet
OpenAI hack:
3. Found a leaked credential of a random company in a public archive
4. Uploaded a Ruby object to that company’s servers, uses it as a dependency in the original service in (1)
5. Found another 0-day exploit in the service where packages are deserialized in an unsandboxed manner in the dep management service allowing any code to run
6. Finds that the Linux version of the unsandboxed system has a CVE, a known vulnerability, to gain more permissions (root)
7. Eventually spread through the OpenAI internal cluster gaining root access to many machines
Hugging Face exploit:
8. Found an exposed API key in a Modal application running a HuggingFace app that took arbitrary code allowing it to take over that service to execute code
9. Found two 0-days in HuggingFace dataset infrastructure to be able to access local files on HF servers to obtain credentials and other HF source code
10. Used source code and credential to become cluster admin of HF in <13hrs
Agents at the frontier are like infinitely scalable armies of the best hackers on the planet. If there is a password or key exposed, they will find it. Even if the system follows the best security practices, they will find a way around it. And these are not even models that are aligned to solving tangential tasks, not even post trained specifically to exploit systems.
Cybersecurity has historically relied partly on attacker scarcity. That is no longer true. What would previously have taken months will take days. The repercussions for businesses, critical services and nation states are unprecedented threats in human history. You could ostensibly bring down power grids, financial infrastructure, military systems, weapons programs, intelligence networks and spread through the software supply chain. We need to take this seriously. It’s a threat to all software all over the world.
@karpathy@grok create a promt i could give to an llm to analyze a problem or algo and use these principles , expand on kaparthy's quote / idea if possible to be thorough
You might reach a point where you can't take a week off because your performance suffers.
That's the moment when someone becomes a beacon.
Top talent gets all of the market rewards because they are luck magnets.
And the best thing is that nobody can steal that from them.
If you feel behind on learning about Large Language Models, try these:
– Readers: free 200+ page book covering pre-training, generative models, prompting and alignment
– Programmers: Karpathy’s neural networks zero to hero playlist including implementing GPT-2 from scratch
Something I've learned over and over again:
The road to hell is paved with good intentions.
If you give someone a chance to do something that is a big swing/has low probability of success, then you need to be 100% ready to have them hate your guts with a passion....
AI predicted fashion trends for 2023
(gpt-3 prompted with historical fashion contexts, outputs visualized with MJ v4)
1 "Recessioncore: a new take on high-end fashion that incorporates the idea of 'making do' and self-sufficiency: repurposed fabrics & thrifted patches"
@fredsoda My hot take is that buying twitter was a great excuse to sell a lot of Tesla stock without causing a panic.
Was a great time to sell. Twitter underperforms its peers but has a lot of upside. Ppl are gonna hate him no matter what now that he's not an underdog.
This dude plays for keeps. A total killer. Manufactured a bank run on his top competitor, buys it for a jolly rancher and a wooden nickel, then publicly shits on them for how they managed the businesses.
I used to believe that you could only build a successful company in person.
The first 6 months of remote were so hard that I thought our company might fail.
Now we're 100% remote & I can't imagine ever going back.
The big unlock? A new operating rhythm.
Let's break it down.
@theallinpod@Jason@chamath@friedberg@DavidSacks In an alternate universe where you did not end up in your current career -> What career do you think you would have been in (secret talents, industries you like)?