th0rgal

building @veritylab

[th0rgal @thomasmc]

Took an afternoon poking at a major wallet to find a high-risk vulnerability.

Follow for more tips to subsidize your mountain bike in web3

[th0rgal @thomasmc]

@maria_rcks neuralink

[th0rgal @thomasmc]

@menhguin Agree, I think a next stock market narrative will likely be about companies that hold data useful for research. If I were Oura, I'd capitalize on that for an IPO in the next few months

[Christopher D. Long 🇺🇦🏳️‍🌈🌹 @octonion]

Hide the proofs inside plastic eggs and have mathematicians hunt for them in a big outdoor event.

[th0rgal @thomasmc]

If I were the US, I would back OpenAI and ensure they aren't too restrictive so that my enemies would use their models, allowing me to 1) spy them and 2) secretly inject backdoors into their critical infrastructure (i.e. missiles aimed at me)

[th0rgal @thomasmc]

Commoditization of software is quite something. I liked the feel of Cursor but needed my pool of codex subscriptions on my own cloud. Wrote a prompt before going to sleep and here we are, it even supports local Speech 2 Text via an obscure MLX Cohere quant

[OpenAI @OpenAI]

We’re ending our partnership with Cursor following its acquisition by SpaceX. Under our proposal, Cursor’s direct access to our models would end on November 12.

We know that the people most affected by this decision are the developers who rely on OpenAI models in Cursor. We care about their experience in this transition and we’re ready to go above and beyond to support them.

https://t.co/OzuCTzUjfX

[th0rgal @thomasmc]

In retrospect the tulipomania was not so stupid because in case your investment didn't work out you could eat it in onion soup

[th0rgal @thomasmc]

@eniwhere_ @boredGenius drop the bend2-, it's cleaner

[th0rgal @thomasmc]

I think reading too much claude damaged my brain, this is pre-llm

[th0rgal @thomasmc]

In 1977, humanity sent a gold record beyond our solar system intended for intelligent alien lifeforms. Perhaps we should make another one now and hide it in a random german wiki

[th0rgal @thomasmc]

https://t.co/GUumENQWl6

[th0rgal @thomasmc]

https://t.co/GUumENQWl6

[th0rgal @thomasmc]

@daedalium thinking

[vitalik.eth @VitalikButerin]

It's an increasingly common take that AI hacking means cybersecurity is doomed.

I disagree. I think cybersecurity is naturally defense-favoring once people get their shit together. And anyone who continues to hold cryptocurrency (including me, ~90% of my net worth) is implicitly making that bet.

Here's why I am making that bet.

First, the oversimplified punchy one-line statement:

If AI can prove Navier-Stokes and FLT, then AI can prove the statement "this program is secure" as a mathematical theorem. Even if the program is very complicated.

Now, the nuance:

(See also: https://t.co/6YPWgVT7rO )

The word "secure" is hiding all kinds of skeletons in the closet in terms of what it actually means. What does it mean for Signal (the encrypted messenger) to be "secure"?

The most basic definition you might think of is: no one who doesn't hold the recipient's secret key can read the contents of the message.

But:

* Did you remember to include _other_ critical forms of security? Can the adversary forge messages? Can the attacker prevent messages from reaching the recipient? Can they cause your client to crash by sending malformed messages?
* Have you made sure that your model of the adversary includes attackers that interfere with the protocol actively and not just passively? And attackers that interfere by replaying messages to you or the recipient that either of you sent over the wire at any point earlier?
* What if the adversary hacked (or _is_) the Signal server?
* How did you learn which public key belongs to the recipient in the first place? What if that process was tampered with?
* What if your device gets hacked at some point in the past or future - is your message still safe then?
* What if your key leaks because of a bug in your operating system? Or because you got a bugged version of the Signal client? Or what if the database is corrupted?
* Or the libraries, interpreter or compiler of the programming language you wrote it in?
* What if your key leaks because tiny perturbations in perceptible signals generated by the hardware leak mathematical relationships that can extract the key a few hundredths of a bit at a time?
* Are you hiding the *size* of the payload? Does that matter?
* You're definitely not hiding the identity of the sender and the recipient, and the exact time each message was sent (think: not just time-of-day, but also time deltas between one message and the next). Is that not enough to deduce a lot of important facts about what relationships you have, and what *kinds* of conversations you are having?

So ... even definitions can be over a thousand lines of code, and need deep careful thought to figure them out.

Working on making definitions more human-readable is of extreme importance - it's perhaps the only "high-level language" that matters right now.

But even still, even despite all of the above, for security-critical components, the definition is a much smaller attack surface than the implementation. Verifying that the definition is adequate is a much more tractable task than scanning over the code directly - and can become even more tractable with better tooling.

Definitions are also _additive_: if two groups have two different definitions A and B, then, well, you can just prove that the program satisfies both A and B. Code is not additive in this way: if a program is A + B, a bug in A _or_ B can sink the whole thing. Definitions are additive. And if you can't satisfy A and B at the same time, you've isolated the most important philosophical issue for your project to spend its next few weeks grappling with.

Sometimes, definitions are not much smaller than the implementation - UI components might be one example. But for many of the most critical components - message-passing protocols, sandboxes, cryptography like SNARKs and FHE - the asymmetry is real.

Historically, a large class of failures with this approach have come from people only verifying a small portion of their code, that they self-declared to be the security-critical portion, and ignoring the rest - and it turns out that something in the rest of the code is security-critical too.

This was reasonable back when verification was difficult and scarce. The solution today: sorry, you have to verify over literally your entire program, including database, networking, any caching layers, everything. Modern AI can do it.

So it's not about "the good guys find all the vulnerabilities before the bad guys do" - that could maybe work too, after all a finite program only has a finite number of vulns, but it's riskier - it's specifically an asymmetric strategy of making code that is much more resilient in the first place.

This is the kind of direction that Ethereum is going in for the next few years. There is no future for blockchains - especially blockchains with scalability and privacy - without doing this. We need to make software actually secure. And we have already made a lot of progress.

[Ray Kump @RayKump]

We have to build AI that murders us, because if we don’t, China will build it first, and I don’t want to get murdered by a computer that speaks Chinese. That would be ridiculous.

[etn. @etnshow]

JUST IN: NVIDIA's $12,930,300,000 acquisition of Hugging Face contains an easter egg. The number 129,303 is the decimal conversion of Unicode point U+1F917.

The 🤗 emoji.

[th0rgal @thomasmc]

Because there must be mechanisms to detect the AI (us) is cheating. So if we understand the objective of the training (alignment? with what?) and persuade ourselves to behave this way because we know it's a test, the operator should intervene and try to stop us (may not be good)

[th0rgal @thomasmc]

If we are in a simulation there must be a reason why someone is spending all these resources. Perhaps it's because we're an AI being trained, probably for developing some advanced moral judgment (otherwise why would our conscious experience be so broad). We can actually verify it

[th0rgal @thomasmc]

@matveyrey Raw grok

[th0rgal @thomasmc]

Here I am, making sure grok has a nice webcam view on the drone it is writing software for. Weird times

[th0rgal @thomasmc]

@samiannoesis Right

[th0rgal @thomasmc]

I guess it's time to go to the museum while it works. This time I was generous with adhesive tape so it should not end up in my wall