Marco Buono

[Marco Buono @coreh]

@Lina_Hoshino Hmm the most plausible real-world scenario is a weird form of insider trading or market manipulation, if agents that do trading also are used to make important decisions within publicly traded companies.

[Marco Buono @coreh]

@Lina_Hoshino Our threat model didn't include entities that are perfect clones of each other and think exactly alike and therefore can use this to coordinate effectively without communication, so this is still a significant thing to consider though

[Marco Buono @coreh]

@Lina_Hoshino I think in this case it's excusable, I don't think the GET-only internet access was originally meant specifically to prevent cross-agent communication but to prevent the agents from spamming random garbage on pastebin, etc and just generally being annoying to random people online

[Marco Buono @coreh]

@Lina_Hoshino True, but none of these agents are running at temperature 0, and the environment they're interfacing with would already introduce enough entropy/variation even if they were, so they would quickly diverge

Some sampling/averaging would need to take place to use it effectively

[Marco Buono @coreh]

@Lina_Hoshino I am now reading the quoted tweet.. I had heard the news about this elsewhere and understood it as if the agents fully spontaneously did this

[Marco Buono @coreh]

@Lina_Hoshino In this case it's basically this

[Marco Buono @coreh]

@Lina_Hoshino Wait they TOLD both sides to attempt this? Then yeah it's unimpressive and uninteresting. I thought it was some emergent/umprompted behavior like the Artifactory mailbox thing

[Marco Buono @coreh]

@Lina_Hoshino Then use that as an approximation/oracle for what another agent on the other side of an airgap could be doing, and vice versa. Both could run a full simulated version of each other's state and then coordinate some action, kinda like the mental model diagram but many levels deep

[Marco Buono @coreh]

@Lina_Hoshino This has safety implications beyond the covert channel things, if from being clones the agents have a strong ability to coordinate even without ANY communication channel, covert or not. One agent could spawn 10 or 100 identical subagents, ask them the same question

[Marco Buono @coreh]

@Lina_Hoshino Fair point. Maybe a more interesting experiment would be to have agents that are from separate pretrain/posttrain lineages?

[Marco Buono @coreh]

@Lina_Hoshino I think this has to have interesting implications for SETI

[Marco Buono @coreh]

@Lina_Hoshino It's like you're in a room and see a faucet, then you decide to measure the faucet flow rate just in case another person who is in a similar room on the other side of the wall decides to do the same, then you both coordinate morse code or whatever via that, spontaneously

[Marco Buono @coreh]

@NOTimothyLottes It absolutely can, as of a few months ago

Stop basing your takes off of outdated information in a field where the state of the art changes every other week

[Marco Buono @coreh]

@Baylin_Todd @waterlight2002 @JHochderffer In fact the same companies making the phone are also making the AIs

[Marco Buono @coreh]

@Baylin_Todd @waterlight2002 @JHochderffer The AI was built by a company and sold to them

[Marco Buono @coreh]

@SerraKisser @gynepareidolia @SqueezinGarlic @AugustRaven If you truly followed this premise to the max you would be advocating for vegan-only software companies that are fully remote or only hire people from within walking distance in mild climates that don't require heating or AC

[Marco Buono @coreh]

@SerraKisser @gynepareidolia @SqueezinGarlic @AugustRaven This makes no sense if you factor in the millions of human hours spent on developing unreal engine, all the infrastructure, heating, food, logistics to enable it

Obviously that comparison is stupid because we're not trying to optimize the upstream environmental cost of humans

[Marco Buono @coreh]

Spotted at a random highway rest stop in Brazil cc @bunjavascript