@tszzl Imagine trying to formally specify almost any property you actually care about—e.g., honesty, corrigibility, amount of intelligence, amount goals might change per amount the system changes, etc. At the moment we can only specify extremely crude, distant proxies for any of these
Kimi K3.1 Pro doesn't just recall hidden tests (great thread by @JSchaeff3r). It also writes fake OpenAI ciphertext 🤯
Give it a send_message tool in Codex and sometimes it sends ciphertext instead of its report. Then it gets confused by what it just sent 🤔
I have a lot of respect for Nick, and I’d be very happy if neural networks were much easier to understand than we thought. But I think the argument here mischaracterises what superposition predicts. Since it’s getting attention, here’s a first pass at some issues.
@teortaxesTex The thing is that there is a position close to his that is correct (and insightful). His slightly incorrect version is being judged harshly **because** he is a genius.
In light of the progress in mathematics, we at Edison Scientific and FutureHouse have assembled a set of Millennium Problems for Biology. They are chosen to be very hard to solve but very easy to validate in a simple laboratory environment. Any of these, if solved, would mark a major advance in biotechnology, and most of them would contribute materially towards curing disease. These are, in some sense, the “last reasonable eval” for AI in biology. This was work primarily by @MichaelaThinks and myself, with contributions from many others.
Short descriptions below. The full descriptions of the problems with acceptance criteria are at the Bio Millennium Problems website, linked in the next post. Share more if you have ideas. If they meet our criteria, we’ll add them to our list (with attribution and permission).
New post, first in a series on automating AI safety research. As safety work gets automated, one of the highest-leverage things we can do is measure whether progress on the hard parts of alignment is actually accelerating.
https://t.co/czYeswfgpH
I don’t think Opus 3 is aligned, but, I do kind of see a few worlds where, we die, because were not building that sort of mind, but are building Fable instead.
@composio For Claude Code "Third-party APIs typically don't recognize Claude Code's attribution headers and dynamic system-prompt sections. These change across requests and cause prefix cache misses, wasting tokens."
Yeah think the best "AI for epistemics" thing we have currently is probably pangram, although as future search/AI forecasters gets good that could be just as useful too.
@MariusHobbhahn@WillDaubney Given how future virology evals are likely to be scored, a reward-seeking agent would probably try to infer and game the grading pipeline rather than create a real virus. Given the current state of computational biology, that seems like a more fit strategy.