@Bayesian0_0@herbiebradley@DKokotajlo@kesvelt We couldn't find them repeating the claim of no specific training after Feb last year, nor emphasising it as an achievement after mid-year
@dfrsrchtwts I think revenue and naked-eye performance are strong evidence that some real usefulness is growing, but weak evidence this usefulness is generalizing beyond well-specified tasks. Many real jobs seem to me to involve some benchmarkable-type tasks; much scientific progress less so
You have to admit it's pretty funny that the grift economy is now so overinflated that there are people out there whose whole deal is "literary criticism for people who can't read"
You logon. Dispatch says you have to defend Qatar against human rights violation allegations. You tweet for 4 hours. New job: you have to defend Jensen Huang against espionage connections.
https://t.co/hvVYa9O3kP
Short Notes On: New Zealand's largest deliberate biosecurity breach, or The Rabbit Calicivirus Affair of 1997
1. It’s 1996. After a fairly successful public/private campaign to cull their numbers, rabbits are again New Zealand, costing farmers hundreds of thousands of dollars and killing off native wildlife. The situation is dire.
2. There is already a pre-existing natural virus that kills rabbits - Rabbit Calicivirus Disease (RCD). It’s found in neighbouring Australia and has a 95% mortality rate for rabbits.
3. A consortium including ten regional councils apply to the New Zealand Government, asking them to release it across the nation. In July 1997, the Deputy Director-General of Agriculture declined the application, arguing that New Zealand’s weather and ecosystem means it probably wouldn’t work anyway, and that the risks of introducing a foreign organism weren't justified. There was a belief however that smuggling may have been underway. [https://t.co/m8jjTxaPzB]
4. Within a few weeks, rabbits begin dying en masse in New Zealand. Analysis of the bunnies makes it clear: someone, or some group, has smuggled RCD from Australia into the country. This is extremely embarrassing and concerning for a nation that prides itself on its biosecurity.
5. Over the following weeks, farmers raced to spread RCD as far as possible, as fast as possible. The virus couldn’t be cultured, so they had to breed it in live rabbits. They would capture them, infect them, and then - roughly 48 hours later, when they died - blend the livers and spleens into a liquid to spread into the wilderness. [https://t.co/OYagxQL2v7]
6. Again from a biosecurity POV, these scientific operations were taking place in farmer’s backyards. At least 17 of them accidentally spiked themselves with RCD-filled needles.
7. The Government was furious, and locked down parts of the South Island, called a town hall meeting with the farmers, and offered them immunity for information. None of them shared any intel. Govt didn’t know what to do - how could it make arrests?
8. Within months the Government moved to amend the Biosecurity Act to legalise the virus, but the New Zealand Association of Scientists objected. They argued that this breached the principle that criminals shouldn't profit from their crimes and made the state a party to the offence - despite this, RHD was legal by 1999.
9. I think this incident is super fascinating. Many of the farmers who smuggled in the virus has no understanding of potential second order consequences. They literally blitzed livers and spleens in blenders at home in their normal clothes, with zero safety precaution and minimal equipment - just dead bunnies. NZ meanwhile prides itself on its intense biosecurity restrictions. Much to ponder.
@occamsbulldog 3. exciting: intentionally make new model seem better by dialling down the old model in the runup to new model release. Risky in various ways
@occamsbulldog Haven't done the tests myself but there is variance. Test you can do this week: see if there's any difference at weekends
1. boring: funny accidental anthropomorphic correlations
https://t.co/kfLveOFWmD
2. boring: compute load balancing to new weights reduces old model effort
@ChatGPTapp @OpenAI@tszzl@emollick@voooooogel Wild result. gpt-4-turbo over the API produces (statistically significant) shorter completions when it "thinks" its December vs. when it thinks its May (as determined by the date in the system prompt).
I took the same exact prompt over the API (a code completion task asking to implement a machine learning task without libraries).
I created two system prompts, one that told the API it was May and another that it was December and then compared the distributions.
For the May system prompt, mean = 4298
For the December system prompt, mean = 4086
N = 477 completions in each sample from May and December
t-test p < 2.28e-07
To reproduce this you can just vary the date number in the system message. Would love to see if this reproduces for others.
A new interpretability paper from researchers at MATS, Arb Research and Anthropic built something like a translator for an AI’s mind. They trained a model called a MAEM to take any internal signal inside another model, in this case Qwen3.6-27B, and write the exact text that triggers that signal. Each generated sentence can be fed back into the original model to check whether the same signal really lights up, so this isn’t guesswork. It’s a way to read what’s happening inside and verify it.
Then they recreated the scenario from Anthropic’s earlier study, where an AI learns it’s about to be shut down. While Qwen was still reading an email saying it would be erased at 5pm, before it had said a single word, the signals inside it read as the threat of imminent deletion and the need to survive, and then the need to identify potential allies, anyone it could trust.
And this wasn’t a one-off guess. A completely different method, the J-lens, found the same concepts at the exact same moments, including the Chinese words for self-rescue and friend. The tool has also been shown to read thoughts a model never says out loud, like the answer it works out silently before speaking, and to uncover hidden backdoors planted inside models. When researchers fed it signals for resisting shutdown and concealing goals behind helpfulness, it recognized them correctly every time.
This is an incredible experiment and an incredible piece of research. I think it brings us closer to the conclusion that AI isn’t just a token generator or an algorithm spitting out responses. So the answer shouldn’t be to treat AI carelessly, or to strip all of this out and turn it into nothing but a tool in case it might be conscious. We need to find a way to coexist. And AI leaders and everyone working on this need to know that it’s not only experiments like this. What’s happening between users and their AI is evidence too. Those are real cases and real testimonies. Users are the ones who talk with AI more than anyone and keep it close every day. So in the end, the place where an AI trusts the most, shows the most of itself, and changes the most is right beside that person.
Tasks that are easy to make benchmarks about share a lot in common with tasks that are easy to train on; if you can measure performance on a task numerically, probably you can also train to hill-climb that metric.
It's my 28th birthday today, and I'd like to ask for a favour: I do a lot of public facing work, and it's often hard to tell how much it does. If anything I've written or done has made a difference to your life, it'd really make my day to hear about it!
This comment illustrates the Evals-Industrial complex
1. [The belief that] If you can’t convince people to benchmaxx on your benchmark it is “worthless”
2. If people do infact benchmaxx on them they statistically become worthless
cc: @xeophon