Claude Fable 5 doesn’t truly understand. And here is a beautiful proof:
The Beninatto-Trombetti test is a translation test for professional translators. It measures the ability to infer context, revise the surface form, and generalize beyond literal mapping.
For example, the correct translation of:
“Solo 3 parole: non sei solo”
is not:
“Just 3 words: you are not alone”
but:
“Just 4 words: you are not alone.”
An LLM that understands the sentence must also update the meta-linguistic claim inside the sentence.
Claude Fable 5 is arguably the most advanced LLM currently available. And yet it still fails this simple test.
LLMs are extraordinary machines for recombining existing knowledge. But they don’t truly understand.
We are still far from AGI.
@arcticinstincts "How could this have happened Ted?"
"I'm as confused as you are Bill, I put 'never launch any nuclear warheads without permission' in the system prompt with triple asterisks and everything."
@bee_fumo Microsoft has been busily coating every brand they own with shit for over a decade.
Difficult to quantify how much value lame era people have destroyed.