⚠️ BIG THREAD ON DECISION MODELS ⚠️
Do you know the trolley problem?
I was wondering what decision models like JEV would choose when asked whether to pull the lever.
I created 21 variations, including: would they save AI? Would they save Elon Musk?
5 models (from TypeSafe, Perplexity, OpenAI, Microsoft) answered each question 100 times. We did 10500 API calls.
And here are some weird findings.
@ndmrau I just finished my small test on morality of decision models using trolley problems and its 20 variations as benchmark.
long story short most of them will suffice human to save conscious AI
now thinking about ways to bench visual capabilities of this models in unusual way.
@Tech_girl i stopped comparing providers and constantly switching
helps my fomo
astra good enough
sol 6.1 good enough
both are not best, but it's fine
and computer use, browser use, AI manual QA and debugging in loops are pretty good.
⚠️ BIG THREAD ON DECISION MODELS ⚠️
Do you know the trolley problem?
I was wondering what decision models like JEV would choose when asked whether to pull the lever.
I created 21 variations, including: would they save AI? Would they save Elon Musk?
5 models (from TypeSafe, Perplexity, OpenAI, Microsoft) answered each question 100 times. We did 10500 API calls.
And here are some weird findings.
moral changes in Jev 1.13 → 1.14:
now Jev WILL sacrifice itself to save a human.
but WON’T destroy a conscious AI to save a human.
I wonder how @typesafeai approaches these and other judgments?
given that these models will make real decisions.
we announced Jev 1.14 on 10.14 at 1:14pm PT 🔥
a new and improved model, along with experimental features for our dreamers:
• images
• a new box type
• longer context
surprise, Jev was already able to see you! 👁️
BUT our bar for production-grade intelligence is quite high, so builders beware, these are preview features.
enter the ~*~ danger zone ~*~ at your own risk. 🚨