First enterprise benchmark that explores world modeling for enterprise tasks. Also the first to have tasks with a lot of hidden workflows that can throw off the agent and tests true enterprise reliability for AI agents. Check it out.
Can we trust AI agents with critical enterprise tasks? Absolutely not.
Introducing Wow (World of Workflows), the first Agentic Safety benchmark that proves that frontier LLMs fail miserably under safety constraints at enterprise tasks.
🧵 WoW demonstrates that LLM agents are “dynamically blind”. They fail to track the downstream ripple effects of their actions against complex enterprise rule sets. In an enterprise, that’s a safety and compliance hazard.
Our research shows how the future of enterprise AI requires proactive agent architectures and Wow is just a starting point.
📌 It’s now available to all researchers at: https://t.co/qIcT5kRzhU
Full blog here: https://t.co/n1cdJdzp9v
The first real evidence that the days of LLM Scaling laws are over.
Introducing SCOPE: the world's most efficient Neural Planner. 🔍📊
We tested SCOPE vs Frontier LLMs for planning tasks on TextCraft (text version of Minecraft) and here are the results: ⤵️
- SCOPE Runs 55x faster than GPT 3.5 (3 seconds vs 164 seconds)
- SCOPE is 160,000 smaller than GPT 4o (11M parameters vs 1.8T parameters)
- SCOPE is more accurate on Planning tasks (56%) than frontier LLM models
The age of efficient AI models starts now.
🔗📌 Read the full write up here: https://t.co/n3DDSqshRm
In all the world modeling papers I've read (whether that is JEPA, Dreamer and many more), most approaches try to model world dynamics in a purely neural way. This paper's approach on neurosymbolic seems pretty novel to me from a world modeling perspective.
Can LLMs be world models in Enterprise?
We actually tested it: and the answer is no.
While everyone was playing MAPs, we were secretly running something else inside it.
WALL-E, a LLM based world model, failed instantly.
So today we’re launching CASSANDRA: the first causal world model built for business decision making.
Research blog -> https://t.co/kReoPn2cHY
@skyfallai Interesting and somewhat expected to see that a purely neural model would fail to capture the environment's complexity. I like the idea of offloading the deterministic environmental elements via code-grounding and a casual network for stochasticity.
The first real evidence that LLMs cannot operate businesses, not even tiny ones.
At Skyfall, we’re building toward Enterprise Super Intelligence: AI systems that can one day run entire enterprises, coordinate teams and make long-horizon decisions, just like a CEO.
To get there, we need to understand what AI cannot do yet.
So we built MAPs, a theme park business with real operational constraints and let GPT5 run it.
Here’s what happened: (spoiler: it fails miserably)
@skyfallai This is great. The next 5 years are going to be very fun with increasing research focus like this on AI for long-horizon business planning. Waiting for AI CEOs to be a thing we consider normal one day 😼
Calling all tech enthusiasts: We’re giving away three AMD Ryzen 7800X3D CPUs to three lucky followers. For a chance to win:
- You must be following our account.
- RT & like this post.
- Comment “7800X3D” below.
Entries close on 06/03/2025.
Winners announced on 07/03/2025
Good luck! 😎
🚨We’re celebrating Spring Festival with a sweet Nvidia 4070ti Super giveaway🥳
For a chance to win👇
1⃣Follow us
2⃣Like, RT & comment on this post
3⃣Random winner picked Feb 5
Good luck🤞
#DeltaForce#DeltaForceGame#DeltaForcePC#SeasonStarfall
🔦 @NVIDIAGeForce RTX 4080 SUPER GIVEAWAY 🔦
TO ENTER:
✅ Like and RT
✅ Tag 2 Friends you're watching the #VCTSpotlightSeries with 👀
You have 7 days to enter!