Mahmoud

Technical Lead | AI & Machine Learning Engineer | Building and scaling with Generative AI, LLM architectures | Montreal 🇨🇦

[Mahmoud @MKhordoo]

Jev can control a swarm of 15 autonomous drones in real time!

I built a simulation to test it navigating an asteroid field:

• Decision latency: < 300ms
• Survival rate: 100% (all 15 reached the destination)

Built with @typesafeai by @CompleteSkeptic

[Mahmoud @MKhordoo]

@michael_chomsky Milvus tested this already. According to their findings, Jev beat Qwen on nDCG, but was 10.2× slower and 6.7× more expensive, and a 0.5 filter dropped 18.8% of relevant docs.
Link: https://t.co/534YRs5l73

[Milvus @milvusio]

𝗖𝗮𝗻 𝗝𝗲𝘃 𝗿𝗲𝗽𝗹𝗮𝗰𝗲 𝗮 𝗿𝗲𝗿𝗮𝗻𝗸𝗲𝗿 𝗶𝗻 𝗥𝗔𝗚?

We tested three setups on the same Milvus shortlist: 𝗻𝗼 𝗿𝗲𝗿𝗮𝗻𝗸𝗶𝗻𝗴, 𝗾𝘄𝗲𝗻𝟯.𝟳-𝘁𝗲𝘅𝘁-𝗿𝗲𝗿𝗮𝗻𝗸, 𝗮𝗻𝗱 𝗝𝗲𝘃.

On 80 SciFact queries, qwen3.7-text-rerank improved nDCG@10 by 𝟬.𝟬𝟰𝟰𝟲 over no reranking, while Jev improved it by 𝟬.𝟬𝟳𝟳𝟴. Jev ranked best of the three, but its P50 reranking latency was 𝟭𝟬.𝟮× 𝗵𝗶𝗴𝗵𝗲𝗿 𝘁𝗵𝗮𝗻 𝗾𝘄𝗲𝗻, with an estimated cost per run of 𝟲.𝟳×.

Why does Jev behave differently?
Its interface is built around 𝘀𝘁𝗮𝘁𝗲 + 𝗾𝘂𝗲𝘀𝘁𝗶𝗼𝗻𝘀. For reranking, we keep the relevance question fixed and change the state for each query-candidate pair: “𝗜𝘀 𝘁𝗵𝗶𝘀 𝗰𝗮𝗻𝗱𝗶𝗱𝗮𝘁𝗲 𝗿𝗲𝗹𝗲𝘃𝗮𝗻𝘁 𝘁𝗼 𝘁𝗵𝗲 𝗾𝘂𝗲𝗿𝘆?” Jev returns a structured Yes/No judgment with probabilities, and we use 𝗣(𝗬𝗲𝘀) as the relevance score for sorting.

In this experiment, 30 candidates meant 30 independent Jev requests running concurrently, so the latency includes that API overhead.

We also tested using probability directly for filtering. At a 𝟬.𝟱 𝘁𝗵𝗿𝗲𝘀𝗵𝗼𝗹𝗱, top-5 precision reached 𝟰𝟵.𝟰%, but 𝟭𝟴.𝟴% 𝗼𝗳 𝗴𝗼𝗹𝗱-𝗿𝗲𝗹𝗲𝘃𝗮𝗻𝘁 𝗱𝗼𝗰𝘂𝗺𝗲𝗻𝘁𝘀 𝘄𝗲𝗿𝗲 𝗳𝗶𝗹𝘁𝗲𝗿𝗲𝗱 𝗼𝘂𝘁. That makes Jev attractive for offline retrieval, data cleaning, and evaluation, but much harder to use as-is for latency-sensitive online RAG.

Next, we’re testing whether batching multiple candidate questions into one Jev request can bring that latency down.

[Mahmoud @MKhordoo]

This is exactly what I was trying to capture here
https://t.co/wABsyK2Fst

[Mahmoud @MKhordoo]

This project cost $23 to build on Jev.

The same token volume on Sonnet 5 or Terra would be about $1,100. On Sol, about $2,258.

There’s always a hidden tax on new ideas, which is the cost of finding out whether they’re even feasible.

At $1,000, the first prototype often never happens. You talk yourself out of it. At $23, you just build it, watch it fail, change the loop, and run it again.

This sim exists because the idea tax was low. One successful 15-drone run made 201 decisions for about $0.24.

That’s a tenth of a cent per decision.

Jev probably doesn’t win by making ChatGPT obsolete or matching frontier-model intelligence.

It wins by making a different class of decisions cheap enough that new use cases become possible. The ones that need millions of small decisions you would never have paid frontier prices for.

[Mahmoud @MKhordoo]

Jev could also significantly reduce the tax of ideation, allowing people to imagine and experiment with use cases they might never have considered practical before.

When the cost of failure and iteration becomes this cheap, you can afford to explore a much wider space of ideas.

[clem 🤗 @ClementDelangue]

What I like about Jev: for years, large generalist models sucked almost all the oxygen out of AI.

But there are huge opportunities in much more specialized and customized models, built for specific tasks and languages and as a result, orders of magnitude cheaper, faster and more optimized. There are 3 million of them publicly available on @huggingface.

Let's build a much more diverse AI ecosystem!

[Mahmoud @MKhordoo]

@npceo_ Nice! Driving a single car should be a breeze for Jev! It already ran 15 autonomous drones at the same time.
https://t.co/T0Gahg0bA4

[Mahmoud @MKhordoo]

Jev can control a swarm of 15 autonomous drones in real time!

I built a simulation to test it navigating an asteroid field:

• Decision latency: < 300ms
• Survival rate: 100% (all 15 reached the destination)

Built with @typesafeai by @CompleteSkeptic

[Mahmoud @MKhordoo]

@MaziyarPanahi Welcome to the club. I already spent 3x that just iterating on whether they’d dodge the rocks or each other.😁
https://t.co/wABsyK27CV

[Mahmoud @MKhordoo]

This project cost $23 to build on Jev.

The same token volume on Sonnet 5 or Terra would be about $1,100. On Sol, about $2,258.

There’s always a hidden tax on new ideas, which is the cost of finding out whether they’re even feasible.

At $1,000, the first prototype often never happens. You talk yourself out of it. At $23, you just build it, watch it fail, change the loop, and run it again.

This sim exists because the idea tax was low. One successful 15-drone run made 201 decisions for about $0.24.

That’s a tenth of a cent per decision.

Jev probably doesn’t win by making ChatGPT obsolete or matching frontier-model intelligence.

It wins by making a different class of decisions cheap enough that new use cases become possible. The ones that need millions of small decisions you would never have paid frontier prices for.

[Mahmoud @MKhordoo]

@m_newhaus @benchmarkheaven The whole appeal of Jev is that finally you have a “general purpose” classfier that you don’t need to “fine tune”.
https://t.co/iOpDqPCAs3

[Mahmoud @MKhordoo]

Yes, classifiers existed. BERT, DeBERTa, GLiNER, fine-tuned heads. Most of them were specialized. Pick the labels, train, ship. Changing the rules at runtime meant another training run.

What we did not have was a general-purpose classifier at Terra / Sonnet 5 level intelligence.

You could fake it by asking an LLM for JSON, but that was slow, expensive, and poorly calibrated.

Jev is that missing piece. Labels defined in code, typed answer plus confidence, no extra LLM call.

Classification is not new. What is new is a general-purpose classifier this intelligent, this fast, and this cheap.

[Mahmoud @MKhordoo]

@JorgeCastilloPr Laya’s performance seems to be lower based on this bench.
https://t.co/KKf7ydRJ9e

[Florian S @airesearch12]

JevBench v1.2.6 is live!

Jev still 👑, but how long?

Top 3:
🥇Jev @CompleteSkeptic
🥈SemIf @theoleecj
🥉djev @mmastrac

New entrants:
#4 openJev Verdict 1.4 @heman10x
#9 SimpleJev Qwen3.8-27B @FeatherlessAI
#16 SimpleJev Qwen3.6-35B-A3B

https://t.co/hFQ5fX4JEb

[Mahmoud @MKhordoo]

@itsolelehmann The real unlock is lowering the idea tax. At $23 you just test it. At $1,100 you talk yourself out of the prototype.
https://t.co/wABsyK27CV

[Mahmoud @MKhordoo]

This project cost $23 to build on Jev.

The same token volume on Sonnet 5 or Terra would be about $1,100. On Sol, about $2,258.

There’s always a hidden tax on new ideas, which is the cost of finding out whether they’re even feasible.

At $1,000, the first prototype often never happens. You talk yourself out of it. At $23, you just build it, watch it fail, change the loop, and run it again.

This sim exists because the idea tax was low. One successful 15-drone run made 201 decisions for about $0.24.

That’s a tenth of a cent per decision.

Jev probably doesn’t win by making ChatGPT obsolete or matching frontier-model intelligence.

It wins by making a different class of decisions cheap enough that new use cases become possible. The ones that need millions of small decisions you would never have paid frontier prices for.

[Mahmoud @MKhordoo]

@damonchen Why should they? Users only care if the changes benefit them directly. They might not appreciate or even notice the model behind it, but they’ll definitely be happier with a better experience, faster workflows, or lower prices if you pass those cost savings down to them.

[Mahmoud @MKhordoo]

I used this hybrid approach in the demo below. The useful part is that Jev gives us its confidence score, which is a good signal to handoff the task to a larger model if needed.
You use Jev for quick and fast system 1 decisions and fall back to a more intelligent slower system 2 llm when it’s unsure. The code is in the replies.
https://t.co/qp6fXvlJp3

[Mahmoud @MKhordoo]

Jev can control a swarm of 15 autonomous drones in real time!

I built a simulation to test it navigating an asteroid field:

• Decision latency: < 300ms
• Survival rate: 100% (all 15 reached the destination)

Built with @typesafeai by @CompleteSkeptic

[Mahmoud @MKhordoo]

@joelgrus Exactly
https://t.co/7ay0IuDEJc

[Mahmoud @MKhordoo]

Yes, classifiers existed. BERT, DeBERTa, GLiNER, fine-tuned heads. Most of them were specialized. Pick the labels, train, ship. Changing the rules at runtime meant another training run.

What we did not have was a general-purpose classifier at Terra / Sonnet 5 level intelligence.

You could fake it by asking an LLM for JSON, but that was slow, expensive, and poorly calibrated.

Jev is that missing piece. Labels defined in code, typed answer plus confidence, no extra LLM call.

Classification is not new. What is new is a general-purpose classifier this intelligent, this fast, and this cheap.

[Mahmoud @MKhordoo]

By that logic, ChatGPT was just a 2022 product built on transformer architecture introduced in 2017 and packaged differently.

The primitive existing before doesn’t make the new operating point uninteresting.

[victor @stokebuilder]

So jev is basically 2022 machine learning, without the need for a minimum training dataset.

[Mahmoud @MKhordoo]

@george_onx @w0lframk0 @PatronusBen Do you have the calibration benchmark behind the claim above? JevBench v1.2 reports Jev at 82.7 vs GLiNER2.5 multi at 56.1 on its calibration axis. Happy to look at a different eval if that’s not the one.

[Mahmoud @MKhordoo]

Yes, classifiers existed. BERT, DeBERTa, GLiNER, fine-tuned heads. Most of them were specialized. Pick the labels, train, ship. Changing the rules at runtime meant another training run.

What we did not have was a general-purpose classifier at Terra / Sonnet 5 level intelligence.

You could fake it by asking an LLM for JSON, but that was slow, expensive, and poorly calibrated.

Jev is that missing piece. Labels defined in code, typed answer plus confidence, no extra LLM call.

Classification is not new. What is new is a general-purpose classifier this intelligent, this fast, and this cheap.

[1LittleCoder💻 @1littlecoder]

I'm genuinely surprised people didn't know Classification models before!

[Mahmoud @MKhordoo]

Jev does not have vision capability. You need to feed the required data or parameters as context to the state and ask it for a decision.

Here’s my demo. It avoids the asteroid without needing vision, using the same concept. It receives distance to obstacles, speed, etc. via sensors. At the end of the day you need a picture to identify that information, so if you feed that data directly into the context you don’t need to consume vision.
https://t.co/qp6fXvmheB

[Mahmoud @MKhordoo]

Jev can control a swarm of 15 autonomous drones in real time!

I built a simulation to test it navigating an asteroid field:

• Decision latency: < 300ms
• Survival rate: 100% (all 15 reached the destination)

Built with @typesafeai by @CompleteSkeptic

[Mahmoud @MKhordoo]

@burkov The useful case is not “faster than an LLM.” It’s replacing the fine-tuned classifier you already run for routing/triage, without another training run every time the labels change.
https://t.co/iOpDqPCAs3

[Mahmoud @MKhordoo]

Yes, classifiers existed. BERT, DeBERTa, GLiNER, fine-tuned heads. Most of them were specialized. Pick the labels, train, ship. Changing the rules at runtime meant another training run.

What we did not have was a general-purpose classifier at Terra / Sonnet 5 level intelligence.

You could fake it by asking an LLM for JSON, but that was slow, expensive, and poorly calibrated.

Jev is that missing piece. Labels defined in code, typed answer plus confidence, no extra LLM call.

Classification is not new. What is new is a general-purpose classifier this intelligent, this fast, and this cheap.

[TypeSafe AI @typesafeai]

Jev is now available to everyone. No waitlist.
Start using it here: https://t.co/5MfN4rIif5

[Mahmoud @MKhordoo]

@vwapster Layla looks interesting. I’m going to run the same loop on it today and compare the results. Stacking them for AGI can wait.

[Mahmoud @MKhordoo]

This project cost $23 to build on Jev.

The same token volume on Sonnet 5 or Terra would be about $1,100. On Sol, about $2,258.

There’s always a hidden tax on new ideas, which is the cost of finding out whether they’re even feasible.

At $1,000, the first prototype often never happens. You talk yourself out of it. At $23, you just build it, watch it fail, change the loop, and run it again.

This sim exists because the idea tax was low. One successful 15-drone run made 201 decisions for about $0.24.

That’s a tenth of a cent per decision.

Jev probably doesn’t win by making ChatGPT obsolete or matching frontier-model intelligence.

It wins by making a different class of decisions cheap enough that new use cases become possible. The ones that need millions of small decisions you would never have paid frontier prices for.

[Mahmoud @MKhordoo]

Jev can control a swarm of 15 autonomous drones in real time!

I built a simulation to test it navigating an asteroid field:

• Decision latency: < 300ms
• Survival rate: 100% (all 15 reached the destination)

Built with @typesafeai by @CompleteSkeptic

[Mahmoud @MKhordoo]

@polymorfaisal Exactly, adding output tokens makes the gap much larger. That’s what actually changes what you bother to try.

[Mahmoud @MKhordoo]

@EvanOtero I’ve already built a fast System 1 control loop with Jev for a 15-drone simulation. I’d be happy to be an early tester for Gemma Diffusion and run it on the same workload to compare latency and decision behavior.

[Mahmoud @MKhordoo]

@Ajion @typesafeai @CompleteSkeptic Fair question. This wasn’t about shipping a one-off algorithm. It was to show Jev’s sub 300ms judgment is good enough to fly 15 drones live, with an optional larger LLM as System 2.

[Mahmoud @MKhordoo]

Jev can control a swarm of 15 autonomous drones in real time!

I built a simulation to test it navigating an asteroid field:

• Decision latency: < 300ms
• Survival rate: 100% (all 15 reached the destination)

Built with @typesafeai by @CompleteSkeptic