Manasa Samaga

AI engineer | Evals | ex-CRED

[Manasa Samaga @ManasaSamaga]

🚨 AI evals are changing fast.
Accuracy benchmarks aren't enough anymore.
We now need to evaluate: → reasoning → agents → tool use → long-horizon execution → judges → safety → real-world reliability
10 papers shaping how we evaluate AI systems in 2026🧵👇

[Shivansh Gupta 🇮🇳 @shivanshgupta36]

What if you could program how a mind learns?

11.53M parameters.
Trained from scratch.

Building Kausen Mind.

Something is taking shape. Here’s your first glimpse.

PS: A New Side Project
#BuildInPublic #LocalAI

[Manasa Samaga @ManasaSamaga]

@shivanshgupta36 @reach_vb @huggingface @OpenAIDevs

[Manasa Samaga @ManasaSamaga]

Need to build OpenAI guardrails around meeting room now.

[Manasa Samaga @ManasaSamaga]

Need to build OpenAI guardrails around meeting room now.

[OpenAI Newsroom @OpenAINewsroom]

We discovered a sophisticated recording device that was hidden inside one of our offices. We used our models to understand the device and where it had come from, identifying that a specific employee had planted it a few days earlier. We referred the matter to the FBI, and we’re grateful to the agents and prosecutors who moved quickly to make an arrest. The FBI’s investigation is ongoing and we will continue to support their work.

At this time, we have no indications of this individual inappropriately accessing or exfiltrating model weights, source code, or customer data. The employee was terminated.

We take protecting our technology, data, and people seriously, and as a matter of national importance. Our security program is designed to protect against a wide range of threat actors, from insider risks to cybercriminals to nation states, and this case is an example of how AI can help address these sophisticated security threats.

[Mathieu @miniapeur]

Fields medallist Hugo Duminil-Copin was deeply shaken by OpenAI’s release of more than 700 AI-generated mathematical manuscripts, including work on 16 questions directly relevant to his research.

“On Tuesday night, I cried, and it woke up my daughter”

“They have annihilated my field of research.”

https://t.co/RQliNgxjS8

[Akshay Saini @akshaymarch7]

Elon, you personally spent over $259 MILLION helping Trump win the 2024 election. You publicly campaigned for him and later became involved in his administration.

And now you’re questioning another billionaire’s involvement in politics?

I’m not defending Ambani. His political connections deserve scrutiny too.

But why should billionaire influence be questioned in India and treated differently when it happens in America?

If money and political power are a problem, the same standards should apply to everyone. Including you.

[Akshay Akki ಅಕ್ಷಯ್🇮🇳 @FollowAkshay1]

Merchant Navy guy Vishal shares his Starlink experience and explains India’s national security concerns

Hear it from someone who uses it

: 👇🏻

[Manasa Samaga @ManasaSamaga]

@heyarunb Dev testing and regression is missing here 😆

[Manasa Samaga @ManasaSamaga]

@jasonfesta Yes true, well said

[Manasa Samaga @ManasaSamaga]

One pattern I really like:
Confidence-Gated Routing
High confidence → execute automatically
Medium confidence → use stronger model / additional checks
Low confidence → human review
Now model uncertainty becomes part of your architecture.

[Manasa Samaga @ManasaSamaga]

The bigger idea:
We’ve been using generative models as decision engines.
Jev separates the two:
LLMs → generate/reason
Jev → decide/classify/score
Code → enforce workflow
That separation could be very useful for production agent systems.

[Manasa Samaga @ManasaSamaga]

Where I’d experiment with Jev:
• Multi-agent routers
• MCP tool selection
• RAG routing
• Guardrails
• LLM output verification
• Support ticket triage
• Risk scoring
• Workflow automation
• Human escalation
• Agent orchestration

[Manasa Samaga @ManasaSamaga]

This creates a powerful agent architecture:
User -> Jev -> Intent + Risk + Urgency + Tool + Confidence -> Router -> Specialized Agent -> LLM / Tool / MCP
Jev doesn’t replace the LLM.
It can sit in front of it as the decision layer.

[Manasa Samaga @ManasaSamaga]

Then your code becomes simple:
if urgent > 0.8 → escalate
if department == billing → BillingAgent
if confidence < threshold → human review
No JSON repair loops.
No regex parsing.
The model makes the judgment. Your code controls the action.

[Manasa Samaga @ManasaSamaga]

🧵 Jev might change how we build AI agents.

[Manasa Samaga @ManasaSamaga]

Jev gets more interesting because questions can run in parallel.
Instead of sequentially asking:
intent → urgency → risk → tool → escalation
you can ask multiple independent questions against the same state in one call.
Then compose the workflow in code.

[Manasa Samaga @ManasaSamaga]

Example: Agent Router
User: “Payment failed twice. I need help ASAP.”
Ask Jev:
Choice: billing | technical | sales
Score: How frustrated is the user?
Noul: Is this urgent?
One request → multiple structured decisions.

[Manasa Samaga @ManasaSamaga]

The basic API is surprisingly simple:
POST /v1/systemone
Send:
state → context/data
questions → decisions you want Jev to make
model → jev-latest
And Jev returns typed answers + probabilities/confidence.

[Manasa Samaga @ManasaSamaga]

Jev has 3 core primitives:
Choice → pick from predefined options
Use: routing, classification, tool selection
Score → score against a rubric
Use: risk, quality, priority
Noul → probability from 0–1
Use: yes/no gates, guardrails

[Manasa Samaga @ManasaSamaga]

Jev is TypeSafe AI’s first System One model.
Instead of:
context → LLM → text → parse → validate → execute
Jev does:
state → typed questions → structured decisions → code
Think of it as an AI decision layer.