Sandeep Kumar

CEO @MindzKonnected
Production AI for enterprises, deployable in your own cloud , Flagship: AI contract management

[Sandeep Kumar @sandeep_MindzK]

@devmalixx Sure! We are open to discussion.

[Sandeep Kumar @sandeep_MindzK]

We just launched BOA, an agent that researches your list of target companies, flags which are worth pitching, and suggests how to approach them.

It’s in early development, so we’d love testers and honest feedback 🙏
https://t.co/Sg1hW9K0sF

[Sandeep Kumar @sandeep_MindzK]

@SeraAndroid Counting malformed responses as infra failures is a great call.
We do something similar in our text-to-SQL evals: harness bugs are tracked apart from agent mistakes, and answers that claim success without running a query get their own bucket.

More here: https://t.co/OWRv8pzPqR

[Sandeep Kumar @sandeep_MindzK]

Last month, we wrote about MindzEval, the eval suite we built for our text-to-SQL agent.
The core idea: don’t grade the agent’s SQL letter by letter. Instead, grade whether it returned the right answer. 🧵
https://t.co/VmVIXZKXwx

[Tim Messerschmidt @SeraAndroid]

Building agents? You need to know what the model can actually do with tools before you trust it in production.

That's why I built tool-eval-bench: 69 real-world tool-calling scenarios (97 with Hard Mode) you can run against any model, local or cloud, across quants and serving engines.

2.8 just shipped, focused on confidence in the results:

- stricter graders catch fabricated success claims
- optional decision-model audits flag disagreements for review
- malformed server responses count as infra failures, not model mistakes

Link below.

[Sandeep Kumar @sandeep_MindzK]

@MatznerJon Here’s one we’re running right now, on our text-to-SQL agent.

Biggest lesson so far: the eval can be wrong too. Our agent “failed” a test, and it turned out our reference answer was the mistake, not the agent. Check out this thread:

https://t.co/fKfbG8N4iI

[Sandeep Kumar @sandeep_MindzK]

Last month, we wrote about MindzEval, the eval suite we built for our text-to-SQL agent.
The core idea: don’t grade the agent’s SQL letter by letter. Instead, grade whether it returned the right answer. 🧵
https://t.co/VmVIXZKXwx

[Sandeep Kumar @sandeep_MindzK]

@simonlin Exactly. A long running agent can quietly drift, and the worst case is coming back to confident work built on a wrong understanding it picked up while you were gone.

[Sandeep Kumar @sandeep_MindzK]

does the codex session actually keep context over voice, or does it drift once youre away from the screen and cant see what its doing? curious how usable that really is vs demo-nice

[Riley Brown @rileybrown]

A computer is a completely different thing now.

I’ve reached a point where what I want, I get. I don’t need to ask any developer, I just ask ai to do it.

As we speak codex is building a new Mac mini case via 3D printer with a phone mount… for bluey. Why am I doing this? because I can! It’s fun.

and Opus 5.5 is making improvements to my internal tool app (which has 6 apps built within it). I’ve cancelled 6 tools because of this.. and it’s better for our workflow.

When I need keyboard shortcuts changed in any app I just ask codex trusting that it’ll figure it out with computer use.

I wanted a cool background animation app for my dual monitors which are in the background of my videos, it never once occurred to me to go look for an existing app. I took the 40 seconds to yap into my computer and it made a desktop app pinned to my dock that I can use for cool background animations.

When I want to go outside and continue working I CAN. I can put in air pods and talk to an AI in real time voice that can spin up codex sessions. While walking I can have that AI summarize all the emails I need to respond to. And then the AI can actually respond based on what I tell it. I can have it go through all my files, notes or videos on my computer for context.

This are just examples of the past 3 hours. Every day I’m surprised by what is actually possible by talking to a computer.

[Sandeep Kumar @sandeep_MindzK]

@brandonlbarros @mastra Given what you are building at Mastra, we would really value your guidance. And if you would be open to trying MindzEval, your feedback would mean a lot.

[Sandeep Kumar @sandeep_MindzK]

Last month, we wrote about MindzEval, the eval suite we built for our text-to-SQL agent.
The core idea: don’t grade the agent’s SQL letter by letter. Instead, grade whether it returned the right answer. 🧵
https://t.co/VmVIXZKXwx

[Sandeep Kumar @sandeep_MindzK]

@brandonlbarros @mastra Thanks for the reply, Brandon.

Your point about grading the result as a whole, then adding turn-by-turn evals, is close to how MindzEval works. It grades the final answer, and also checks each stage of the pipeline, so we can see which step broke.

[Sandeep Kumar @sandeep_MindzK]

really clean idea. An addition i thought, a crawler or LLM reading raw page text will drop details, but if a system one model already has those fields as clean structured data, the LLM can check its read against that and patch the gaps. small model as ground truth, LLM reconciles

[elvis @omarsar0]

Another great application of System One models.

I think much of what we call hallucination in web agents comes from misreading the page.

When a page is flattened to text, a small ✗ in a pricing table disappears, and the agent answers incorrectly with full confidence.

Expand Fetch reads the rendered page, returns the underlying data as clean JSON, and keeps a replay of exactly what your agent saw.

[Sandeep Kumar @sandeep_MindzK]

The lesson: when your agent and your test disagree, the test might be the one that’s wrong. Part 2 on MindzEval, read here:
https://t.co/71nncO4wqt

[Sandeep Kumar @sandeep_MindzK]

So MindzEval now tests its own golden queries before trusting any score:

1. Flag queries that come back empty
2. Compare each count with the one we recorded
3. Remove leftover links to deleted records, and check whether the answer changes

[Sandeep Kumar @sandeep_MindzK]

@AxelMtn5 hey, happy to connect. working on AI agents and a couple of tools in the space. alongside a day job myself so i feel that. what are you building these days?

[Sandeep Kumar @sandeep_MindzK]

@clicknify hey, happy to connect. we're in the agent space, building and shipping agentic systems into production for clients. always up to meet other founders here.