sohit kumar

Building autonomous SDLC paltform for agents - - engineering@ @toplynehq, Goldman Sachs, Oracle

[sohit kumar @ksohit]

Introducing Prinevo Memory - a long-term memory engine built on typed-edge capture, bi-temporal knowledge-graph supersession, and hybrid retrieval, with one engine shared across domains through a pluggable ontology layer.

Benchmark numbers, published today: LoCoMo 83.89% (mem0's own judge, n=1,986) and LongMemEval 82.60% (its own official judge, n=500) — full methodology, disclosed judges, and raw per-question results are public.

Concretely - when a fact changes, the old one is retired, not overwritten, so we can answer both "what's true now" and "what did we believe was true back then." And because relationships are typed edges, multi-hop questions - "what transitively depends on X," full blast-radius - get answered by walking the graph directly, not by hoping a bigger context window surfaces the right chunk.

Nothing enters the graph without a source, either - an unsourced claim is rejected at write time, so "why did the agent believe this" stays answerable.

The bet underneath it - retrieval is only as good as what you captured. No amount of reranking or prompt tuning fixes a system that didn't write down the right entities and relations upstream.

That's why domain understanding - an actual ontology, not just embeddings over raw text - matters so much for agent performance, and it's the problem Prinevo memory is built to solve.

These are early, measured numbers, not a finished story. We're continuing to track how this holds up as the same shared context graph + ontology layer gets used across a multi-agent system, and we'll keep publishing results as that develops.

Link in comment

[sohit kumar @ksohit]

@RhysSullivan Need a faster CI.

[sohit kumar @ksohit]

@RhysSullivan I am spending some time evey couple of weeks to optimze the run time of my test suite.

[sohit kumar @ksohit]

@boringdev77 Contextual understanding of system is also nededed along with dependency graph to figure out the right blast radius.

[sohit kumar @ksohit]

We will need a much faster and smarter CI layer.

Coding agents have made writing code faster, but along with that they are also generating a lot more test cases.

And in my opinion, that is needed.

If you want to trust AI writing code, the only way is to have more complete test coverage across scenarios.

But this also means test suites are getting blown out of proportion.

I am already running 8,000+ test cases, and before every deployment a large chunk of them needs to run. It is slowing down CI.

Every couple of weeks, I find myself spending time optimising and parallelising my testcase run because it starts slowing me down.

As we move towards more and more autonomous coding, and more code written by AI

More code → more test cases → more verification → slower CI.

Software factory can solve for the smart layer we don't need to everything everytime but run testcase in the blast radius of the chnage.

Looking out for something that will help up with speed.

[sohit kumar @ksohit]

@kushbhuwalka 2 is one of the feedback signal to your harness, what should change in harness to make it more effective.

[sohit kumar @ksohit]

@diptanu I have to spin up different sandbox for my QA agent - as I want it to bring up the services and do the testing.

If I can vertically scale the same sandbox where other agents are running that will be quite useful.

[sohit kumar @ksohit]

@haxzie_ I am building this and its being run by few teams and it has for sure expedited the delivery by at least 50%.

[sohit kumar @ksohit]

@bibryam @cursor_ai One pattern teams are using - run agents inside a sandbox and use GitHub as the filesystem to manage state/access across different sessions. This unlocks running agents at scale inside sanbox.

[sohit kumar @ksohit]

@bibryam @cursor_ai One pattern teams are using - run agents inside a sandbox and use GitHub as the filesystem to manage state/access across different sessions. This unlocks running agents at scale inside sanbox.

[sohit kumar @ksohit]

@araseb_ Switched to codex

[sohit kumar @ksohit]

@V1rendra_ I will add skill discovery(proxy to tool discovery) to the list as well, if you are building complex agentic system you will have too many skills.

You need to ensure you discover and load relevant skill and then call it.

[sohit kumar @ksohit]

Self evolving Harness

If you are making changes to harness, you are following some setup of principle in your head - when to add agent, when to add skill, how to handle context engineering, how to handle tool calling etc ?

If you can come up with set of principle that needs to be followed then after every run an agent can follow the principles and can come up with suggestion for you.

Principle of how to figure out changes ?
Principle of how to make the changes ?
Principle to change the principle itself ?

Observe -> Follow the evolution principles -> Evolve the harness.

Agents can then recommend
- you need to add new skill or agent
- you need to handle context better because compaction is happening too frequently.

And then follow the principles to go ahead and make the changes.
- How to create ?
- Where to create. ?
- How to update ?
- When to update ? etc..

But what if you need to change the principle itself?

Then you need to have guidelines to evolve the principle itself, this is where things get tricky where you have to keep it open to add new principles
but it has to be closed how that principle should be added.

I have evolved, figured out issue in my harness this way,

I have an evolve command which just goes follow these steps and make the changes for me.

[sohit kumar @ksohit]

@sandislonjsak Feel the same that overall software quality has degraded lately.

[sohit kumar @ksohit]

@danielkleach Very frequent scenario in data pipelines, where you just assume the data in front of you is correct - where in reality it may be stale, wrong, not present etc.

[sohit kumar @ksohit]

Don't make agent responsible for everything.
If the system already knows the fact, state or next step, keep it deterministic.

Use the agent when there is a real judgment call.
building agentic system doesn't mean every layer needs an agent.

This one simple thing will make your system much more reliable.

[sohit kumar @ksohit]

@peer_rich More headcount you add more "cordination tax" you need to pay.
You cant always avoid it but if you can you should.

[sohit kumar @ksohit]

@DanielLockyer Have been using Daytona, quite easy to setup and use.

[sohit kumar @ksohit]

@gwenshap This is the judgment we talk about is not going to get replaced by AI. 🔥