John De Oliveira

AI Architect | Former VP of Product & AI | Self-learner & Inventor | Work over ego 🚀

[Rohan Paul @rohanpaul_ai]

This paper asks whether AI agents have a real memory system yet, and finds the answer is mostly no.

The problem is that AI agents now need memory that can store, search, update, and clean up information across long tasks.

The authors say current tests mostly check final answers, so they miss whether the memory system itself is fast, reliable, or good at handling changed facts.

They split agent memory into 4 parts: how memories are stored, how facts are extracted, how useful memories are found, and how old or conflicting memories are maintained.

They tested 12 memory systems across 5 workloads and 11 datasets, including long conversations, multi-session recall, database tasks, and update-heavy settings.

The main result is that no memory design wins everywhere, because graph memories help with linked facts, hybrid systems help with filtered search, and raw traces help when exact action history matters.

----

Link – arxiv. org/abs/2606.24775

Title: "Are They Ready For An Agent-Native Memory System?"

[Akshay 🚀 @akshay_pachaar]

Claude Code fully dissected!

Researchers from UCL reverse-engineered the leaked Claude source. What they found changes how you should think about agent design.

Only 1.6% of the codebase is AI decision logic.

The other 98.4% is operational infrastructure. Permission gates, tool routing, context compaction, recovery logic, session persistence. The model reasons. The harness does everything else.

This is the opposite of what most agent frameworks do today.

LangGraph routes model outputs through explicit state machines. Devin bolts heavy planners onto operational scaffolding. Claude Code gives the model maximum decision latitude inside a rich deterministic harness, and invests all its engineering effort in that harness.

The core loop is a simple while-true. Call model, run tools, repeat.

But the systems around that loop are where the real design lives:

A permission system with 7 modes and an ML classifier. Users approve 93% of prompts anyway, so the architecture compensates with automated layers instead of adding more warnings.

A 5-layer context compaction pipeline. Each layer runs only when cheaper ones fail. Budget reduction, snip, microcompact, context collapse, auto-compact.

Four extension mechanisms ordered by context cost. Hooks (zero), skills (low), plugins (medium), MCP (high). Each answers a different integration problem.

Subagents return only summary text to the parent. Their full transcripts live in sidechain files. Agent teams still cost roughly 7x the tokens of a standard session.

Resume does not restore session-scoped permissions. Trust is re-established every session. That friction is the point.

The bet behind all of this is simple. As frontier models converge on raw coding ability, the quality of the harness becomes the differentiator, not the model.

Paper: Dive into Claude Code (arXiv:2604.14228)

In the next tweet, I've shared an article I wrote on Agent Harness and what every big company is building. Do check.

[Rahul @sairahul1]

As an AI Engineer. Please learn

>Harness engineering, not just prompt engineering

>Context engineering, not just long prompts

>Prompt caching vs. semantic caching tradeoffs

>KV cache management, eviction, reuse, and memory pressure at scale

>Prefill vs. decode latency and why they optimize differently

>Continuous batching, paged attention, and throughput optimization

>Speculative decoding vs. quantization vs. distillation tradeoffs

>INT8, INT4, FP8, AWQ, GPTQ, and when quantization hurts quality

>Structured output failures, schema validation, repair loops, and fallback chains

>Function calling reliability, tool contracts, argument validation, and idempotency

>Agent guardrails, loop budgets, tool budgets, and termination conditions

>Model routing, graceful fallback logic, and degraded-mode UX

>RAG architecture: chunking, embeddings, hybrid search, reranking, and freshness

>Retrieval evals: recall, precision, grounding, attribution, and citation quality

>Evals: golden sets, regression tests, adversarial tests, LLM-as-judge, and human evals

>LLM observability as a first-class discipline: traces, spans, tokens, latency, errors, and drift

>Cost attribution per feature, workflow, tenant, and user journey not just per model

>Safety engineering: prompt injection defense, data leakage prevention, and permission boundaries

>Multi-tenant isolation, cache safety, and cross-user context contamination prevention

>Fine-tuning vs. in-context learning vs. RAG vs. distillation and when each is the wrong tool

>Latency, quality, cost, and reliability tradeoffs across the full inference stack

>Production failure modes: hallucinated tool calls, malformed JSON, stale retrieval, runaway agents, and silent eval regressions

[Akshay 🚀 @akshay_pachaar]

from weights → context → harness engineering

(evolution of agent landscape from 2022-26)

the biggest shift in AI agents had nothing to do with making models smarter.

it was about making the environment around them smarter.

here's how agent engineering evolved in just 4 years, across three distinct phases:

𝗽𝗵𝗮𝘀𝗲 𝟭: 𝘄𝗲𝗶𝗴𝗵𝘁𝘀 (𝟮𝟬𝟮𝟮)

everything was about the model itself. bigger models, more data, better training. scaling laws told us that progress = more parameters.

RLHF and fine-tuning shaped behavior. if you wanted a better agent, you trained a better model.

this worked great for single-turn tasks. ask a question, get an answer.

but it hit a wall fast. updating one fact meant retraining. auditing behavior was nearly impossible. and personalization across millions of users from one frozen set of weights? not happening.

𝗽𝗵𝗮𝘀𝗲 𝟮: 𝗰𝗼𝗻𝘁𝗲𝘅𝘁 (𝟮𝟬𝟮𝟯-𝟮𝟬𝟮𝟰)

the realization: you don't always need to change the model. you can change what the model sees.

prompt engineering, few-shot examples, chain-of-thought, RAG. suddenly the same frozen model could behave completely differently based on what you put in front of it.

developers stopped fine-tuning and started iterating on prompts and retrieval pipelines instead. it was cheaper, faster, and surprisingly effective.

but context windows are finite. long prompts get noisy. models attend unevenly (the "lost in the middle" problem is real). and every new session starts fresh with zero memory of what happened before.

context made agents flexible. it didn't make them reliable.

𝗽𝗵𝗮𝘀𝗲 𝟯: 𝗵𝗮𝗿𝗻𝗲𝘀𝘀 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 (𝟮𝟬𝟮𝟱-𝟮𝟬𝟮𝟲)

this is where we are now, and the shift is fundamental.

the question changed from "what should we tell the model?" to "what environment should the model operate in?"

the model is no longer the sole location of intelligence. it sits inside a harness that includes persistent memory, reusable skills, standardized protocols (like MCP and A2A), execution sandboxes, approval gates, and observability layers.

the model stays the same. what changes is the task it's being asked to solve.

a concrete example: a coding agent asked to implement a feature, run tests, and open a PR.

without a harness, the model must keep repo structure, project conventions, workflow state, and tool interactions all inside a fragile prompt.

with a harness, persistent memory supplies context, skill files encode conventions, protocolized interfaces enforce correct schemas, and the runtime sequences steps and handles failures.

same model. completely different reliability.

𝘁𝗵𝗲 𝗽𝗮𝘁𝘁𝗲𝗿𝗻 𝗮𝗰𝗿𝗼𝘀𝘀 𝗮𝗹𝗹 𝘁𝗵𝗿𝗲𝗲 𝗽𝗵𝗮𝘀𝗲𝘀 𝗶𝘀 𝘀𝗶𝗺𝗽𝗹𝗲:

- weights encoded knowledge in parameters (fast but rigid)
- context staged knowledge in prompts (flexible but ephemeral)
- harnesses externalized knowledge into persistent infrastructure (reliable and governable)

each phase didn't replace the previous one. it layered on top. weights still matter. context engineering still matters. but the center of gravity has moved outward.

the most consequential improvements in agent reliability today rarely come from changing the base model.

they come from better memory retrieval, sharper skill loading, tighter execution governance, and smarter context budget management.

building better agents increasingly means building better environments for models to operate in.

there's a great paper on this:

Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering

paper: https://t.co/TeldoC3dS8

i also published this deep dive (article) on agent harness engineering, covering the orchestration loop, tools, memory, context management, and everything else that transforms a stateless LLM into a capable agent.

[Akshay 🚀 @akshay_pachaar]

Your agent remembers everything and understands nothing.

Most agent memory systems optimize for recall. The harder problem is what to forget, or more precisely, what to never store in the first place.

The default agent memory pipeline hands an LLM raw text and asks it to extract entities and relationships. The model decides the types, the labels, the attributes, all on its own.

The result is a knowledge graph that behaves like an expensive vector store. Entity types collapse into generic labels. Relationships flatten into a single "RELATES_TO."

The graph has the data, but no query can reach it with precision.

The problem is not retrieval. It is structure. And the fix is the same pattern that already works everywhere else in the AI stack: constrain the output space before generation, not after.

𝗘𝗻𝘁𝗶𝘁𝗶𝗲𝘀 define what the agent is allowed to remember. Pydantic models with typed fields and descriptive docstrings replace the LLM's guesswork with domain vocabulary it was never trained on.

𝗘𝗱𝗴𝗲𝘀 define how things connect. Source/target constraints on relationship types mean the graph can only form valid connections. If your schema has no edge connecting Project to Competitor, that relationship cannot exist in memory.

𝗧𝗲𝗺𝗽𝗼𝗿𝗮𝗹 𝗿𝗲𝘀𝗼𝗹𝘂𝘁𝗶𝗼𝗻 handles what was true versus what is true. Fact resolution invalidates outdated edges while preserving history, so the graph never silently serves stale state.

The schema guides extraction at two points in the pipeline (entity extraction and fact extraction) while resolution and temporal processing run automatically downstream.

You define what to look for. The system handles deduplication, contradiction detection, and time-windowing without additional configuration.

A useful constraint: 10 entity types, 10 edge types, 10 fields per type. That forces you to model the 80% that matters rather than attempting completeness. Start with 3-4 of each and expand only when retrieval fails.

Zep AI's Graphiti does all of this as a fully open-source temporal knowledge graph library. Pydantic-based ontology definition, schema-guided extraction, entity resolution, fact resolution, and temporal windowing out of the box.

If you are building agent memory with any kind of domain specificity, it is worth looking at before rolling your own.

Check this out: https://t.co/8CboBlWffX

(don't forget to star 🌟)

Agent memory without schema discipline is storage without structure. The schema is what turns a pile of facts into a queryable model of your domain.

I covered this topic in more depth in the article quoted below.

[The All-In Podcast @theallinpod]

Anthropic is kicking OpenAI’s ass: Insights from the largest revenue explosion in tech history

Brad Gerstner on how Anthropic dominated the last 90 days, and could they hit a $100B run rate by EOY!?:

“Anthropic was literally counted out of the game last year, and they've kicked OpenAI's ass over the last 90 days.

Bam, you have the largest revenue explosion in the history of technology.

So you have to ask, what's going on?

The first thing, for me, is that model and product capability just hit this threshold near AGI, whatever the hell you want to call it.

And everybody, like Altimeter, said, ‘Damn, this is so good. I have to have it.

This is no longer about my IT budget. This is about labor augmentation and labor replacement.

Turns out that the TAM for intelligence is radically different than anything we've seen before.

And I think the best example of this, right, this is millions of self-interested parties, consumers, enterprises, a thousand now over $1M, right?

It's not that there was some great go to market at Anthropic that all of a sudden they snuck up and blew everybody away.

No, it was companies demanding the product. They're getting throttled on the product.

Why? Because it's so good. It makes them better at their business.

We knew intelligence was going to scale on the exponential.

The question was whether revenue will scale on the exponential, and that's what we're seeing.

And remember, they're doing this with only 1.5-2 gigawatts of compute, and the models are only getting better.

So I think when you look out toward the end of the year, I would not be shocked if you see Anthropic exiting this year at $80-$100B in revenue.”

[The All-In Podcast @theallinpod]

Chamath: Anthropic's Mythos Warning Is Theater

@Jason:

“Chamath, is it the Boy who Cried Wolf, or is this the real deal now?”

@Chamath:

“I think it's mostly theater.

In February of 2019 when Dario was still at OpenAI, they did the same thing with GPT-2. That was a 1.5 billion parameter model, which sounds like a total fart in the wind in 2026. But at that time, this model was supposed to be the end of days. And at the end of it, it was a huge nothingburger.

If you actually think that Mythos is capable of doing what it says it can do, two things are true. One is, a very sophisticated hacker can probably do those things right now with Opus. And two, if these exploits are this easy to find, whether you use Opus or whether you use Mythos, the reality is you'd have to shut down the internet for about five years to patch them all.

So when you see a large multi-trillion dollar GSIB bank, it's a bit of theater. Why?

What do you think they can actually accomplish in two months? Do you actually think that if there's these vulnerabilities, it's all going to get fixed? Let's give them six months, let's give them nine months.

So I do think that Sacks is right, that they have figured out a very clever go-to-market muscle here that activates hyper attention and hyper usage, and so I give them tremendous credit.

But we've seen it before, we saw it when these folks were the principal architects at OpenAI, and we're now seeing the same playbook here.

The reality is that capitalism moves forward, the funding needs moves forward, and the need for these guys to build adoption moves forward. And that's going to supersede what this is.”

[Chamath Palihapitiya @chamath]

AI is stripping companies down to three key elements:

1) Expert Knowledge - this is the vertical specific understanding of what to do

2) Tribal Knowledge - this is knowledge of the fuzzy grey details that are poorly document and is what creates a dispersion in performance in companies doing roughly the same things

3) Hardware and Software - this is the glue that humans use to stitch Expert and Tribal Knowledge in order to perform

If 1 and 2 are documented well, it allows you to rebuild how you do work. This isn't about replacing humans - in fact I think it allows you to grow faster and hire more people because with a great understanding of what to do, it requires more and more people to express judgement.

This is why we built Software Factory and its being used by many established and large enterprises.

[Chamath Palihapitiya @chamath]

Here is a good walkthrough of Software Factory. It’s getting smarter and more useful every day.

[John De Oliveira @johndeo]

@MarioNawfal 🤡

[John De Oliveira @johndeo]

@RepMTG Did you discuss this with David Sacks? I think you are jumping to conclusions about this.

[John De Oliveira @johndeo]

@elonmusk it would be irresponsible for you not to incorporate this information into your overall assessment.

[MMarksHQ @MMarksHQ]

@elonmusk

[John De Oliveira @johndeo]

@flurryhq @elonmusk Elon, can you comment on this?

[John De Oliveira @johndeo]

@_MAGA_Patriot @elonmusk Where did @elonmusk provide the examples of good and bad? Can someone point me to it? I can't find it. Thanks.

[John De Oliveira @johndeo]

@bindureddy That's your conclusion? For real?

[John De Oliveira @johndeo]

@stepfanie When you say quantum physics adjacent, it makes me think of Federico Faggin. I just came across him. Have you heard of him?

[John De Oliveira @johndeo]

@FairweatherPhD @bariweiss What do you mean "redistribute"? Who would do the redistributing? The government?

[John De Oliveira @johndeo]

@keshavchan @karpathy Why is she always sleeping? 🤔