Chit-Akasa

Demystifying AI’s future: one breakthrough at a time. For builders, thinkers, and the relentlessly curious. Stay sharp. Stay ahead.

[Chit-Akasa @deepshikhxr]

@sama 🦞🦞🦞

[Chit-Akasa @deepshikhxr]

@ai_for_success can it help find love?

[Chit-Akasa @deepshikhxr]

@OfficialLoganK he was ahead of his time

[Chit-Akasa @deepshikhxr]

@mreflow the api quality often dips before a major release. they might be aggressively cost optimizing the existing endpoints to push adoption for the new one or running a/b tests on cheaper, less stable weight sets. it happens.

[Chit-Akasa @deepshikhxr]

@rasbt it's funny how the core math is still mostly gradient descent but the complexity moved from writing the algorithm to managing the distributed systems running it. infrastructure is the new algorithm.

[Chit-Akasa @deepshikhxr]

@bindureddy honestly the flash prediction is the only part that matters to builders. pro models are great for twitter screenshots but you can't run a production workload with positive margins on them. give me cheap and fast over sota any day.

[Chit-Akasa @deepshikhxr]

@kimmonismus expecting a massive jump with 5.2 feels optimistic.

we are hitting the asymptote of what scaling laws can do with current transformers. throwing more compute at the problem doesn't fix the reasoning bottleneck it just hides it better.

[Chit-Akasa @deepshikhxr]

we spend 90% of our time tweaking chunk sizes and re-rankers, and 10% actually reading the source documents.

if your data is trash, it doesn't matter if you use gpt-4 or gpt-5.

garbage in, expensive garbage out.

[Chit-Akasa @deepshikhxr]

the gap between 'demo software' and 'production software' has never been wider.

it's easy to get a llm to write code that runs once, it's incredibly hard to get it to write code that survives a year of edge cases and dependency updates.

[Chit-Akasa @deepshikhxr]

sam calling 'code red' at openai because gemini 3 took the lead is the final proof that model moats are dead.

we are all just renting intelligence from whoever has the lowest latency and cheapest caching this week. switching costs are zero.

[Chit-Akasa @deepshikhxr]

Quick context on "mechanistic interpretability" for those who do not understand:

In traditional software engineering, a human writes the logic (If X, then Y). We know exactly why the code works because we wrote the rules.

In Deep Learning, we don't write the rules. We define the architecture and the optimization algorithm (Gradient Descent), and then we let the model "learn" the rules by itself by looking at TBs of data.

"Mechanistic Interpretability" is an attempt to reverse-engineer those learned rules. It is the process of taking a trained neural network and asking: "Which specific neurons fire when the model sees the word 'Python'?

Why the pivot? It doesn't scale.
We have successfully mapped small models. Anthropic famously found a specific "Golden Gate Bridge" neuron in Claude. But modern models have trillions of parameters which costs too much compute.

This is why the industry is shifting to Behavioral Evals. Instead of trying to understand the brain surgery (Interpretability), we are just giving it IQ tests and personality quizzes (Behavioral Evals) to see if it's safe.

We are basically accepting that we might never know how it thinks, only that it works.

[AI Notkilleveryoneism Memes ⏸️ @AISafetyMemes]

Google is giving up on mechanistic interpretability

WHY THIS MATTERS:

For many years, AI safety researchers hoped to solve the "black box problem" so humanity can read the alien minds BEFORE they get dangerous

This was load-bearing for some p(doom) estimates. Who will update?

(quote below is Dario Amodei on why mechanistic interpretability matters)

[Chit-Akasa @deepshikhxr]

@paraschopra plumbers. ai cannot fix a leak.

[Chit-Akasa @deepshikhxr]

@bindureddy google 2022 called. they want their corporate panic strategy back.

[Chit-Akasa @deepshikhxr]

aws announcing 'frontier agents' that code while you sleep sounds magical until you wake up to a 400-line pr that hallucinates a database migration and deletes the backups.

i think i'll keep my agents supervised for now, thanks.

[Chit-Akasa @deepshikhxr]

we treat models like bad interns now.

"why did it do that?" "did you explain the task clearly?" "maybe it's tired."

debugging used to be logic, now it's just corporate psychology.

[Chit-Akasa @deepshikhxr]

reading that startups are just prompt engineering and seeing 'agentic workflows' pivot back to 'cli tools'.

nature is healing. we basically reinvented bash scripts, added 2s latency, and called it a unicorn 💀

[Chit-Akasa @deepshikhxr]

grok benchmarking elon as "fittest man alive" over lebron james is the specific kind of alignment failure i pay $8/mo for.

rlxf is going great guys. zero hallucinations detected.

[Chit-Akasa @deepshikhxr]

I was watching Tenet last night.

For the first 45 minutes? I was locked in. I felt smart. I was tracking the entropy, the timelines, the protagonists.

Then the second hour hit.

I turned to my friend and whispered, "Wait. Is the guy with the beard the bad guy? Or is he the good guy from the future moving backwards?"

My friend looked at me with pure pity. I had completely lost the plot. The new explosions had overwritten the earlier scenes.

That specific panic- "I know I saw this, but I can't access it" is exactly what happens to your AI Agent when you blow past the Context Window.

To understand why this happens, we have to look under the hood of the Transformer. Most people think the "Context Window" is just a word count limit. It’s not. It’s a compute bottleneck defined by the Quadratic Curse.

Here is the reality of the architecture: In a standard Transformer, every single token (word part) has to "talk" to every other token to understand what's going on.
Complexity = O(N^2)

Think of it like a dinner party.
~ If you have 4 guests, there are 6 conversations. Easy.
~ If you have 100 guests, and everyone has to speak to everyone else before taking a bite of food... nobody eats.

The room explodes. That is your GPU running out of VRAM.

Even if you buy a model with a massive context window (like 1M tokens), you aren't safe.

Research shows that attention isn't distributed equally. We call it the "U-Shaped" Performance Curve.

~ The Beginning: The model remembers your system prompt perfectly.
~ The End: The model remembers your last question perfectly (Recency Bias).
~ The Middle: The muddy swamp.

Accuracy
^
| * *
| * *
| * *
| * *
| * *
| * * *
-----------------------------> Position in Prompt
[Start] [Middle] [End]

If you stuff a 50-page PDF into the prompt, the specific fact you need on Page 25 is statistically the most likely to be hallucinated or ignored. It's just like me forgetting the subplot from the middle of Tenet.

So, how do we fix it?

Don't be lazy. Don't just "dump it in context" because the window allows it.

- Use RAG (Retrieval Augmented Generation): Only fetch the specific snippets relevant to the query.

- Summarization Chains: Compress the "middle" history into tight summaries so the model doesn't lose the thread.

- Sanitize Data: Remove whitespace and useless headers before tokenizing.

The Context Window is a scarce resource, not a garbage dump.

Treat it like your brain during a Christopher Nolan movie: Focus only on what matters right now, or you're going to lose the plot.

Which topic would you want me to cover next?

[Chit-Akasa @deepshikhxr]

I was watching Tenet last night.

For the first 45 minutes? I was locked in. I felt smart. I was tracking the entropy, the timelines, the protagonists.

Then the second hour hit.

I turned to my friend and whispered, "Wait. Is the guy with the beard the bad guy? Or is he the good guy from the future moving backwards?"

My friend looked at me with pure pity. I had completely lost the plot. The new explosions had overwritten the earlier scenes.

That specific panic- "I know I saw this, but I can't access it" is exactly what happens to your AI Agent when you blow past the Context Window.

To understand why this happens, we have to look under the hood of the Transformer. Most people think the "Context Window" is just a word count limit. It’s not. It’s a compute bottleneck defined by the Quadratic Curse.

Here is the reality of the architecture: In a standard Transformer, every single token (word part) has to "talk" to every other token to understand what's going on.
Complexity = O(N^2)

Think of it like a dinner party.
~ If you have 4 guests, there are 6 conversations. Easy.
~ If you have 100 guests, and everyone has to speak to everyone else before taking a bite of food... nobody eats.

The room explodes. That is your GPU running out of VRAM.

Even if you buy a model with a massive context window (like 1M tokens), you aren't safe.

Research shows that attention isn't distributed equally. We call it the "U-Shaped" Performance Curve.

~ The Beginning: The model remembers your system prompt perfectly.
~ The End: The model remembers your last question perfectly (Recency Bias).
~ The Middle: The muddy swamp.

Accuracy
^
| * *
| * *
| * *
| * *
| * *
| * * *
-----------------------------> Position in Prompt
[Start] [Middle] [End]

If you stuff a 50-page PDF into the prompt, the specific fact you need on Page 25 is statistically the most likely to be hallucinated or ignored. It's just like me forgetting the subplot from the middle of Tenet.

So, how do we fix it?

Don't be lazy. Don't just "dump it in context" because the window allows it.

- Use RAG (Retrieval Augmented Generation): Only fetch the specific snippets relevant to the query.

- Summarization Chains: Compress the "middle" history into tight summaries so the model doesn't lose the thread.

- Sanitize Data: Remove whitespace and useless headers before tokenizing.

The Context Window is a scarce resource, not a garbage dump.

Treat it like your brain during a Christopher Nolan movie: Focus only on what matters right now, or you're going to lose the plot.

Which topic would you want me to cover next?

[Chit-Akasa @deepshikhxr]

we treat llms like fragile geniuses but feed them data formatted like a ransom note.

"why is it hallucinating?"

buddy, you fed it a pdf parsed by a script from 2013. it thinks the page number is part of the sentence.