David Jimenez

I am a technology enthusiast | MSc Student at TUM - Munich | Computer Engineering

[David Jimenez @jimezsa]

I prefer to use the term “Responsible vibe coding” instead of agentic engineering.xD

[David Jimenez @jimezsa]

@tri_dao very nice work guys!

[David Jimenez @jimezsa]

Finally finished this blog on a Flixbus from Munich to Freiburg.

The current research direction in efficient attention is genuinely fascinating a lot of recent work is actively pushing away from the transformer architecture, and it feels like the right way to go.

As part of my master's thesis, I'm sharing a short literature review of where the field is heading.

If the concepts are new to you, video 1 at the end is a great place to start.

https://t.co/lKQ72InxFL

[David Jimenez @jimezsa]

Nice visual summary of current sub-quadratic sota approaches.

[elie @eliebakouch]

gated deltanet 2 compared to previous linear attention methods (kimi delta attention, gated deltanet, mamba2)

each new variant adds finer control over what to decay, erase, and write in the state matrix

[elie @eliebakouch]

gated deltanet 2 compared to previous linear attention methods (kimi delta attention, gated deltanet, mamba2)

each new variant adds finer control over what to decay, erase, and write in the state matrix

[David Jimenez @jimezsa]

@ahatamiz1 Very nice work 🚀

[Yifan Zhang @yifanzhang_]

Higher-Order Linear Attention Models Are RNNs/SSMs:

Generalizing State-Space Duality to higher-order linear attention.

It’s getting wild.

https://t.co/vUBN3nDFMy

[David Jimenez @jimezsa]

SSMs are the thing. Very excited about all the research happening in this direction.

[David Jimenez @jimezsa]

@jbhuang0604 I love your videos prof. Jia-Bin, thanks for sharing.
Greetings from Munich

[David Jimenez @jimezsa]

@cmpatino_ @andrew_n_carr Este es mi github parce https://t.co/fhLDgOJgHY

[David Jimenez @jimezsa]

@cmpatino_ @andrew_n_carr Chimba tu trabajo Carlos, estoy trabajando en algo parecido tambien :)

[David Jimenez @jimezsa]

codex is not slow, is really good. I love it.

[David Jimenez @jimezsa]

@pmarca hey Marc, what are you thinking ?

[David Jimenez @jimezsa]

General architecture diagram

[David Jimenez @jimezsa]

Over the past few weeks, I’ve been intensively participating in the OpenAI Parameter Golf Challenge:

Train a small language model under 16MB in less than 10 minutes on 8× NVIDIA H100 GPUs.

A few numbers from the journey:
-> 112+ experiments run
-> +1.000EUR burned on GPUs via RunPod
Thank you @OpenAI for the financial support. :)

My approach was a hybrid autoregressive architecture with a latent diffusion component, where the diffusion part acted as a regularizer.

For the challenge.

I used my own project, OpenColab, to collaborate with multiple AI agents:

The Professor: Opus 4.6 in Claude Code
Set project direction and coordinated the overall strategy (Human taste and judgment remain essential for setting project direction).

Scout : Gemini 3.1 Pro Preview in Gemini CLI
Handled deep research on papers, and experiment spec generation.

The Engineer: Codex 5.4 xhigh in Codex CLI
Implemented scout’s spec.\

AutoResearch — Opus 4.6
Ran experiments overnight. See example below.

I didn’t manage to set a new record, but I came close and got some very interesting results. The network was able to learn faster with fewer steps than the autoregressive architecture, but because my hybrid implementation was slower per step, the autoregressive architecture was still able to outperform it overall.

OpenColab is inspired by @OpenClaw, @NanoClaw, and @Karpathy’s AutoResearch.

[Garry Tan @garrytan]

OpenClaw is the most significant innovation since ChatGPT and Claude Code

It’s the third shockwave that will be felt by all

[David Jimenez @jimezsa]

@garrytan thanks for sharing Garry.

[David Jimenez @jimezsa]

@karpathy Why does a question from two months ago appear in the context windows?

[Deepak Pathak @pathak2206]

We hosted Prof. Alyosha Efros (UC Berkeley) at @SkildAI! He didn't believe that robots could actually cook eggs reliably. :)
Tested back-to-back 5times without fail! One batch of scrambled eggs every ~2.5mins nonstop. The same model assembles a GPU on a server rack too.

[David Jimenez @jimezsa]

@pathak2206 @chris_j_paxton @SkildAI Beautiful 🔥