David Rose 🫧

Randomness and dynamical systems. ML engineer.

[David Rose 🫧 @drose101]

AWS prices have been surprisingly stable recently. Getting close to a pivot point away from local or rented hardware. I’m seeing around up to 30/40% saturation needed now to make a lower level investment worth it at this point rather than just paying spot prices as needed.

[David Rose 🫧 @drose101]

@eisokant @PengmingWang @NikolayZinov @pmarca I like to imagine I have the same great thoughts, but I’m just not driven enough to get up and put in the effort to write them down. Like my dreams, just built up and torn back down each day that goes by.

[David Rose 🫧 @drose101]

I haven’t read my code in months (over a year?). “Hey Claude review this code make sure it’s not stupid” literally what I would be doing. Just stack more layers like that old neural network jester meme.

Reading the code is injecting yourself as another layer of review, that’s good! But you can just automate that too. The key is a NEW model without the context bias of the model that wrote the code. I built a system called Hatch that works well. Just ask my agent to hatch a few reviews and it passes the task to a few agents in parallel to return a steelman or something like that.

You should balance the different models too. If im coding with opus then a deploy or merge requires a sol and k3 review.

[David Rose 🫧 @drose101]

I haven’t read my code in months (over a year?). “Hey Claude review this code make sure it’s not stupid” literally what I would be doing. Just stack more layers like that old neural network jester meme.

Reading the code is injecting yourself as another layer of review, that’s good! But you can just automate that too. The key is a NEW model without the context bias of the model that wrote the code. I built a system called Hatch that works well. Just ask my agent to hatch a few reviews and it passes the task to a few agents in parallel to return a steelman or something like that.

You should balance the different models too. If im coding with opus then a deploy or merge requires a sol and k3 review.

[Guillermo Rauch @rauchg]

If you’re not reading the code, whether explicitly or through agentic inquiry, one or more of these is true:

○ You’re a beginner
○ Software is throwaway
○ You’re prototyping
○ You have no users / revenue
○ You’re taking on debt & risk
○ Your problems are basic

And btw. All of this is fine. But the reality is that models are still not at the “full autonomy” stage yet.

They make rookie mistakes, they go down bad architectural paths. I just had the best model in the world add a nonsensical 700ms delay to “settle” something and it told me “you’re right, I was cargo-culting” 🤨

I am on the camp that this need will diminish more and more. Most code is indeed going to be assembly-like. But we also have the global internet and software infrastructure riding on these models and narrative, and we have to respect that.

[David Rose 🫧 @drose101]

The world if deepseek flash had vision capabilities.

[David Rose 🫧 @drose101]

The backplane to power my vehicle local AI has completed first draft. I will soon have 157 trillion INT8 ops per second in my car!

[David Rose 🫧 @drose101]

Picked up a USB cable tester. This thing is going to save me so much frustration!

[David Rose 🫧 @drose101]

I kept trying to play out this situation with the latest chat models , they seemed so confident that it would not be a reliable path of advantage. - - Disagree. The responses had weird waffling and then I just sort of moved on.

But tldr ya the entire trained model is just sitting there in each car? Like just download it? Despite the PSAs of yore I would download a car

[David Rose 🫧 @drose101]

@mattryanxyz @andrewmccalip This is new to me! Looks awesome thanks

[David Rose 🫧 @drose101]

Interesting peek behind the curtain in Gemini. Appears the LLM generation itself produces (elicits) follow up queries that the user can tap on in the UI, along with further reasoning on why.

Here it rendered into the assistant message object itself.

[David Rose 🫧 @drose101]

The next phase of my self driving car project begins. Hardware has all arrived, will run it on bench and then migrate to the vehicle with more permanent wiring.

[David Rose 🫧 @drose101]

with the confluence of 3d printing, send cut send, and the latest AI models using CAD software it feels like whole new worlds are opening up.

Needed a case for a little DC converter board, posted a pic and the dimensions to @adamdotnew extension in Fusion and 30 minutes later I had it!

Unfortunately the single prompt to create this case used my entire $15 balance.

[David Rose 🫧 @drose101]

I must inform that after two days of using Luna max it makes dumb decisions and doesn’t follow through with tasks well enough. Back to Sol!

[David Rose 🫧 @drose101]

With the gpt 5.6 Luna 80% price drop I figured what if I switched my tasks to 'max' effort Luna in Codex?

Previously I ignored this model for work, just figured i could run sol on low effort and call it a day. But today I ran some side-by-side tasks in Opus High and Luna Max and differences exist but are mostly immaterial. i like it.

also its so much faster so feels better when watching it stream into the terminal

[David Rose 🫧 @drose101]

My intuition is that the real gist of an agent session is just the user+ai messages, while tool calls/responses make up almost the entire bulk of the context window.

seems like room for optimization! I analyzed 176B tokens from my agent transcript logs and came up with this

[David Rose 🫧 @drose101]

A trend to watch as small models keep improving is that you can't keep compressing knowledge and fact recall, or can you?

There is a long tail of world knowledge the models have a tough time storing based on current training paradigms (as most effort goes into RL/post-training now at least). The smaller models way underperform on evals like FACTS Parametric and SimpleQA-Verified, even when they are close on the reasoning evals!

But maybe this won't always be true, I found one paper that said under sufficient training, roughly 2 knowledge-bits per parameter is achievable https://t.co/NRa97NCBKU while another said 3-4 bits https://t.co/npTqerQK7Z, though those are quite favorable ideal scenarios.

With strong compression you could get english wikipedia down to ~4-5GB which seems to easily be stored in the 167GB of mixed FP4/FP8 parameters in Deepseek v4 Flash

that makes me wonder... why dont we just set aside some of the parameter budget for a fact-like info storage system directly integrated into the architecture

[Artificial Analysis @ArtificialAnlys]

DeepSeek V4 Flash 0731 is now open weights!

@deepseek_ai has just released the weights for its new flash tier model, DeepSeek V4 Flash 0731. With a score of 50 on the Artificial Analysis Intelligence Index, it lands among the top 3 open weights models on the leaderboard. The weights are released under the MIT license, allowing unrestricted commercial use and modification.

DeepSeek V4 Flash 0731 shares identical architecture and pricing with the earlier DeepSeek V4 Flash. At a size of 284B total parameters (13B active), released in mixed FP4/FP8 precision at ~167GB total file size, it lands on our Pareto frontier for Intelligence Index vs. Total Parameters. Among open weights models, DeepSeek V4 Flash 0731 delivers a significant leap in intelligence for its size class. DeepSeek V4 Flash 0731 is also available now through DeepSeek's first-party API.

Check out Artificial Analysis to compare DeepSeek V4 Flash 0731 with other leading open weights and proprietary models: https://t.co/zeUIrzHIOC

[David Rose 🫧 @drose101]

My laptop is back at the Airbnb with my AI agents building for me, while I sit in a Tesla I rented off Turo driving me around with its own internal AI. I love technology

[David Rose 🫧 @drose101]

@Aella_Girl @misraetel Like the prompts we use for our AI. We keep adding rarely subtracting. Each addition has targeted benefits but subtle diffuse externalities

[David Rose 🫧 @drose101]

We are in a weird point in time where many agents still tie together compute and interface on the same device.

Codex does this right, despite a nice little cli command it has a backend server and separate TUI you interact with. Claude fails.

Ya it’s easy to just run everything on your laptop now but I’m growing weary of energy usage. Compute should be fungible. It’s all just a function processing inputs. Inputs are messages. Messages are the context and latest user prompt for the turn. Serverless kind of makes sense here too. The labs are starting to push their cloud offerings. But we can do better. I continue to think on this