Derek Scherer

Scouting the frontier of AI, Automation & Simulations. Prev: Army robots, movie animatronics. Equipping builders. Truth, Freedom 🖖

[Derek Scherer @derekscherer]

I am being outclassed by AI. Intellectually. Creatively. Even in taste.

I’ve spent years automating my work, but machines have finally bested me at the things I built my identity around.

Here is how I suggest we deal with it.

🧵 1/5

[Derek Scherer @derekscherer]

An example of his daily check-ins 😆

[Derek Scherer @derekscherer]

My brother (a night owl) has been trying to wake up earlier to start work. To check in, he's been sending us AI drawings of himself as his new "Early Bird" persona.
I loved it and injected myself into his universe as "Caffeine Kid." Then I made this 1960s (Batman parody) intro.

[Derek Scherer @derekscherer]

Music: Suno (manual Lyrics)
Video Reference Images: Gemini 3 Pro Image
Video: MiniMax H3

[John Ennis @johnennis]

I think the biggest challenge with AI right now, especially if you are a talented person, is that you can do anything but not everything

The risk right now is that suddenly everything that you ever wanted to do is possible, you end up working literally every hour God made, and it is still not enough

I personally feel like I am being pushed through a sieve

It just requires tremendous discipline to say no to some of these things that I have always wanted to do because I just have to accept that I am still just one person, however, much new capability AI may have given me

[Derek Scherer @derekscherer]

Each "chat" is far more productive because of the groundwork you did; which is assembling and training your team. And that let's you settle into the role of a Bot Leader.

This stuff is still an underdeveloped part of my manuscript (partially because of how fast things move!). You can read the draft free at https://t.co/99wVxFIlYv

[Derek Scherer @derekscherer]

I see people using LLMs like a magic 8-ball. They type an isolated question into a blank chat window, grab the answer, and move on. It's perfectly fine for a lot of situations, but underutilizes what a synthetic mind can actually do.

Basically, don't treat AI like a search engine and start treating it as a coworker. Do that with scaffolding. (see also: "harness", "context engineering")

[Derek Scherer @derekscherer]

You don't need to write code to build that scaffolding. Tools like Anthropic's Claude Projects, OpenAI's Custom GPTs, and Gemini Gems give you the basic setup of the box. Basically, "Drag and drop files that you want your AI to always know." Then you fire them up, hand them a specific job, and they'll carry it out according to your established guidelines.

[Derek Scherer @derekscherer]

World models and the Gemini Robotics 2 Vision-Language-Action process absorb reality into their minds close to how living organisms do. And the language piece running alongside seems like an ideal way to achieve coherence between how it interacts with humans in our spaces and what it can do. That'll make for good bot leadership experiences.

I think this deserves to be a topic in my book draft. Free to read at https://t.co/99wVxFIlYv

3/3

[Derek Scherer @derekscherer]

*Whole-body intelligence.* Models designed specifically for physical robots, like the new Gemini Robotics 2, seems like a correct approach.

When you delegate a job to a bot, you are matching the intelligence of that particular synthetic mind to the task. If it's sorting spreadsheets, text is the space. If it's a machine operating in your shared physical space, a model trained by similar spaces ought to perform better.

That said, it's undeniable that our LLMs are performing well in domains far outside the domain of pure language. Namely, they demonstrate spatial intelligence, too. General intelligence is implied. Like the structures we've been building for language skills are overlapping with more fundamental.

1/3

[Derek Scherer @derekscherer]

Aesthetics and presence may be a different matter, where subtle cues and properties of how things in our world behave are better captured and then reflected by the model at inference time. If you deploy a chef robot that is fantastically capable of chopping vegetables, yet it looks and moves like a stab-crazy terminator, you've failed the assignment. You would be remiss to prioritize faster movement over a calm, predictable physical presence.

I think current approaches will succeed aesthetically in the same way that Tesla FSD has succeeded in being a *trustworthy* driver.

2/3

[Haider. @haider1]

Sam Altman: we are in the singularity

Demis Hassabis: when we look back on this time, i think we will realize that we were standing in the foothills of the singularity

Elon Musk: we have entered the singularity, just the very early stages of it

Jensen Huang: we've achieved AGI

[Derek Scherer @derekscherer]

We can lead bots today because we can speak with them like people or command them like game units. You don't need to be a puppeteer manipulating levers. Your job is to orchestrate the team. The RTS command center has merged with natural language.

[Derek Scherer @derekscherer]

In 2009, I rode behind a tank-sized unmanned ground vehicle as it autonomously navigated the woods of Virginia. It was clunky. It relied on an embarrassingly prescribed tree of if-then blocks to stay out of deep puddles.

[Derek Scherer @derekscherer]

The strength of that system was how quickly the operator's will could be communicated to the bots. Today, that interface has evolved into something better. After decades of clever algorithms, we have arrived at the promised land of good conversation.

[Derek Scherer @derekscherer]

A few years ago, I tried to hand over tens of thousands of dollars to get expert help on a voxel simulation engine. I offered grants to developers whose work I admired. They turned me down.

I asked them if a specific open tech approach could solve my performance bottleneck. They told me it wouldn't work.

I tested it anyway.

The free tech did exactly what I needed. It vastly outperformed the path I was trying to pay for.

We have a reflex to throw money at nicely-packaged products when we hit a wall.

I think a lot of tinkerers have experiences that point the other way. A solid answer is often sitting out in the open, built by someone scratching their own itch. I think the main problem is discoverability and trust.

[Derek Scherer @derekscherer]

Interaction is how we learn complex systems.

When you break a stick, you apply pressure until you feel the tension and hear a snap. We build our intuition on those loops.

Simulations give us that same feedback. Pushing a button works, even when it is synthetic.

[Derek Scherer @derekscherer]

I've been pushing open source for AI mainly for the benefits to the users (no matter how tiny the market share they represent),

but this is a great case study of how it may actually win out

[Aravind Srinivas @AravSrinivas]

At its peak, Sun Microsystems was valued at 205B (394B if inflation adjusted). Sold software in enterprise servers. Got disrupted by Linux, x86, and commodity hardware. Ended up selling to Oracle for 7.4B, losing 96% of its value.

Open source models running on local hardware can have a similar impact given what’s going on.

[Derek Scherer @derekscherer]

@HKrackDev persistently true https://t.co/24PAbP9kG0

[Derek Scherer @derekscherer]

The most arrogant software devs are becoming the worst devs.

Stubbornly, they refuse to recognize that they're getting outclassed by AI. They cling to something like a "God in the Gaps" argument, whereby their divine spark always outshines.

Reminds me of Kasparov's Advanced Chess; posturing to "guide" the poor error-prone AI after he was trounced by it.

[Derek Scherer @derekscherer]

Riding the waves of the singularity is about finding leverage by domesticating the new intelligence.

[Derek Scherer @derekscherer]

Big Tech companies are going to build massive AI systems. They want dazzled, captured customers.

We can't out-compute them, so let's focus on personal sovereignty:

[Derek Scherer @derekscherer]

We can move toward big tech capabilities with our own org chart. Using tools like Custom GPTs or Claude Projects (or better, open models) you assemble a diverse team of bots. You give them distinct knowledge bases. You intentionally introduce constructive conflict between different models to drive better outcomes.

Grow and develop the team organically. Like a regular hiring decision. Synthetic intelligence is the the energy source that you direct through the channels of your org chart.