Frederik Schubert

Computational Scientist at Flexcompute

[Frederik Schubert @fschubert]

@badlogicgames @bentlegen Prompting gpt 5.5 to “push back” works ok-ish. opus otoh can’t help but find your idea interesting no matter how much you tell it to push back

[Seungwook Han @seungwookh]

Can language models learn useful priors without ever seeing language?

We pre-pre-train transformers on neural cellular automata — fully synthetic, zero language. This improves language modeling by up to 6%, speeds up convergence by 40%, and strengthens downstream reasoning.

Surprisingly, it even beats pre-pre-training on natural text!

Blog: https://t.co/Pni0RsIcxL

(1/n)

[Mario Zechner @badlogicgames]

recommended reading. complements @mitsuhiko and @thorstenball

https://t.co/ZbwVufwp6a

[Frederik Schubert @fschubert]

@thsottiaux 1. Make the session id available to the model so that it can reference it in commits.
2. A capable Hook system, eg in plan mode I would like the model to narrate (TTS) context while presenting the multiple choice questions.
3. Automatic forking in chats to preserve tokens.

[Frederik Schubert @fschubert]

@badlogicgames Yes, from a technical standpoint this is straightforward. But I wonder what would need to happen for us to trust them more so that they can be this autonomous. At which point is the model behavior reliable enough that we can tell them some safeguards and let them roam?

[Frederik Schubert @fschubert]

@deredleritt3r @petergostev What a sentence when viewed from 2 years ago, yet here we are… computation/search in natural language still baffles me.

[Frederik Schubert @fschubert]

@giffmana @crystalsssup I always thought that was a bit simplistic. At least it hinges on the scope of understanding. Just because someone can’t explain the inner workings of something doesn’t imply not understanding its function. But that functionality might be hard to convey verbally.

[Frederik Schubert @fschubert]

@giffmana @RichardSSutton @eigenrobot I think this breaks down to the definition of state and action. Either the state is described by the language or can be inferred from it (no pure imitation learning) or not (imitation of actions).

[outside five sigma @jwt0625]

sharing this to remind both you and myself, put your work out there openly, put it on your own website so you control the process. Share knowledge in all forms, from making short-form videos that capture a few second in someone's attention span, to marble sculptures that last centuries. Knowledge only lives by being shared.

[RL_Conference @RL_Conference]

RLC accepted papers are out! Check them out or come see them in August:
https://t.co/g06D7bOdkQ

[George Salafatinos @GSalafatinos]

@aryehazan @bremen79 The way I did it was progressively asking it to breakdown the problem into self-contained sub-questions that could be used to make a determination. I'd feed each one to a fresh context gemini and report back.

This particular subquestion produced the counter example in response.

[Georgi Gerganov @ggerganov]

Retweet if you want llama.cpp added here 👇

[Frederik Schubert @fschubert]

@francoisfleuret @freddiekarlbom I think RL subsumes bandits and denotes learning by trial and error, while reinforcing successful (i.e. high reward) actions in one or more states.

[Niels Rogge @NielsRogge]

Unpopular opinion: benchmarks like these are moving the field in the wrong direction

No I don't want an AI to be able to memorize (useless?) questions like "How many paired tendons are supported by a sesamoid bone?" in its weights

I want the "intern", as @karpathy is suggesting

[Frederik Schubert @fschubert]

@paulgauthier Maybe something along the lines of https://t.co/OyacjZxDrk

[Taelin @VictorTaelin]

IT WORKS! Demo time 🥳
(next 10x productivity bump is here?)

Suppose you must refactor a large codebase, e.g.:

> "use I32 instead of U32 as the native number type"

That task, by itself, is simple enough to be automated by Sonnet alone, right? Problem is: what if the codebase is too large to fit on the AI's context, and RAG solutions / editors like Cursor can't precisely isolate which parts require edit (because it needs some semantic reasoning)? What now? Do we fallback to human coding, as did the old Mayans?

Of course not! We just split the codebase in chunks, and send each chunk to ~500 DeepSeek instances in parallel (!). Each instance then decides if given block should be edited or not. This is viable because DeepSeek is smart, fast and really cheap. Finally, we send the blocks that need edit to Sonnet, in a single prompt, and let it work normally. The entire process takes less than a minute, is cheap enough, and very effective. We conclude with a 'git diff' for manual review.

In short, AI has came to grep search & replace. Since this script feels like dropping a bomb in your codebase, I'm calling it AoE, and it is available on VictorTaelin/AI-Scripts. Enjoy!

Here's the demo:

[Frederik Schubert @fschubert]

@ymahlau We also evaluated fdtdx against other popular FDTD solvers and found it to decrease the gap in performance to proprietary solutions such as tidy3d.
See the whitepaper for more information: https://t.co/t7i23qrq47

[Frederik Schubert @fschubert]

📢 Introducing fdtdx: A #jax based FDTD solver that combines blazing-fast performance with automatic differentiation for electromagnetic simulations and inverse design!

[Frederik Schubert @fschubert]

Have a look at our preprint https://t.co/UvzHyN1iay for more results or watch out for the talk of my colleague @ymahlau at the Photonics West conference in January 2025 https://t.co/Km4so0ErIg