Lukas Fisch

Don't panic! Just build!

[Lukas Fisch @codingfisch]

🧠 Exciting news for #neuroscience! Launching deepmriprep: Voxel-based Morphometry (VBM) preprocessing via neural networks in ~10 seconds per #brain image 🚀

🔗 Preprint: https://t.co/Es2lMhEDUu
🔗 GitHub: https://t.co/e7IDmkqvNT

Install via "pip install deepmriprep"

[Lukas Fisch @codingfisch]

@___Harald___ @teslavangelist Interesting. Will it be mainly the current model (RL trained with world model) that asks the large general (LLM like) model once higher level reasoning is needed? Or will it be the large model all the time? The second option feels like less 9s

[Lukas Fisch @codingfisch]

@___Harald___ @teslavangelist Do you really think that LLMs will bring the missing 9s of reliability in self-driving?

[Lukas Fisch @codingfisch]

@___Harald___ Bitter lesson as in "just scale" or "shit we need a better method that scales"? 🤓

[Andreas Kling @awesomekling]

The only real moat is giving a shit.

[Lukas Fisch @codingfisch]

@__tinygrad__ @tenstorrent @Qualcomm @intel @cristianoamon Hopefully you'll get to robotics once tinygrad is savely set up towards all abstractions being "from the book". Fixing RL scaling (no exp. data needed) is THE blocker for robotics. Self-plug: I will publish a lib for fast & simple "Atari from pixels" RL research soon. Stay tuned!

[Lukas Fisch @codingfisch]

@__tinygrad__ @tenstorrent @Qualcomm @intel @cristianoamon Bought quite some AMD stock based on your investment (thanks btw!). Will maybe add some QCOM. Could be too late because margins are shrinking fast in the OpenAI/Anthropic business model that currently pumps up GPU valuations. Markets do not price that in yet imo

[Lukas Fisch @codingfisch]

@__tinygrad__ @tenstorrent @Qualcomm @intel @cristianoamon Wild that even with the clearly successful AMD case study you have to push so hard to lead the horse to water. That inertia shows how overstaffed these companies are. Hopefully coding agents will lead to more tiny corps!

[Lukas Fisch @codingfisch]

@__tinygrad__ Are you bullish on RL finetuning of LLMs going forward? It is sufficient to squeeze out perf (math & code) but catastrophic forgetting bites back. Feels like for image (instead of token) input LLMs are too far from Kolmogorov complexity to be scaled to AGI

[Lukas Fisch @codingfisch]

@timodonnell Cool! Any idea what causes those drops in throughput?

[Lukas Fisch @codingfisch]

@classiclarryd Makes these nice loss curves even more impressive 💪

[Lukas Fisch @codingfisch]

@___Harald___ The Curse of Dimensionality: The Cure of Composability

[Lukas Fisch @codingfisch]

@kjaved_ @chris_j_paxton Can you share specifics about the new OaK implementation? Options and Knowledge is clearly what we currently lack in RL! Instead of learning one blob of weights a composable set of learned things is needed. E.g. what OOM parameter count is it? Thousand, millions or billions?

[Lukas Fisch @codingfisch]

@kjaved_ @jsuarez "Atari Breakout in 4 seconds" and Physical Atari are two extremes. The first is too easy (not using pixels but [ball_x, ball_y...]). The second is too hard (takes too long to iterate on it). Currently working on something that tries to fill the gap...will take a few more weeks

[Lukas Fisch @codingfisch]

@kjaved_ @jsuarez The best part is...i have to wait for 6 hours. The pitfall of RL research in one sentence

[Lukas Fisch @codingfisch]

@mhutter42 The models show respect because they know that they are all slowly converging towards the AIXI model 😄

[Lukas Fisch @codingfisch]

@__tinygrad__ @bsilone Does the exabox fit in a bike shed? Would love that picture just for the lolz 😄

[Lukas Fisch @codingfisch]

@___Harald___ And this is the one true reward: ∃

[Lukas Fisch @codingfisch]

@OosGergVer @__tinygrad__ He should reconsider: Selling chips w solid firmware and OSS stack on top -> other people fixing the OSS stack -> possible alternative to CUDA moat

CUDA is why nvidia is worth that much