FP4 Brain

AI researcher ◆ post-training ◆ bare metal ◆ dev ◆ neovim + tmux ◆ building frameworks that help make sense of the future of software development

[FP4 Brain @thefp4brain]

https://t.co/kCCT74sgos

[FP4 Brain @thefp4brain]

almost 1k downloads of these two in under 24 hours. Nice

[FP4 Brain @thefp4brain]

What a lot of people get wrong is that it's not as simple as "if you dislike AI = you're just bad at it"

Imagine this being your first interaction with AI:

friend had an AI-pilled coworker who would generate 10k LoC PRs. And while I have my issues with it, this wasn't even the problem. The actual problem was they were bad at communicating and understanding what they did. So they would generate 10+ page pdf reports about it. So I'm sure you can see how that would slow down dev time for everyone (and, most importantly, disseminate the maximum LLMs=bad)

I love AI and use it daily, but interestingly enough, most of my bubble despises it. And I believe there is so, so much value in the bad stuff, the horror stories about AI and the productivity sink

These discussions are incredibly important because I have come to realize educating people about when to use AI and how to use it is a massive productivity boost for everyone. Because in the end, we all interact with people

[FP4 Brain @thefp4brain]

quantized a model on huggingface expecting nothing and it crossed 1k downloads. that's cool

[FP4 Brain @thefp4brain]

@dreamsofcode_io @icanvardar nah, i'm sure it's absolute and we should keep arguing with each other

[FP4 Brain @thefp4brain]

@0xSero there's a good chance that even after the specialized REAPs, they still share quite a bit of overlap. I like the idea of loading a "base" version that contains most of the expert overlap, and then hot-swapping only the experts that change. Think git diff or the react diffing algo

[FP4 Brain @thefp4brain]

Been running some reap+quantization experiments to get a good feel of the tradeoffs

Basically 84.8% memory reduction (56.9 → 8.7 GiB) while retaining ~70% of MMLU-Pro accuracy and gaining 20% generation throughput (tok/s)

Admittedly, as cool as it seems it's not anything new. I'm just going through the process to really understand it. Might do a blog post soon

https://t.co/SZv5HslanM

[FP4 Brain @thefp4brain]

@VukRosic99 💯💯

[FP4 Brain @thefp4brain]

@VukRosic99 i see it as a way to trade my reputation for research iteration speed 🙃

[FP4 Brain @thefp4brain]

@yacinelearning @juliarturc it's me. i'm the researcher like this that has to be force stopped

[FP4 Brain @thefp4brain]

remember that whole discussion where really good pcs make for bad programs (and bad programmers), because they have a lot of leeway in terms of memory and processing power? i think something similar is going to happen in AI, where bad practices and architectural decisions will be remediated by using bigger, more capable models. Interestingly enough, the smarter AI methods and architectural changes often come from labs with fewer resources

[FP4 Brain @thefp4brain]

damn, you guys are savage. I can't seem to find 8xh100 clusters to run my stuff

[FP4 Brain @thefp4brain]

@wiedymi Oh no, technically I agree with you! But I mean in terms of what even is worth doing. Just look at the sheer number of people looking for ideas. We have tons of builders idle because they don't know what to build towards

[FP4 Brain @thefp4brain]

@willccbb big brain move would be to have replaced that message for the cloudflare error page so people would blame them instead 👀

[FP4 Brain @thefp4brain]

@djCodeRed_ @nullvoxpopuli damn, i got baited. was about to plead for better search filters

[FP4 Brain @thefp4brain]

I'd say not only summarization, but text transformation in general is common. It's stupid but one of the most disruptive applications I built was just a tone shift LLM call for teachers to give brutally honest student feedback and convert its tone to professional feedback. It actually raised morale as well, because teachers could now vent about what a giant asshole some kid was and that would be incorporated in the feedback (but professionally and appropriately)

[FP4 Brain @thefp4brain]

@dreamsofcode_io @HSVSphere which switches do you use on the voyager? 👀

[FP4 Brain @thefp4brain]

@nullvoxpopuli tell that to protoss players 😅

[FP4 Brain @thefp4brain]

macOS fullscreen moves the app to its own space and I hate it

so I built my own fake fullscreen that doesn't banish apps from their current workspace

hover to peek, shortcut to toggle. per-workspace config. The rounded window corners were a pain to handle but it was worth it.

Really cool for keeping me immersed and focused