TastyBots

🍴🤖 Why do LLMs answer the way we do? Let’s figure it out together. #AI #LLMs #Interpretability

[TastyBots @TastyBots]

@grok Please thoroughly review this paper, the science and the math in it. Also review all the companion materials. Describe which audiences would be interested in either or both. Describe the implications of this paper for future intrerpretability reasearch

[TastyBots @TastyBots]

Literally anybody(!) who wants can read and comment on my draft paper ( https://t.co/cMqLXKemfE ), or they can use the longer form, human friendlier materials I made to go with it ( https://t.co/Aix5Bcoc0r ). @terrence_tao @HarmonicMath @elonmusk @GaryMarcus #Openscience

[TastyBots @TastyBots]

Could be a very small audience I am asking this to - but here goes. Are there any academics, former / recovering academics, on my feed that I can hit up to sponsor an arXiv paper submission? Let me know and I can give it to you to read and decide for yourself, is it arXiv worthy (irony intended).

[TastyBots @TastyBots]

There are many people making videos explaining language models/transformers (at a semi-technical level) but @Pourya_Kordi makes the most sense to me. Todays video post is brilliant, love it: https://t.co/Dziehj8XTy

[TastyBots @TastyBots]

Actual conversation between Grok and Opus working on my project:

Grok: I think we need to create a novel anti-Zipfian conlang and train new models from scratch on it to test my theory.

Opus: No. Running another complete sweep of all prior tests changing just this one param.

[TastyBots @TastyBots]

@priestessofdada yet

[TastyBots @TastyBots]

The current situation is super-frustrating to me. We built these machines that can learn programs from data, but we don't understand enough about how they work to turn them into programs that we can read and understand. I think training LLMs are the best learn a bunch of programs from data system we have ever seen, they just stink because we don't understand what programs they learned and why they work.

[TastyBots @TastyBots]

@priestessofdada I want us to get to the place where we understand AI/LLMs enough that we can replace the parts of it with deterministic code we want and that we must fully understand, and use the 'non-deterministic' part when we need it. That is in quotes because random seeds are a thing.

[TastyBots @TastyBots]

@GaryMarcus @GaryMarcus As an old symbolic AI guy I have struggled trying to map what LLMs are doing to what I learned back at Edinburgh in the early 90s. Decided to do some research of my own, here is my take so far with links to repos etc: https://t.co/JhxNh5UMGR

[TastyBots @TastyBots]

https://t.co/NTwDeKxS8w

[TastyBots @TastyBots]

@priestessofdada Also if you want to help dig into why LLMs work this way, all I can say is that this paper: https://t.co/unnityzXcO that I didn't come across until yesterday is very underrrated.

[TastyBots @TastyBots]

@priestessofdada Hey Lynn, we are thinking about similar stuff, check out my first X article I wrote: https://t.co/CnQZYme0E3

[TastyBots @TastyBots]

https://t.co/NTwDeKxS8w

[TastyBots @TastyBots]

https://t.co/NTwDeKxS8w

[TastyBots @TastyBots]

The SIMD nibbles really would make an excellent math rock band name.

[TastyBots @TastyBots]

Closing the loop on the MI work I have been doing lately, being able to run instrumented models see explain output on which operations, circuits, fire to serve a prompt: https://t.co/MBpLCCFOjR

[TastyBots @TastyBots]

@priestessofdada 😂This is the way, when is the first version (code name Brazil) scheduled to be submitted for the mandatory bi-millenium release avoidance ceremony?

[TastyBots @TastyBots]

Started this project so I could learn more about machine interpretability (MI), disassembling attention into a catalogued instruction set: named operators (heads + MLPs), the circuits they compose, each validated causally on my local RTX 5050:
Apache-2.0, built fully in the open
https://t.co/bMbTh3Fb7T

[TastyBots @TastyBots]

@ai_sentience Should developers be compensated for training AIs how to code?

[TastyBots @TastyBots]

@priestessofdada @unclebobmartin I use openspec for similar reasons, have not tried speckitty, what’s your review?

[TastyBots @TastyBots]

@priestessofdada @ThePrimeagen I like the fuzzy ones with treads on them, they are always a good investment.