Aistis Raulinaitis

[Taelin @VictorTaelin]

Yet another massive breakthrough (the last one?)

This is probably getting annoying at this point, but I must report that a new discovery led to yet another 10x-20x speedups, and more.

This time, it came from noticing that certain parameter was affecting the interaction count. My intuition said it shouldn't. I spent 4 days thinking about this and making experiments. This led me to change key parts of the algorithm, and it worked. The result of this is: we're now able to search by increasing pattern-match complexity! This makes ALL functions be found *much* quicker.

Examples:
- 'inc' is down from 1500k to 51k interactions (20x)
- 'eq' is down from 580k to 94k interactions (6x)
- 'draw' is down from 800k to 100k interactions (8x)

Moreover, this new shape *allows us to search non-linear terms*. Functions like `mul`, which were previously unreachable (for not being linear), can now be synthesized perfectly fine (!!!). Technically, non-bizarre sorting functions like mergesort and quicksort should be synthesizable now. I wanna try it next!

Yes, before this update, we were searching with an unlimited pattern-match depth. Similar tools like Myth have a fixed bound. Now, NeoGen is still unlimited, but is capable of searching by increasing pattern-match complexity, which is only possible because optimal evaluation let us fully scan a given depth in finite time. This makes everything much faster, while remaining expressive.

Now I have a strong feeling this was the last breakthrough. The algorithm looks and feels like it is in the "optimal shape" and what's forward is a lot of engineering to micro-optimize, deploy, and explore. I really wanted this to work, and it does. Now is time to ask the next big question: what the hell do I do with that? 😅

[effectfully @effectfully]

"a sum type is only visually different than a product type" has got to be one of the dumbest takes I've seen on this platform

[Aistis Raulinaitis @Sheganinans]

@atroyn My thoughts on making base 60 numerals useful in a modern context: Pen & paper numeral forms, the old clay tablet numerals are useless. I spent a few hours tinkering with the idea, but I think it can be done way better. Ideally with better symmetries for all it's divisors.

[DHH @dhh]

If you're not doing over a million dollars in ARR for your SaaS app, you almost certainly don't need more than a single server. Chasing a 99.99% uptime in those early days by prematurely optimizing your resilience is a vast of effort that could have fueled features and bug fixes.

[Aistis Raulinaitis @Sheganinans]

@men_igus @PhysInHistory That would be an inverse Dirac. Since that is measuring the rate of speech. You'd want to measure the rate of silence.

[Nathan Marz @nathanmarz]

Static typing is better for large codebases if your development team is mediocre or weak. Dynamic typing is far superior for a strong team. If you think static typing is better, then there are huge improvements you can make in how you structure your code and how you test.

[Manuel Simoni @msimoni]

Uhm, wow. "Practical compilation of fexprs using partial evaluation: Fexprs can performantly replace macros in purely-functional Lisp" https://t.co/JP9semPlA9

[Aistis Raulinaitis @Sheganinans]

@JeTmAn1981 @ChShersh It's really easy, you just use records

[Aistis Raulinaitis @Sheganinans]

@BKAngryKing @ChShersh What is that reason?

[Aistis Raulinaitis @Sheganinans]

@PresAlpha @MartinShkreli Yup, I've got an 11 year old project on my GitHub which is a WebGL project (the predecessor to WebGPU): https://t.co/bUNRlbplQt

I mostly focus on backend development, but I've done everything from frontend, to embedded development (FPGAs), to AI/ML, and everything in between :)

[Andrej Karpathy @karpathy]

These 94 lines of code are everything that is needed to train a neural network. Everything else is just efficiency.

This is my earlier project Micrograd. It implements a scalar-valued auto-grad engine. You start with some numbers at the leafs (usually the input data and the neural network parameters), build up a computational graph with operations like + and * that mix them, and the graph ends with a single value at the very end (the loss). You then go backwards through the graph applying chain rule at each node to calculate the gradients. The gradients tell you how to nudge your parameters to decrease the loss (and hence improve your network).

Sometimes when things get too complicated, I come back to this code and just breathe a little. But ok ok you also do have to know what the computational graph should be (e.g. MLP -> Transformer), what the loss function should be (e.g. autoregressive/diffusion), how to best use the gradients for a parameter update (e.g. SGD -> AdamW) etc etc. But it is the core of what is mostly happening.

The 1986 paper from Rumelhart, Hinton, Williams that popularized and used this algorithm (backpropagation) for training neural nets:
https://t.co/f52IcDNitR
micrograd on Github: https://t.co/GaTd16jRnB
and my (now somewhat old) YouTube video where I very slowly build and explain:
https://t.co/EPGG6kd5Yz

[Aistis Raulinaitis @Sheganinans]

@remdesivoir @JenFiftyNine @MartinShkreli I misspoke, I worked with Martin during March-May.

[Naomi Saphra @nsaphra]

Modern generative models are trained to imitate human experts, but can they actually beat those experts? Our new paper uses imitative chess agents to explore when a model can "transcend" its training distribution and outperform every human it's trained on. https://t.co/oKsIh5nVBk

[Francis Stokes @fstokesman]

This is a thread about the CORDIC algorithm, and why it's been living rent-free in my brain for the last couple of weeks 🧵

[Jacob Pfau @jacob_pfau]

Do models need to reason in words to benefit from chain-of-thought tokens?

In our experiments, the answer is no! Models can perform on par with CoT using repeated '...' filler tokens.
This raises alignment concerns: Using filler, LMs can do hidden reasoning not visible in CoT🧵

[Vitalist Doxometrist @doxometrist]

I hate to break it to y'all, but Iron is never going to replace Bronze.
First of all, there's just not enough iron. Its only usable form is found in rare lumps that we don't know the source of. (Some say that they fall from the sky as a gift from the gods, but this has not been verified). These lumps are not large enough to work into anything much larger than a dagger, so we wouldn't be able to use it to replace all of bronze's uses even if it fell from the sky like rain.
Secondly, fire just isn't hot enough to fully melt iron, just to soften it, so that also limits what we can do with it. Some have suggested extracting iron from impure sources in rocks, but that would require an even more impossibly hot fire, plus other processes we wouldn't even know, and only the gods would have the ability to perform.
The thing is, we've had metallurgy for two thousand years, with many generations of smiths dedicating their whole lives to mastering and improving it. If it was possible to mass produce iron, somebody would have figured it out by now.
We are living at the pinnacle of humanity. Be grateful for what we have, we've come so far since the hunter gatherer times. The so-called "Iron Age" of the future is just a science fiction fantasy.

[LambdaConf 2025 @lambda_conf]

Join @arcfide's talk at #LambdaConf2024!

Discover the challenges encountered while building the Co-dfns APL compiler and how Nano-parsing emerged as a groundbreaking solution.

Join us: https://t.co/BcQ0ZYV2wy

#APLCompiler #ParsingProblems #NanoParsing #ErrorHandling

[Hamilton Greene 🐷🦀 @SIRHAMY]

A Brief Comparison of Modern Programming Languages:

* TypeScript
* Golang
* Elixir
* Rust
* F#

https://t.co/egj9l03F39

#typescript #golang #elixir #rust #fsharp