kfish

grim

[kfish @conradparker]

With great bugs comes great understanding

[Nikias Molina @NikiasMolina]

Apple’s Most Unique Event Ever 🤯🍏‼️

Apple flew me to Sydney for their 50th anniversary and it felt like stepping inside the future of creativity.

Watching the Sydney Opera House come alive with art made on iPad + Procreate was unreal.

11 local artists. One iconic canvas. Technology turned into emotion.

This is why we create.

#MadeOniPad #Apple50 #InvitedByApple

[Tim Cook @tim_cook]

Now this is breathtaking! Amazing to see the sails of @SydOperaHouse lit up with Australian artists’ designs made with @Procreate on iPad. We’re proud to support the next generation of Aussie talent and continue to champion creativity. #Apple50

[Procreate @Procreate]

Procreate Dreams 2.0.2 is now available 🎉 This patch update includes:
🔄Reverse Keyframes
✍️Flipbook track renames and highlight
🔈Separate Audio and Video
and more.

Watch the full video on YouTube for more details, and update through the AppStore today!

#ProcreateDreams

[Procreate @Procreate]

Watch our new intro video for Procreate Dreams 2, and learn everything you need to get moving.

Now live on our YouTube channel.

#Procreate #ProcreateDreams

[Procreate @Procreate]

Find out about everything arriving in the free Procreate Dreams 2 update, available now.
Head to our YouTube channel for the full video.

#ProcreateDreams #Procreate

[Procreate @Procreate]

It's almost time! Procreate Dreams 2 lands tomorrow.

[kfish @conradparker]

and humans will still play chess
and humans will still write software
and humans will still do math
and we'll dance, play sports and love
like we always did
for the love of it

[Taelin @VictorTaelin]

in a post-AGI world, people will simply get used to the fact that computers can solve cognitive problems quickly, and beat us in any cognitive domain, just like we're used to computers multiplying large numbers quickly now, or kicking our asses in chess and go. software and math will lose all their scarcity.

"pls create a MMORPG that is like Zelda: Breath of the Wild but with Pokémon instead, let me fly around the world as a Charizard and dive on the water like a Gyarados. give me an executable that I can send to my friends and we'll all be connected to the same world"

you press a button and, 3 minutes later, done

I doubt that won't work by 2027

"pls create a blockchain exactly like Bitcoin except it uses quantum resistant signatures like Lamport, and you can deploy smart contracts in a Lean-like language, and contracts are only accepted if they're formally verified to be correct w.r.t the following specs..."

you press a button and done, you have a hack-proof chain

"in three space dimensions and time, given an initial velocity field, there exists a vector velocity and a scalar pressure field, which are both smooth and globally defined, that solve the Navier–Stokes equations"

you press a button, and done, you get a solution

math is fundamentally easy, and this will break some ppl's worldviews. currently, math seems mystically hard, like chess once was, because we're are animals that struggle with it, only a few of us are capable of adding fractions, let alone working on the edge, so hard problems stand for a long time unsolved, we praise our geniuses.

but it isn't

once computers are doing it, that won't be a thing anymore. theorem proving will be as trivial as multiplying large numbers. the "uh duh but godel?" folk will still be confused. and computers will come up with incredibly simple, clean Lean proofs for impossibly hard problems. and mathematicians will yell that it is just some trick to satisfy the checker, that it isn't real math if we can't understand it. but then we'll ask the AI and it will kindly reveal the nature of a surprisingly clever mathematical structure that is so alien for our brains to come up with

and life will go on

automation will increase 100-fold
food and goods will be abundant
the price of everything will crash
other than things that can't be copied
like human time and attention
which will be on all time high

and humans will still play chess
and humans will still write software
and humans will still do math
and we'll dance, play sports and love
like we always did
for the love of it

software and math will lose their scarcity
computers will be truly general solvers

and we'll get used to it faster than you think

and life will go on

[kfish @conradparker]

@AtherionGG @HustleBitch_ Meditating in anticipation of its future memory

[kfish @conradparker]

@Pixel0Symphony @grok I want my feed to be posts like this all day forever (also math)

[Kelsey Piper @KelseyTuoc]

In civics class at the microschool I recently ran a game where I taught a lesson to half the kids and then they had to teach a lesson to the other half, with prizes based on the second batch of kids' mastery. It was a followup to a couple months of lessons we'd done about the topic of how education itself works.

For a lot of those lessons, I declared that we were now a different school: Oakland STRICT, or Oakland NICE, or Oakland SILLY. Oakland STRICT did drills all day. Oakland NICE always told you you were a genius, whether you got the right answer or not. At Oakland SILLY I had no classroom control, etc etc. The kids would talk after the lessons about how much they learned, how much fun they had, and whether that's how they'd want a school to teach if they really wanted to learn the material.

Some of the kids ended up concluding that a strict school was the best way to learn; some felt that a fun school was (as long as classroom control was maintained); none actually liked the nice school that pretended you were doing well when you weren't. When they were trying to run a lesson for their classmates, and highly motivated to get their classmates to master the material, they tried to be strict, because it seemed to them that that was what worked best.

(Interestingly, they're really bad at being strict - they yell at their classmates, and had to be told not to physically sit on them to get them to pay attention, but they hadn't seemed to notice that a lot of 'strict' classroom management is about gravitas, instantaneous response to disruptions, and rapid-fire constrained engagement, not about 'being mean', so when they tried to imitate it it didn't work.)

I do a lot of civics lessons about how learning works/how schools work because to the kids it's immediately and apparently among the most interesting subjects. They go to schools; they have special access to information about what makes learning work for them; 'how do kids learn' is a question that lends itself to learning through experimentation and practice and which really, really matters.

All of these kids would be appalled if I proposed making their education fair by expecting less of all of them and giving them good grades whether they work hard or not. Because that's not fair at all, actually; because they take fierce pride in working hard and seeing results; because our job is to challenge them and set high expectations they rise to. It is a betrayal of these kids to expect less of them.

[Gerard Rennick @RennickGBR]

Just a short message to let you know the Online Safety Bill has been referred to a Senate Inquiry. Unfortunately you only have 24 hours to make a submission and the inquiry will finish by the 26th which is next Tuesday.

If you do have time to lodge a submission please visit the link below to do so.

I will be voting against the bill.

https://t.co/rHZ5KVcR2J

#auspol

[kfish @conradparker]

@jaspervdj Love it, will be porting all enterprise code to turnstyle. It's graphical so it should be trivial to debug, PMs love it.

[kfish @conradparker]

Excellent general advice for designing software APIs

[kfish @conradparker]

@karpathy 🐐🐐🐐

https://t.co/B5XpeNmVMK

[kfish @conradparker]

Today I'm releasing a C++ port of the code and explanations for @karpathy's micrograd, and episode 1 of Neural Nets: Zero to Hero.

https://t.co/0PkjaqXZBT

Check it out for:
🚧 Details of implementing autodiff
🤯 Gratuituous use of templates and concepts
🌱 Fresh vibes on AI

[Rohan Paul @rohanpaul_ai]

This is really a 'WOW' paper. 🤯

Claims that MatMul operations can be completely eliminated from LLMs while maintaining strong performance at billion-parameter scales and by utilizing an optimized kernel during inference, their model’s memory consumption can be reduced by more than 10× compared to unoptimized models. 🤯

'Scalable MatMul-free Language Modeling'

Concludes that it is possible to create the first scalable MatMul-free LLM that achieves performance on par with state-of-the-art Transformers at billion-parameter scales.

📌 The proposed MatMul-free LLM replaces MatMul operations in dense layers with ternary accumulations using weights constrained to {-1, 0, +1}. This reduces computational cost and memory utilization while preserving network expressiveness.

📌 To remove MatMul from self-attention, the Gated Recurrent Unit (GRU) is optimized to rely solely on element-wise products, creating the MatMul-free Linear GRU (MLGRU) token mixer. The MLGRU simplifies the GRU by removing hidden-state related weights, enabling parallel computation, and replacing remaining weights with ternary matrices.

📌 For MatMul-free channel mixing, the Gated Linear Unit (GLU) is adapted to use BitLinear layers with ternary weights, eliminating expensive MatMuls while maintaining effectiveness in mixing information across channels.

📌 The paper introduces a hardware-efficient fused BitLinear layer that optimizes RMSNorm and BitLinear operations. By fusing these operations and utilizing shared memory, training speed improves by 25.6% and memory consumption reduces by 61% over an unoptimized baseline.

📌 Experimental results show that the MatMul-free LLM achieves competitive performance compared to Transformer++ baselines on downstream tasks, with the performance gap narrowing as model size increases. The scaling law projections suggest MatMul-free LLM can outperform Transformer++ in efficiency and potentially in loss when scaled up.

📌 A custom FPGA accelerator is built to exploit the lightweight operations of the MatMul-free LLM. The accelerator processes billion-parameter scale models at 13W beyond human-readable throughput, demonstrating the potential for brain-like efficiency in future lightweight LLMs.

[varepsilon @var_epsilon]

this looks interesting

[Andrej Karpathy @karpathy]

Have you ever wanted to train LLMs in pure C without 245MB of PyTorch and 107MB of cPython? No? Well now you can! With llm.c:
https://t.co/PoGTZIwASL

To start, implements GPT-2 training on CPU/fp32 in only ~1,000 lines of clean code. It compiles and runs instantly, and exactly matches the PyTorch reference implementation.

I chose GPT-2 to start because it is the grand-daddy of LLMs, the first time the LLM stack was put together in a recognizably modern form, and with model weights available.

[kfish @conradparker]

@ixahmedxii alias gtfo="git fetch --prune origin"

[nisten🇨🇦e/acc @nisten]

Gen-z is now using deepfakes to teach each other calculus 🤣 @OnlockLearning 👏