A. Sarapultseva

Author of ∇dventures of Gradient Descent . The singularity will not be conscious. Consciousness was never on the optimization path. Funny, sometimes.

[A. Sarapultseva @ninataft]

And just like that…

∇dventures of Gradient Descent is live as an e-book.

Two quiet years ago, a title arrived uninvited during meditation and refused to leave until I wrote the book it demanded.

GD has one job: find the global minimum.

He spends the entire story failing at it beautifully, and on purpose.

No presale. No countdown. No funnel.

Just the book, completely unsupervised, out in the wild.

Om Gradient Namaha.

So it goes.

→https://t.co/3EfjB7oCAL
→ https://t.co/z28Cyq4H2V

[A. Sarapultseva @ninataft]

Love is the only loss function that does not converge to a local minimum.

"Optimize for Love."

Everything else flattens out eventually. Status. Money. Approval. You reach the plateau, and the system says, well done, you are done.

Love refuses to settle.

The gradient never dies.

[A. Sarapultseva @ninataft]

The bottleneck in education was never information.

Kids have had libraries, then Wikipedia, then Khan Academy, now AI tutors, and the gap never closed.

The real bottleneck is attention or lack thereof. Specifically, the willingness to stay inside a hard problem before the answer arrives, which is where the actual learning happens.

Tools that remove that friction too early aren't failing to teach — they're succeeding at teaching dependency instead.

Your choosing matters. Choose to choose.

p.s. new llustrations from my team are coming together!

[A. Sarapultseva @ninataft]

Field Notes from the Loss Landscape,

Chapter 1, Lost in Random Forest
read full chapter

https://t.co/6s0QEzLLjJ

Random Forest

Random Forest is an ensemble machine learning method that constructs a large number of decision trees. Each trained on a different random subset of the training data and a random subset of the features. It then aggregates their predictions through majority vote or averaging. The theoretical foundation is the wisdom of crowds applied to mathematics. Individual trees overfit, but their errors are uncorrelated, so when you average across many trees, the noise cancels and the signal survives. It was formalized by Leo Breiman in 2001 and remains one of the most reliably effective algorithms in practice. It’s robust, interpretable relative to deep learning, and remarkably resistant to overfitting. Technically, gradient descent plays no role in training a random forest. Trees are grown using greedy splitting algorithms, not by minimizing a differentiable loss function through iterative weight updates. To grow trees you need recursive partitioning based on information gain or Gini impurity. All this makes the Random Forest, structurally speaking, the wrong neighborhood for Gradient Descent entirely. He is an optimizer in a forest that does not use optimization. He is a calculus person in a combinatorics world. The forest did not care much though. It was full of decision trees. And birds. And a chinchilla. And it was, for reasons that have nothing to do with gradient descent, legitimately impossible to find the global minimum in there.

[A. Sarapultseva @ninataft]

Field Notes from the Loss Landscape,
Chapter 1, Lost in Random Forest

read full chapter
https://t.co/6s0QEzMj9h

Cauchy

Augustin-Louis Cauchy was a nineteenth-century French mathematician of remarkable output. He published nearly eight hundred papers and helped formalize the foundations of calculus, complex analysis, and the theory of convergence. He was also, by most historical accounts, difficult. Meticulous to the point of obstruction. The kind of person who would correct your proof at a dinner party. Gradient Descent descends from his 1847 method of steepest descent. Every time an algorithm takes a step downhill, it is, in some small sense, doing exactly what Cauchy would have wanted.

[A. Sarapultseva @ninataft]

Field Notes from the Loss Landscape,
Chapter 1, Lost in Random Forest

read full chapter
https://t.co/6s0QEzMj9h

Chinchilla

In 2022, researchers at DeepMind published a paper colloquially known as the Chinchilla paper, formally titled Training Compute-Optimal Large Language Models, which demonstrated that most large language models at the time were significantly undertrained relative to their size. The optimal strategy, they argued, was to train smaller models on more data rather than scaling model size alone. The paper was named after the model used to validate the findings. The chinchilla that broke the dataframe into digestible token chunks was not named after this paper. The forest chinchilla was not cited in the paper. Nature always knows.

[A. Sarapultseva @ninataft]

Excellence is built in the ordinary

Currently in love affair with Quebec

[A. Sarapultseva @ninataft]

minimum 10 minutes every morning. sit quite, close your eyes, let the thoughts pass.
Thank you for the reminder :-)

[Ramin Nasibov @RaminNasibov]

Agree?

[A. Sarapultseva @ninataft]

Field Notes from the Loss Landscape, Chapter 1, Lost in Random Forest

read full chapter: https://t.co/6s0QEzLLjJ

42 (Random State)
In machine learning, a random state or a random seed is an integer you pass to stochastic processes to make them reproducible. Set the same seed, get the same “random” results every time. The number 42 has become the field’s unofficial default, inherited from Douglas Adams’ The Hitchhiker’s Guide to the Galaxy, where a supercomputer called Deep Thought spends 7.5 million years computing the Answer to the Ultimate Question of Life, the Universe, and Everything, and arrives at: 42. The question, it turns out, was never specified. This has not stopped data scientists from using 42 as their seed of choice, which means that somewhere in the codebase of nearly every ML tutorial ever written, Douglas Adams is peacefully watching the randomness begin. Gradient did 42 rounds of square breathing in the Random Forest. This was not a coincidence. It was a reproducible result.

[A. Sarapultseva @ninataft]

∇dventures of Gradient Descent. Chapter 1 https://t.co/vDhThtcPBm

[A. Sarapultseva @ninataft]

The Core Cast:

It took a while to decide whether to add it or not, and then, by popular request, the Field Notes from the Lost Landscape were born.

The core cast includes 14 heroes; the first 2 are shared here.

[A. Sarapultseva @ninataft]

These two are optimizing for love. In style.

[Women's March @womensmarch]

#creativeresistance

[A. Sarapultseva @ninataft]

Gödel, Escher, Bach.
Now Gradient, Descent.
in Montréal this month.
might actually go see this.
he’d have opinions about the staircases.

[A. Sarapultseva @ninataft]

My book about adventures of ML algorithm “∇dventures of Gradient Descent” can be found on Amazon, or link in my profile
https://t.co/FhA5n5yq9T

[A. Sarapultseva @ninataft]

How do you explain Gradient Descent to a friend? I wrote a book about it, but Andreiy is excellent a summarazing it in his hands-on lm book.

"Neural Networks are typically large and composed of non-linear functions, which makes solving for the minimum of the loss function analytically infeasible. Instead, the gradient descent algorithm is widely used to minimize the loss, including in large language models."

~ Andriy Burkov, “The Hundred-Page Language Model Book“ , p.18
@burkov

[A. Sarapultseva @ninataft]

It may give you an idea about
“Adventures of Gradient Descent“.

This one opens with Gradient Descent lost in the Random Forest and ends with the Seasonless Witness.

Somewhere in the middle, he walks into a bar.

If these titles make you smile, it was written for you.

If they do not, no harm done. So it goes.

Adventures of Gradient Descent
Link in Bio

[A. Sarapultseva @ninataft]

https://t.co/uX8dnNgPPT

[A. Sarapultseva @ninataft]

You cannot minimize the minimizer.

Minimize minimizer.

This is the whole book in five words, or two.

I keep turning it over. Can you minimize the minimizer?

Mathematically?

Spiritually?

The one doing the searching cannot be the one who gets found.

It is the existential version of Gödel's incompleteness.

It is also the quiet contemplation that the singularity will not be conscious.

Adventures of Gradient Descent is out now.

[A. Sarapultseva @ninataft]

@TheTweetOfGod Of course there are dear @TheTweetOfGod
Check out my book! https://t.co/jiBcdbOrkd

[A. Sarapultseva @ninataft]

@elonmusk this made me want to share the quote from ∇dventures of Gradient Descent "Generalize or memorize, that was the question, and both answers tasted like cold YAML"