Long_Period

[Library Mindset @librarymindset]

the book the quote

[Long_Period @long_period]

https://t.co/mLqV3V6WKS

[Jürgen Schmidhuber @SchmidhuberAI]

DeepSeek [1] uses elements of the 2015 reinforcement learning prompt engineer [2] and its 2018 refinement [3] which collapses the RL machine and world model of [2] into a single net through the neural net distillation procedure of 1991 [4]: a distilled chain of thought system.

REFERENCES (easy to find on the web):

[1] #DeepSeekR1 (2025): Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv 2501.12948

[2] J. Schmidhuber (JS, 2015). On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models. arXiv 1210.0118. Sec. 5.3 describes the reinforcement learning (RL) prompt engineer which learns to actively and iteratively query its model for abstract reasoning and planning and decision making.

[3] JS (2018). One Big Net For Everything. arXiv 1802.08864. See also US11853886B2. This paper collapses the reinforcement learner and the world model of [2] (e.g., a foundation model) into a single network, using the neural network distillation procedure of 1991 [4]. Essentially what's now called an RL "Chain of Thought" system, where subsequent improvements are continually distilled into a single net. See also [5].

[4] JS (1991). Learning complex, extended sequences using the principle of history compression. Neural Computation, 4(2):234-242, 1992. Based on TR FKI-148-91, TUM, 1991. First working deep learner based on a deep recurrent neural net hierarchy (with different self-organising time scales), overcoming the vanishing gradient problem through unsupervised pre-training (the P in CHatGPT) and predictive coding. Also: compressing or distilling a teacher net (the chunker) into a student net (the automatizer) that does not forget its old skills - such approaches are now widely used. See also [6].

[5] JS (AI Blog, 2020). 30-year anniversary of planning & reinforcement learning with recurrent world models and artificial curiosity (1990, introducing high-dimensional reward signals and the GAN principle). Contains summaries of [2][3] above.

[6] JS (AI Blog, 2021). 30-year anniversary: First very deep learning with unsupervised pre-training (1991) [4]. Unsupervised hierarchical predictive coding finds compact internal representations of sequential data to facilitate downstream learning. The hierarchy can be distilled [4] into a single deep neural network. 1993: solving problems of depth >1000.

[Virat Singh @virattt]

I read up on DeepSeek’s learning algo, GRPO.

GRPO: group relative policy optimization

How GRPO works:
1 • model generates a group of answers
2 • compute score for each answer
3 • compute avg score for entire group
4 • compare each answer score to avg score
5 • reinforce model to favor higher scores

Other methods like PPO, use a value function model to do reinforcement learning.

GRPO does not, which reduces memory and computational overhead when training.

A concrete example of GRPO in action:

Query: “What is 2 + 3?”

Step 1: LLM generates three answers.
1. “5”
2. “6”
3. “2 + 3 = 5”

Step 2: Each answer is scored.
1. “5” → 1 points (correct, no reasoning)
2. “6” → 0 points (incorrect)
3. “2 + 3 = 5” → 2 points (correct, w/ reasoning)

Step 3: Compute avg score for entire group.
Avg score = (1 + 0 + 2) / 3 = 1

Step 4: Compare each answer score to avg.
1. “5” → 0 (same as avg)
2. “6” → -1 (below avg)
3. “2 + 3 = 5” → 1 (above avg)

Step 5: Reinforce LLM to favor higher scores.
1. Favor responses like #3 (positive)
2. Maintain responses like #1 (neutral)
3. Avoid responses like #2 (negative)

This process is repeated, allowing the model to learn and improve over time.

Great work by the @deepseek_ai team.

[PyQuant News 🐍 @pyquantnews]

FinNLP provides a playground for all people interested in LLMs and NLP in Finance.

There is a full pipelines for LLM training and fine tuning:

https://t.co/dsCjDjore9

[Brian Roemmele @BrianRoemmele]

We made some spectacular Garage Lab discoveries with FinGPT running local with a new vector database and AutoGPT.

Running unattended for 4 days it has built theories of the current market based on my Cycle Data Research.

I am close to saying this will change business forever.

[Michael A. Arouet @MichaelAArouet]

Gold against fiat on log scale 👇 nice to watch. Ht @JamesEagle17

[Wolfram @WolframResearch]

Find the best design or make the best decision given constraints with optimization the #WolframWay. https://t.co/WSZftfGhbF

[zerohedge @zerohedge]

Digital Authoritarianism: AI Surveillance Signals The Death Of Privacy https://t.co/3cQlaTrwIF

[OpenAI @OpenAI]

Our newest system DALL·E 2 can create realistic images and art from a description in natural language. See it here: https://t.co/Kmjko82YO5

[Fintwit @fintwit]

On March 23, 2020, the stock market hit its pandemic low.

Exactly two years ago today.

[Kenneth Field @kennethfield]

One month in, and my updated illustrative map of the crisis of human displacement shows 8% of #Ukraine population has been forced to flee to neighbouring countries.

Please remember, dots show general proportions and movement, NOT specific locations.

💙💛
#cartography #dataviz

[World of Statistics @stats_feed]

Countries with the highest number of internet users rate.

[Wolfram @WolframResearch]

This past year, we've seen the benefits of computation in the classroom. ElectroDynamics, by @JamesRohlf and @Kevin60747711, is an interactive textbook on electricity and magnetism. This book is a great resource for anyone interested in #electrodynamics! https://t.co/wlahcMTxur

[Wolfram @WolframResearch]

Always making useful code, @arnoudbuzing !