Nikola Sivacki

Co-Founder of @greyparrotai. Machine Learning. Neural Networks. Optimising Recycling with Deep Learning.

[Lucas Beyer (bl16) @giffmana]

You may have recently heard claims that video generation models are "dumb" about physics, and only "world models" (V-JEPA, specifically) have a valid internal model of physics.

This turns out to be false. In a recent paper, researchers show that a LINEAR probe of diffusion videogen models predict various "physics" very well, significantly better than V-JEPA or VideoMAE (and plain VAE just sucks).

This is noteworthy, because a *linear* probe being this accurate shows that the model has a pretty explicit internal representation of the physics!

[Nikola Sivacki @nsivacki]

I'll be at @CVPR in Denver, Colorado this year! If you want to talk about computer vision, self supervised learning, domain adaptation, robust learning, let me know!

[Nikola Sivacki @nsivacki]

@fchollet The Hellenistic Empire was far more inspiring than the Roman Empire, IMO, even if being a short lasting supernova. Universalism, not uniformity. A deeper link to science and philosophy. Libraries (Alexandria!), not legions. Connection, not domination.

[Ryohei Sasaki@engineer @rsasaki0109]

[NeurIPS 2025] Official Implementation of DINO-Foresight: Looking into the Future with DINO
https://t.co/6xvidSNE6C

[Nikola Sivacki @nsivacki]

Interestingly, normalisation layers can be replaced with 80's style sigmoids without any loss in performance. Isn't this funny 🙂

[Zhuang Liu @liuzhuang1234]

New paper -

Transformers, but without normalization layers (1/n)

[Nikola Sivacki @nsivacki]

The study of crystals inspires models that inspire the study of statistics to further inspire the cutting edge AI of today.

Interestingly, the motivation to understand how the brain works has produced amazing results in AI, yet we still don't know how the brain works.

[The Nobel Prize @NobelPrize]

BREAKING NEWS
The Royal Swedish Academy of Sciences has decided to award the 2024 #NobelPrize in Physics to John J. Hopfield and Geoffrey E. Hinton “for foundational discoveries and inventions that enable machine learning with artificial neural networks.”

[Haoru Xue @HaoruXue]

🎉 Diffusion-style annealing + sampling-based MPC can surpass RL, and seamlessly adapt to task parameters, all 𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴-𝗳𝗿𝗲𝗲!

We open sourced DIAL-MPC, the first training-free method for whole-body torque control using full-order dynamics 🧵

https://t.co/wIaaT5CTEH

[Dan Hendrycks @hendrycks]

We've created a demo of an AI that can predict the future at a superhuman level (on par with groups of human forecasters working together).
Consequently I think AI forecasters will soon automate most prediction markets.

demo: https://t.co/r5NlsnAvG4
blog: https://t.co/X3eIFSPaGh

[Nikola Sivacki @nsivacki]

Competition comes from scarcity.
In a world of increasing abundance, cooperation will be more impactful.

[Nikola Sivacki @nsivacki]

When in a state of competition, open your eyes and focus on competence, not competitiveness. Competitiveness wastes your energy and narrows your vision.

[David Pfau @pfau]

I’m beyond thrilled to share that our work on using deep learning to compute excited states of molecules is out today in @ScienceMagazine! This is the first time that deep learning has accurately solved some of the hardest problems in quantum physics. https://t.co/TWagWjXMV6

[Jon Evans @rezendi]

"The researchers found that 24 of the brain samples, which were collected in early 2024, measured on average about 0.5% plastic by weight." https://t.co/PkKYZ71pq5

[Nikola Sivacki @nsivacki]

@fchollet @erikbryn Sufficiently abstract interpolation is indistinguishable from extrapolation

[Nikola Sivacki @nsivacki]

@fchollet @erikbryn For all we know, human innovation may be 'just interpolation' in some higher abstraction space, combined with random perturbations. We have zero proof that innovation is extrapolation, even if it may look like it.

[Pessimists Archive @PessimistsArc]

2024: The Eiffel Tower is a romantic symbol of the good old days

1897: The Eiffel Tower is a techno-dystopian vanity project built by a tech-bro engineer

https://t.co/XNFy7UdC2O

[Nikola Sivacki @nsivacki]

Surprising. If only it was that simple. The 'winning mentality' prevents innovation, as motivations end after a few wins and complacency sets. It also clouds the development of deeper intrinsic motivations. Winning is a mere side-effect of those. Rediscover the core @ericschmidt

[Jonathan Fischoff @jfischoff]

This paper is wild.

Create a camera with cheap terrible camera lens, but train a diffusion model to recreate a much better image.

https://t.co/Q2Hd5PUa54

[Sakana AI @SakanaAILabs]

Introducing The AI Scientist: The world’s first AI system for automating scientific research and open-ended discovery!

https://t.co/jC7g5GPVsE

From ideation, writing code, running experiments and summarizing results, to writing entire papers and conducting peer-review, The AI Scientist opens a new era of AI-driven scientific research and accelerated discovery.

Here are 4 example Machine Learning research papers generated by The AI Scientist.

We published our report, The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery, and open-sourced our project!

Paper: https://t.co/lTQ8UenFHk
GitHub: https://t.co/Im53whVeAq

Our system leverages LLMs to propose and implement new research directions. Here, we first apply The AI Scientist to conduct Machine Learning research. Crucially, our system is capable of executing the entire ML research lifecycle: from inventing research ideas and experiments, writing code, to executing experiments on GPUs and gathering results. It can also write an entire scientific paper, explaining, visualizing and contextualizing the results.

Furthermore, while an LLM author writes entire research papers, another LLM reviewer critiques resulting manuscripts to provide feedback to improve the work, and also to select the most promising ideas to further develop in the next iteration cycle, leading to continual, open-ended discoveries, thus emulating the human scientific community. As a proof of concept, our system produced papers with novel contributions in ML research domains such language modeling, Diffusion and Grokking.

We (@_chris_lu_, @RobertTLange, @hardmaru) proudly collaborated with the @UniOfOxford (@j_foerst, @FLAIR_Ox) and @UBC (@cong_ml, @jeffclune) on this exciting project.

[Tsarathustra @tsarnick]

Richard Sutton says AI safety advocates are creating the opposite of what they seek - we should not try to solve the AI control problem because a decentralized system of independent agents is better than everything being aligned under one goal

[Science Is Strategic @scienceisstrat1]

The world’s solar capacity hit 1,419 gigawatts in 2023, radically more than forecast by any major forecaster 🚀🚀