A Numerical Approximation Method for the Fisher–Rao Distance Between Multivariate Normal Distributions — Free PDF
Explore the Fisher–Rao distance, Information Geometry, multivariate normal distributions, Jeffreys divergence, and numerical approximation methods in this research work.
📚 Key Topics
Fisher–Rao Distance
Information Geometry
Multivariate Normal Distributions
Fisher Information
Jeffreys Divergence
KL Divergence
Statistical Manifolds
Geodesics
Symmetric Positive-Definite (SPD) Matrices
Numerical Approximation
Mahalanobis Distance
Machine Learning
🎓 Useful For
Data Scientists, Machine Learning Researchers, Statisticians, Mathematicians, AI Researchers, and students studying advanced probability, statistics, and Information Geometry.
📥 Free PDF
Read the complete article and access the free PDF here:
https://t.co/5JpOvboWun
Aaaaaand it’s officially on @arxiv 🥳🥳
https://t.co/eA7nQew4JB
Wanna do Fisher, XY Fisher or DALI with a chosen derivative method backend in a simple sleek Python way? Look no further, my friend. We got you!
Simple examples here
https://t.co/wH2T46XwJr
🚀 We present the first large-scale Lean 4 formulation of Statistical learning theory from scratch!
Led by my student @yuanhezhang6 and collaborated with @jasondeanlee
📄 Paper: https://t.co/8q5laY8KYp
💻 GitHub: https://t.co/mTpkr92TMQ
🤗 Dataset: https://t.co/K8wDD6us4k
If you're looking for a comprehensive guide to LLM finetuning, check this!
a free 115-page book on arxiv, covering:
> fundamentals of LLM
> peft (lora, qlora, dora, hft)
> alignment methods (ppo, dpo, grpo)
> mixture of experts (MoE)
> 7-stage fine-tuning pipeline
> multimodal finetuning & challenges
> industrial frameworks (hf, sagemaker, openai)
everything you need to know in one place!
I have shared it in the replies.
If you have the time during the holiday season, I highly recommend you to check out ''Mathematics As Metaphor'', a wonderful collection of highly profound essays, by one of my personal inspiration, Yuri Manin.
Mathematicians, computer scientists, logicians, philosophers, linguists and physics won't want to miss this one, I promise you!
A massive new study on peak performance included 34,000 international top performers: Nobel laureates, renowned classical music composers, Olympic champs, and the world’s best chess players. It shows early specialization is a trap, and the road to greatness is long and varied.
How did people in 1913 see the world? How did they think about the future? We trained LLMs exclusively on pre-1913 texts—no Wikipedia, no 20/20. The model literally doesn't know WWI happened. Announcing the Ranke-4B family of models. Coming soon: https://t.co/KOjbdLlH3S
An information-theoretic framework finds and removes redundancies in a hypergraph representation of a network, allowing for more efficient analysis of higher-order elements https://t.co/Kd6Sxe3RLT
Discrete lattice Gaussian or normal distributions maximize Shannon entropy.
Thus they form a discrete exponential family (DEF) with cumulant function related to Riemann theta function:
👉https://t.co/rci5VvcQQn
https://t.co/13cz7Wm5MF
I wrote a review paper about statistical methods in generative AI; specifically, about using statistical tools along with genAI models for making AI more reliable, for evaluation, etc. See here: https://t.co/0aq8hJqXzo!
I have identified four main areas where statistical thinking can be helpful. These are just a subset of what is out there; other topics have been well-covered in other reviews.
1. Designing "statistical wrappers" around a model, for instance, changing behavior of a trained model (e.g., abstaining), where a score, e.g., an "unsafety score" is too high. The key connection to statistics is to use the quantiles of the loss (on a calibration set) to set the critical threshold, thus enabling conformal-type high probability guarantees.
2. Closely related, methods for uncertainty quantification, which enable the model to express uncertainty in an answer. A crucial component here is "calibration", whereby the uncertainty is required to reflect reality.
3. Statistical methods for AI evaluation: Specifically, tools for statistical inference (e.g., confidence intervals) on model performance. Exciting recent work proposes careful statistical models for leveraging a very small high-quality dataset, possibly combined with much larger low-quality datasets, for accurate evaluation.
4. Experiment design and interventions. Careful AI experiments to understand and steer models may require interventions such as modifying experimental settings in a controlled manner. This brings up connections to classical experimental design in statistics. This connection has largely remained implicit so far, and my review aims to make it more explicit; hoping that experimental design principles will become useful here.
This review references the work of many, including @HamedSHassani@obastani@tatsu_hashimoto @yuekai_sun @CsabaSzepesvari@ml_angelopoulos@stats_stephen@yaniv_romano@yaringal@KilianQW@_onionesque +their teams, and some work that I was also involved in.
Hopefully, my review will be helpful to orient yourself in this exciting area. Nonetheless, since the area is rapidly expanding, it is possible that I missed important references. Please feel free to let me know of anything that I should add/change!
Random projections reduce high-dimensional data by multiplying it with a random matrix, preserving distances with surprising accuracy due to the Johnson–Lindenstrauss lemma. In probability, they help analyze concentration phenomena and simplify high-dimensional geometry. In machine learning, random projections speed up nearest-neighbor search, clustering, large-scale regression, and feature reduction while keeping core structure intact. They enable fast preprocessing for text, image, and genomic data, where dimensions can be huge. In real life, random projections appear in compressed sensing, privacy-preserving data sharing, signal processing, and large database search, allowing efficient storage, transmission, and querying without heavy computation. They offer a powerful, simple way to tame high dimensionality while retaining essential information.
I will be giving an invited talk at the Non-Euclidean Foundation Models and Geometric Learning workshop @NeurIPSConf on
Geometry-aware generative models for scientific data generation
[papers: https://t.co/8FmnobDR2j, https://t.co/LClAwNpo20]
I was going to do this, with Dr Gülen (the turbine expert who helped with the Kollmann memiors), as co-author, however soon afterwards Dietrich Eckardt published this >>>
Eckardt is a very acomplished German turbine designer, so Dr Gülen and I saw this and thought we could not surpass it.
However, Jet Web has its own focus, and I certainly have a lot of details about the British jet program which are not in his book, so I would not say that Jet Web is exactly the same as a book I`d write on it, but I would need to be careful about the focus to make sure its looking at areas not really covered by Dietrich, as it would be utterly pointless overlapping with Jet Web, it is an extremely good book, much like THE SECRET HORSEPOWER RACE, its written by a very capable specialist engineer IN the topic he`s writing about, and the result is predictably superlative.
Its also far from a dry tome about thermodynamics, its full of political intrigue and who-exchanged what information-when.
Very expensive, but DO buy it.
I encourage you to read this article, in which we describe the current situation and the directions in which, in our view, mathematics is heading. Many thanks to Ken Ono for including me in this extraordinary project. I look forward to a wide-ranging discussion and will be sharing more about activities related to this article.
https://t.co/kd84ApmvLO