Data Scientist II, Time series analysis, Causal AI
๐ฟ๐๐๐๐๐๐๐๐๐๐๐ ๐๐๐๐๐๐๐๐ @DCU ๐ฐ๐๐๐๐๐๐๐ข๐๐๐๐๐๐ - ๐ณ๐๐๐ ๐ผ๐๐๐๐๐
Physics and Philosophy: The Revolution in Modern Science written by Nobel Prize-winning German physicist Werner Heisenberg.
It is derived from his celebrated Gifford Lectures, the book acts as an accessible, non-mathematical portal exploring how quantum mechanics dismantled classical concepts of reality.
Chatterjee Correlation, introduced by Sourav Chatterjee in 2020, is a powerful measure of dependence between two random variables. Unlike Pearson correlation, which primarily captures linear relationships, Chatterjee correlation can detect a broad range of nonlinear and nonmonotonic dependencies. A remarkable property is that it equals 0 when variables are independent and approaches 1 whenever one variable is nearly a deterministic function of the other.
The measure is rank-based, computationally efficient, and does not require strong distributional assumptions. This makes it attractive for modern data analysis where complex nonlinear relationships are common.
In statistics, it is used for dependence testing, feature screening, and exploratory data analysis. In machine learning, Chatterjee correlation can identify informative variables beyond linear correlations, improving feature selection and interpretability. In deep learning, it helps analyze learned representations and hidden-layer dependencies. In reinforcement learning, it can be used to study relationships between states, actions, and rewards, helping uncover hidden structure in complex environments.
The broader lesson is that modern data often contains rich nonlinear patterns, and Chatterjee correlation provides a principled way to detect them without imposing restrictive modeling assumptions.
New interactive simulation in my online textbook illustrating publication bias in meta-analyses, and some techniques to model bias-adjusted effect sizes. None of these work perfectly. We need an unbiased literature to draw valid inferences. https://t.co/Tr1mqiBJT1
[Submitted on 26 Nov 2025 (v1), last revised 10 Feb 2026 (this version, v2)]
Causal Inference: A Tale of Three Frameworks
Linbo Wang, Thomas Richardson, James Robins
https://t.co/FmTf3tJ7eu
Misinterpretation of correlation and causation is a common issue in data analysis. Correlation measures the strength and direction of a relationship between two variables, but it does not imply that one variable causes the other. Misunderstanding this distinction can lead to several problems:
โ Incorrect conclusions: Assuming that a correlation between two variables means one causes the other can lead to erroneous conclusions.
โ Misleading reports: Presenting correlated data as causative can mislead decision-makers and the public.
โ Poor decision-making: Basing decisions on incorrect causal assumptions can result in ineffective or harmful policies and actions.
โ Oversimplified analyses: Focusing on correlation without considering other factors can oversimplify complex relationships.
Consider the statement, "More storks lead to more babies!" This humorous example illustrates the fallacy of assuming causation from correlation. In reality, both variables might be influenced by a third factor, for example rural areas often have more storks and more families with children. The image below uses synthetic data to show how such indirect relationships can create a misleading appearance of causation.
For regular tips on data science, statistics, Python, and R programming, check out my free email newsletter. Check out this link for more details: https://t.co/ktUcWo9XpO
#database #StatisticalAnalysis #Data #statisticians
The Machine That Learns The Law Behind The Data
A very very interesting US Patent US10963540B2 - Physics Informed Learning Machine describes a learning system that does not begin with data alone. It begins with a physical model, usually written as a differential equation (or PDE)
dx/dt = f(x,t)
A normal Machine Learning model sees scattered data and tries to fit it.
A physics-informed learning machine starts with a law.
Then it treats the data as evidence that updates what the model believes about the physical system.
For this application, I use the patent idea on NASA C-MAPSS Turbofan engine data. The machine watches multivariate telemetry from a degrading engine and infers a hidden health state that is not measured directly. From that posterior belief, it estimates the engineโs remaining useful life.
In the main 3D scene, the engine lifetime is turned into a tunnel. The spiral ribbons are real sensor channels evolving over cycle-time. The glowing core is the inferred health state. The surrounding cloud is uncertainty. The orange wall ahead is the predicted failure horizon. So the big picture is: sensor evidence comes in, posterior belief tightens, and the machine moves from uncertainty toward a concrete failure prediction.
The inset posteriors make that explicit. The health posterior shows where the model believes the hidden engine condition sits at the current moment, and how sharply it believes it. The RUL posterior shows the same idea for remaining life... early on it is broad, later it shifts left and narrows as the machine becomes more certain about how close failure is.
This idea is not limited to engines. The same idea can apply to data centers, CPUs, GPUs, cooling systems, power grids, robotics, batteries, and any machine that produces telemetry while obeying physical constraints. In an age where machine learning runs on massive hardware infrastructure, this kind of model matters: it can turn noisy sensor streams into early warnings before expensive systems fail.
"The Laws of Thought: The Quest for a Mathematical Theory of the Mind" - explores the quest to use mathematics to describe the ways we think, the ideas behind modern AI systems, and the ways in which they still differ from human minds.
Get it at https://t.co/fb3AXBVvZ0
In AI, causal learning without backpropagation means learning causal direction through local interactions instead of gradient-based training.
This paper moves away from gradient-based training and shows how causal direction can emerge from neural assemblies through local plasticity and sparse co-activation.
Directionality is not inferred after training, but reflected in asymmetric synaptic structure shaped by local interactions. In controlled settings, the model recovers the underlying causal graph.
It feels less like optimization and more like structure emerging from local interactions.
https://t.co/7YdyiFpPQ4
Elements of Structural Equation Models (SEMs) by Kenneth A. Bollen
Combining theoretical foundations with practical applications, this book offers a rigorous, accessible guide to structural equation models.
๐ https://t.co/ZrXW4qpAG4