Amin Sadou

Statistical learning & Causality.
#rstat, Data Science.

[Amin Sadou @aminsaadou]

HR Analytics report steps https://t.co/wYnocR7jmi

[elvis @omarsar0]

Everyone is talking about this new OpenAI paper.

It's about why LLMs hallucinate.

You might want to bookmark this one.

Let's break down the technical details:

[Lior Alexander @LiorOnAI]

AWS released an open-source framework that lets you orchestrate multiple AI agents and handle complex conversations.

Can be deployed locally on your computer.

[Poonam Soni @CodeByPoonam]

Google just changed note taking forever.

NotebookLM got massive upgrades.

7 Powerful features you can't afford to miss:

[Lior Alexander @LiorOnAI]

NVIDIA just made Pandas 150x faster with zero code changes.

It is now directly integrated in Google Colab.

All you have to do is:
%load_ext cudf.pandas
import pandas as pd

Their RAPIDS library will automatically know if you're running on GPU or CPU and speed up your processing.

You can try it here: https://t.co/oq7qkVWztY

[Alex Reibman 🖇️ @AlexReibman]

Meta launched Llama 3 to show the world what’s possible with open source LLMs.

500+ AI engineers just spent 24 hours straight putting it to the test.

Here’s what we saw at the @AIatMeta x @cerebral_valley #Llama3Hackathon (🧵):

[Valeriy M., PhD, MBA, CQF @predict_addict]

Conformal prediction and SHAP are two developments that data scientists in 2023 should know.

One might also be interested in Conformal SHAP that was presented in COPA conformal conference couple of years ago.

https://t.co/kHDpOmysxb

https://t.co/IKFwm74Q5b

And in case you are curious about how conformal prediction is changing the landscape there is now Conformal Quantile Regression, Conformal Monte Carlo, Conformal VaR and more.

#conformalprediction

[Jeremy Howard @jeremyphoward]

I've spent the last few months interviewing >60 experts in law, economics, AI, alignment, etc, on the impacts of AI, and safety interventions.

Today I'm publishing my first article, showing regulation designed to increase AI safety may backfire badly!
https://t.co/NsZZ5H2BVH

[Valeriy M., PhD, MBA, CQF @predict_addict]

Personally I quite like scikit-learn but, I am not convinced that the probability calibration section in scikit learn is well-written.

TL;DR when it comes to classifier calibration scikit learn page is not a good source of information on the topic as it is neither up to date or factual.

The issue with unfortunate naming of predict_proba is well known, Samuele Mazzanti has written a great article about it 'Python’s «predict_proba» Doesn’t Actually Predict Probabilities (and How to Fix It)' https://t.co/bquHDO2GX1

The part that is confusing users of scikit-learn users is that 'predict_proba'does not produce probabilities of the classes (it produces class scores). The estimates of class scores are not class probabilities by definition because they don't meet the basic definition and properties of probabilities as anyone who has studied Probability 1.01 knows.

The other confusing part is that the authors mention that models producing poor estimates of the 'class probabilities' (should be 'class scores' instead) are not good, but don't mention explicitly anywhere that most if not all machine learning models produce poor estimates of class probabilities (deep learning is notorious but neither boosted trees nor other types of popular models produce calibrated provabilities either).

The authors then suggest isotonic regression as a 'solution' but isotonic regression is an hoc tool that overfits on small datasets and assumes ROC AUC of 1 on the test set (correct ranking) which of course never happens in real life.

Scikit learn then cites papers that are either 20 years old (Platt's scaler, isotonic regression), or papers that were laters debunked in their conclusion (e.g. 2005 classical paper by A. Niculescu-Mizil claimed that shallow neural networks were well calibrated by they aren't).

TL;DR The scikit-learn page on Probability calibration is not an informative or correct source of information in 2023 and requires major revisions.

#scikitlearn # #machinelearning #deeplearning #python #calibration

[Valeriy M., PhD, MBA, CQF @predict_addict]

What is cooking in @nvidia R&D lab that has 300 PhDs?

At GTC23 two weeks ago Nvidia chief scientist and SVP of research Bill Dally presented a behind-the-scenes peek at Nvidia’s R&D.

One of the key 🔑 areas Better Uncertainty Prediction for Safer Driving using Conformal Prediction!

“AI technology, of course, is a basic staple of autonomous driving. It’s what lets autonomous vehicles – using their cameras, radars and Lidar – interpret the environment around them and make predictions. Dally noted traditional machine learning will have a network and run a model against images and produce a single classification say a pedestrian or cyclist or car, and then make determinations about how it’s moving.

What we’d really like to do is be able to output a [data] set where we can say it could be a pedestrian with some probability, could be a cyclist with some probability. If we’re judging the distance, rather than saying it’s a pedestrian at 20 meters, we could say it’s a pedestrian or possibly or a cyclist, and it’s 20 meters plus or minus one meter, because we need to make sure that we’re not going to miss this object. If our estimation is off by a meter that could wind up being a lot in difficult circumstances. So rather than getting a single output from our neural network, we’d like to have a set output, a set of possible classes (objects) and a set of possible positions usually centered around a mean estimate position. Given that output set, we can run our planner, so we can guarantee a Safe Action for any element of that set,” said Dally.

A technique called conformal prediction is used to accomplish the task (see slide below). Broadly, this approach includes a non-conformance function to inject a range probability or confidence around many possible solutions.”

https://t.co/djhHz6Wc2I

#NVIDIA #conformalprediction #machinelearning #technology

[Michael C. Frank @mcxfrank]

People are testing large language models (LLMs) on their "cognitive" abilities - theory of mind, causality, syllogistic reasoning, etc. Many (most?) of these evaluations are deeply flawed. To evaluate LLMs effectively, we need some principles from experimental psychology.🧵

[Allen Downey @AllenDowney]

Exploring the effect of researcher choices on statistical results: 73 teams estimate the same effect size with the same data, and generate 1,253 different results.

As always, statistical results depend on modeling decisions.

https://t.co/xLMK2qLl15

[François Chollet @fchollet]

The gap between generating words and generating meaning will prove hard to close. Generating meaning requires *a model of the things being talked about* (language as communication), while generating words only requires a model of the structure of text (language as statistics)

[Miguel de Carvalho @mbcarvalho]

Data Science is a misnomer; we should be talking about Data Sciences. Stats, ML, AI, Psychometrics, Econometrics, are examples of Data Sciences. The plural form unites the fields.

[Stacey Hendriks, PhD @SHendriks_PhD]

Top 5 tools that I used during my #PhD and continue to use throughout my early #research career.

A thread 🧵:

@OpenAcademics @PhDVoice @PostdocVoice #AcademicTwitter #AcademicChatter

If you have tools you find helpful I'm always looking for new ways to improve my workflow 👌

[Posit PBC @posit_pbc]

📢 Announcing keynote and talk recordings from rstudio::conf(2022)!

⬡ Keynotes: https://t.co/4cvPNAokhn
⬡ Talks: https://t.co/HTLwKREkEF

Keep an eye out for a blog post that summarizes conf materials 👀

[Cédric Scherer @CedScherer]

"Graphic Design with #ggplot2" 👨‍💼👩‍💻🧑‍💻

Do you want to recap the 2-day workshop at #rstudioconf? Or do you feel sad you've missed it?

🔥 All course material incl. latest updates can be found on the workshop webpage—9 sessions, 760 slides, 314 ggplots!

👉 https://t.co/TyBRWQOW7R

[Mine Çetinkaya-Rundel @minebocek]

Abstinence-only education doesn't work, not for anything and not for spreadsheets. Here are some tips for better spreadsheets from myself and a few others, instead of the not-so-helpful advice of "avoid spreadsheets".

[Davis Blalock @davisblalock]

"Language Models Can Teach Themselves to Program Better"

This paper changed my thinking about what future langauge models will be good at, mostly in a really concerning way. Let's start with some context: [1/11]

[Philippe Massicotte @philmassicotte]

Nice to see the daily build of #rstudio supporting colour preview #rstats