Bhaskar Aryal

An Engineer, Interested in World History and Philosophy !

[UMPH AI @__umph__ai]

UMPH Phase 1 is live.

Scan or upload a selfie and get your skin score, 7 skin metrics, your skin age, and a personalized skincare and treatment plan in under 60 seconds.

We're starting with skin analysis.

Next phases will turn UMPH into a full skin health journey:

• Mobile app with weekly skin tracking
• Your personal AI skin advisor
• Discover clinics and book treatments directly
• Build custom treatment stacks with up to 50% OFF

To celebrate the launch: the first 50 people who like, comment, and retweet get free scan coupons.

See your score at https://t.co/LzGfelruXY

[Bhaskar Aryal @aryalbhaskar]

@neemeshe @ManUtd @ManCity says thanks!!

[MIT CSAIL @MIT_CSAIL]

9 distance measures in data science w/algorithms (v/@MaartenGr).

[Joachim Schork @JoachimSchork]

As your data grows, so do the challenges of interpreting p-values effectively! With large data sets, p-values can often become misleading, as even negligible effects may appear statistically significant. This happens because larger samples reduce the margin of error, narrowing confidence intervals and amplifying the apparent importance of small differences. The key is distinguishing statistical significance from practical importance.

Key Takeaways:
✅ Larger sample sizes reduce the margin of error, increasing the likelihood of small effects appearing significant.
✅ In regression models, all variables may show significant p-values, limiting their interpretive value.

What can you do?
1️⃣ Focus on effect sizes: Evaluate coefficients and confidence intervals for practical relevance.
2️⃣ Standardize variables: Scaling your data allows for better comparison of variable importance based on coefficients.
3️⃣ Use complementary metrics: Incorporate adjusted R² and other model performance measures for a more robust evaluation.

The visualization originally shared by Serhat Simsek clearly demonstrates how confidence intervals shrink and p-values drop with increasing sample sizes. Thanks to Serhat for sharing this valuable content and inspiring the idea behind my post.

#datascience #statistics

[Eyo Eyo, PhD @Eyowhite3]

Key aspects of Exploratory Data Analysis (EDA) beginners should follow.

[Bhaskar Aryal @aryalbhaskar]

@Deepakrajjoshi7 Welcome to K-State Dr. Joshi!

[Bhaskar Aryal @aryalbhaskar]

@AndrewBolis send

[Matt Dancho (Business Science) @mdancho84]

Bayesian data analysis is a fundamental concept in data science. But it took me 2 years to understand its importance. In 2 minutes, I'll share my best findings over the last 2 years exploring Bayesian Modeling. Let's go.

1. Why Bayesian Data Analysis? Bayesian modeling is a powerful tool in statistics and data science, especially where traditional approaches fall short. It avoids arbitrary assumptions and provides distributions of possible values instead of just point estimates.

2. Bayes Theorem: Bayesian modeling is based on Bayes’ theorem. Bayes' Theorem provides a mathematical formula to update the probability for a hypothesis as more evidence or information becomes available. It describes how to revise existing predictions or theories in light of new evidence, a process known as Bayesian inference.

3. Simplification of Bayes’ Theorem: Since X (data) is not dependent on θ (the model) and can be hard to calculate, Bayes’ theorem is often simplified to P(θ|X) ∝ P(X|θ) × P(θ), meaning the posterior distribution is proportional to the likelihood times the prior.

4. From Bayesian Theorem to Bayesian Modeling: Bayes’ Theorem provides a process for constructing a Bayesian model. Combining key ingredients: Likelihood and Prior distributions to produce Posterior Distributions.

5. Calculating the Posterior Distribution: There are two main methods: direct calculation using complex equations, and simulation methods which create samples from the posterior distribution for summarizing information about parameters. Many software programs like PyMC, Brms, and Stan use sampling methods such as Markov Chain Monte Carlo (MCMC).

6. Advantages of Bayesian: The Bayesian approach allows for direct inclusion of prior knowledge, transparency in modeling steps, and provides broad information about the problem, including risks, uncertainty, and variability.

7. Business Cases: Any time knowledge of uncertainty is a business requirement, Bayesian modeling can benefit the business. This includes Demand Forecasting, Pricing Strategy Optimization, Customer Analysis, Credit Scoring and Financial Modeling. Businesses need to know not only a point estimate but the risk or confidence of the prediction.

===

Need help applying data science to business?

I'd like to help. Here's how:

👉 My Free 5-Day Course (How to Solve Business Problems with Data Science): https://t.co/YXG4pL97ZN

👉 Learn ChatGPT for Data Science (Live Workshop): https://t.co/EaMpKrJiqX

If you like this post, please reshare ♻️ it so others can get value (follow me, 🔥 Matt Dancho 🔥 for more data science concepts).

[Bhaskar Aryal @aryalbhaskar]

The monk and the martial artist https://t.co/o29Ydm3S7x via @YouTube

[Bhaskar Aryal @aryalbhaskar]

Introducing the Kansas State University Institute for Digital Agriculture... https://t.co/vfzPP8iW0c via @YouTube

[Matt Dancho (Business Science) @mdancho84]

When I was first exposed to the Confusion Matrix, I was lost. And there was a HUGE mistake I was making with False Negatives. It took me 5 years to fix it. I'll teach you in 5 minutes. Let's dive in.

1. A confusion matrix is a tool often used in machine learning to visualize the performance of a classification model. It's a table that allows you to compare the model's predictions against the actual values.

2. Correct Predicitions: True Positives (TP): These are cases in which the model correctly predicts the positive class. True Negatives (TN): These are cases in which the model correctly predicts the negative class.

3. Model Errors: False Positives (FP, Type I Error): These are cases in which the model incorrectly predicts the positive class. False Negatives (FN, Type II): These are cases in which the model incorrectly predicts the negative class.

4. My Big Mistake: In machine learning we're taught to optimize for model performance. I listened. I said OK, let's optimize for F1 Score. That's the gold standard right?

5. The Problem with F1 Score: The problem with F1 is that it weights False Positives (Type 1) and False Negatives (Type 2) Errors equally. But in business this is RARELY the case. False Negatives are normally 10X to 100X more costly to a business like Netflix. Let me explain.

6. Why minimizing False Positives is worth LESS: If a predictive model (False Positive) incorrectly predicts a customer is going to leave, and Netflix decides to send them preventative actions like a discounted deal. The customer takes it and saves 10%. But they would have stayed anyway. Over a year that costs Netflix $12.

7. Why minimizing False Negatives is worth MORE: Now on the flip side, if Netflix's model incorrectly classifies some one that is on the edge of leaving as predicted to stay, Netflix does nothing. That customer leaves. Over a year that costs Netflix $120. And over the lifetime that could be $500+.

8. What they don't teach you: Expected Value (EV). I'll have a post on that soon. It's how you optimize Machine Learning models for $$$ instead of F1.

===

Ready to learn Data Science for Business?

I put together a free on-demand workshop that covers the 10 skills that helped me make the transition to Data Scientist: https://t.co/LR39RJ5XKB

And if you'd like to speed it up, I have a live workshop where I'll share how to use ChatGPT for Data Science: https://t.co/EaMpKrJiqX

If you like this post, please reshare ♻️ it so others can get value.

[Valeriy M., PhD, MBA, CQF @predict_addict]

It is almost 2024, one should not confuse confidence intervals with prediction intervals.

So what is the difference? This slide describes perfectly. #conformalprediction

[Matt Dancho (Business Science) @mdancho84]

When I was first learning data science, one of the things that tripped me up the most was Cross Validation. In 5 minutes, I'll share 5 years of experimentation with dozens of Cross Validation techniques. Let's dive in.

1. Goal: Cross-validation is a statistical method used to estimate the accuracy of machine learning models. It's also used to measure the stability of models when combined with hyperparameter tuning of machine learning models.

2. Principle: The main principle behind cross-validation is partitioning a sample of data into complementary subsets, performing the analysis on one subset, and validating the analysis on the other subset (called the assessment set).

3. Types of Cross Validation: There are many ways to perform cross validation. Some of the most common are K-Fold, Stratified K-Fold, Leave One Out, Group K-Fold, and Time Series Cross Validation. We'll tackle these one at a time.

4. K-Fold Cross-Validation: The data set is divided into 'k' number of subsets (folds). The holdout method is repeated 'k' times, with each of the 'k' subsets serving as the test data one by one and the remaining 'k-1' subsets as the training data. The average of the 'k' testing experiments is used as the overall result.

5. Stratified K-Fold Cross-Validation: Similar to K-Fold, but the sampling method ensures that each fold of the dataset has the same proportion of observations with a given label. This is particularly useful for imbalanced datasets.

6. Leave-One-Out Cross-Validation (LOOCV): A special case of k-fold cross-validation where 'k' is equal to the number of data points in the dataset. I never use this method because it's very time-consuming. I prefer K-fold and stratified k-fold.

7. Time Series Cross-Validation (TSCV): In time series data, the sequence of observations is important. A common approach is to use a "rolling" or "expanding" window for training and testing. Important point, if your model does not require the sequence to be kept intact, it's sometimes better to use K-Fold. I've seen this with XGBoost, where date features are used rather than lags. K-fold outperforms TSCV. But for ARIMA, TSCV is needed because of the algorithm depends don't the sequence of the time series being maintained.

8. Group K-Fold Cross-Validation: The data is split into groups, and these groups are used to ensure that a group is entirely in the training or test set. This is useful for problems where the data is naturally divided into groups (e.g., customers from different store locations).

Happy Holidays,
-Matt

===

Ready to learn Data Science for Business?

I put together a free on-demand workshop that covers the 10 skills that helped me make the transition to Data Scientist: https://t.co/LR39RJ5XKB

And if you'd like to speed it up, I have a live workshop where I'll share how to use ChatGPT for Data Science: https://t.co/EaMpKrJiqX

If you like this post, please reshare ♻️ it so others can get value.

[Bhaskar Aryal @aryalbhaskar]

Digital Ag & Advanced Analytics Symposium #DA3Symposium https://t.co/J2hhgUnmCY via @YouTube

[Selçuk Korkmaz @selcukorkmaz]

A Simple Guide for Generalized Additive Models (GAMs)

1/ 🧵Let's dive into the world of #GeneralizedAdditiveModels or #GAMs! These are flexible regression models that can capture non-linear relationships. Perfect for when life (and data) isn’t just a straight line! 📈➰

2/ At its core, GAM is a generalization of the linear model. Instead of fitting a straight line (or plane), GAMs fit smooth curves to the data. Think of it as letting the data guide the shape of the relationship, rather than forcing it into a straight jacket!🕺

3/ Why use GAMs? 🤔
• Your scatter plot suggests a wavy pattern
• Residual plots from linear models show patterns (they shouldn't!)
• You have complex temporal or spatial data
• You want flexibility without manually creating polynomial terms

4/ How do GAMs achieve this? 🧐 Through smoothing functions. These are mathematical constructs that allow for bends and twists in the relationship between predictors and the outcome. Splines are a common choice for these functions!

5/ One beauty of #GAMs is that they can handle multiple types of data distributions. Whether you're predicting a continuous variable, binary outcomes, or counts, there’s a GAM for that! It's like GLMs but with added flexibility. 🎯

6/ Now, while GAMs sound dreamy (and often they are!), there's a balance. More flexibility can sometimes lead to overfitting. You know, when your model is TOO tailored to your training data & performs poorly on new data. It's like memorizing answers for a test but failing the real exam. 📚✖️

7/ Fortunately, GAMs have built-in penalties to control for overfitting. It's like having an internal check, making sure the model isn't getting too carried away with wiggles and bends! 🙌

8/ Another plus? Interpreting GAMs can be pretty intuitive. You get visual plots showing the effect of each predictor. Instead of squinting at coefficients, you can see the shape of the relationship directly! 📊

9/ In practice, tools like R's mgcv package make it pretty straightforward to fit GAMs. But, as with all models, understanding the underlying mechanics & assumptions can really up your GAM game! 🛠️

10/ In summary:
• Use GAMs for non-linear relationships
• They're flexible but have checks to prevent overfitting
• Visual interpretations are a boon!

11/ So next time your data doesn’t play nice with straight lines, consider giving #GeneralizedAdditiveModels a spin. They're a powerful tool in a statistician’s toolbox, ready to tackle those wavy, bendy data challenges! 🌀🔍

12/ Liked this thread? Found it useful? Feel free to like, share, and comment with your experiences or questions about GAMs! Let’s keep the #StatsChat going! 🗣️👥🎉

#Statistics #DataScience

[DA3_Symposium @DA3_Symposium]

⚠️ CALL FOR POSTERS ⚠️

👉🏼Be part of the Poster Session at the #DA3Symposium

👉🏼Submit your abstract at
https://t.co/h3XLmJaodm

Abstract Submission Deadline:
Now extended to October 1, 2023

[Bhaskar Aryal @aryalbhaskar]

🌽Mark your calendar Ag Enthusiasts! 📈

[Ajay Sharda @KSU_PrecisionAg]

Kansas State Digital Ag team is organizing "DA3: Digital Ag & Advanced Analytics Symposium" on October 23, 2023. More information about speakers, participation and presenting poster is available at https://t.co/vDSCFRvDtD
https://t.co/qQ57sb3evH

[DA3_Symposium @DA3_Symposium]

#PosterSession
We invite you to submit your abstract!

👉🏼Share your research in our poster session at the Corteva Symposia Series "Digital Agriculture & Advanced Analytics Symposium (DA3)" at the Alumni Center at @kstate

👉🏼To know more visit
https://t.co/2ebY6Unxgk

[Bhaskar Aryal @aryalbhaskar]

@durapada Congratulations Didi!

[Bhaskar Aryal @aryalbhaskar]

Too many zeros and/or highly skewed? A tutorial on modelling health behaviour as count data with Poisson and negative ... https://t.co/zNazhN6qBp