Hi everyone!
Have you ever wondered about:
Why is data cleaning crucial?๐ค
What does dirty data mean?๐ค
And how can we clean our data to be tailored for our goals? ๐ฏ
Let's get into this data-cleansing thread๐งน๐ก #DataCleaning#DataScience
Understanding these missing value mechanisms is vital for an accurate approach to handling missing data in our data sets so that we can decide whether to apply data imputation or deletion. ๐๐#DataScience
2๏ธโฃ MAR (Missing At Random):
Scenario: Participants skip or elderly ones forget weight based on self-reported height and age (X)
Explanation: Missing weights depend on observed variable (X), like height, age. Varying heights may lead to more skips. ๐๐ค #DataScience
3๏ธโฃ MNAR (Missing Not At Random):
Scenario: Participants hide weight, linked to their weight itself (Y).
Explanation: Missing weights are tied to the variable being measured. Those with a higher body weight might be more likely to skip. ๐ก#DataScience
1๏ธโฃ MCAR (Missing Completely At Random):
Scenario: Gym scale malfunctions during survey.
Explanation: Participants face random weight value gaps, unrelated to any characteristics (X or Y). A glitch, not tied to survey factors. ๐๏ธโโ๏ธ#DataScience
In other words, clean data equals clean decisions. Having clean data will ultimately boost overall productivity and provide the highest quality information for your decision-making.
As for dealing with typos in ๐ฐ๐ฎ๐๐ฒ๐ด๐ผ๐ฟ๐ถ๐ฐ๐ฎ๐น ๐ฐ๐ผ๐น๐๐บ๐ป๐, "if it is a lot and varying", the Levenshtein distance metric could be a good option for fixing misspellings.
๐ช๐ต๐ ๐ถ๐ ๐ฑ๐ฎ๐๐ฎ ๐ฐ๐น๐ฒ๐ฎ๐ป๐ถ๐ป๐ด ๐ฐ๐ฟ๐๐ฐ๐ถ๐ฎ๐น?
Data cleaning is highly important because it ensures the accuracy, reliability, consistency, compatibility, and completeness of our data resulting in meaningful insights and sound decisions to achieve our goals.
Hi everyone!
Have you ever wondered about:
Why is data cleaning crucial?๐ค
What does dirty data mean?๐ค
And how can we clean our data to be tailored for our goals? ๐ฏ
Let's get into this data-cleansing thread๐งน๐ก #DataCleaning#DataScience