@criccomini@AbeGong@glebmm@leowmjw@datafoldcom@expectgreatdata@anomalo_hq Makes total sense @criccomini. Reliability of EL is done by data/analytics engineering (via different test types). Quality (validation) is done outside of the pipeline and is the responsibility of producers & consumers, which know the data & can actually fix record-level issues.
@criccomini@glebmm@leowmjw@datafoldcom@expectgreatdata@anomalo_hq Got it. I've just never heard of assertions in a business context. Assertions are needed to test what you expect before you transform data. It's bad practice however to encode business rules as assertions in pipelines. Unless you want to go back to the 90s where we did just that.
Don't miss @MahdiKarabiben's detailed overview of the past and present of data validation, and the workflows that will ensure data quality at scale. https://t.co/rcz0i7DPIl
We’re excited to announce our partnership with @sodadata, which brings powerful data quality metrics and insights to the Metaphor platform! Read more here: https://t.co/9R6NT9oGUz #data#datateams#dataobservability
@peeterskris@zevav@sodadata Yes! Hey Zev, Soda SQL & Soda Cloud to the rescue. With Soda SQL you can collect all those metrics. With Soda Cloud you can store the results and dashboard over time (https://t.co/Ij9yFdANou). We also have a reporting API for Soda Cloud if you want to dashboard in e.g. Tableau.
This is really a no brainier but it has to be stressed out - @AndrewYNg wants the ML community to focus more on data than models; he emphasized the importance of MLOps to build and deploy machine learning models more systematically. https://t.co/D1Eo9tfcGZ
Join me live at today at 5.15pm GMT for 'Deconstructing the Raise' with @masscheleinm Co-Founder & CEO of @sodadata who talks us through the two funding rounds he did in 2020 and 2021. Join us live here: https://t.co/9GlfEMV48F
@sarahcat21 We've been going back and forth on this as well at Soda. We've found that making it easy for the engineer to quickly iterate and involve non-tech users (SMEs) via the UI works really well for our use-case.