Dedupe.io

is a powerful tool that learns the best way to find similar rows in your data.

[Dedupe.io @DedupeIo]

if you are in the panda-verse, check out this project that makes it easy to use dedupe with pandas.

https://t.co/b2FRvzxtTA

[Dedupe.io @DedupeIo]

hi #NICAR22, if you have data, then you have messy data. journalists can use our web-based record matching service for datasets up to 250,000 rows.

https://t.co/UTmGoYlZx5

[Dedupe.io @DedupeIo]

to get even more control, you can use our open-source record linking engine directly in your python code.

https://t.co/bCNO5e5VjZ

[Dedupe.io @DedupeIo]

.@ING_news's Financial Crimes Analytics group uses the dedupe library to monitor transactions https://t.co/SDdafnZ8rX

[Dedupe.io @DedupeIo]

hi NICARians, do you need to join up messy data or find fuzzy duplicates (you know you do)? we offer up to 250,000 free rows to journalists.

@IRE_NICAR #NICAR21

https://t.co/UTmGoY4oFx

[MaryJo Webster @MaryJoWebster]

#NICAR21: Join us at 11 CT for a live panel on using data to hold police accountable. @SimoneJWei , @eads and I have way too much information for you!

[Enrico A @iEriii]

I've added gazetteer deduplication support to pandas-dedupe.
Deduplicate by matching messy data against canonical data set (i.e. gazette).

📝 Blog post: https://t.co/NE6LtOewwg
👨‍💻GitHub: https://t.co/PKSG2DvCk3

@DedupeIo #pandas #deduplication

[Enrico A @iEriii]

New blog post: record linkage in Pandas.
✍️: https://t.co/68OzWFx0ou

Fast and reliable record linkage using pandas-dedupe.

@DedupeIo #deduplication #Pandas

[Nacho G. del Amo @ngdelamo]

@JaimeObregon Cuando que he tenido que hacer esto casi siempre he tenido que implementar yo mi propia solución. Es un tema difícil de estandarizar. Las pocas veces que el problema era más canónico esta librería es, con diferencia, la que mejor me ha funcionado: https://t.co/Z5hipfUDoi

[Natzir @natzir9]

dedupe: Python library that uses Machine Learning to perform fuzzy matching, deduplication and entity resolution quickly on structured data
https://t.co/4wvegWcAuL

[Scott Stanfield 🇺🇸🧢🌻 @seesharp]

@Gwunhar @DataMadeCo Woah, yeah, that was 2+ years ago in Florida.

I love what @DataMadeCo has done for civic / data journalism. Their https://t.co/nh0PVqtSph is so solid.

[Injustice Watch @injusticewatch]

We used @dedupeio to help us match these messy charges to clean citations from the Illinois State Police. And that was just the beginning. Read more in our methodology: https://t.co/bpo5GIC3Ed (4/6)

[Dedupe.io @DedupeIo]

it would be nice to see some work on how other active learning regimes affects power, and might better approximate a labeled random sample.

[Dedupe.io @DedupeIo]

.@tahamonster, @zubinjelveh, @ajc730, @shiyan_cj, & @benconomics have a nice paper on how using exact matches to link administrative data leads to statistically underpowered analyses.

probabilistic linkage methods, like dedupe, can restore that power.
https://t.co/zuy6YIOmP1

[Dedupe.io @DedupeIo]

they show that the best results come from training a model with a large, presumably random? sample of pairs already labeled as matches or distinct.

this labeled data is typically not available, and they show that dedupe's active learning approach significantly improves power

[Dedupe.io @DedupeIo]

thanks for the shout out, @dataeditor

https://t.co/CTRwKtcW37

[Steven Rich @dataeditor]

@forestgregg seriously it's so good and so necessary

[NPR visuals team @nprviz]

🚨 New blog post: How We Approached Data Cleaning for Our Listeners' Favorite Albums of 2017 by @jjelosua and @BritRenee_ https://t.co/DG5EGtg0lL

[Dedupe.io @DedupeIo]

.@jjelosua and @BritRenee (when they were part of @nprviz) used dedupe to organize and clean the songs that listeners submitted for "All Songs Considered"

https://t.co/BlZZz8zPO6