@artie_labs id love to talk with someone on the team about opportunities and also about backfills some of the problems ive been working through and how you approach them at artie :)
Seam
an online backfill engine for postgres cdc. the goal is to copy a table while writes are still happening without the snapshot overwriting newer changes or bringing deleted rows back.
most of the work is around reconciling snapshot rows with the live wal stream. updates can race the snapshot, deletes can happen before a row is copied, events can replay, and the process can crash halfway through.
checkpoints and ordering are persisted so it can restart without pretending the backfill happened exactly once.
still early , @artie_labs was a main reference point while i was working through this