#datanengineering - Twitter Hashtag

over 3 years ago

✍️Blogged: Aligning mismatched #Parquet schemas in #DuckDB 🦆 https://t.co/MRsBaUAiKl #datadon #data #datanengineering #xp

2

24

6

7

8K

Joseph Machado

@startdataeng

almost 4 years ago

JSON (VARIANT) fields are great for data pipelines. + You can dump each JSON record into a single field + Schema changes in the JSON record do not affect ingestion - Size limitations (DB specific) - No inherent type information, like protobuf #datanengineering #datamodeling

0

3

0

Joseph Machado

@startdataeng

almost 4 years ago

Data pipelines break! It is the data team's responsibility to fix and make them resilient. Some common causes 1. Data size too large 2. Unaccounted data skew 3. Bug in code 4. Insufficient permissions 5. SQL division by zero, missing commas, etc #datanengineering #data #SQL

3

7

0

SeattleDataGuy @SeattleDataGuy

almost 5 years ago

These videos are all competing for the top video 5 hours after posting! Consider checking them out. #datascience #sql #programming #data #datanengineering #python

SeattleDataGuy's tweet photo. These videos are all competing for the top video 5 hours after posting! Consider checking them out. #datascience #sql #programming #data #datanengineering #python https://t.co/PYMLt75XH5

3

8

0

1

0

Top Tweets for #datanengineering

Last Seen Hashtags on Sotwe

Trends for you

Most Popular Users