#datapiplines - Twitter Hashtag

about 1 year ago

Tech Stack: Python Airflow Gen AI and LLMs SQL Pyspark GitAction Rest APIs #dataengineer #ArtificialIntelligence #datapiplines #RemoteJobs

1

2

0

1

159

Malcolm Chisholm

@MDChisholm

about 3 years ago

When #CICD (Continuous Integration / Continuous Deployment) is applied to #datacentric projects, there can be a #datagovernance problem. If the #dataengineers run the #datapiplines then they have become #dataoperations. They will gradually spend more and more time running rather than developing. Running regularly scheduled batch jobs in production, distributing outputs, troubleshooting, liaising with users, etc. is a specialization that should be left to those who do it well.

MDChisholm's tweet photo. When #CICD (Continuous Integration / Continuous Deployment) is applied to #datacentric projects, there can be a #datagovernance problem. If the #dataengineers run the #datapiplines then they have become #dataoperations. They will gradually spend more and more time running rather than developing. Running regularly scheduled batch jobs in production, distributing outputs, troubleshooting, liaising with users, etc. is a specialization that should be left to those who do it well.

0

2

0

51

Michael Meyer @dataguyatheart

about 3 years ago

Curious about #datapiplines and what it takes to make a robust one? Check out this latest blog from @Alation - https://t.co/U3FOWo9QBm

0

2

0

182

Luke Farrell @TheLukeFarrell

over 4 years ago

Join @Fivetran & @Databricks for a live, hands-on workshop! We'll present a step-by-step guide to building #automated #data pipelines on the #lakehouse - power your #datamovement #dataintegration #datapiplines https://t.co/TTHNuTJilR

0

2

1

0

Luke Farrell @TheLukeFarrell

over 4 years ago

Join @Fivetran & @Databricks for a live, hands-on workshop! We'll present a step-by-step guide to building #automated #data pipelines on the #lakehouse - power your #datamovement #dataintegration #datapiplines https://t.co/VbR4H0RHHu

0

2

1

0

Luke Farrell @TheLukeFarrell

over 4 years ago

Join @Fivetran & @Databricks for a live, hands-on workshop! We'll present a step-by-step guide to building #automated #data pipelines on the #lakehouse - power your #datamovement #dataintegration #datapiplines https://t.co/dhcuPXHFGj

0

1

0

Emtec Inc. (Now Bridgenext) @EmtecInc

almost 5 years ago

Learn the steps for a #DataOps implementation and check out a case study on how you can utilize Azure/AWS to achieve your DevOps objectives. https://t.co/fq9fGPS0Un #implementingdataops #cloud #Devops #datapiplines #analyticinsights #emtecdigital

0

2

3

0

Astronomer

@astronomerio

almost 5 years ago

You asked we deliver! In this blog post, you'll learn all about #DataPiplines, their types, key components, best practices, and how #Airflow can make them better! https://t.co/n4dcWTftV5

0

2

0

Astronomer

@astronomerio

almost 5 years ago

You asked we deliver! In this blog post, you'll learn all about #DataPiplines, their types, key components, best practices, and how #ApacheAirflow can make them better! https://t.co/n4dcWTftV5

0

2

1

0

Luke Farrell @TheLukeFarrell

about 5 years ago

98% of companies asked said "#datapiplines break frequently" Is that YOU? Check out @McCabeJesse of @Fivetran & @kellykohlleffel of @hashmapinc discussing - Benefits of the Modern Data Stack and ELT, on @YouTube! #hashmapontap #data #datastack #ELT https://t.co/WLGXTs460q

0

1

0

Luke Farrell @TheLukeFarrell

about 5 years ago

98% of companies asked said "#datapiplines break frequently" Is that YOU? Check out @McCabeJesse of @Fivetran & @kellykohlleffel of @hashmapinc discussing - Benefits of the Modern Data Stack and ELT, on @YouTube! #hashmapontap #data #datastack #ELT https://t.co/0Rqqz9kUAA

0

1

0

Luke Farrell @TheLukeFarrell

about 5 years ago

98% of companies asked said "#datapiplines break frequently" Is that YOU? Check out @McCabeJesse of @Fivetran & @kellykohlleffel of @hashmapinc discussing - Benefits of the Modern Data Stack and ELT, on @YouTube! #hashmapontap #data #datastack #ELT https://t.co/nPu7vEiRmT

0

1

0

Luke Farrell @TheLukeFarrell

about 5 years ago

98% of companies asked said "#datapiplines break frequently" Is that YOU? Check out @McCabeJesse of @Fivetran & @kellykohlleffel of @hashmapinc discussing - Benefits of the Modern Data Stack and ELT, on @YouTube! #hashmapontap #data #datastack #ELT https://t.co/cbRWzsss6G

0

1

0

CIOTechOutlook @CIOTech_Outlook

almost 6 years ago

#GDPR and the Right to be Forgotten: Lessons for India Inc in Times of #Pandemic The GDPR was published two years ago. At that point in time nobody had foreseen the outbreak of the #COVID19 pandemic. Read More: https://t.co/JuKlveQe8B @druvainc #edgecomputing #datapiplines