Asgarde

Library that simplifies error handling with @ApacheBeam

[Asgarde @AsgardeBeam]

#Asgarde library simplifies error handling with @ApacheBeam with less code and more concise and expressive code.

Don’t hesitate to contribute to this library and give us feedback.

If you like this project, don’t hesitate to add a #Github star :

https://t.co/lWLjGDKtsO

[Mazlum Tosun 💙 #GCP @MazlumTosun3]

Take a look 👀 to my @kotlin #Asgarde library based on extensions functions.

@AsgardeBeam allows to simplify error handling in @ApacheBeam pipelines and apply Dead Letter Queue.

✅File on @github : https://t.co/OImSCi5oxJ

✅Article : https://t.co/HoVWYwnLIZ

#GoogleCloud #DLQ

[Mazlum Tosun 💙 #GCP @MazlumTosun3]

@ApacheBeam @GoogleCloudTech @AsgardeBeam 👨🏻‍💻👨🏻‍💻 The library is developed in #Java #Kotlin and #Python

🐝🐝 Feel free to test it, give me feedback and to contribute to the project.

🙏🏻🙏🏻 Please also boost the project and add a @github star ⭐⭐

- Java & Kotlin : https://t.co/JllV3aMPox
- Python : https://t.co/nQ0CKo4FYS

[Mazlum Tosun 💙 #GCP @MazlumTosun3]

❓❓ Do you use @ApacheBeam and deal with errors in data processing pipelines?

If it's the case, you know how important this topic is

💡➡ In this case, the best practice and recommended approach is to apply the #DeadLetterQueue principle in your pipelines🧵

@GoogleCloudTech

[Mazlum Tosun 💙 #GCP @MazlumTosun3]

Si vous utilisez @ApacheBeam pour faire du processing de la donnée en mode #streaming et #batch ou vous vous intéressez au sujet, j’ai créé une librairie open source appelé #Asgarde

Cette lib permet de simplifier la gestion d’erreurs avec le principe de #DeadLetterQueue

👇🏻

[Mazlum Tosun 💙 #GCP @MazlumTosun3]

@ApacheBeam @AsgardeBeam

[Mazlum Tosun 💙 #GCP @MazlumTosun3]

If you are interested by @ApacheBeam #BigData, error handling on Data Pipelines and #DeadLetterQueue,
check our @AsgardeBeam libraries for #Java #Kotlin #Python

#Asgarde #Java was downloaded more than 6500 time from Maven Central

#Asgarde #Python more than 2000 from PyPi

👇🏻

[Mazlum Tosun 💙 #GCP @MazlumTosun3]

I published 3 #GoogleDevLibrary 👇🏻

Asgarde Java : https://t.co/5iZW3nJk1U

Asgarde Python : https://t.co/HwJHgrgKOa

Midgard : https://t.co/Ws4a4Vgzr3

@GoogleCloudTech @googlecloud #gcp #data @ApacheBeam #beam @java @kotlin #python

[Mazlum Tosun 💙 #GCP @MazlumTosun3]

I written a new article on #Medium:

"Dead letter queue for errors with @ApacheBeam, @AsgardeBeam, Dataflow and alerting in real time"

https://t.co/wcroJeLtgb

@googlecloud @GoogleCloudTech @googledevgroups @GoogleDevEurope @GoogleOSS @GoogleCloud_FR @GoogleDevExpert #GCP #beam

[Mazlum Tosun 💙 #GCP @MazlumTosun3]

Thank you for allowing me to present these libraries at Beam Summit in Austin. It was a very nice experience !! @AsgardeBeam @ApacheBeam @java @kotlin #python #dlq #errorhandling

[Mazlum Tosun 💙 #GCP @MazlumTosun3]

@AsgardeBeam @java #Python libraries was added to @Advocu_com

https://t.co/5iZW3nJRRs

https://t.co/HwJHgrhiDI

@GoogleCloudTech @googlecloud @ApacheBeam @googledevgroups @GoogleOSS #data #pipeline #dql #gcp #beam

[Mazlum Tosun 💙 #GCP @MazlumTosun3]

I share an example of Dead Letter Queue with @ApacheBeam pipeline #kotlin and @AsgardeBeam #opensource lib.

https://t.co/6STTIbr3QW

Links to the lib :
#java #kotlin : https://t.co/JllV3aNne5
#python : https://t.co/nQ0CKo5dOq

Help us with a Github star ⭐

@GoogleCloudTech

[Mazlum Tosun 💙 #GCP @MazlumTosun3]

@ApacheBeam @AsgardeBeam @GoogleCloudTech - #java and #kotlin versions : https://t.co/JllV3aMPox
- #python version : https://t.co/nQ0CKo4FYS

https://t.co/WVcSkSqHOQ

https://t.co/HoVWYwnLIZ

@googlecloud @GoogleOSS @googledevgroups @java @kotlin #python #data #deadLetterQueue #errorHandling #opensource #beam

[Mazlum Tosun 💙 #GCP @MazlumTosun3]

Example of error handling and dead letter queue with @ApacheBeam :
- Left side, error handling in each step with #Beam native
- right side, error handling in each step with @AsgardeBeam library

Don't hesitate to contribute and support us with a #Github star

@GoogleCloudTech

[Mazlum Tosun 💙 #GCP @MazlumTosun3]

Example of error handling and a #deadLetterQueue with @ApacheBeam #Python and @AsgardeBeam lib👇🏻

https://t.co/KwR7BDM5Je

Links :

#python : https://t.co/nQ0CKo5dOq
#java : https://t.co/JllV3aNne5

#gcp #beam

Don’t hesitate to contribute to the project and add #Github ⭐️

[Mazlum Tosun 💙 #GCP @MazlumTosun3]

I written a new article "Remove duplicates in Bigquery batch pipeline with Airflow and Dataflow"

@ApacheBeam @ApacheAirflow #bigquery #data #dag #DataWarehouse #pipeline #sql #python #cloudDataflow #shell #gcloud #gcp @googlecloud @GoogleCloudTech

https://t.co/0TZuRZZ1gE

[Mazlum Tosun 💙 #GCP @MazlumTosun3]

Demain soir je ferai un replay de mon talk sur la gestion d’erreur avec @ApacheBeam et @AsgardeBeam donné au @BeamSummit, aux adhérents de @GroupBees1 mais en Français cette fois

Au programme 👇🏻

[Mazlum Tosun 💙 #GCP @MazlumTosun3]

I am really happy 😊, @AsgardeBeam #Python library helped a developer on @ApacheBeam to solve an issue with #deadLetterQueue pattern and #errorHandling :👇🏻
https://t.co/nVBicxXoFD

The links to the libs :
Python : https://t.co/nQ0CKo4FYS
Java : https://t.co/JllV3aMPox

#GCP

[Mazlum Tosun 💙 #GCP @MazlumTosun3]

My talk Migration #Spark to #Beam #CloudDataflow with #hexagonalArchitecture and #DDD given at @BeamSummit is now available online

https://t.co/iTyy1biWNG

@ApacheBeam @GoogleCloudTech @googlecloud #data #pipeline

[Mazlum Tosun 💙 #GCP @MazlumTosun3]

My talk Error handling with #Beam and #Asgarde library given at @BeamSummit is now available online

https://t.co/dEZQjuzFPF

@ApacheBeam @GoogleCloudTech @GoogleOSS @AsgardeBeam @GroupBees1 @googlecloud #data #pipeline #errorhandling #java #kotlin #python #opensource

[Asgarde @AsgardeBeam]

The version 0.19.0 of Asgarde Java was released

This version uses the 2.40.0 version of @ApacheBeam

https://t.co/XKBLo7uNvh

#beam #errorhandling #GoogleCloud

Any contributions and feedback are welcomed

If you like the project, support us with a #Github star ⭐️