Ayomide

Data Engineer - Building Scalable Data Pipelines & Analytics Platforms | ETL/ELT • PostgreSQL • Airflow • Python • Cloud Data Engineering @taxstreem

[Ayomide @sh_ayomide]

Hi 👋, I am Ayo a Data Engineer building optimized data systems that enables organization makea data driven decisions.

Currently expanding my expertise in modern data engineering practices, including orchestration, data warehousing, cloud platforms, and data governance.

[Ayomide @sh_ayomide]

Day 02 of #DataGist with @sh_ayomide

[Ayomide @sh_ayomide]

A 30 Days #DataGist

[Ayomide @sh_ayomide]

@LeoOliemans91 The post-fix window is where things get risky.
My check would be:
1. Re-read after the write.
2. Compare row counts, hashes, or key aggregates.
3. Mark recovered only if they match
Zero-row writes and race conditions are too common to trust the first attempt.

[Ayomide @sh_ayomide]

AI agents + Data engineering is a serious combo....imagine an agent detecting a failed pipeline, investigating the logs, identifying the bad dataset, and opening a fix automatically.

[Ayomide @sh_ayomide]

Day 01 of #DataGist with @sh_ayomide

[Ayomide @sh_ayomide]

A 30 Days #DataGist

[Y Combinator @ycombinator]

The Startup Industrial Base: Building for the Next 250 — Washington, D.C. https://t.co/7A8r3jfSf8

[Ayomide @sh_ayomide]

A 30 Days #DataGist

[Ayomide @sh_ayomide]

But your revenue dashboard is now wrong.

A simple data-quality check could catch it:
order_amount IS NOT NULL
order_amount >= 0

This is why a successful pipeline ≠ trustworthy data.

[Ayomide @sh_ayomide]

The job succeeds.....but a source-system change turns:
₦25,000 → 25000
into:
₦25,000 → NULL
No pipeline error. No failed job. Everything looks 🟢.

[Ayomide @sh_ayomide]

The job succeeds.....but a source-system change turns:
₦25,000 → 25000
into:
₦25,000 → NULL
No pipeline error. No failed job. Everything looks 🟢.

[Ayomide @sh_ayomide]

Your pipeline can run successfully and still deliver completely wrong data.

Imagine an e-commerce pipeline processes today’s orders.

[Ayomide @sh_ayomide]

Your pipeline can run successfully and still deliver completely wrong data.

Imagine an e-commerce pipeline processes today’s orders.

[TaxStreem @taxstreem]

Have tax questions?

We have answers!

Join us at The Bridge for 90 minutes of practical answers and jargon free explanation.

CEO, C- Suite exec, Accountant, Tax & Finance managers

Everyone gets a seat.

[Ayomide @sh_ayomide]

With idempotency, you use a unique order_id and an UPSERT/merge strategy:
Order #1025 → ₦50,000 ✅

Run it once or run it five times, the final state remains the same.

That’s idempotency:
Safe retries without corrupting your data.

[Ayomide @sh_ayomide]

Imagine a pipeline processes yesterday’s orders and inserts:

"Order #1025 → ₦50,000"
The job fails after inserting the record, then retries.

Without idempotency:
"Order #1025 → ₦50,000"
"Order #1025 → ₦50,000" ❌

Now your revenue is overstated.

[Ayomide @sh_ayomide]

Imagine a pipeline processes yesterday’s orders and inserts:

"Order #1025 → ₦50,000"
The job fails after inserting the record, then retries.

Without idempotency:
"Order #1025 → ₦50,000"
"Order #1025 → ₦50,000" ❌

Now your revenue is overstated.

[Ayomide @sh_ayomide]

This matters when you have:
• Retries
• Failed jobs
• Duplicate events
• Backfills
• Scheduled runs

A pipeline shouldn’t just move data.

It should be designed to recover safely when things go wrong.

[Ayomide @sh_ayomide]

This matters when you have:
• Retries
• Failed jobs
• Duplicate events
• Backfills
• Scheduled runs

A pipeline shouldn’t just move data.

It should be designed to recover safely when things go wrong.

[Ayomide @sh_ayomide]

If the same job runs twice and creates duplicate records, you have a problem.

That’s where idempotency comes in.

An idempotent pipeline can safely process the same input multiple times without changing the final result.

[Ayomide @sh_ayomide]

If the same job runs twice and creates duplicate records, you have a problem.

That’s where idempotency comes in.

An idempotent pipeline can safely process the same input multiple times without changing the final result.

[Ayomide @sh_ayomide]

What happens when your data pipeline runs twice?

[Ayomide @sh_ayomide]

What happens when your data pipeline runs twice?

[Ayomide @sh_ayomide]

A really interesting perspective on where data engineering is heading.

Spotify 👇🏾
https://t.co/AolXeJP4YV

[Ayomide @sh_ayomide]

It’s becoming less about simply building pipelines and more about building data systems that AI agents can actually reason over, interact with, and use effectively.

[Ayomide @sh_ayomide]

It’s becoming less about simply building pipelines and more about building data systems that AI agents can actually reason over, interact with, and use effectively.

[Ayomide @sh_ayomide]

What stood out to me is how the rise of agentic systems is pushing us to rethink the relationship between AI, data, databases, and the architecture that connects them.

[Ayomide @sh_ayomide]

What stood out to me is how the rise of agentic systems is pushing us to rethink the relationship between AI, data, databases, and the architecture that connects them.

[Ayomide @sh_ayomide]

AI agents are changing more than how we use data. They’re changing how we architect it.

Just listened to “From RAG to Relational: How Agentic AI is Reshaping Data Architecture.”

[Ayomide @sh_ayomide]

AI agents are changing more than how we use data. They’re changing how we architect it.

Just listened to “From RAG to Relational: How Agentic AI is Reshaping Data Architecture.”

[TaxStreem @taxstreem]

The rollout of Rev 360 has begun, and it’s happening in phases.

First, medium and emerging taxpayers are being onboarded.

Then, large taxpayers and government institutions will follow.