Most “data quality checks” are just fancy COUNT(*) WHERE column IS NULL.
Wrapped in a UI.
With an alert.
And a dashboard that says ✅ “Passed.”
That’s not a framework. That’s a sanity check.
Real data quality = logic, consistency, trends, drift.
Let’s aim higher.
Want to stand out in a SQL interview?
Talk about QUALIFY.
It filters window functions—like HAVING, but for ROW_NUMBER(), RANK(), etc.
No CTEs. No subqueries.
Just clean, readable SQL.
Not ANSI, but supported in Snowflake, BigQuery, and Databricks.
Most teams treat databases like static assets—manual edits, risky changes, no history.
There’s a better way: Database as Code.
✅ Schema in Git
✅ CI/CD for DB changes
✅ Zero-copy clones for testing
✅ Instant rollbacks
More control. Less chaos. Game changer.
Corporate data engineering in regulated industries be like:
🚫 You scoped the project
🧠 Wrote the SQL in your head
��� But… no data access
Finance, healthcare, telecom (especially in Europe)?
Access takes weeks—sometimes months.
That first query hits different after 47 approvals.
What school got wrong about coding:
📝 Code on paper
✅ Real world: Know your IDE
📚 Perfect syntax
✅ Real world: Let the linter fix it
⏳ 3 weeks per project
✅ Real world: Done by EOB
It’s not about memorizing—it’s about solving problems fast.
Unpopular opinion as a Data Engineer:
I use Excel every single day.
✅ Quick checks
✅ Debugging
✅ Sharing data
✅ Stakeholder sanity
No one asks about it in interviews—
But every project touches Excel at some point.
WHERE 1=1 in SQL
Useless or genius?
At first glance, WHERE 1=1 looks pointless.
Why filter rows where 1 always equals 1?
But if you’ve worked with dynamic SQL or pipeline queries, you know it actually makes life easier.
It simplifies dynamic filtering.
Instead of checking if a WHERE clause already exists, you can safely append conditions without worrying about syntax errors.
SQL tricks like WHERE 1=1 look unnecessary until you need them.
Have you seen other creative SQL workarounds?
What’s a “typical” day for a Data Engineer?
Start the day with… a failed ETL job.
A report’s delayed.
Something looks off in yesterday’s load.
Scheduled job didn’t run.
Every fix makes the system better.
Every fire teaches you something.
SQL Extensions: Helpful or Harmful?
SQL extensions make queries short and fast, but can lock you into one database.
ANSI SQL works everywhere, easy to move later.
Best tip:
Mostly use ANSI.
Use SQL extensions only if really needed.
Which do you pick?
Writing complex SQL in dbt?
End your query with:
SELECT * FROM last_cte
✅ Easier to debug
✅ Easier to test each step
✅ Easier to extend later
Stop stacking filters at the end. Wrap it in a final CTE and keep it clean.
My tech stack went from simple:
- Excel
- SQL
To:
- Excel
- SQL
- Python
- Snowflake
- Talend / Alteryx
- AWS
- CI/CD
- dbt
And all that in a last few years.
I hope you like to learn because in tech you need to learn a lot.
You know what feels better than writing the perfect SQL query?
Not needing to write it at all.
Because the data was modeled right.
Documented well.
And built to be reused.
Great data teams don’t reinvent—they enable.
Reusability > Reinvention. Every time.
If you work with data, you know this:
What they say ≠ What’s actually true.
“They said it’s a primary key” → You found duplicates
“They said the schema won’t change” → It did
“They said the sample matches prod” → It doesn’t
Trust, but verify. Always.
“I want to be a Data Engineer… but there are too many tools.”
Totally get it. It’s overwhelming.
But here’s the truth:
You don’t need to learn everything.
Start with:
SQL
Data modeling
One cloud
ETL basics
Python
Master the fundamentals. The tools will follow.
As a Data Engineer, saying “yes” to everything feels helpful.
Real-time dashboards. Schema changes. Overengineered “scalable” setups.
But every yes has a cost.
Sometimes, the best thing you can say is:
“No—or at least, not like this.”
What’s the last thing you wish you’d said no to?
School taught you to write perfect pipelines.
The real world says: Ship fast, fix later.
Real-world data work isn’t about long queries or complex flows.
It’s about impact:
Fast insights. Simple pipelines. Scalable SQL.
Still following the old playbook? Time to update it.
Broken dashboards. Conflicting KPIs. Auditors asking, “Where did this number come from?”
You don’t have a data quality problem.
You have a lineage problem.
Map your dependencies.
Track ownership.
Use tools that show you where data really comes from.
Every data team has that pipeline.
Breaks weekly.
Nobody understands it.
Been “temporary” for 2 years.
When is it time to rebuild?
It can’t scale
Fixes cause more bugs
Zero tests
Everyone avoids it
Rebuild it boring.
Rebuild it right.
Daily struggles of a Data Engineer:
Pipeline slow? Turn it off/on.
NULL has 17 meanings.
Spark wants more RAM.
Schema changed... again.
Logs? Oh, they are the errors.
API died. No one knows why.
Data consistency? Just pick a version and believe in it.
Still want this job?
Most data engineers never use these 5 SQL functions—and it slows them down:
1️⃣ COALESCE – Handle NULLs easily
2️⃣ DATE_TRUNC – Clean time-based grouping
3️⃣ ARRAY_AGG – Pack values into one row
4️⃣ LAG – Compare to the previous row
5️⃣ IFF (Snowflake) – Simpler IF logic
Small tricks. Big time savers.