Most âdata quality checksâ are just fancy COUNT(*) WHERE column IS NULL.
Wrapped in a UI.
With an alert.
And a dashboard that says â âPassed.â
Thatâs not a framework. Thatâs a sanity check.
Real data quality = logic, consistency, trends, drift.
Letâs aim higher.
Want to stand out in a SQL interview?
Talk about QUALIFY.
It filters window functionsâlike HAVING, but for ROW_NUMBER(), RANK(), etc.
No CTEs. No subqueries.
Just clean, readable SQL.
Not ANSI, but supported in Snowflake, BigQuery, and Databricks.
Most teams treat databases like static assetsâmanual edits, risky changes, no history.
Thereâs a better way: Database as Code.
â Schema in Git
â CI/CD for DB changes
â Zero-copy clones for testing
â Instant rollbacks
More control. Less chaos. Game changer.
Corporate data engineering in regulated industries be like:
đŤ You scoped the project
đ§ Wrote the SQL in your head
��� But⌠no data access
Finance, healthcare, telecom (especially in Europe)?
Access takes weeksâsometimes months.
That first query hits different after 47 approvals.
What school got wrong about coding:
đ Code on paper
â Real world: Know your IDE
đ Perfect syntax
â Real world: Let the linter fix it
âł 3 weeks per project
â Real world: Done by EOB
Itâs not about memorizingâitâs about solving problems fast.
Unpopular opinion as a Data Engineer:
I use Excel every single day.
â Quick checks
â Debugging
â Sharing data
â Stakeholder sanity
No one asks about it in interviewsâ
But every project touches Excel at some point.
WHERE 1=1 in SQL
Useless or genius?
At first glance, WHERE 1=1 looks pointless.
Why filter rows where 1 always equals 1?
But if youâve worked with dynamic SQL or pipeline queries, you know it actually makes life easier.
It simplifies dynamic filtering.
Instead of checking if a WHERE clause already exists, you can safely append conditions without worrying about syntax errors.
SQL tricks like WHERE 1=1 look unnecessary until you need them.
Have you seen other creative SQL workarounds?
Whatâs a âtypicalâ day for a Data Engineer?
Start the day with⌠a failed ETL job.
A reportâs delayed.
Something looks off in yesterdayâs load.
Scheduled job didnât run.
Every fix makes the system better.
Every fire teaches you something.
SQL Extensions: Helpful or Harmful?
SQL extensions make queries short and fast, but can lock you into one database.
ANSI SQL works everywhere, easy to move later.
Best tip:
Mostly use ANSI.
Use SQL extensions only if really needed.
Which do you pick?
Writing complex SQL in dbt?
End your query with:
SELECT * FROM last_cte
â Easier to debug
â Easier to test each step
â Easier to extend later
Stop stacking filters at the end. Wrap it in a final CTE and keep it clean.
My tech stack went from simple:
- Excel
- SQL
To:
- Excel
- SQL
- Python
- Snowflake
- Talend / Alteryx
- AWS
- CI/CD
- dbt
And all that in a last few years.
I hope you like to learn because in tech you need to learn a lot.
You know what feels better than writing the perfect SQL query?
Not needing to write it at all.
Because the data was modeled right.
Documented well.
And built to be reused.
Great data teams donât reinventâthey enable.
Reusability > Reinvention. Every time.
If you work with data, you know this:
What they say â Whatâs actually true.
âThey said itâs a primary keyâ â You found duplicates
âThey said the schema wonât changeâ â It did
âThey said the sample matches prodâ â It doesnât
Trust, but verify. Always.
âI want to be a Data Engineer⌠but there are too many tools.â
Totally get it. Itâs overwhelming.
But hereâs the truth:
You donât need to learn everything.
Start with:
SQL
Data modeling
One cloud
ETL basics
Python
Master the fundamentals. The tools will follow.
As a Data Engineer, saying âyesâ to everything feels helpful.
Real-time dashboards. Schema changes. Overengineered âscalableâ setups.
But every yes has a cost.
Sometimes, the best thing you can say is:
âNoâor at least, not like this.â
Whatâs the last thing you wish youâd said no to?
School taught you to write perfect pipelines.
The real world says: Ship fast, fix later.
Real-world data work isnât about long queries or complex flows.
Itâs about impact:
Fast insights. Simple pipelines. Scalable SQL.
Still following the old playbook? Time to update it.
Broken dashboards. Conflicting KPIs. Auditors asking, âWhere did this number come from?â
You donât have a data quality problem.
You have a lineage problem.
Map your dependencies.
Track ownership.
Use tools that show you where data really comes from.
Every data team has that pipeline.
Breaks weekly.
Nobody understands it.
Been âtemporaryâ for 2 years.
When is it time to rebuild?
It canât scale
Fixes cause more bugs
Zero tests
Everyone avoids it
Rebuild it boring.
Rebuild it right.
Daily struggles of a Data Engineer:
Pipeline slow? Turn it off/on.
NULL has 17 meanings.
Spark wants more RAM.
Schema changed... again.
Logs? Oh, they are the errors.
API died. No one knows why.
Data consistency? Just pick a version and believe in it.
Still want this job?
Most data engineers never use these 5 SQL functionsâand it slows them down:
1ď¸âŁ COALESCE â Handle NULLs easily
2ď¸âŁ DATE_TRUNC â Clean time-based grouping
3ď¸âŁ ARRAY_AGG â Pack values into one row
4ď¸âŁ LAG â Compare to the previous row
5ď¸âŁ IFF (Snowflake) â Simpler IF logic
Small tricks. Big time savers.