Albert Joseph

365 days of Data Platform Engineering
▓▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ 01/365

Building telemetry pipelines & data integrity gates.
SQL ➔ Python ➔ Kafka ➔ Spark

[Albert Joseph @__AlbertJoseph]

Day 05/365
Covered core storage & database objects:
Views vs CTAS (Virtual vs Materialized speed)
Temp Tables for session-based staging
Stored Procedures & Triggers (DML/DDL events)
Intro to Indexing for performance

#DataEngineering #SQL

[Albert Joseph @__AlbertJoseph]

Views vs. CTEs vs. Tables:
Views: Persisted SQL logic stored in the database for multi-query reusability without duplicating physical storage.
CTEs: Temporary "on-the-fly" query abstraction with zero maintenance overhead.

Tables: Physical data storage with high performance

#SQL

[Albert Joseph @__AlbertJoseph]

Day 04/365.

Today was all about moving beyond basic queries into complex data transformations, query modularity, and database architecture. Here is a breakdown of what I covered:

[Albert Joseph @__AlbertJoseph]

4. Views & 3-Level Database Architecture

Explored the 3-level database architecture: Physical Layer (raw storage/files/indexes), Logical Layer (tables/relationships), and External/View Layer (virtual abstraction for applications).

[Albert Joseph @__AlbertJoseph]

Day 03/365

SQL Subqueries.

A subquery is a query nested inside another query, pulling data from DB tables or subquery results.

Categories:
• Dependency(Correlated and Non-correlated)
• Result Type
• Clause Location(SELECT, FROM, JOIN, WHERE)

#SQL

[Albert Joseph @__AlbertJoseph]

Day 02/365

Deep dive into DB Architecture & Storage Mechanics.

• Engine vs Storage: Disk (high capacity/permanent) vs Cache (fast/temporary)
• 3 Disk areas: User data, System Catalog(metadata schema) & Temporary Storage.

#DataEngineering #SQL

[Albert Joseph @__AlbertJoseph]

Onto the next chapter!

#DataEngineering #SQL #BuildInPublic

[Albert Joseph @__AlbertJoseph]

1️⃣ Aggregate Functions: COUNT(), SUM(), AVG(), MAX(). MIN()
Use cases:
- COUNT(): Quick dataset snapshots & detecting nulls vs total cases.
- SUM(): Overall analysis, group totals & part to whole analysis.

[Albert Joseph @__AlbertJoseph]

3️⃣ Value Functions: LEAD(), LAG(), FIRST_VALUE(), LAST_VALUE()
Use cases:
- LEAD() & LAG(): Time series analysis(YoY and MoM trends) & customer retention.
- FIRST_VALUE() & LAST_VALUE(): Performance comparison against extremes.

[Albert Joseph @__AlbertJoseph]

Day 01/365
Started learning SQL via the Data with Baraa course on YouTube. I’m currently stepping into Advanced SQL which is Chapter 9.

Committing to posting daily progress updates! The last chapter covered was Window Functions, spilt into 3 core groups:

[Albert Joseph @__AlbertJoseph]

Day 3 of learning java:
- Start Error Handling today.
- Learnt about exceptions and error classes, Exception handling, bubbling up and stack traces.
- Finishing the other parts tomorrow and build two projects.

[Albert Joseph @__AlbertJoseph]

Day 2 of learning Java:
- Spent today learning about OOP in java.
- Learnt about classes, methods, object creation, object destruction, packages, and advanced class design.
- Starting Error Handling tomorrow.

[Albert Joseph @__AlbertJoseph]

Struggled a bit with argument passing/reference and constructors. Right now I only have the theory part down. I'm planning to build a small bank account and a student grading system after I do error handling tomorrow. hoping building those two will make the concepts stick more.

[Albert Joseph @__AlbertJoseph]

Day 2 of learning Java:
- Spent today learning about OOP in java.
- Learnt about classes, methods, object creation, object destruction, packages, and advanced class design.
- Starting Error Handling tomorrow.

[Albert Joseph @__AlbertJoseph]

Day 1 of learning to be a backend/systems engineer, with AI/ML engineering as the long term goal.

- Spent today learning java with the book Learning java 6th edition. Chapters 2-4(variables, loops, conditionals).
- Built a basic calculator.
- Built a number guessing game.

[Albert Joseph @__AlbertJoseph]

Day 1 of learning to be a backend/systems engineer, with AI/ML engineering as the long term goal.

- Spent today learning java with the book Learning java 6th edition. Chapters 2-4(variables, loops, conditionals).
- Built a basic calculator.
- Built a number guessing game.