Hi, I’m Venkat!
This is where I share my journey into tech:
-Learning Python & SQL
-Exploring cloud & data pipelines
-Exploring data engineering tools & cloud
-Sharing what I learn along the way
->Documenting my journey as I grow & build
I am done with my Capstone project! I didn't wanna use the traditional ETL pipeline, instead I chose medallion architecture.
Medallion architecture is flexible and allows a clear path for data validation.
#DataEngineering#DataEngineeringStudy
Starting my morning by working on my ongoing Capstone project! Data engineering is more than just using tools it's filled with fundamentals and business outcome optimization and reasoning.
#DataEngineeringStudy#Saturday
Building a capstone project using DBT and Airflow for an imaginary e-commerce platform. Currently working on the medallion architecture of the ETL pipeline.
@ApacheAirflow@getdbt
Wrapping up my day after covering some topics in Databricks for my Databricks Engineer certification prep.
- Autoloader
- Copy Into Command
- Databricks SQL
#learning#databrickssql
Finished a hands-on dbt project with Databricks—set up env, connected to a SQL Warehouse, built Bronze→Silver→Gold models, declared sources, wrote Jinja macros, added seeds, tests (generic/custom/singular), and snapshots. Thanks to @AnshLambaJSR tutorial. #dbt#Databricks
Attended Zero to Snowflake with James DuHart today.
Hands-on labs helped me explore:
• Data warehousing
• Automated pipelines
• Data Governance
• Snowflake Cortex AI
A great starting point for my #Snowflake journey.
Thank you, @Snowflake, for hosting this online workshop.
Recently worked on a #DataEngineering project:
Crypto Whale Detection ETL Pipeline using #AWS + #Python
• Extracted blockchain data daily
• Stored & transformed in S3
• Automated with Lambda + CloudWatch
• Queried with Glue + Athena
Repo: https://t.co/nisAickXvU