Top Tweets for #PySpark
@X Say hi and tell me what you're currently learning or building! 🚀
#DataEngineering #SQL #PySpark #Azure #DataCommunity
Hey @X algorithm 👋
I'm looking to connect with more Data Engineers and data professionals.
If you're interested in:
📊 Data Engineering
🗄️ SQL
⚡ PySpark
☁️ Azure / AWS / GCP
🏗️ Data Warehousing & Lakehouses
Say hi
#DataEngineering #SQL #PySpark #Azure #DataCommunity
Me: I'll finish my Data Engineering project this weekend.
Also me:
- Watches 5 hours of SQL tutorials
- Organizes folders
- Renames files
- Creates a new GitHub repo
Project progress: 0.1% 😂
#DataEngineering #SQL #PySpark
Hiring: Python Developer (Big Data & SAS Migration Engineer)
#PythonDeveloper #SingaporeTech #BigDataEngineer #PySpark #Hadoop #SASMigration #DataEngineering #Hive #SingaporeJobs #DataPipelines
https://t.co/NtDB9Fvnkv
Just updated my EVM blockchain log decoder pipeline!
Added unified support for @compoundfinance (V1, V2, V3) alongside @Aave.
Refactored the core logic to be completely protocol-agnostic. Ready to scale and easily onboard more money markets!
#DeFi #PySpark #DataEngineering
Decoding blockchain data for money-market protocols.
In my new Blockchain Decoding Series, I walk through exactly how to take raw, unreadable Ethereum event logs from AWS public datasets and transform them into clean, structured data for Aave V1/V2/V3 using Apache Spark!
🧵 Spark AQE Parallelism Dilemma:
Why your "optimized" job is still slow Most data engineers enable Adaptive Query Execution and call it a day.
But there's a hidden setting destroying your performance. Here's what you need to know 👇
#ApacheSpark #PySpark
Exciting opportunity awaits!
Send in your application at [email protected] if you fit the criteria.
Apply today!
#Hiring #LeadDataEngineer #DataEngineering #PySpark #Databricks #AWS #BigData #CloudComputing #PythonDeveloper #SQL #TechJobs #ITJobs #CareerOpportunity
I just published How to identify Outliers and treat them in PySpark https://t.co/FizvvJVqqt
#pyspark #datascience #dataanalysis #python
How does a tiny 50 MB table crash a massive PySpark driver node during a broadcast join? 🤯
If your answer is "it shouldn't," you're missing how Spark handles memory under the hood.
The hidden architecture trap explained below... 👇 #DataEngineering #PySpark
Completed my Medallion pipeline in Databricks! 🚀
Integrated multi-source S3 data using PySpark for schema checks, and optimized the Gold layer with Delta MERGE upserts. Scalable architecture built from scratch. 🧠
Code: https://t.co/m6yrWvQY8I
#DataEngineering #PySpark #sql
✅ Builds stronger foundations for scalable data engineering
Whether you prefer SQL syntax or PySpark DataFrame operations, both approaches ultimately achieve the same business goal. Transforming data into actionable insights. #SQL #PySpark #SparkSQL #DataEngineering #BigData
I just published How to find Duplicates and deal with them in PySpark https://t.co/gqZ273VVuf
#pyspark #python #datascience
I just published How to Deal with Missing Values and Handle Them in PySpark https://t.co/iJEaysmzCa
#pyspark #python #datascience
We are #Hiring for #India #Positions
#JobTitle : #Sr. #AWS #Data #Engineer - #Airflow + #ETL ( #AWS, #PySpark, #Databricks)
#Location : #Hyderabad, #Telangana (#Hybrid)
https://t.co/8J3CCb4VhK
Learning updates 🛠️
• Raw S3 CSVs now move from /landing to /processed only AFTER a successful Unity Catalog Bronze write (https://t.co/ZlmpLkpBml) 🎯
• Used PySpark regex (rlike, regexp_extract) to parse and clean messy date strings
#DataEngineering #PySpark #Databricks #S3
Using Jupyter’s AI integration backed by a local Ollama instance to break down PySpark code.
No data leaving the machine, full code explanations on demand. What a time to be alive! 🚀
#PySpark #DataEngineering #OpenSource #Ollama #Jupyter
We are #Hiring for #India #Positions
#JobTitle : #Sr. #AWS #Data #Engineer - #Airflow + #ETL ( #AWS, #PySpark, #Databricks)
#Location : #Hyderabad, #Telangana (#Hybrid)
https://t.co/211hyYMnb6
We are #Hiring for #India #Positions
#JobTitle : #Sr. #AWS #Data #Engineer - #Airflow + #ETL ( #AWS, #PySpark, #Databricks)
#Location : #Hyderabad, #Telangana (#Hybrid)
https://t.co/Iq4jlw4avj
Most Popular Users
Elon Musk 
@elonmusk
240.1M followers
Barack Obama
@barackobama
119.3M followers
Donald J. Trump
@realdonaldtrump
111.6M followers
Cristiano Ronaldo
@cristiano
108.8M followers
Narendra Modi
@narendramodi
106.9M followers
Rihanna
@rihanna
97.2M followers
NASA
@nasa
92.1M followers
Justin Bieber
@justinbieber
90.5M followers
KATY PERRY
@katyperry
86.7M followers
Taylor Swift
@taylorswift13
80.5M followers
Lady Gaga
@ladygaga
72.1M followers
Kim Kardashian
@kimkardashian
69.3M followers
YouTube
@youtube
68.6M followers
Virat Kohli
@imvkohli
68.4M followers
Bill Gates
@billgates
63.4M followers
The Ellen Show
@theellenshow
62.5M followers
CNN
@cnn
61.9M followers
Neymar Jr
@neymarjr
60.9M followers
X
@x
60.9M followers
CNN Breaking News
@cnnbrk
59.9M followers