Data Ops Guy

Data Ops | Data Engineer | Data something

[Denis Magda @denismagda]

Let Postgres own the Iceberg catalog and delegate analytics to DuckDB. The result => transactional lakehouse updates with fast analytical queries.

This isn’t a concept. It’s exactly what pg_lake delivers today.

pg_lake combines a set of extensions and components that let you query and modify Iceberg tables (and other lakehouse formats) directly from Postgres. DuckDB is used to accelerate analytical queries and runs in a sidecar process called pgduck_server, which communicates with Postgres during query execution.

How it works (diagram below):

1. An application sends a query to Postgres to calculate unrealized PnL (Profit and Loss) for the Disney ticker.

2. Postgres parses the query and identifies the part that computes the average price from historical lakehouse data.

3. That part is forwarded to pgduck_server for accelerated execution.

4. pgduck_server delegates execution to DuckDB, which queries the lakehouse (reusing cached data if available).

5. DuckDB computes the average price and returns it to Postgres.

6. Postgres joins the result with local portfolio data and computes the unrealized PnL.

7. The final result is returned to the application.

[Data Ops Guy @DataOpsGuy]

Listen to Quantum Computing Study Notes on Spotify

https://t.co/KGCoVh7NwY
Mathematical Architecture of Quantum Mechanics
@DataOpsGuy
#DataPipelines #Qubits

[Data Ops Guy @DataOpsGuy]

Listen to Quantum Computing Study Notes on Spotify for Creators https://t.co/yjUJ7D8FSy

#RenovablesEnergy #GRID @SolarEnergyOptimization

[Data Ops Guy @DataOpsGuy]

Listen to DBR DataOpsGuy on Spotify for Creators https://t.co/vx7bwyUASR

[Databricks @databricks]

Data modeling on the Lakehouse looks different than it did a few years ago.

Which warehouse patterns still hold up? What’s evolved? And how does the medallion architecture fit alongside newer, more flexible modeling approaches?

This post breaks down common myths and shares practical guidance for building relational and dimensional models with keys, constraints, and BI-grade performance on Databricks: https://t.co/qJIpPmNrZ6

[Data Ops Guy @DataOpsGuy]

#DatabricksAutoLoader
#AutoLoader @DataOpsGuy

Listen to DBR DataOpsGuy on Spotify for Creators https://t.co/vx7bwyUASR

[Data Ops Guy @DataOpsGuy]

Listen to Quantum Computing Study Notes on Spotify for Creators Week 4 2026 https://t.co/lcgandhCa6

[Data Ops Guy @DataOpsGuy]

#qubits
https://t.co/fjYXxySU2c

[Data Ops Guy @DataOpsGuy]

Supercharging Apache Spark https://t.co/4CMLOfjlfp via @YouTube
#Databricks #DataEngineer

[Data Ops Guy @DataOpsGuy]

@Databricks

https://t.co/XWg9GxgXGi

[Data Ops Guy @DataOpsGuy]

quantum-centric supercomputing (QCSC)
https://t.co/0iLczY6ANg
@Qiskit , @IBM

[Data Ops Guy @DataOpsGuy]

#Databricks #Datamasking
Read “How To Implement Column Masking in Databricks To Protect Sensitive Data“ by Lorena Gongang on Medium: https://t.co/pWpgQj8mYA

[Data Ops Guy @DataOpsGuy]

#Databricks #Lakebase Listen to DBR @DataOpsGuy on Spotify for Creators https://t.co/T5m6rh4Tta

[Data Ops Guy @DataOpsGuy]

Listen to "Episode 7: How Quantum Computers are Slashing Energy Waste" by Quantum Computing Study Notes. https://t.co/2y9owlmph7

[Data Ops Guy @DataOpsGuy]

#Databricks, #DataEngineering, #DatabricksCertified, #DataEngineerAssociate, #AzureDatabricks, #Spark, #ApacheSpark, #DeltaLake, #AutoLoader #DatabricksConnect, #Debugging, #ExamPrep,
https://t.co/MHCh0e7UA8

[Databricks @databricks]

How to balance rapid AI adoption with strong risk management and governance.

We’ve created a practical framework to help teams manage AI risks while enabling collaboration and innovation. This step-by-step approach helps identify, assess, and mitigate risks so you can maximize AI’s full potential while maintaining trust.

https://t.co/irq8GK1bQb

[Data Ops Guy @DataOpsGuy]

Databricks Certified Data Engineer Associate Exam - Section 2: Developme... https://t.co/Qp56jDwwvc via @YouTube

[Data Ops Guy @DataOpsGuy]

Ready for the Databricks Certified Data Engineer Associate exam? This is all updated for the new exam version after July 25, 2025. Let's get certified! 🧑‍💻🔥 - Section 1: Databrick... https://t.co/XH4nnOnOIO via @YouTube

[PVLDB @pvldb]

Vol:18 No:9 → STsCache: An Efficient Semantic Caching Scheme for Time-series Data Workloads Based on Hybrid Storage https://t.co/uuI8HjVF58