Daily Databricks

@databricks @azuredatabricks tips straight to your twitter feed! For Beginners to Advanced users with snippets and examples.

#databricks by @fusionet24

[Daily Databricks @DailyDatabricks]

There's a really rich #R package for databricks from databricks-labs. You should give it ago. If you've previously used Sparklyr or SparkR this is a worth while package to pick up for your platform.

I've jumped into a quick overview below https://t.co/kg3I5xlJFi

[Daily Databricks @DailyDatabricks]

Did you know about With Columns Renamed? You can pass in Key Vaule Pairs of
{old_name: new_name}

https://t.co/CKTtVPYSsp

[Daily Databricks @DailyDatabricks]

Anyone still here?

[Daily Databricks @DailyDatabricks]

NEW YEAR NEW YOU I get it but there's still time to think about this super awesome @databricks tip from last year

[Daily Databricks @DailyDatabricks]

1/6 🎄 Launching #Decembricks!

24 days of @databricks tips to supercharge your data engineering.

Day 1: The hidden power of IPython events - your ticket to professional-grade notebook engineering 🚀

Check this one out seriously! The code is at the bottom

[Daily Databricks @DailyDatabricks]

4/4 Read the full guide here https://t.co/03MICjFH2x
#Databricks #DataEngineering #Python #Ipython #Spark

[Daily Databricks @DailyDatabricks]

1/4 🧵 Just published a guide on creating Custom IPython Magics in Databricks!

First, we'll cover the basics:
Line magics (%) vs Cell magics (%%)
Creating custom commands
Registering your magics

[Daily Databricks @DailyDatabricks]

3/4 This is part of the #Decembricks2024 series, where we're sharing a new Databricks tip every day of December!

[Daily Databricks @DailyDatabricks]

💡 Quick fix: Try 'UNDROP TABLE WITH ID' instead of table name. And if you've got naming conflicts, use 'ALTER TABLE RENAME' first to clear the path for restoration. #TechTips #DatabaseAdmin 3/3

https://t.co/U0n9vugy5g

[Daily Databricks @DailyDatabricks]

🔄 Getting the INVALID_STATE.RESTORATION_PERIOD_EXPIRED error in #Databricks when trying to undrop tables? Even within the 7-day window? Here's what you need to know...

[Daily Databricks @DailyDatabricks]

🔍 Pro tip: Use 'SHOW TABLES DROPPED' to get the exact table ID and drop time. Sometimes the issue isn't about timing - it could be naming conflicts or incorrect table references. #DatabricksSQL #DataOps 2/3

[Daily Databricks @DailyDatabricks]

Ever fancied using a managed identity for your #Azure #Databricks workspace that already exists and to start using it only requires a few lines of code?

Find out how to use this from your Managed Resource Group https://t.co/5uIYWaHr4E

[Daily Databricks @DailyDatabricks]

4/ Ready to transform your Databricks testing workflow? Check out the full post here: 🔗https://t.co/g0G2VEIAGc #Databricks #PythonTesting #Unittest #DynamicTesting

[Daily Databricks @DailyDatabricks]

🧵 Dynamically Loading Unit Tests in Databricks Notebooks 🚀

1/ Tired of the hassle of manually maintaining test suites?

Want to keep your tests closer to your code?

Our latest tip dives into dynamically loading unit tests using Python's unittest library. 🐍🧪

[Daily Databricks @DailyDatabricks]

3/ We break down the process step by step, from setting up the TestSuite to filtering and loading the discovered tests. 📝 Code samples included! 💻

[Daily Databricks @DailyDatabricks]

2/2 By defining a window spec once & referencing it by name, you can:

🔹Reuse complex definitions

🔹Improve query readability

🔹Avoid errors

Check out the blog for a quick guide with examples! 🔗 https://t.co/gzqgbts7fH

#DataWithDatabricks #SparkTips

[Daily Databricks @DailyDatabricks]

1/2 Are verbose window functions bogging down your @databricks SQL & PySpark code? 😰 Named window specs to the rescue! 🚀

[Daily Databricks @DailyDatabricks]

We are over on 🟦☁️ as @Dailydatabricks.tips

[Daily Databricks @DailyDatabricks]

https://t.co/26WT7LkPU1

[Daily Databricks @DailyDatabricks]

💡#Decembricks Day 2
Use #DeltaLake's custom metadata to:
- Track data quality metrics
- Maintain audit trails
- Document pipeline versions
- Monitor data transformations

Full guide with code examples below 👇
@Databricks #DataEngineering #Databricks