Michael Cade

Global Field CTO | Lead Technologist @Veeam - Kubernetes, Cloud-Native, DevOps & Data #90DaysOfDevOps 👨🏻‍💻

[Michael Cade @MichaelCade1]

Learning the fundamentals of DevOps principles and processes may seem daunting or feel like a huge mountain to climb. I personally don’t think it should or needs to be that way. https://t.co/BuPfGemTzb the community interactions around this repo is mind blowing! #90DaysOfDevOps

[Michael Cade @MichaelCade1]

Today we are going to be build an assistant. Everything I need bubbled into one place.

Written in golang
Runs local

Needs to hit Outlook, Jira, Confluence, newsletters, notion, some forums

[Michael Cade @MichaelCade1]

Change of plans this week!

Going to be warm here in the UK apparently.

[Michael Cade @MichaelCade1]

@_ediri @gopherconuk I have found this event the last few years. The sessions have been really informative and interactive. Something you can’t get from the hallway (although always the best track)

For me it’s also educational and great to ask my beginner noob questions

[Michael Cade @MichaelCade1]

@gopherconuk do you have an expected date for the agenda?

I want to attend but also want to make sure the sessions are useful for me before I purchase a ticket.

[Michael Cade @MichaelCade1]

@user_ops I think this was the databricks announcement this morning as well. So not just DBaaS but also compute.

[Michael Cade @MichaelCade1]

Intrigued to see this play out.

A platform of platforms.

Interesting day one keynote

[Databricks @databricks]

Welcome to the era of agentic data engineering.

Today at the Data + AI Summit, we are excited to announce the next major evolution of Databricks Lakeflow:

- 𝗟𝗮𝗸𝗲𝗳𝗹𝗼𝘄 𝗖𝗼𝗻𝗻𝗲𝗰𝘁: Connect to 100+ enterprise data sources across SaaS, databases, cloud storage, and more.
- 𝗭𝗲𝗿𝗼𝗯𝘂𝘀 𝗜𝗻𝗴𝗲𝘀𝘁: Stream high-volume event data via multiple interfaces, including OTEL and Kafka-compatible APIs.
- 𝗦𝗽𝗮𝗿𝗸 𝗗𝗲𝗰𝗹𝗮𝗿𝗮𝘁𝗶𝘃𝗲 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲𝘀: Batch and streaming, SQL and Python, and now real-time streaming with e2e latencies as low as 5ms.
- 𝗚𝗲𝗻𝗶𝗲 𝗖𝗼𝗱𝗲 & 𝗚𝗲𝗻𝗶𝗲 𝗭𝗲𝗿𝗼𝗢𝗽𝘀: Accelerate authoring and reduce operational toil with agentic development.
- 𝗟𝗮𝗸𝗲𝗳𝗹𝗼𝘄 𝗝𝗼𝗯𝘀: Orchestrate complex data pipelines and AI agents seamlessly with 50+ integrations.

https://t.co/tX3vhxDqzG

[Michael Cade @MichaelCade1]

Is MCP dead? Dying?

[Michael Cade @MichaelCade1]

@virtualhobbit Absolutely! This is my concern! Crazy.

[Michael Cade @MichaelCade1]

I am not against under 16s not being allowed on social media.

I am cautious and interested how a government imposes the identity checks.

It must be levels of airport and bank security which then means it’s in the hands of the Metas and Google?

Seems fishy to me?

[Michael Cade @MichaelCade1]

@GaryWilliamsIT @TheAutisticTech I mean the government and skills not the kids. I am fully aware they can learn and will be steps ahead of whomever the government pays billions to try and implement something.

[Michael Cade @MichaelCade1]

@marius402 You have to believe that this would be covered as another social media platform. There are so many that will need policing.

[Michael Cade @MichaelCade1]

Birmingham airport trying their best for me to miss my flight. Or maybe it’s the passengers that don’t know how to do anything!

[Michael Cade @MichaelCade1]

@user_ops This is my concern

[Jamin Ball @jaminball]

Awesome keynote by @alighodsi at the @databricks summit.

Some takeaways below. TLDR - Databricks is now a data processing platform, a data platform, an agents platform and an apps platform

Architecture updates
Iceberg / table formats
- v3 is now GA: the unified data layer. Delta & Iceberg files are laid out identically on disk, no rewriting to share across formats
- v4 (targeting Q4 '26) finishes the job by unifying metadata. After that, Delta and Iceberg are effectively one format end-to-end

Lakeflow ingestion (3 modalities, all GA):
- ZeroBus: kills the need for Kafka. Hit one API endpoint with bursty/tiny-row data at high rate, no buffering, no millions of small files
- Spark Real-Time Mode: sub-10ms latency in open-source Spark, closing the gap with Flink (Spark's old micro-batch floor was ~1s)
- Lakeflow Designer: Alteryx-style drag-and-drop; talk to Genie, it generates inspectable, version-controllable Spark under the hood
- 100+ connectors now (Salesforce, Workday, NetSuite, Meta, Google Analytics, and more)

Lakebase + LTAP:
- Lakebase: open-source Postgres on the lake. Serverless autoscaling to zero, plus branching: clone a petabyte DB in <1s via copy-on-write (agents love it)
- LTAP: unifies the transactional (Lakebase) and analytical (Lakehouse) layers, a breakthrough the industry has chased for 40 years
- Reyden: new query engine hitting tens-of-ms latency

Unity AI Gateway: one pane of glass for all AI
- Single entry point for every agent, harness & model
- Commit spend to Databricks, buy tokens directly from OpenAI / Anthropic / Gemini on any cloud
- Budgets + alerts down to the individual level
- Guardrails, auditing, identity management for every agent
- Register any MCP server, authenticate once
- Free + open source (part of Unity Catalog + MLflow)

Genie: The Agents Platform, powered by Ontology
Instead of agents for-looping through your data live (slow, expensive, inaccurate), Genie Ontology runs in the background and builds a knowledge graph of your most important assets across lakehouse + Drive + SharePoint + email + many other sources. It ranks importance using "OntoRank" - basically PageRank for enterprise assets. Databricks' own instance has 4.5M ontology snippets. That context feeds four agents:

Genie One: universal interface for any business user to ask questions across all data
Genie Agents: turn any Genie conversation into a deployable autonomous agent
Genie Code: coding agent that's elite at data engineering + ML/data science
Genie Zero Ops: monitors pipelines, fixes a 2am break in an isolated branch, pings you for one-click approval

New vertical apps
Lakewatch: agentic SIEM / "security lakehouse" (also acquiring Panther Labs)

Customer Lake: agentic CDP with LLM-powered identity dedupe + one-to-one "infinity campaigns"

[Michael Cade @MichaelCade1]

@marius402 But also your top point is my biggest concern and what I mentioned in the tweet.

[Michael Cade @MichaelCade1]

@marius402 Why is everyone going on about Bluesky?

I would be more worried about the next social media for kids that is hidden away and easily accessible.

[Michael Cade @MichaelCade1]

@UK_Daniel_Card And speaking

[Michael Cade @MichaelCade1]

@TheAutisticTech And now seeing sparks of VPN control, feels all a little off.

Equally I don’t see them having any real skill to do this.

[Michael Cade @MichaelCade1]

@UK_Daniel_Card By order of the Peaky VPN blinders…..