Naga Mahadevan

Data Engineering, Love Basketball & food

[Snowflake @Snowflake]

Context is a recurring cost. Tool definitions, skill entries, and prior turns all get re-sent as the session grows.

CoCo trims what the agent doesn't need. Tools load on demand, calls get bundled, and big outputs stay out of the prompt. Cost per query drops 33 to 45%. CoWork supplies what the agent does need. In our benchmark, that took a query from $1.76 to $0.59 and accuracy from 24% to 86%.

The deep dive👇
https://t.co/PyLGHtI8dr

[sridhar @RamaswmySridhar]

Sensitive data needs to be accessible without becoming easy to exfiltrate.

Snowflake Data Movement Policies let you govern how data moves — by user, channel, context, and volume — without taking access away.

Security built right into the AI Data Cloud.

https://t.co/a7mOqnmVxl

[Andrew Ng @AndrewYNg]

The most important skills in Building and Deploying AI Applications. https://t.co/IyWIKLIzeM

[sridhar @RamaswmySridhar]

We created data-eng-bench with @bespokelabsai and we’re open-sourcing it.

There are plenty of model performance benchmarks for code generation. But for data engineering, the harness matters just as much as the model.

We built a benchmark that asks agents to build and fix real pipelines, then grades them on whether the output actually works.

The results: Using the same Opus 5 model, @Snowflake CoCo achieves 73.8% Pass@1 at 3.9× lower cost than Claude Code. With Sonnet 5, CoCo delivers the same quality at 2.3x lower cost.

Now anyone can use data-eng-bench to evaluate model + harness combinations on real data engineering workflows. Run your own tests and share what you find.

https://t.co/wlFwOZ3Ne1

[sridhar @RamaswmySridhar]

Bringing @OpenAI's latest open‑source models to @Snowflake-same day. Excited to see what you build. 🚀❄️

[Snowflake @Snowflake]

Introducing Snowpark Connect for Apache Spark™, now in Public Preview.

Building on the decoupled architecture of Spark Connect, Snowpark Connect allows you to run all your compatible Spark SQL, DataFrame, and UDF code directly on the Snowflake platform. This means you get the best of both worlds: the power of Snowflake's vectorized engine combined with the familiarity of your existing Spark code.

Experience faster performance and cost savings compared to managed Spark, as Snowflake automatically handles all performance tuning and scaling. This frees your developers from operational overhead and establishes robust, unified governance upstream. Plus, it works seamlessly with Apache Iceberg tables for your open lakehouse initiatives.

Accelerate development, lower costs, and simplify your data pipelines: https://t.co/ZSl3NO0n7V

[sridhar @RamaswmySridhar]

With Snowpark Connect for Apache Spark™, data teams can keep the tools they know – and gain the performance, efficiency, and governance of the Snowflake platform.

Faster insights. Lower costs. No trade-offs.

Incredible to see how we’re helping businesses move faster and unlock more value from their data than ever before.

[Andrew Ng @AndrewYNg]

There’s a new breed of GenAI Application Engineers who can build more-powerful applications faster than was possible before, thanks to generative AI. Individuals who can play this role are highly sought-after by businesses, but the job description is still coming into focus. Let me describe their key skills, as well as the sorts of interview questions I use to identify them.

Skilled GenAI Application Engineers meet two primary criteria: (i) They are able to use the new AI building blocks to quickly build powerful applications. (ii) They are able to use AI assistance to carry out rapid engineering, building software systems in dramatically less time than was possible before. In addition, good product/design instincts are a significant bonus.

AI building blocks. If you own a lot of copies of only a single type of Lego brick, you might be able to build some basic structures. But if you own many types of bricks, you can combine them rapidly to form complex, functional structures. Software frameworks, SDKs, and other such tools are like that. If all you know is how to call a large language model (LLM) API, that's a great start. But if you have a broad range of building block types — such as prompting techniques, agentic frameworks, evals, guardrails, RAG, voice stack, async programming, data extraction, embeddings/vectorDBs, model fine tuning, graphDB usage with LLMs, agentic browser/computer use, MCP, reasoning models, and so on — then you can create much richer combinations of building blocks.

The number of powerful AI building blocks continues to grow rapidly. But as open-source contributors and businesses make more building blocks available, staying on top of what is available helps you keep on expanding what you can build. Even though new building blocks are created, many building blocks from 1 to 2 years ago (such as eval techniques or frameworks for using vectorDBs) are still very relevant today.

AI-assisted coding. AI-assisted coding tools enable developers to be far more productive, and such tools are advancing rapidly. Github Copilot, first announced in 2021 (and made widely available in 2022), pioneered modern code autocompletion. But shortly after, a new breed of AI-enabled IDEs such as Cursor and Windsurf offered much better code-QA and code generation. As LLMs improved, these AI-assisted coding tools that were built on them improved as well.

Now we have highly agentic coding assistants such as OpenAI’s Codex and Anthropic’s Claude Code (which I really enjoy using and find impressive in its ability to write code, test, and debug autonomously for many iterations). In the hands of skilled engineers — who don’t just “vibe code” but deeply understand AI and software architecture fundamentals and can steer a system toward a thoughtfully selected product goal — these tools make it possible to build software with unmatched speed and efficiency.

I find that AI-assisted coding techniques become obsolete much faster than AI building blocks, and techniques from 1 or 2 years ago are far from today's best practices. Part of the reason for this might be that, while AI builders might use dozens (hundreds?) of different building blocks, they aren’t likely to use dozens of different coding assistance tools at once, and so the forces of Darwinian competition are stronger among tools. Given the massive investments in this space by Anthropic, Google, OpenAI, and other players, I expect the frenetic pace of development to continue, but keeping up with the latest developments in AI-assisted coding tools will pay off, since each generation is much better than the last.

Bonus: Product skills. In some companies, engineers are expected to take pixel-perfect drawings of a product, specified in great detail, and write code to implement it. But if a product manager has to specify even the smallest detail, this slows down the team. The shortage of AI product managers exacerbates this problem. I see teams move much faster if GenAI Engineers also have some user empathy as well at basic skill at designing products, so that, given only high-level guidance on what to build (“a user interface that lets users see their profiles and change their passwords”), they can make a lot of decisions themselves and build at least a prototype to iterate from.

When interviewing GenAI Application Engineers, I will usually ask about their mastery of AI building blocks and ability to use AI-assisted coding, and sometimes also their product/design instincts. One additional question I've found highly predictive of their skill is, “How do you keep up with the latest developments in AI?” Because AI is evolving so rapidly, someone with good strategies for keeping up — such as reading The Batch and taking short courses 😃, regular hands-on practice building projects, and having a community to talk to — really does stay ahead of the game.

[Original post: https://t.co/I3alxNs0vn ]

[Snowflake @Snowflake]

The Apache Iceberg™ v3 table spec is officially ratified! A huge thank you to the entire open source community for this collaborative effort.

This unlocks powerful new capabilities for developers, including native row lineage for reliable CDC workflows, client-side table encryption, and more efficient deletion vectors. We're excited to be part of the community building this.

See the full spec deep-dive: https://t.co/p5H3uHVrOa

[sridhar @RamaswmySridhar]

. @Snowflake is more than a platform. It’s an ecosystem.

Our AI Data Cloud brings together an incredible community of partners and customers, and it was energizing to spend time with so many of them last week at #SnowflakeSummit.

Of our 500+ sessions, over 480 featured our customers telling their stories. @MarriottIntl, @WHOOP, @luminate_data, and so many other world-leading companies and innovators showcased how they’re ingesting data faster, building agentic applications, and putting enterprise AI into action.

Thanks to everyone who joined us! Great to spend time with you!

[sridhar @RamaswmySridhar]

Just announced: @Snowflake has agreed to acquire @crunchydata, bringing open-source Postgres tech into the AI Data Cloud.

With this news, we will be introducing Snowflake Postgres: enterprise-grade, AI-ready, and fully managed.

Run your most critical, AI-powered apps on Postgres, inside Snowflake.

Let’s go. 🚀 #SnowflakeSummit

https://t.co/QfqFWPPM2F

[sridhar @RamaswmySridhar]

In just a few hours, we’re hosting our @Snowflake Summit Opening keynote! ❄️

20K+ attendees are joining us at #SnowflakeSummit for 4 days, over 500+ sessions, hands-on workshops, luminary speakers and much more.

You won’t want to miss it. Tune in here: https://t.co/jXqang1pNM

[sridhar @RamaswmySridhar]

Dev Day is back.
🚀 Bigger. 💡 Bolder. 💻 Built for devs.

We’re bringing the community together—again—for a free day of building, learning, and pushing limits.

World-class speakers. Live demos. Real innovation.

Let’s shape the future—one line of code at a time. See you there!

#SnowflakeSummit #SnowflakeDevDay
https://t.co/S0fPpfLjz4

[sridhar @RamaswmySridhar]

Excited to see @AnthropicAI's Claude Opus 4 & Sonnet 4 are available now in @Snowflake Cortex AI

Snowflake is at the center of today’s enterprise AI transformation and it’s this flexibility and choice to bring  best-in-class AI capabilities to your data that helps our customers lead the way in the era of AI. Can’t wait to see these models in action.

https://t.co/GpATF56jAs

[sridhar @RamaswmySridhar]

Big news: @SnowflakeDB has achieved @DeptofDefense IL5 authorization!
We’re now cleared to deliver secure, mission-critical, AI-powered data solutions to the DOD.

To support this milestone, we’re launching Snowflake Public Sector, Inc.

Details: https://t.co/gjq8Id7YOQ
#GovTech #DefenseTech #IL5 #DataSecurity

[Snowflake @Snowflake]

🚨 We’re excited to announce Amazon S3 Tables is now integrated with Snowflake — bringing powerful, secure, and seamless access to open data directly from Snowflake.

With this integration, you can:
- Query Amazon S3 Tables using Apache Iceberg
- Skip the ETL — your data stays in place
- Use vended credentials for secure, federated access
- Leverage Snowflake’s performance, governance, and simplicity

This is another step forward in making Snowflake the best place for Apache Iceberg.

Try it today: https://t.co/hn9bt0jD9x

[Snowflake @Snowflake]

Apache Polaris (incubating) 0.9 is here!

Lead Developer Advocate OSS, Danica Fine, shares the exciting first official release of Apache Polaris 0.9 — a major milestone for this open, vendor-neutral, multi-format catalog built for open lakehouse architectures.

✨ Highlights:
✅ 1,400+ GitHub stars
✅ 60+ contributors across 10+ companies
✅ Fully open development & governance

This release proves that the Polaris Community can ship software the Apache way — and sets the stage for an upcoming v1.0. Learn more: https://t.co/RtWVJOOOoQ

[sridhar @RamaswmySridhar]

Snowflake Cortex Agents are here. A game-changer for AI-driven workflows:

✅ Search quality that outperforms anything else out there.
✅ Accurate retrieval - not just guesswork - from structured + unstructured data, seamlessly.
✅ Enterprise-grade privacy & control.

Built natively into @SnowflakeDB Cortex AI, powered by @AnthropicAI models. Check it out!

https://t.co/KKfPEENdvy

[sridhar @RamaswmySridhar]

Big news ‼️ Over the past 5 years, SnowConvert— @SnowflakeDB's native code conversion solution for data warehouse migrations—has converted over 2B+ lines of code and 46M+ database objects, with an impressive 96%+ conversion rate. Now, we’re making it easier than ever for organizations of all sizes to move to Snowflake—faster, cheaper, and hassle-free–by making SnowConvert available to everyone: prospects, partners, and customers.

What’s more, we’re taking it up a notch by adding support for Amazon Redshift migrations to Snowflake. Check it out: https://t.co/w9lBV3QXXc

[sridhar @RamaswmySridhar]

🌎 Introducing @SnowflakeDB's One Million Minds + One Platform - where we’re empowering one million minds with AI and data skills 🧠

I couldn’t be prouder to support the next generation of builders, AI users, developers, and data experts with this initiative. 🚀

https://t.co/fFLt4cJ7Z5