Streamkap

Streamkap is a serverless real-time change data capture platform. Scalable, fault tolerant and simple pricing.

[Streamkap @Streamkap]

Managing Kafka shouldn’t feel like an endless battle. Yet, for many organizations, it does:

🔹 Setting up clusters, brokers, and configurations
🔹 Tuning for performance and scaling
🔹 Managing downtime, maintenance, & troubleshooting

[Willy Lulciuc @wslulciuc]

Connecting @Streamkap to @OleanderHQ has never been easier.

Stream your data directly into oleander's fully managed Iceberg REST catalog, then query it with intelligent data compute routing. No clusters. No infrastructure to manage.

See it live 👉 https://t.co/CxjVnm1EVN

[Streamkap @Streamkap]

Cons - Less flexible for schema drift, new columns are excluded by default so you need to monitor new columns and add them if desired. Not supported before PG 15.

[Streamkap @Streamkap]

Pros - Great for least-privilege policies; data won't even be sent to the publication, let alone downstream consumers.

[Streamkap @Streamkap]

Postgres Publication List - If you're on PG 15 or later, you can create a safelist approach on the source side and only send the columns you want to the replication slot in the first place.

[Streamkap @Streamkap]

Flink transformations - Drop or mask relevant data using more powerful Flink transformations; generally used in complex cases SMTs can't handle (e.g. a JSON field which contains a combination of important non-PII data and selected PII info which needs to be dropped)

[Streamkap @Streamkap]

Cons - In addition to the considerations with SMTs above, Flink is another separate distributed system which comes with costs (though it's pretty cost-effective with Streamkap and we remove all the admin overhead).

[Streamkap @Streamkap]

Pros - can remove deeply nested sensitive data amongst required useful data.Flink also allows for flexibility around whether it acts like a blocklist or allowlist and alerting around that.

[Streamkap @Streamkap]

Cons - If Streamkap is deployed as SaaS, data leaves your environment (but is not stored long term), if you have least access policies, it's worth considering that even with BYOC, data is being read out from the source database

[Streamkap @Streamkap]

Pros - The data in question is never delivered to Snowflake, easy to configure with either a blocklist or safelist pattern depending on desired behavior, if Streamkap is deployed as BYOC data never leaves your environment. With a blocklist, you get schema drift for new columns.

[Streamkap @Streamkap]

Single message transformations (SMTs)- We can drop or mask columns in the connector before it's delivered to the destination

[Streamkap @Streamkap]

More and more folks are asking about masking/dropping CDC data from Postgres before it lands in @Snowflake or other destinations.

There are a few options we've seen/enabled Streamkap, each with its pros and cons:

[dltHub @dltHub]

Agentic Analytics Demo Night lands in San Francisco this Wednesday 🚀

We're joining @inngest, @lightdash_devs, and @streamkap for an evening of demos from teams building at the intersection of AI, analytics, and data infrastructure.

🧵

[Streamkap @Streamkap]

npm install -g @ streamkap/tools

@Streamkap CLI is live. Have your agent check it out and tell us what it thinks!

[Streamkap @Streamkap]

At @Streamkap our customers are ready for the future. Are you?

Agents can't make the right decisions without fresh data.

Do your customer and operational agents know what's happening now?

[Streamkap @Streamkap]

@Snowflake We've been excited to partner with a bunch of shared customers to help power real-time agents and analytics on @Snowflake

[Snowflake @Snowflake]

Real-time data is the heartbeat of enterprise AI, but traditional pipeline complexity often stalls the pulse. 🫀

@Streamkap brings zero-ops streaming to the Snowflake AI Data Cloud, turning slow batches into instant, contextually relevant AI. ❄️

Build products, not plumbing.

Learn more: 👉 https://t.co/4ulYXg4pL0

[Paul Dudley @PaulFDudley]

My favorite use case for AI lately?

Excalidraw-style architecture diagrams. See a few recent ones attached here.

I've been working through a backlog of customer case studies for @Streamkap and wanted visualizations to accompany them.

Creating architecture diagrams that show before and after is a pain. I'm not great at it; it would have taken me like an hour per image.

Then, I thought, let's see if nano-banana can handle this. Boy, can it ever.

@GeminiApp Nano Banana Prompt:

"Create an excalidraw style before and after diagram based on this case study."

Then paste into the full text of the case study.

Make sure you check the nano banana/image creation button.

[Streamkap @Streamkap]

@fivetran @DBTLabs @confluentinc @databricks @Snowflake check out the full newsletter on substack https://t.co/I4pvWK1rRS

[Streamkap @Streamkap]

The newsletter is back this week with a recap of a very acquisitive year in the data space. @Fivetran & @DBTLabs merge, @confluentinc goes to IBM ($11B), and @databricks /@Snowflake buy Postgres. @Streamkap keeps shipping and growing

[Streamkap @Streamkap]

@ClickHouseDB Full guide is here - Check the full docs page here - https://t.co/TLzbQIitkg

[Streamkap @Streamkap]

Struggling with SQL Server CDC to @ClickHouseDB? The latest @Streamkap guide is live on the ClickHouse docs with a full how to. Check it out below: