GlassFlow

Run any data transformation at TB scale for ClickHouse
🌐

[GlassFlow @glassflowdev]

Full breakdown + upgrade notes → https://t.co/xch3FsCRtp

[GlassFlow @glassflowdev]

Before v3.1.0, when a downstream stage couldn't keep up, GlassFlow had no coordinated way to slow down. Events would pile up, NATS memory would fill, pipeline would fail. That's fixed now. 🧵

[GlassFlow @glassflowdev]

Also in v3.2.0: OTLP receiver concurrency caps, operator reconciles 4 pipelines in parallel, and full backpressure signals from every component to the control plane.

[GlassFlow @glassflowdev]

Most teams default to OTel → Kafka → ClickHouse for observability.
Kafka is powerful. But is it always necessary?
We broke down when it's worth the overhead and when it isn't.
👉 https://t.co/KdQz7pIUvI
#ClickHouse #OpenTelemetry #DataEngineering #Kafka #OTel

[GlassFlow @glassflowdev]

ReplacingMergeTree deduplicates eventually in #ClickHouse.
Your counts might be wrong until the merge runs.
Fix it upstream with GlassFlow → https://t.co/Os0EBSSyHu

#DataEngineering #GlassFlow #OpenSource #DataPipelines

[GlassFlow @glassflowdev]

If you're sending #OTel data to ClickHouse without a processing layer in between, you're probably storing 15–30% duplicate logs and fighting nested JSON schema issues.
We built GlassFlow to fix this:dedup, enrichment, & schema mapping for #ClickHouse.
→ https://t.co/YnEvwtOmRK

[GlassFlow @glassflowdev]

Still looking for a Founding Engineer 🚀🚀
Not a "senior dev" role. Not a backlog ticket role. A "here's the problem, let's build for it" role.
→ Low-latency #eventstreaming infra
→ Billions of events
→ GlassFlow stack: #Go, #Kafka, #NATS
Apply here: https://t.co/3BkOo0hrBZ

[GlassFlow @glassflowdev]

Your Kafka → #ClickHouse pipeline shouldn't be a 2 AM nightmare.
ReplacingMergeTree doesn't deduplicate on insert. It merges "eventually", leading in messy count()s
GlassFlow fixes this by handling dedup, PII masking, and more upstream
➡️ Details: https://t.co/Os0EBSSyHu
#Kafka

[GlassFlow @glassflowdev]

Stop writing glue code to connect #OTel to #ClickHouse. 🛑
GlassFlow now ingests OTLP data natively: deduplicating, masking PII, and delivering traces/logs/metrics query-ready.
✅ No #Kafka
✅ No custom transforms
✅ Just clean data
See how: https://t.co/5c5kc7cBk9
#OpenTelemetry

[GlassFlow @glassflowdev]

Read more here: https://t.co/NyORU1iJ3l

[GlassFlow @glassflowdev]

ClickHouse for #observability is great, until you have to deal with:
• Duplicate spans from collector retries
• PII leaking into trace attributes

New guide: #OTel Collector → GlassFlow → #ClickHouse
Dedup, PII masking, tail sampling.
🔗 Link in replies

#OpenTelemetry

[GlassFlow @glassflowdev]

Working with #ClickHouse and running into the "Too Many Parts" errors? Let the server do the heavy lifting with Asynchronous Inserts.

Check our article on Asynchronous Inserts in ClickHouse here: https://t.co/utDEhSaQ2h

#DataOps #Analytics #DataEngineering #RealTimeData

[GlassFlow @glassflowdev]

We just shipped native #OTel ingestion in GlassFlow. 🚀
The OTel Collector exports. ClickHouse stores. Nobody handles what happens in between.
Now GlassFlow does: deduplication, enrichment, schema mapping, before the first row hits your DB.
https://t.co/5c5kc7cBk9
#OpenTelemetry

[GlassFlow @glassflowdev]

https://t.co/UR4WBOuMY4
#DataEngineering

[GlassFlow @glassflowdev]

Moving to #ClickHouse? Don't let denormalization slow you down!
While it’s a standard hack for row-based DBs, in ClickHouse it causes:
📦 Bloated storage
✍️ Write amplification
⏳ Ingestion friction
The solution?
https://t.co/UR4WBOuMY4
#DataEngineering

[GlassFlow @glassflowdev]

ClickHouse #MaterializedViews are powerful, but they aren't a silver bullet.
Learn how they work and when they might be killing your ingestion performance.

Read more: https://t.co/GuzYqjyAqQ
#ClickHouse #DataEngineering #OpenSource

[GlassFlow @glassflowdev]

Backfilling PB-scale #AI data is usually where the "standard" approach breaks.

If you're managing real-time embeddings or feature stores in #ClickHouse, you need a strategy that won't kill your production uptime.
Here\s how to handle the switch:
https://t.co/LyjgLkZuZu
#AIInfra

[GlassFlow @glassflowdev]

ICYMI: GlassFlow just broke the 500k events/sec barrier. 🚀

We’ve achieved linear horizontal scaling within a single pipeline. No architectural rework, no complexity—just add resources and go.

Read the benchmark breakdown:
https://t.co/EuW03fXqKX

#DataEngineering #ClickHouse