Yaroslav Tkachenko

Founder at Irontools. Data Streaming Advocate. Building data platforms. Writing about data engineering, real-time data and distributed systems.

[Yaroslav Tkachenko @sap1ens]

The Advanced Apache Flink training is now available as a self-paced, on-demand course!

[Yaroslav Tkachenko @sap1ens]

@almoggavra

[Yaroslav Tkachenko @sap1ens]

@richardartoul We gave a lot of content for free and it didn’t turn out well…

[Yaroslav Tkachenko @sap1ens]

@BenGamble7 Would love to learn more!

[Yaroslav Tkachenko @sap1ens]

“It is built in Rust and leverages the Apache DataFusion query engine”

Any new database these days

[Ankush Gola @ankush_gola11]

We built SmithDB: the database purpose built for agent observability workloads that now powers many parts of LangSmith.

Agent observability presents a challenging data problem. Agent traces can contain tens of thousands of intermediate spans and large, unbounded payloads. These characteristics are a direct result of agents running for longer time horizons and LLM context window sizes growing.

Traditional data infrastructure was not built to handle the complexities associated with storing and querying this data.

SmithDB brings LangSmith up to 12x performance improvements across access patterns most important for agent observability. I’ve been working on SmithDB directly with an amazing team over the past few months, and I’m incredibly proud of the results we’re seeing.

I wrote a bit more about the story and engineering challenges behind SmithDB in this blog.

Additionally, if you’re a systems engineer interested in building the future of agent observability, please reach out!

[Yaroslav Tkachenko @sap1ens]

@gwenshap See you soon!

[Yaroslav Tkachenko @sap1ens]

https://t.co/3xQNHK2j2i

[Yaroslav Tkachenko @sap1ens]

@richardartoul Make it public? 🙂

[Yaroslav Tkachenko @sap1ens]

https://t.co/UOsuxtvdVa

[Yaroslav Tkachenko @sap1ens]

@emaxerrno 7+ hours of sleep is a non-negotiable!

[Yaroslav Tkachenko @sap1ens]

@mitchellh > As an anti example, I would not PR this (without prior warning) to another project.

That's the problem. Many would.

[Yaroslav Tkachenko @sap1ens]

@splix 🙃

[Yaroslav Tkachenko @sap1ens]

@diptanu All my writing is 0% AI 😉

[Cameron @cameron_p_m]

https://t.co/zfTK1IlWN5

[Yaroslav Tkachenko @sap1ens]

Last night 🤟🥁

[Yaroslav Tkachenko @sap1ens]

https://t.co/zdPS8pQwFV

[Yaroslav Tkachenko @sap1ens]

https://t.co/ioHMG3ZMzY

[Yaroslav Tkachenko @sap1ens]

Sharing some thoughts this morning about different paths you can take as a software engineer who's looking for more independence, and perhaps, eventually becoming a solopreneur.

It's not just about building a product or becoming a consultant.

https://t.co/PvhhD8BRiy

[Yaroslav Tkachenko @sap1ens]

Data Streaming Academy has tutorials now! The first tutorial covers reading and writing Kafka consumer offsets in Apache Flink using the State Processor API.

https://t.co/GoBloaZcln

[Yaroslav Tkachenko @sap1ens]

Recently, I spent a lot of time benchmarking various Iceberg sinks (Spark, Flink, Kafka Connect) and trying to beat Supermetal for moving data from Postgres to Iceberg. Here's my report: https://t.co/lOmu6XN5w4

[Yaroslav Tkachenko @sap1ens]

Never gets old