Erik Sipsma

Software Engineer @dagger_io

[Dagger @dagger_io]

How can we make containerized development and deployment more efficient and user-friendly? @gerhardlazu hosts @ESipsma on his Podcast discussing using the combination of #BuildKit and #Dagger to automate almost everything.

https://t.co/XvbFg8pnra

[Airbyte @AirbyteHQ]

We at Airbyte are excited to share our journey of optimizing our CI/CD process with the Dagger platform. As an open-source ELT platform managing hundreds of Docker containers, we faced challenges with our original CI system, which was a maze of GitHub Actions, YAML, and shell scripts.

It was not only slow but also challenging to maintain. However, with @dagger_io introducing its Python SDK in 2022, we saw an opportunity and started transitioning to a more streamlined CI/CD setup by early 2023.

With Dagger Cloud, we've achieved a 2-5x boost in our CI pipelines and managed to cut our costs by a whopping 75%. This "Daggerized" CI system has transformed our workflow. With Dagger Cloud's intelligent caching, our CI processes are significantly faster.

By integrating Dagger into a Kubernetes cluster with auto-scaling runners, we've optimized our resource usage, leading to considerable savings. As we look to the future, our plan is to further expand our CI infrastructure with Dagger and encourage our teams to adopt and contribute to this efficient platform.

Our collaboration with Dagger has been pivotal, and we're eager to see where this partnership will take us next!

[Darren Shepherd @ibuildthecloud]

Dagger is probably the thing I'm most excited about right now. I'm having a lot of fun with it. Like normal, I'm not really using it the right way, but I see so much potential in it. I like where it's headed.

[Erik Sipsma @ESipsma]

@ibuildthecloud The universe has always been a DAG, we’re just giving you the API for it 🙂

[Dagger @dagger_io]

Introducing the Dagger Node.js SDK: develop your CI/CD pipelines in Typescript or Javascript, run them in containers anywhere. https://t.co/8P5wALai3b

[Dagger @dagger_io]

Introducing the Dagger Python SDK: develop your CI/CD pipelines in Python, run them in containers anywhere. https://t.co/X8YY59WTqz

[Dagger @dagger_io]

Dagger Go SDK 0.4.0 is released! https://t.co/rxtvbHjHCR

[Erik Sipsma @ESipsma]

@zefexdeveloper @dagger_io @solomonstre @golang We updated our docs with more info on this: https://t.co/fxFhXYycO7

Thank you for the suggestion!

[Dagger @dagger_io]

Introducing the Dagger Go SDK: a new way to develop your CI/CD pipelines as code, and run them in containers anywhere. https://t.co/7riHAwUSMO

[Dagger @dagger_io]

Build your own automation platform in 20mn, one artisanal script at a time. @solomonstre just showed us how with #ProjectCloak at #VMwareExplore.

Check out the recording here: https://t.co/EnTQhcVwUx

[Dagger @dagger_io]

Join the Livestream of @solomonstre's talk at #VMwareExplore at 11 am PST. He'll demo how to build an automation platform in 20 mn, one artisanal script at a time! #ProjectCloak https://t.co/qS82rWMDWy

[Dagger @dagger_io]

Introducing Dagger for VS Code, with native language server support for @cue_lang . Featuring syntax highlighting, hover, and jump to definition. It’s never been easier to develop CI/CD pipelines in your favorite IDE. https://t.co/dkVjMy820E

[Erik Sipsma @ESipsma]

@gerhardlazu Thanks Gerhard! Glad I could spread the good word 😄

[Erik Sipsma @ESipsma]

@luca_cloud Thanks Luca! Appreciate you having me :-)

[Erik Sipsma @ESipsma]

Check out my talk at #PlatformCon about modeling CI/CD workflows with the power of BuildKit, CUE and Dagger! https://t.co/hOpczAhSTj

[Erik Sipsma @ESipsma]

@ForSpareParts @dagger_io e.g. there could be a reusable library for converting a list of apk packages to LLB, or one for converting an npm package.json to LLB. Merge+Diff enable these libraries to be as efficient as possible at the cost of the effort needed for deep integration.

[Erik Sipsma @ESipsma]

Wrote about a new way to build DAGs with BuildKit. I'm excited by use cases like constructing container images as lazy package merges. Between the low-level efficiency of BuildKit and high-level power of tools on top of it like @dagger_io, a lot to look forward to in this space!

[Docker @Docker]

Complex builds that once took an hour @netflix now take three minutes with MergeOp, a new feature released in #BuildKit v0.10. Engineer @ESipsma explains how features like MergeOp & DiffOp enable BuildKit to maximize cache reusability & unlock use cases: https://t.co/bhELy4mdb2

[Erik Sipsma @ESipsma]

@ForSpareParts @dagger_io Exactly, shell commands are a universal interface but lossy by wrapping the DAG in a black box. The goal for merge+diff would be to create high-level interfaces that are simple but underneath the hood integrate with tools to maximize cacheability.

[Erik Sipsma @ESipsma]

@ForSpareParts @dagger_io Not yet, I’ve done some early investigation with Alpine but had to put that on the backburner temporarily due to time constraints. Nix is promising too, some discussion here: https://t.co/s39NAZVkhU

[Erik Sipsma @ESipsma]

@jonjonsonjr Thanks, glad there's an audience out there that these otherwise extremely niche topics appeal to!