Co-authored a post w/ @danieljameskay featuring not one, but two of my favorite technologies! #KafkaConnect and #Elasticsearch of course. It walks you through everything you need to know to get started with the Kafka Connect Elasticsearch connector: https://t.co/ATihzo4ku4
How do you get your organization to adopt streaming technologies? I had a great conversation with @tlberglund about how I did it at Stitch Fix. https://t.co/MIZnmMxhN3 Spoiler alert: #KafkaConnect is how
How do you get your team to adopt @apachekafka? @zzbennett joins @tlberglund on #StreamingAudio to share about how she introduced Kafka to @stitchfix_algo, which led to the creation of The Data Highway. Hear about her experience in our latest episode: https://t.co/DZ9jNKegKY
@steven_cuthill One big improvement was the VPC deployments, which fixed all the authentication complexity. They also added slow logs and a few other nice features. The tuning seems better as cluster migrations don't kill indexing and searching anymore. Still fundamentally the same product tho
@lostcol0ny@valencia_andrez Honestly, it depends on what you need it for. We used it for years and although it was occasionally unstable, it did the job well enough and is a pretty cost effective way to run an Elasticsearch cluster
@valencia_andrez No second part. It's basically the same product with the same issues. They added some improvements like VPCs. Tuning seems better now with fewer issues with master nodes getting overwhelmed during full cluster migrations, which still happen when you change literally anything.
@chrilves We only retain the data in Kafka for a few days. The permanent archive of the data is in the data warehouse. We are considering having a few topics as compacted topics so that we can use them for streaming analytics but we haven't set that up yet.
@biophetik@stitchfix_algo We're not using lambdas anymore in our event data pipelines, although we still use lambdas for a few miscellaneous things across the data platform. You can use schemas with vanilla Kafka since you can configure arbitrary key and value converters.
I wrote a blog post on extracting meaning from typically black box recommender systems. Thanks @iPancreas for the beautiful data visualization describing it! https://t.co/HwSLxjJYJi
Stitch Fix just opened sourced Flotilla! It makes running containerized jobs really, really, really really, really easy and simple. I literally use this everyday and I'm so excited that we open sourced it. Check it out! https://t.co/8OTnXbZsSG via @stitchfix_algo
Read all about what I've been up to this last year at Stitch Fix! DZone just published my latest post, Solving Data Integration at Stitch Fix https://t.co/F7js5eJIlB