Atharva Inamdar

Principal Data Engineer.

[Atharva Inamdar @atharvai]

@LNRailway This is a good reminder of why I've avoided trains for the last 5+ years. Yet the ticket prices keep going up and no valuable maintenance of the trains.

[Atharva Inamdar @atharvai]

@LNRailway 1839 train from Euston is delayed and the air con is blasting heat because it is broken due to hot weather 😱 according to driver/manager announcement. Can I get refund for my ticket?

[Atharva Inamdar @atharvai]

@NetworkRailEUS @nationalrailenq seems like the displays need to be swapped. Useful information is on a tiny board and not readable from distance and distracting ads are huge and readable.
Someone forgot this is a train station and not an advert station.

[Atharva Inamdar @atharvai]

I just published Warehouse or Lake House? What to build and what actually matters? https://t.co/Vt7Me6F9Mh

[Atharva Inamdar @atharvai]

@ben11kehoe Deequ library. OSS and integrated with Glue Data Quality.

[Atharva Inamdar @atharvai]

@alexbdebrie If only everyone used the same architecture. You could spin up a graviton instance and run docker there. Both are ARM architectures I think 🤔

[Atharva Inamdar @atharvai]

Actually a very important distinction.

[Alex DeBrie @alexbdebrie]

❗️Wrote something new today:

Event-driven architectures vs. event-based compute in serverless applications.

I see these terms used interchangeably a lot, but they are different, and the implications are important!

https://t.co/MRBtdamY76

[Atharva Inamdar @atharvai]

@boyney123 Is step functions turning into a data processing engine and the orchestration being side feature?

[Atharva Inamdar @atharvai]

@boyney123 @meilisearch How did you decide between DDB and S3 for event store? Perf v cost?

[Atharva Inamdar @atharvai]

@boyney123 The main thing is to get started. The examples are not clear enough for me tbh. With schemas it is usually just bad design allowing any type. Generally loose structure kind of following just the code class structure.

[Atharva Inamdar @atharvai]

@boyney123 Pretty much flow lineage, cataloguing and schema versioning/evolution.

[Atharva Inamdar @atharvai]

I just published Understanding `Optional` and `Nullable` properties https://t.co/F4ly7gxdbz

[Atharva Inamdar @atharvai]

@alexbdebrie Yep. That's why producers can define any partitioning strategy including custom partitioner. A lot of producer and consumer behaviour is what is commonly interpreted as Kafka broker behaviour

[Atharva Inamdar @atharvai]

@boyney123 I was talking about testing rules and input transform in target. This document helps but unfortunately no easy way to test. Especially when using predefined variables ( `https://t.co/zjGvKV8h7v.event.json` ).

[Atharva Inamdar @atharvai]

I just published Two skills I value and look for https://t.co/ldxrFDGAA6

[Atharva Inamdar @atharvai]

@boyney123 Sounds like cataloguing and lineage/tracing of events would help?

[Atharva Inamdar @atharvai]

@boyney123 Btw how did you generate that video? I would like to play around

[Atharva Inamdar @atharvai]

@boyney123 Would be great. Even for Step Functions Amazon States Language and input/results path and processors.