dataorc

Data Engineering and Product development at scale

[dataorc @DataOrc]

Processing millions of transactions daily? 🚀 Ditched OLTP & Athena for Apache Spark + Hudi—now doing incremental updates instead of scanning 30TB+ daily.
✅ 90% cost savings ✅ Faster queries ✅ Better scaling
Full breakdown: https://t.co/iA1J1M8uww
#BigData #Hudi

[dataorc @DataOrc]

✨ Our fresh new website is live, and it’s looking gorgeous—built effortlessly using @lovable with just a few prompts! 🚀
Check it out ��� https://t.co/g5P9My8SNI
#AI #DataOrc #LovableDev

[dataorc @DataOrc]

New blog: How ONDC secures open commerce network with cryptography!
✍️ By @Ad1tya97
🔑 Ed25519 + Libsodium for auth
📜 Public key discovery via registry
🔒 Asymmetric encryption at scale
Exciting to see Open Networks secured strong! 🚀 #ONDC #Cryptography https://t.co/IjuIuYSaI9

[dataorc @DataOrc]

We used Postgres as a queue to handle extremely large payloads of data. It worked great for us, and we're seeing some great benefits. Check out our blog post to learn more: https://t.co/QcmwBWEWIv

[dataorc @DataOrc]

What's common to @quickheal @dyte_io @helpshift & many more is a petabyte-scale, a battle-tested data platform. Here is our blog on how to handle petabyte-scale leveraging the industry's best data engineering practices
https://t.co/DfSaZttCD2

[dataorc @DataOrc]

Upcoming blogs:
1. How we migrated our polling based ONDC apps to server side events
2. Signing and authentication in ONDC

[dataorc @DataOrc]

We've been fortunate to be associated with @ONDC_Official since beginning. In coming weeks we'll be publishing series of blogs which will cover depths & breaths of ONDC, pls review our latest blog: "ELI5: What is ONDC & developers preview" https://t.co/GeA8ql72TU #ondc #ecommerce

[dataorc @DataOrc]

TIL: Embed collapsible markdown gist to reduce the size of code snippets used in #medium blogs. e.g.
https://t.co/LjilNhyuHy

[dataorc @DataOrc]

@firesofmay @bargava Thanks @firesofmay, @bargava we are building dataplatforms for likes of Helpshift, quickheal, https://t.co/UzjB6aoHzP, and Sql is central to our architecture, can we do short catch up, do checkout our blogs too at https://t.co/XSZNYJKkGm

[Aditya Patil @Ad1tya97]

We @DataOrc just published Google Maps Integration (regardless of which CRS it is) https://t.co/OariTVdZzC

[dataorc @DataOrc]

read about massive parallel processing via #awslambda, our learnings, how did we end up with the huge bill, and yet it’s still our most favored mode of computing. #serverless #aws #terraform

https://t.co/OOCVnTRi0M

[dataorc @DataOrc]

used recently open-sourced https://t.co/MPPBIZyPpc , though still miles to go but no #yaml handling, beautiful abstraction to build complex charts and build it in #Python3. check out one of our charts https://t.co/bo9Tsn8AOQ #Kubernetes #byebyeyaml #AWS #k8s

[dataorc @DataOrc]

scale of internet 101 https://t.co/D1LSX5wz1Y

[dataorc @DataOrc]

https://t.co/Kj7IxC9jBk

[dataorc @DataOrc]

Loadbalancer of loadbalancer, for our across cloud deployment, we found cloudflare loadbalancer to be super quick to set up, intuitive and hassle free, #faulttolerant #cloudflare #cloudnative

[dataorc @DataOrc]

I just published Scalable and Cost-effective ML Pipeline on Kubernetes https://t.co/1cx8RtxlQS

[dataorc @DataOrc]

Storage is cheap, querying is not! 🙂

[dataorc @DataOrc]

Check out our latest blog on our Custom Serverless Data platform
https://t.co/Tg58n2LC94
architecture is so stable that the client has forgotten about the deployment :) #dataplatform #serverless #awscloud #terraform #clojure #startups

[dataorc @DataOrc]

Startups against Covid, we are doing our bit
Check out these useful resources from Startups vs COVID for startups in India who are solving challenges caused by COVID-19. https://t.co/Ev6ysJurmA #StartupsvsCOVID #91springboard #Startups

[dataorc @DataOrc]

If you love #Statistics and
#cricket, this will make your day, @just22yards published an article on elo rating first time applied on ODI teams over the complete course of its history
https://t.co/bDMJDlhxJ1
#cricketforall #analytics #cricketforlife