@neverblinkai Its good article, thanks for sharing. Few more things can be added like in what scenarios we should use what technique. IMO every technique have its own pros/cons as per the conditions and usecase.
Pagination in opensearch is very interesting. There are many technique which can do the pagination and each technique serve its different purpose like "scroll" is used for the reindex (A reindex reads every matching document once, from start to end, with one fixed query, by one consumer). In case of migrations, PIT (point in time) + search_after is really good for resiliency like if job crashes at 60% -> scroll context expires -> start over but in case of the PIT application save the last sort values ["2026-08-07", "PIT-XYZ"] after each page -> job crashes -> resume from the saved point (with a new PIT if the old one expired; the cutover still needs a catch-up step).
A GPU can do billions of calculations, but if memory canโt feed it data fast enough, the compute units sit idle.
70 GB/s => data movement
5 TFLOP/s => computation
The bridge is:
Arithmetic Intensity = FLOPs / Bytes
Decode โ 1 FLOP/byte => memory-bound.
Thatโs why quantization + reducing memory reads can make such a huge difference.
The GPU isnโt weak.
Itโs waiting for data.
Rule of thumb which I understand so far is
- Use the built-in _reindex, or a simple one-off full dump -> scroll (cheapest, and it's what the tooling uses).
- Use a custom, long-running, or checkpoint-and-resume migration job -> PIT + search_after with a stable sort key.
Pagination in opensearch is very interesting. There are many technique which can do the pagination and each technique serve its different purpose like "scroll" is used for the reindex (A reindex reads every matching document once, from start to end, with one fixed query, by one consumer). In case of migrations, PIT (point in time) + search_after is really good for resiliency like if job crashes at 60% -> scroll context expires -> start over but in case of the PIT application save the last sort values ["2026-08-07", "PIT-XYZ"] after each page -> job crashes -> resume from the saved point (with a new PIT if the old one expired; the cutover still needs a catch-up step).
Lately, Iโve started paying a little extra to my Rapido/Uber drivers. It doesnโt really affect my finances, but it might make a meaningful difference to theirs. And honestly, thereโs a different kind of satisfaction in knowing youโve contributed, even in a small way, to someoneโs life.
rovo by atlassian is a game changer [solving all on-call issues from rovo only while working on project in parallel] - structural engineering is really solved by @Atlassian
A good manager and a good company can improve your mental well-being in ways you never imagined.
@Atlassian has been a blessing for me in so many ways. Alhamdulillah ๏ฟฝ๏ฟฝ๏ฟฝ