@ChShersh Spark 3.5 + Delta:
- Streaming job → custom reader → only hot fields + byteOffset
- Delta table, Z-order (symbol, ts) → 500–900 ms queries
- Full record needed? mmap + jump → <100 ms
350 GB/day, <15 GB RAM/exec. Works at our prop desk.
DM for the 120 LOC InputFormat