every system i've ever monitored did exactly what the code told it to. web, mobile, microservices, gpu clusters.
models are different. it's a live thinking body of neurons, thinking way faster than we can think. & our whole monitoring playbook assumes a human reads the dashboard in time to do something about it.
that assumption is gone. the controls have to sit outside the model & move at its speed, not ours.
i spent years thinking the dashboard was the product.
splunk, then telemetry & reliability for large gpu clusters. in that world, if you could see it & alert on it, you'd done the job.
with models, monitorability is just the means. controllability, safety & alignment are the real benches that get built on top of it. a trace of what the model is doing only matters if something downstream can act on it before the action lands.
most of the conversation right now still stops at "we can see it."
this is a great step into how most of the companies will try to go into production with AI. Monitorability and Observability has truly been essential across each and every workload transition we have had; web2.0, mobile, micorservices, infra, etc. & now we will see a host of companies trying to capitalize across these dimensions as well.
looking forward to what people build next in this space.
@matvelloso whats new or on the roadmap for Docker? heard it's super high churn for the pm's especially because its hard to build consensus internally!