Michael Malis

Co-creator of pgrust: formerly founder of Freshpaint (YC S19)

[Michael Malis @mmalisper]

I'll be giving a talk next week at the @PlanetScale SF Database Meetup about how we made @pgrustdb and how we made it 300x faster than Postgres for analytics

[Michael Malis @mmalisper]

@andreaposti @pgrustdb @JasonSeibel The priorities are set to decay over time so lower latency queries inherently get throttled less

The main priority right now is doing enough testing that pgrust is production ready

[Michael Malis @mmalisper]

In @pgrustdb, @JasonSeibel and I built a solution to one of the most common causes of Postgres outages: runaway queries.

pgrust can selectively throttle queries. By default, we deprioritize long running queries, preventing one big long running query from taking down your db

[Michael Malis @mmalisper]

@andreaposti @pgrustdb @JasonSeibel right now it's based on cpu time, but there's lots of design space for making it better

[Michael Malis @mmalisper]

til in addition to doing COUNT(DISTINCT x) in Postgres, you can also do SUM(DISTINCT x) and AVG(DISTINCT x)

[Michael Malis @mmalisper]

@mrmarkhalonen @pgrustdb @JasonSeibel yep! that's the primary way we've been invisioning people using pgrust initially. It's low risk and is one of pgrust's strengths

[Michael Malis @mmalisper]

@kolovskydev @pgrustdb @JasonSeibel the throttle doesn't do anything smart right now and we'll have to tune it for things like locks over time

[Michael Malis @mmalisper]

@kolovskydev @pgrustdb we try to size tasks so they take on the order of 2ms

[Michael Malis @mmalisper]

My favorite change we made to @pgrustdb is how we rearchitected parallel queries. Postgres decides a fixed degree of parallelism at plan time. For pgrust, we assign cpus to queries dynamically. For beefy machines with many cores, this results in a massive speedup

[Michael Malis @mmalisper]

@gergomoricz @CalebPeffer we have some prototypes that say yes

[Michael Malis @mmalisper]

@zombodb @pgrustdb @JasonSeibel shared locks is a different problem - this is more oriented around you have a couple long running queries doing big table scans that use up all your I/O

[Michael Malis @mmalisper]

@LeoOliemans91 @pgrustdb the main way we tested this was by measuring tail latency of fast queries while simultaneously running slower longer queries

we based the approach on this paper if you want to read more about it: https://t.co/tXjoq6kjWD

[Michael Malis @mmalisper]

@mr_mich_e @PlanetScale @pgrustdb we support both a columnar format and a row format so you can use both

[Michael Malis @mmalisper]

I'll be giving a talk next week at the @PlanetScale SF Database Meetup about how we made @pgrustdb and how we made it 300x faster than Postgres for analytics

[Michael Malis @mmalisper]

@MarioBalukcic @pgrustdb We're still working on it every day. We're getting close to a v0.3 launch (the public version is actually the second release - it was 10x faster than the initial release)

[Michael Malis @mmalisper]

@vamsi2708 @pgrustdb Custom skill and a point and click interface for giving feedback

[Michael Malis @mmalisper]

@camuelg @pgrustdb exactly

[Michael Malis @mmalisper]

@kolovskydev @pgrustdb Yes. We have a fixed sized thread pool that works off of a shared queue of small tasks. More queries means the thread pool spreads across more queries. Fewer queries and the pool will adapt

[Michael Malis @mmalisper]

@ericciarla @pgrustdb feel free to dm

[Michael Malis @mmalisper]

@andrew_dizenzo @pgrustdb I have a skill and a vibecoded point and click interface where I can click on different parts of the animation and give feedback