frevib

Software developer and security engineer

[Phoronix @phoronix]

Axboe Achieves 8M IOPS Per-Core With Newest #Linux Optimization Patches

-- Great work by @axboe in pushing Linux I/O limits!

https://t.co/UYo3nIogfY

[frevib @hielkedv]

@avh what does the 'a' in the astore JVM bytecode instruction stand for? You might know it as you built the compiler :)

[Norman Maurer @normanmaurer]

I know a lot of you have waited for this… I am finally ready to share the first numbers for the #io_uring based #netty transport… While the benchmark is quite limited yet (and done on localhost) it shows the possible gains we can expect 🔥 https://t.co/J2hF6G5B5x

[Norman Maurer @normanmaurer]

After working on the #iouring integration for #netty I am very certain this is the future for IO / syscalls in #linux … If you are curios you can check our current work :

https://t.co/41BHwqfyKE

We also have an io_uring label for PRs related to it.

[frevib @hielkedv]

@daniele_dll Just for archiving, this issue is fixed here https://t.co/z5LuNUDq7f

[Jens Axboe @axboe]

Here's an interesting use-case for kvm ring sharing that allows (basically) native performance:

https://t.co/BfFDtca017

[.Net LibHunt @dotNetLibHunt]

Awesome #.NET Weekly #205 is out https://t.co/wDZs4GqTBW
Featuring @hielkedv @alpinelinux @Aaronontheweb @Nick_Craver

[frevib @hielkedv]

@perfectlydefect @axboe Someone on the ML also had some issues reproducing the performance, maybe have a look: https://t.co/SzDLyl9P0R

[frevib @hielkedv]

io_uring simple echo server: updated using IORING_FEAT_FAST_POLL and IORING_OP_PROVIDE_BUFFERS: https://t.co/WHKHxaCBRk

Performance drops a bit with automatic buffer selection, but possible improvement is in progress: https://t.co/OvVFcO5eF2

[frevib @hielkedv]

@perfectlydefect @axboe FEAT_FAST_POLL should outperform epoll. By how much depends on many variables.A syscall to epoll is already a big save. Even with PROVIDE_BUFFERS I get (a bit less) better performance.These are quick benches but io_uring consistently outperforms epoll.Which bench tool do you use?

[olivier giroux @__simt__]

Hi @axboe, I finally found time to try io_uring again, and just drove one from the GPU today. Checksummed some files from a GPU thread, with no user-space CPU action.

I know this was not the intention, but enabling accelerator programs to make syscalls feels like a milestone.1/2

[frevib @hielkedv]

@maurotortonesi @sadisticsystems @axboe @hamidr_ @pml0pes @VincentFree @jlengrand @normanmaurer @JordyMoos Message sizes are quite arbitrary. Feel free to use the epoll code with non-blocking sockets. These echo servers are kept as simple a possible for ppl to learn io_uring, so no non-blocking/short writes/etc. Critique is OK - better to be correct 👍

[frevib @hielkedv]

#io_uring vs #epoll: simple echo server,some more "serious" raw benchmarks of io_uring's IORING_FEAT_FAST_POLL:https://t.co/HkVY7qCO5N

tldr; io_uring outperforms epoll up to 68% (!)🏆

Next up IOSQE_BUFFER_SELECT to fix buffer selection
@sadisticsystems @axboe @hamidr_ @pml0pes

[frevib @hielkedv]

@axboe There’s also a cost for the app if it’s managing the buffers vs io_uring buffer selection. But even with automatic buffer selection it is much faster that epoll + recv. If we could win some cpu cycles by e.g. reducing CQEs, that would be great :)

[frevib @hielkedv]

@axboe @sadisticsystems @hamidr_ @pml0pes @VincentFree @jlengrand @normanmaurer @JordyMoos Hi Jens, have you thought about adding this? It could solve the problem of "expensive" completions like this: create another ring where in each iteration of the event loop you re-register the buf, chaining them. Then after N chained, one CQE will appear in the other ring.

[Jens Axboe @axboe]

Some storage related benchmarks, pitting io_uring against aio. Looks pretty good to me! Buffered aio will always get slaughtered by io_uring, but O_DIRECT shows nice wins too.

[frevib @hielkedv]

@cl0p3z @sadisticsystems @axboe @hamidr_ @pml0pes @VincentFree @jlengrand @normanmaurer @JordyMoos That would indeed have been better. However, the point here is to show the relative performance improvement to epoll. It should not matter if it's in a VM or not.

[frevib @hielkedv]

@axboe @sadisticsystems @hamidr_ @pml0pes @VincentFree @jlengrand @normanmaurer @JordyMoos This could work. We keep track of N freed up BIDs and chain (link) N SQEs in a N iteration loop. Now we have this loop vs the double iterations with "one CQE for one PROVIDE_BUFFERS". Same no. of iters, but likely(?) cheaper. Further testing will show which approach is better.👍

[frevib @hielkedv]

@axboe @sadisticsystems @hamidr_ @pml0pes @VincentFree @jlengrand @normanmaurer @JordyMoos I was wondering if there are some possibilities here from the kernel side. TOL: give PROVIDE_BUFFERS N (non-sequential) bids to save N-1 cqes. In the end the application itself can also manage the buffers, e.g. with a queue or stack.

[frevib @hielkedv]

@axboe @sadisticsystems @hamidr_ @pml0pes @VincentFree @jlengrand @normanmaurer @JordyMoos If we want to reduce cqes, we can only reuse a whole buffer pool/group when all buffers in it have been used. If one buffer is still in flight (stuck) because of some timeout/network error etc. then the whole pool/group cannot be reused. This is more an application level concern.

[frevib @hielkedv]

@axboe @sadisticsystems @hamidr_ @pml0pes @VincentFree @jlengrand @normanmaurer @JordyMoos Other strategy could be to have multiple buf pools, and switch to a new pool with PROVIDE_BUFFERS to register a whole pool (one cqe). This can easily become more complex and have problems like when a buf is “stuck” in a pool. Maybe the kernel api can assist in any way?

[frevib @hielkedv]

@axboe @sadisticsystems @hamidr_ @pml0pes @VincentFree @jlengrand @normanmaurer @JordyMoos Using N - replenish N; because RECV requests complete async, we have to keep track of the bufs that can be reused. Then we re-add the bufs one by one with PROVIDE_BUFFERS, which will result in N cqes. This won’t reduce the iterations in the event loop. Maybe I miss something?