Performance regression between V2_02_177 and V2_02_180

[David Teigland <teigland@redhat.com>]

On Wed, Oct 30, 2019 at 09:34:51AM +0000, Ben Sims wrote:
> I have not observed any contention directly but the comment in the commit
> 
>     "ca66d520326493311a3c7132b1bcee0807862301
> 
>     io: use sync io if aio fails
>     
>     io_setup() for aio may fail if a system has reached the
>     aio request limit.  In this case, fall back to using sync io..." 
> 
> implies that aio is a resource with a limit and open for contention. I
> observed heavy background aio traffic with perf probes from our virtual
> disk backends, I don't believe the io_submit calls in LVM are blocking
> as they are submitted with O_DIRECT and strace time to return is low.

OK, I thought you were referring to contention that would cause lvm's
async io to suffer, as opposed to simply being unavailable.  If other
programs are consuming all of the available aio, then you can increase
/proc/sys/fs/aio-max-nr, which would hopefully make some available for
lvm.

> This may off course be academic if lvm moves to io_uring. Have there
> been any experiments in this area?

No, but it does sound interesting.  I should check if the userspace
library for that has matured.

Dave