Re: [PATCH] rbd: add queuing delay

[Christian Brunner <chb@muc.de>]

> >> > 10.06.20 22:10:07.337108 b67dcb70 client4136.objecter  pg 3.437e on [0] is laggy: 33
> >> > 10.06.20 22:10:07.337708 b67dcb70 client4136.objecter  pg 3.2553 on [0] is laggy: 19
> >> > [...]
> >> >
> >> > Everything is working fine, though. I think that the large number of
> >> > queued requests is the cause for this behaviour and I would propose to
> >> > delay futher requests (see attached patch).
> >>
> >> It seems that the osd is lagging behind. The usleep might work for you
> >> as you avoid the pressure, but it's also somewhat random and will
> >> probably hurt performance on other setups. I'd rather see a
> >> configurable solution that lets you specify a total in-flight bytes or
> >> some other resizable window scheme.
> >
> > I'm not sure if I understand what "lagging behind" means. If the in-flight
> > bytes are the sum of all requests in the queue, a solution could look like
> > this (although it isn't configurable yet).
> >
> The problem is that the sleep will lead to having the osd being
> underutilized in certain configurations. What you probably need here
> is some mechanism that keeps feeding the osd with pending data
> whenever old data has been cleared. E.g., make use of the async
> callbacks to wake up the sender whenever the amount of pending
> outgoing data has fell below some threshold.

Do you mean replacing the sleep with something like a futex? 

When thinking about this a little bit more I'm not sure if doing this in
the client is the right way to handle this. The client has no way to 
judge if the delay is caused by a general high load or by a single slow osd.
Maybe it would be better if libraos could notify the client to slow down?

Christian
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html