Re: RFC v1: Xen block protocol overhaul - problem statement (with pictures!)

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Mon, Jan 21, 2013 at 12:37:18PM +0000, Ian Campbell wrote:
> On Fri, 2013-01-18 at 18:20 +0000, Konrad Rzeszutek Wilk wrote:
> > 
> > > > E). The network stack has showed that going in a polling mode does improve
> > > > performance. The current mechanism of kicking the guest and or block
> > > > backend is not always clear.  [TODO: Konrad to explain it in details]
> > 
> > Oh, I never did explain this - but I think the patches that Daniel came
> > up with actually fix a part of it. They make the kick-the-other guest
> > only happen when the backend has processed all of the requests and
> > cannot find anything else to do. Previously it was more of 'done one
> > request, lets kick the backend.'.
> 
> blkback uses RING_PUSH_RESPONSES_AND_CHECK_NOTIFY so doesn't it get some
> amount of evthcn mitigation for free?

So there are two paths here - the kick from a) frontend and the kick b) backend
gives the frontend.

The a) case is fairly straighforward. We process all of the rings we and everytime
we have finished with a request we re-read the producer. So if the frontend keeps
us bussy we will keep on processing.

The b) case is the one that is trigger happy. Every time a request is completed (so
say 44kB of data has finally been read/written) we kick the frontend. In the
networking world there are mechanism to modify the hardware were it would
kick the OS (so frontend in our case) when it has processed 8, 16, or 64 packets
(or some other value). Depending on the latency this can be bad or good. If the
backend is using a very slow disk we would probably want the frontend to be
kicked every time a response has been completed.

But if we have a very fast SSD, we might want to batch those kicks up so
that the frontend does not get kicked that often. I don't know the impact
of these 'one request = one kick' is but we could make this a bit more
adaptive - so that it starts scalling down the kicks as it has more responses.
And if there are less responses it notches up the amount of kicks.
I think this is called adaptive interrupt moderation (Or interrupt coalescing)

> 
> > But going forward, this 'kick-the-other-guest' could be further modulated.
> > If we have a full ring and the frontend keeps on adding entries we (backend)
> > should never get an interrupt. As of matter of fact the frontend should be just
> > polling all the time and process them as fast as possible.
> 
> I think the existing req_event/rsp_event ring fields should enable this
> already, assuming the front- and back-ends are using them right.
> 
> Ian.
>