Re: RFC v1: Xen block protocol overhaul - problem statement (with pictures!)

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Wed, Jan 23, 2013 at 03:41:17PM +0000, Ian Campbell wrote:
> On Wed, 2013-01-23 at 15:21 +0000, Konrad Rzeszutek Wilk wrote:
> > > Have you consider variable length requests? This would let the request
> > > size scale with the number of segments required for that request, and
> > > allow you to cache align the ends of the requests without wasting the
> > > extra space that including the worst case number of segments would
> > > imply. e.g. a small write would take 32-bytes (padded to 64 if you must)
> > > and a larger one would take 196 (padded to 256). You should end up with
> > > more efficient use of the space in the ring this way.
> > 
> > Yes. If I recall right, the D) had it as "we could negotiate a proper alignment for
> > each request/response structure.". The idea is pretty much the same as yours
> > - pad to the cacheline if the cacheline is bigger than the request or response.
> 
> It would be interesting to know whether it is cache line bouncing or
> smaller ring capacity (which padding reduces) which becomes the
> bottleneck.
> 
> My money is, unhelpfully, on "both, depending on workload".

and on the guest pCPU distribution potentially. The one workload I've been
doing is to stick guests on different sockets and also turning
hardware prefetching off (and also Turbo Mode for good measure).

This way it is the worst of the worlds - but it can provide me with the
data whether this cache issue is a problem or not.

> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>