From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: RFC v1: Xen block protocol overhaul - problem statement (with pictures!) Date: Thu, 24 Jan 2013 10:11:36 -0500 Message-ID: <20130124151136.GC5448@konrad-lan.dumpdata.com> References: <20121218143109.GA24471@phenom.dumpdata.com> <50F9718A.4040202@citrix.com> <20130118182028.GB11351@phenom.dumpdata.com> <50FA9545.9090406@citrix.com> <20130122194640.GB10733@phenom.dumpdata.com> <1358934794.3279.431.camel@zakaz.uk.xensource.com> <20130123152124.GF3067@phenom.dumpdata.com> <1358955677.17440.51.camel@zakaz.uk.xensource.com> <20130123165930.GC12605@phenom.dumpdata.com> <1359022016.17440.100.camel@zakaz.uk.xensource.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Content-Disposition: inline In-Reply-To: <1359022016.17440.100.camel@zakaz.uk.xensource.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Ian Campbell Cc: "axboe@kernel.dk" , "xen-devel@lists.xensource.com" , Felipe Franciosi , "martin.petersen@oracle.com" , "matthew@wil.cx" , Roger Pau Monne List-Id: xen-devel@lists.xenproject.org On Thu, Jan 24, 2013 at 10:06:56AM +0000, Ian Campbell wrote: > On Wed, 2013-01-23 at 16:59 +0000, Konrad Rzeszutek Wilk wrote: > > On Wed, Jan 23, 2013 at 03:41:17PM +0000, Ian Campbell wrote: > > > On Wed, 2013-01-23 at 15:21 +0000, Konrad Rzeszutek Wilk wrote: > > > > > Have you consider variable length requests? This would let the request > > > > > size scale with the number of segments required for that request, and > > > > > allow you to cache align the ends of the requests without wasting the > > > > > extra space that including the worst case number of segments would > > > > > imply. e.g. a small write would take 32-bytes (padded to 64 if you must) > > > > > and a larger one would take 196 (padded to 256). You should end up with > > > > > more efficient use of the space in the ring this way. > > > > > > > > Yes. If I recall right, the D) had it as "we could negotiate a proper alignment for > > > > each request/response structure.". The idea is pretty much the same as yours > > > > - pad to the cacheline if the cacheline is bigger than the request or response. > > > > > > It would be interesting to know whether it is cache line bouncing or > > > smaller ring capacity (which padding reduces) which becomes the > > > bottleneck. > > > > > > My money is, unhelpfully, on "both, depending on workload". > > > > and on the guest pCPU distribution potentially. The one workload I've been > > doing is to stick guests on different sockets and also turning > > hardware prefetching off (and also Turbo Mode for good measure). > > This last two are constructing a rather artificial scenario though > aren't they? Yes. But they give a great insight on the cache issues. > > > This way it is the worst of the worlds - but it can provide me with the > > data whether this cache issue is a problem or not. > > > > > > > > > > > > > > _______________________________________________ > > > Xen-devel mailing list > > > Xen-devel@lists.xen.org > > > http://lists.xen.org/xen-devel > > > > >