From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: Xen Developer Summit Storage Performance BoF notes Date: Wed, 30 Oct 2013 14:20:51 -0400 Message-ID: <20131030182051.GA25174@phenom.dumpdata.com> References: <5270E79A.1050703@citrix.com> <5270EE70.5080300@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Return-path: Content-Disposition: inline In-Reply-To: <5270EE70.5080300@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Roger Pau =?iso-8859-1?Q?Monn=E9?= Cc: David Vrabel , "Xen-devel@lists.xen.org" List-Id: xen-devel@lists.xenproject.org On Wed, Oct 30, 2013 at 12:33:04PM +0100, Roger Pau Monn=E9 wrote: > On 30/10/13 12:03, David Vrabel wrote: > > [ I forgot to make notes so these are from memory, please respond with > > any corrections or omissions. ] > > = > > Felipe introduced the session, highlighting the change in storage (i.e., > > low latency SSDs and fast SANs) were exposing bottlenecks in the current > > architecture which is designed with slow disks. Refer to his > > presentation from Friday for more details. > > = > > Felipe noted that persistent grants were causing performance regressions > > when the backend did not support them and system where copy cost > map > > cost (e.g., when dom0 has few VCPUs). Roger agreed on restoring the > > zero-copy path in the frontends was a good idea. [He has now posted > > patches for this.] > > = > > Felipe mentioned that persistent grants were most beneficial when using > > user space backend. David pointed out that this is most likely caused > > by a poor implementation of the gntdev device. > > = > > Matt mentioned contention on the m2p override lock as causing > > performance problems and suggested making this a read/write lock. > > = > > David listed some of the key bottlenecks already identified and plans to > > resolve them without any protocol changes. > > = > > 1. Unmap TLB flushes can be eliminated if the mapping is not used. > > Experiments by XenServer suggest grant mapped pages by blkback are never > > accessed thus eliminating all TLB flushes. > > = > > 2. Grant table lock contention can be reduced by finer grained locked. > > e.g., by having buckets of map tracking structures and hashing > > domid+grant ref to a bucket. > > = > > 3. gntdev device does a double map/unmap (for userspace and kernel > > mapping) and does the kernel mapping a page at a time. Userspace > > mappings could be done at page fault time (in the expectation that > > userspace doesn't touch them) and the kernel side should batch grant > > table ops using a new GNTOP_unmap_and_duplicate hypercall sub-op for the > > unmap. Roger said he'd posted patch for the sub-op, but received no > > feedback. > = > Here are the patches, the Linux side was reviewed by Stefano (if my > memory doesn't fail): > = > http://lists.xen.org/archives/html/xen-devel/2013-07/msg02825.html > https://lkml.org/lkml/2013/8/27/189 Could you repost them please based on the v3.12-rc7? Thanks!