From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= Subject: Re: Xen Developer Summit Storage Performance BoF notes Date: Wed, 30 Oct 2013 12:33:04 +0100 Message-ID: <5270EE70.5080300@citrix.com> References: <5270E79A.1050703@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <5270E79A.1050703@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: David Vrabel , "Xen-devel@lists.xen.org" List-Id: xen-devel@lists.xenproject.org On 30/10/13 12:03, David Vrabel wrote: > [ I forgot to make notes so these are from memory, please respond with > any corrections or omissions. ] > > Felipe introduced the session, highlighting the change in storage (i.e., > low latency SSDs and fast SANs) were exposing bottlenecks in the current > architecture which is designed with slow disks. Refer to his > presentation from Friday for more details. > > Felipe noted that persistent grants were causing performance regressions > when the backend did not support them and system where copy cost > map > cost (e.g., when dom0 has few VCPUs). Roger agreed on restoring the > zero-copy path in the frontends was a good idea. [He has now posted > patches for this.] > > Felipe mentioned that persistent grants were most beneficial when using > user space backend. David pointed out that this is most likely caused > by a poor implementation of the gntdev device. > > Matt mentioned contention on the m2p override lock as causing > performance problems and suggested making this a read/write lock. > > David listed some of the key bottlenecks already identified and plans to > resolve them without any protocol changes. > > 1. Unmap TLB flushes can be eliminated if the mapping is not used. > Experiments by XenServer suggest grant mapped pages by blkback are never > accessed thus eliminating all TLB flushes. > > 2. Grant table lock contention can be reduced by finer grained locked. > e.g., by having buckets of map tracking structures and hashing > domid+grant ref to a bucket. > > 3. gntdev device does a double map/unmap (for userspace and kernel > mapping) and does the kernel mapping a page at a time. Userspace > mappings could be done at page fault time (in the expectation that > userspace doesn't touch them) and the kernel side should batch grant > table ops using a new GNTOP_unmap_and_duplicate hypercall sub-op for the > unmap. Roger said he'd posted patch for the sub-op, but received no > feedback. Here are the patches, the Linux side was reviewed by Stefano (if my memory doesn't fail): http://lists.xen.org/archives/html/xen-devel/2013-07/msg02825.html https://lkml.org/lkml/2013/8/27/189