From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: Create a iSCSI DomU with disks in another DomU running on the same Dom0 Date: Fri, 11 Jan 2013 16:09:35 -0500 Message-ID: <20130111210935.GA30742@phenom.dumpdata.com> References: <20121221140320.GD25526@phenom.dumpdata.com> <50D47678.2050903@citrix.com> <20121221173513.GB27893@phenom.dumpdata.com> <50E430B0.3070605@citrix.com> <20130102213621.GA15765@phenom.dumpdata.com> <50EDC3C1.3010100@citrix.com> <20130111150655.GB15353@phenom.dumpdata.com> <50F03680.3020800@citrix.com> <20130111185153.GA29020@phenom.dumpdata.com> <50F06808.40503@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Return-path: Content-Disposition: inline In-Reply-To: <50F06808.40503@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Roger Pau =?iso-8859-1?Q?Monn=E9?= Cc: xen-devel List-Id: xen-devel@lists.xenproject.org On Fri, Jan 11, 2013 at 08:29:12PM +0100, Roger Pau Monn=E9 wrote: > On 11/01/13 19:51, Konrad Rzeszutek Wilk wrote: > > On Fri, Jan 11, 2013 at 04:57:52PM +0100, Roger Pau Monn=E9 wrote: > >> Hello Konrad, > >> > >> I've found the problem, blkback is adding granted pages to the bio tha= t = > >> is then passed to the underlying block device. When using a iscsi = > >> target running on another DomU in the same h/w this bios end up in = > >> netback, and then when performing the gnttab copy operation, it = > >> complains because the passed mfn belongs to a different domain. > > = > > OK, so my original theory was sound. The m2p override "sticks". > >> > >> I've checked this by applying the appended patch to blkback, which = > >> allocates a buffer to pass to the bio instead of using the granted = > >> page. Of course this should not applied, since it implies additional = > >> memcpys. > >> > >> I think the right way to solve this would be to change netback to = > >> use gnttab_map and memcpy instead of gnttab_copy, but I guess this = > >> will imply a performance degradation (haven't benchmarked it, but I = > >> assume gnttab_copy is used in netback because it is faster than = > >> gnttab_map + memcpy + gnttab_unmap). > > = > > Or blkback is altered to use grant_copy. > = > This would not work with the persistent-grants extension, and also when > scaling to a large number of guests will probably have a degraded > performance due to the grant table lock (compared to using persistent > grants). > = > > Or perhaps m2p_override > > can do multiple PAGE_FOREIGN? (So if it detects a collision it will > > do something smart.. like allocate a new page or update the = > > kmap_op with extra information). > = > What we could do is add extra information to m2p_override, containing > the grant_ref_t and domid, so when a FOREIGN_FRAME is detected in > grant_copy (or netback) the grant_ref_t and domid of the passed mfn is > used instead of the mfn (provided that grant_copy can perform a copy > between two grant references of different domains). > = > > = > > = > > And yes, grant_map in netback is much much slower that grant_copy > > (I tested 2.6.32 vs 3.7 using a Xen 4.1.3 with the grant_copy fixes > > that Jan came up with). > = > Yes, I see there's no way we are going to use grant_map instead of > grant_copy. I guess this will no longer be true once netback/front > starts using the persistent grants extension. Hm? Annie posted patches for the persistent grants on netback/netfront and they did not show much improvement (as you are already doing grant_copy= ). Or are you saying change netback to use grant_map and utilize the skb->deconstructor to keep track of it? And then do persistent grant extensions on it?