From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= Subject: Re: Create a iSCSI DomU with disks in another DomU running on the same Dom0 Date: Fri, 11 Jan 2013 20:29:12 +0100 Message-ID: <50F06808.40503@citrix.com> References: <50D41DF3.306@citrix.com> <20121221140320.GD25526@phenom.dumpdata.com> <50D47678.2050903@citrix.com> <20121221173513.GB27893@phenom.dumpdata.com> <50E430B0.3070605@citrix.com> <20130102213621.GA15765@phenom.dumpdata.com> <50EDC3C1.3010100@citrix.com> <20130111150655.GB15353@phenom.dumpdata.com> <50F03680.3020800@citrix.com> <20130111185153.GA29020@phenom.dumpdata.com> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <20130111185153.GA29020@phenom.dumpdata.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Konrad Rzeszutek Wilk Cc: xen-devel List-Id: xen-devel@lists.xenproject.org On 11/01/13 19:51, Konrad Rzeszutek Wilk wrote: > On Fri, Jan 11, 2013 at 04:57:52PM +0100, Roger Pau Monn=E9 wrote: >> Hello Konrad, >> >> I've found the problem, blkback is adding granted pages to the bio that = >> is then passed to the underlying block device. When using a iscsi = >> target running on another DomU in the same h/w this bios end up in = >> netback, and then when performing the gnttab copy operation, it = >> complains because the passed mfn belongs to a different domain. > = > OK, so my original theory was sound. The m2p override "sticks". >> >> I've checked this by applying the appended patch to blkback, which = >> allocates a buffer to pass to the bio instead of using the granted = >> page. Of course this should not applied, since it implies additional = >> memcpys. >> >> I think the right way to solve this would be to change netback to = >> use gnttab_map and memcpy instead of gnttab_copy, but I guess this = >> will imply a performance degradation (haven't benchmarked it, but I = >> assume gnttab_copy is used in netback because it is faster than = >> gnttab_map + memcpy + gnttab_unmap). > = > Or blkback is altered to use grant_copy. This would not work with the persistent-grants extension, and also when scaling to a large number of guests will probably have a degraded performance due to the grant table lock (compared to using persistent grants). > Or perhaps m2p_override > can do multiple PAGE_FOREIGN? (So if it detects a collision it will > do something smart.. like allocate a new page or update the = > kmap_op with extra information). What we could do is add extra information to m2p_override, containing the grant_ref_t and domid, so when a FOREIGN_FRAME is detected in grant_copy (or netback) the grant_ref_t and domid of the passed mfn is used instead of the mfn (provided that grant_copy can perform a copy between two grant references of different domains). > = > = > And yes, grant_map in netback is much much slower that grant_copy > (I tested 2.6.32 vs 3.7 using a Xen 4.1.3 with the grant_copy fixes > that Jan came up with). Yes, I see there's no way we are going to use grant_map instead of grant_copy. I guess this will no longer be true once netback/front starts using the persistent grants extension.