Re: Create a iSCSI DomU with disks in another DomU running on the same Dom0

["Roger Pau Monné" <roger.pau@citrix.com>]

On 11/01/13 19:51, Konrad Rzeszutek Wilk wrote:
> On Fri, Jan 11, 2013 at 04:57:52PM +0100, Roger Pau Monné wrote:
>> Hello Konrad,
>>
>> I've found the problem, blkback is adding granted pages to the bio that 
>> is then passed to the underlying block device. When using a iscsi 
>> target running on another DomU in the same h/w this bios end up in 
>> netback, and then when performing the gnttab copy operation, it 
>> complains because the passed mfn belongs to a different domain.
> 
> OK, so my original theory was sound. The m2p override "sticks".
>>
>> I've checked this by applying the appended patch to blkback, which 
>> allocates a buffer to pass to the bio instead of using the granted 
>> page. Of course this should not applied, since it implies additional 
>> memcpys.
>>
>> I think the right way to solve this would be to change netback to 
>> use gnttab_map and memcpy instead of gnttab_copy, but I guess this 
>> will imply a performance degradation (haven't benchmarked it, but I 
>> assume gnttab_copy is used in netback because it is faster than 
>> gnttab_map + memcpy + gnttab_unmap).
> 
> Or blkback is altered to use grant_copy.

This would not work with the persistent-grants extension, and also when
scaling to a large number of guests will probably have a degraded
performance due to the grant table lock (compared to using persistent
grants).

> Or perhaps m2p_override
> can do multiple PAGE_FOREIGN? (So if it detects a collision it will
> do something smart.. like allocate a new page or update the 
> kmap_op with extra information).

What we could do is add extra information to m2p_override, containing
the grant_ref_t and domid, so when a FOREIGN_FRAME is detected in
grant_copy (or netback) the grant_ref_t and domid of the passed mfn is
used instead of the mfn (provided that grant_copy can perform a copy
between two grant references of different domains).

> 
> 
> And yes, grant_map in netback is much much slower that grant_copy
> (I tested 2.6.32 vs 3.7 using a Xen 4.1.3 with the grant_copy fixes
> that Jan came up with).

Yes, I see there's no way we are going to use grant_map instead of
grant_copy. I guess this will no longer be true once netback/front
starts using the persistent grants extension.