Re: Rebooting domu fails in nfs share exported from another domu on the same dom0

[annie li <annie.li@oracle.com>]

On 2014/7/18 17:07, Konrad Rzeszutek Wilk wrote:
> On Fri, Jul 18, 2014 at 04:31:13PM -0400, annie li wrote:
>> On 2014/7/18 16:22, Konrad Rzeszutek Wilk wrote:
>>> On Fri, Jul 18, 2014 at 04:17:02PM -0400, annie li wrote:
>>>> On 2014/7/18 15:43, Konrad Rzeszutek Wilk wrote:
>>>>> On Fri, Jul 18, 2014 at 03:31:43PM -0400, annie li wrote:
>>>>>> On 2014/7/18 14:53, Konrad Rzeszutek Wilk wrote:
>>>>>>> On Thu, Jul 17, 2014 at 12:56:12PM -0400, annie li wrote:
>>>>>>>> On 2014/7/17 11:49, Roger Pau Monné wrote:
>>>>>>>>> On 16/07/14 22:36, annie li wrote:
>>>>>>>>>> Hi
>>>>>>>>>>
>>>>>>>>>> I hit a problem in such scenario: vm1 is running and export nfs service,
>>>>>>>>>> dom0 mount this nfs, and vm2 is booted in this nfs location. vm1 and vm2
>>>>>>>>>> are running on the same dom0.
>>>>>>>>>>
>>>>>>>>>> When this bug happens, the data flow is:  vm2 blkfront-> vm2 blkback->
>>>>>>> I am a bit confused here. 'vm2 blkfront -> vm2 blkback'? Did you
>>>>>>> mean 'dom0 blkback'?
>>>>>> Yes, Dom0 blkback.
>>>>>>
>>>>>>>>>> loop -> nfs file -> nfs client -> bridge priv1 -> vm1 vif -> vm1 netback
>>>>>>>>>> -> vm1 netfront.
>>>>>>> So both netback and netfront run in the same guest? I think you
>>>>>>> want these two swapped around (netfront -> netback).
>>>>>> No, the netfront in above routine means the one in guest vm1, and this is
>>>>>> network RX path in vm1.
>>>>> OK, so 'dom0 netback' then. As the netback thread is running in the
>>>>> initial domain?
>>>> Netback thread is running in Dom0 as normal, and the two vms are all
>>>> normal vm, not driver domain. Probably my initial description causes
>>>> some confusion.
>>>>
>>>>> So a revised view is:
>>>>>
>>>>>    vm2 blkfront -> dom0 blkback -> loop -> nfs file -> nfs client
>>>>> 	-> bridge priv1 -> vm1 vif -> dom0 netback -> vm1 netfront
>>>>>
>>>>> So with the grant map (blkfront -> blkback) the source is dom0
>>>>> and the destination is vm1.
>>>> You mean destination is vm2 here, right?
>>> Yes :-)
>>>> But I am little confused. Blkfront of vm2 sends out request to
>>>> blkback, and the data page is allocated from vm2. So the grant map
>>>> source is vm2, destination is dom0. Anything I missed here?
>>>>
>>>>>   For the grant copy, the source is
>>>>> dom0 and the destinatation is vm2 right? (or did I get my src
>>>>> and dst confused?).
>>>> You mean vm1 here, right?
>>> Heh.
>>>> If so, yes, source is dom0, and destination is vm1.
>>>> In grantcopy code, it checks whether the source is from dom0, if
>>>> not, then an error is thrown out. In this situation, the source is
>>> >from vm2 which is grant mapped.
>>> In both cases dom0 is part of the equation. In one
>>> case it is the destination and in another it is the source - both cases
>>> for the same page right?
>> Yes.
>> blkfront->blkback, dom0 is destination
>> netback->netfront, dom0 is the source
>>
>> When doing grantcopy for netback, xen requires the source is dom0. However,
>> it is vm2 in this case.
> Would it be possible in the hypervisor to have an extra check where you
> look to see if the source (dom0 in this case) has this page mapped
> from another guest? As in, this is a bit of A->B->C transition
> and we want to do A->C (B is dom0). If you figure out that the PFN
> belongs to A and you are doing a copy of A's page to C's page from B (dom0)
> page (and B PFN is actually mapped to be A's page), then why
> not just copy from A to C directly?
>
> Hmm. Linux kernel could actually also have this information (it does
> already I think) and we could search for that if the grant copy fails
> and if so alter the source and retry the grant copy. Instead of dom0
> being the source it is the C guest (vm2)? Thought I don't know if
> the hypercall allows us to make a grant_copy on behalf of two
> other guests.

Oh... I remember netback did have some grant_copy code on behalf of two 
guests on same server, and it was removed later on. So I think grantcopy 
A->C works for this case with the precondition that we can recognize 
this page is mapped from another guest.

Thanks
Annie
>
>> Thanks
>> Annie
>>
>>>> Thanks
>>>> Annie
>>>>> _______________________________________________
>>>>> Xen-devel mailing list
>>>>> Xen-devel@lists.xen.org
>>>>> http://lists.xen.org/xen-devel
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel