Re: Assigning contiguous memory to a driver domain

[Jeremy Fitzhardinge <jeremy@goop.org>]

 On 09/20/2010 04:42 PM, Dan Magenheimer wrote:
>>  On 09/20/2010 02:41 PM, Konrad Rzeszutek Wilk wrote:
>>> On Mon, Sep 20, 2010 at 01:27:24PM -0700, Jeremy Fitzhardinge wrote:
>>>>  On 09/20/2010 12:48 PM, Konrad Rzeszutek Wilk wrote:
>>>>> Let me expand this. During bootup Xen-SWIOTLB  (which for DomU you
>> have
>>>>> to enable via the 'iommu=soft'), allocated 32 2MB chunks of
>> contingous
>>>>> memory under the 4GB limit. Those chunks stay in DomU and are used
>>>>> during the the runtime of the DomU. They don't go back to Xen
>> unless the
>>>>> domain has been terminated. Any of the DMA operations that any
>> driver
>>>>> does go through the SWIOTLB bufer if the physical (mfn) for the DMA
>> is
>>>>> outside the driver capabilities (say, your ping buffer is allocated
>> above
>>>>> the 4GB, and your r8169 can only do 32-bit, then SWIOTLB would be
>> utilized
>>>>> to "bounce" the memory).
>>>> BTW, are there any hooks to make sure these pages are still
>> contiguous
>>>> after migration/restore?
>>> <shudders>I don't really know. We do save the P2M map, so those
>> mappings
>>> are saved in the guest. But for the MFNs that have been exchanged - I
>> don't
>>> believe the hypervior is notified about which MFNs have been
>> transfered
>>> to another domain? Or these MFNs extracted during save/resume
>> On save the toolstack will rewrite all the mfns in the pagetables and
>> the p2m map to canonical pfn form, then rewrite them back to mfns on
>> resume when it has allocated pages for them all.  I don't believe
>> there's any mechanism to remember what regions were mfn-contiguous and
>> restore that on resume.
> I may be too far behind on this thread, but aren't we only talking about
> driver domains with passthrough?   Is it even possible to migrate
> driver domains yet?

Not with devices attached.  But there should be no problem in principle
with detaching all the devices, migrating, the reattaching everything.

    J