Re: Dom0 hypercall for adding and removing PCI devices

[Keir Fraser <keir.fraser@eu.citrix.com>]

If a device is assigned to a domain (in this case dom0) then that domain's
VT-d pagetables will contain the RMRR mappings for that device. Hence BIOS
can perform DMA in those RMRR-indicated regions without swapping to and fro
between domain tables and fallback RMRR tables. The new fallback RMRR table
would be just that -- a fallback table used for any device not currently
assigned to any domain (and hence those devices should only have DMAs
initiated by the BIOS, if at all).

 -- Keir

On 24/7/08 09:20, "Han, Weidong" <weidong.han@intel.com> wrote:

> I have another concern, when BIOS is initiating DMA operation using
> RMRR, can we use RMRR VT-d page table to replace dom0 VT-d page table?
> Does it result in some DMA failures?
> 
> Randy (weidong)
> 
> Han, Weidong wrote:
>> Espen Skoglund wrote:
>>> [Keir Fraser]
>>>> On 23/7/08 10:26, "Han, Weidong" <weidong.han@intel.com> wrote:
>>>>>> So this would be one extra VT-d pagetable, for the whole system,
>>>>>> which would be the fallback location for RMRR mappings for devices
>>>>>> which are currently not assigned to any domain? Thus allowing
>>>>>> firmware to successfully initiate DMA operations on those devices?
>>>>>> Sounds sensible.
>>> 
>>> Well, the VT-d page table for RMRR devices need not contain the whole
>>> system memory---only those regions specified in the DMAR tables.
>>> 
>>>>> Is it possible that idle_domain owns the RMRR VT-d page table?
>>> 
>>>> If that's a convenient place to stash it then why not? Either way,
>>>> seems you're going to have it special-cased in the code as fallback
>>>> owner for unassigned devices. It's possible that having it stashed
>>>> in the idle domain will simply make the code more confusing. I'm not
>>>> sure though.
>>> 
>>> Right.  I don't see any particular good reason to associate it with
>>> the idle domain.  As noted above, the page table need not even cover
>>> the whole memory, and it will never change after being set up at boot
>>> time.  If special case code is needed anyway, then one might as well
>>> install a custom VT-d page table.
>> 
>> What does "custom VT-d page table" mean?
>> 
>> Due to domain id is not the same with DID field in context, we can
>> reverse a DID for RMRR VT-d page table, it can avoid to associate
>> with idle domain.
>> 
>> Currently we reassign the device from dom0 to target domain when
>> assign a device, and return the device to dom0 when target domain
>> tears down. It's not correct due to some devices may be not assigned
>> to any domain. Current device_assigned() also needs to be changed.
>> Maybe it needs more changes on VT-d.
>> 
>> I have some concerns, maybe I missed something. Why did you use dom0
>> hypercall approach to replace original method? What's the main
>> benefit? I also wonder it's suitable to wrap pci_bus_probe()
>> function.   
>> 
>> Randy (Weidong)
>> 
>>> 
>>> If supported by hardware, the extra page tables can even be skipped
>>> altogether and the device marked as having passthrough access.  That
>>> would give the RMRR device complete access to system memory, though.
>>> 
>>> eSk
>