Re: PCI-passthrough for 32 bit guests and high MMIO addresses

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 21/11/14 15:04, Juergen Gross wrote:
> On 11/21/2014 03:45 PM, Andrew Cooper wrote:
>> On 21/11/14 14:39, Juergen Gross wrote:
>>> Hi,
>>>
>>> again a fallout from my "linear p2m list" tests:
>>>
>>> Trying to do PCI-passthrough with a 32-bit pv-domain I passed the
>>> wrong device to the domain. The MMIO address was too large for a
>>> MFN of a 32-bit system (it was 380003200000-3800036fffff).
>>>
>>> Instead of rejecting the operation Xen tried to perform it resulting
>>> in a (quite understandable) failure in the domU.
>>>
>>> I think either the hypervisor or the tools should refuse to do
>>> PCI-passthrough in this case.
>>>
>>
>> What actually goes wrong?  Does the 32bit guest truncate the MMIO region
>> to 44 bits and try to map that?
>
> I think it is 42 bits (the top 2 bits of the MFN are the "identity" and
> "foreign" bits).

This is a very grey area in desperate need of some clarification.

There is nothing (I am aware of) in the API or ABI, not even an
INVALID_PFN/INVALID_MFN constant.  ~0UL is independently defined in
several locations, but I believe that only Linux has the notion of a
foreign bit in the p2m; Xen and the Toolstack don't appear to have this
concept.

When developing migration v2, there was quite a lot of digging through
history to find out what was safe.  The top bit being set in the guests
p2m used to mean "this is an skb frame and can safely be left behind on
migrate"  (This change was subsequently backed out).  This was back in
the days when max_pfn was 0x1000, and there were plenty of "free" bits
in the top of a 32bit unsigned long.

None of the interfaces like this have been re-evaluated as machines have
become larger.

~Andrew