Re: HVMlite ABI specification DRAFT A

["Roger Pau Monné" <roger.pau@citrix.com>]

El 5/2/16 a les 15:31, Jan Beulich ha escrit:
>>>> On 05.02.16 at 15:27, <roger.pau@citrix.com> wrote:
>> El 5/2/16 a les 14:22, Jan Beulich ha escrit:
>>>>>> On 05.02.16 at 12:50, <roger.pau@citrix.com> wrote:
>>>> El 5/2/16 a les 12:45, Jan Beulich ha escrit:
>>>>>>>> On 05.02.16 at 12:30, <roger.pau@citrix.com> wrote:
>>>>>> El 5/2/16 a les 11:40, Jan Beulich ha escrit:
>>>>>>>>>> On 05.02.16 at 10:50, <roger.pau@citrix.com> wrote:
>>>>>>>> For legacy PCI interrupts, we can parse the MADT inside of Xen in order
>>>>>>>> to properly setup the lines/overwrites and inject the interrupts that
>>>>>>>> are not handled by Xen straight into the hardware domain. This will
>>>>>>>> require us to be able to emulate the same topology as what is found in
>>>>>>>> native (eg: if there are two IO APICs in the hardware we should also
>>>>>>>> provide two emulated ones to the hw domain).
>>>>>>>
>>>>>>> I don't think MADT contains all the needed information, or else we
>>>>>>> wouldn't need PHYSDEVOP_setup_gsi.
>>>>>>
>>>>>> AFAICT, I think we could do something like:
>>>>>>
>>>>>>  - IRQs [0, 15]: edge-trigger, low-polarity.
>>>>>>  - IRQs [16, n]: level-triggered, high-polarity.
>>>>>
>>>>> That's not a valid assumption - I've seen systems with other settings
>>>>> on GSI >= 16 ...
>>>>
>>>> Then we just propagate how the emulated IO APIC pins are setup to the
>>>> real one, this should match reality, and is no different from using
>>>> PHYSDEVOP_setup_gsi. AFAICT it's just a different way of getting the
>>>> same information.
>>>
>>> That won't work either I'm afraid: For one, Dom0 may not even write
>>> RTEs for interrupts it never enables. And even if it did, it would write
>>> them masked, yet we mustn't derive information from masked RTEs -
>>> see commit 669d4b85c4 ("x86/IO-APIC: don't create pIRQ mapping
>>> from masked RTE").
>>
>> In which case, why does Xen need to setup this interrupt/RTE if it's
>> never used by Dom0?
> 
> Because (a) Xen itself may use it and (b) it may be used by a guest
> (for example, Dom0 may not have a driver for a device, causing its
> interrupt to not get enabled, but a guest handed the device would
> very likely then also know how to deal with it).

I would say that in case (a) Xen will have to use polling forever (I
guess our current approach was to switch to an interrupt driven model
once the IRQ was setup).

For (b) I'm quite sure we could force pciback (or whichever driver in
Dom0 gets the device assigned) to perform the IRQ configuration, even if
the device itself is not going to be used.

>>> Also consider e.g. the device IRQ which the
>>> serial driver may be using: We specifically suppress modifications to
>>> RTEs for in-use IRQs in current code and would of course need to
>>> do so in the PVHv2 code too. That way there would be no proper
>>> way to establish the two bits (short of grabbing the data from what
>>> Dom0 tries to write despite us otherwise suppressing the write).
>>
>> For devices in use by Xen itself, like the uart, doesn't Xen already
>> take care of setting the right interrupt configuration? Or else how does
>> the uart work before Dom0 is launched?
> 
> In polling mode.

I guess this is not very common, since most uarts use a GSI < 16. In
which case, couldn't the ones that use a GSI >= 16 just be used in
polling mode _forever_?

>> The plan was to use the STAO ACPI table in order to notify Dom0 that
>> certain devices (like the uart) are not accessible, thus preventing Dom0
>> from setting any interrupts for this devices at all (ie: they should
>> just be ignored/skipped by Dom0 when doing device enumeration).
>>
>> And in any case, writes to pins that are in use by Xen should not be
>> propagated to the physical IO APIC at all, since I would assume Xen has
>> already set them up properly.
> 
> Once again - it can't without Dom0's help if the interrupt isn't in
> the legacy GSI range (below 16).

Which devices is Xen expected to use with a GSI >= 16? I can only think
of the uart, but maybe there are others which I'm missing?

Roger.