Re: [RFC PATCH] hw/arm/virt: Support NMI injection

[Auger Eric <eric.auger@redhat.com>]

Hi Marc,
On 1/28/20 10:25 AM, Marc Zyngier wrote:
> Gavin, Eric,
> 
> On 2020-01-28 08:05, Auger Eric wrote:
>> Hi,
>>
>> On 1/28/20 7:48 AM, Gavin Shan wrote:
>>> [including more folks into the discussion]
>>>
>>>> On Fri, 17 Jan 2020 at 14:00, Peter Maydell <peter.maydell@linaro.org>
>>>> wrote:
>>>>> On Thu, 19 Dec 2019 at 04:06, Gavin Shan <gshan@redhat.com> wrote:
>>>>>> This supports NMI injection for virtual machine and currently it's
>>>>>> only
>>>>>> supported on GICv3 controller, which is emulated by qemu or host
>>>>>> kernel.
>>>>>> The design is highlighted as below:
>>>>>>
>>>>>> * The NMI is identified by its priority (0x20). In the guest (linux)
>>>>>> kernel, the GICC_PMR is set to 0x80, to block all interrupts except
>>>>>> the NMIs when the external interrupt is disabled. It means the FIQ
>>>>>> and IRQ bit in PSTATE isn't touched when the functionality (NMI) is
>>>>>> functional.
>>>>>> * LPIs aren't considered as NMIs because of their nature. It means
>>>>>> NMI
>>>>>> is either SPI or PPI. Besides, the NMIs are injected in round-robin
>>>>>> fashion is there are multiple NMIs existing.
>>>>>> * When the GICv3 controller is emulated by qemu, the interrupt states
>>>>>> (e.g. enabled, priority) is fetched from the corresponding data
>>>>>> struct
>>>>>> directly. However, we have to pause all CPUs to fetch the interrupt
>>>>>> states from host in advance if the GICv3 controller is emulated by
>>>>>> host.
>>>>>>
>>>>>> The testing scenario is to tweak guest (linux) kernel where the
>>>>>> pl011 SPI
>>>>>> can be enabled as NMI by request_nmi(). Check "/proc/interrupts"
>>>>>> after injecting
>>>>>> several NMIs, to see if the interrupt count is increased or not. The
>>>>>> result
>>>>>> is just as expected.
>>>>>>
>>>>
>>>> So, QEMU is trying to emulate actual hardware. None of this
>>>> looks to me like what GICv3 hardware does... If you want to
>>>> have the virt board send an interrupt, do it the usual way
>>>> by wiring up a qemu_irq from some device to the GIC, please.
>>>> (More generally, there is no concept of an "NMI" in the GIC;
>>>> there are just interrupts at varying possible guest-programmable
>>>> priority levels.)
>>>>
>>>
>>> Peter, I missed to read your reply in time and apologies for late
>>> response.
>>>
>>> Yes, there is no concept of "NMI" in the GIC from hardware perspective.
>>> However, NMI has been supported from the software by kernel commit
>>> bc3c03ccb4641 ("arm64: Enable the support of pseudo-NMIs"). The NMIs
>>> have higher priority than normal ones. NMIs are deliverable after
>>> local_irq_disable() because the SYS_ICC_PMR_EL1 is tweaked so that
>>> normal interrupts are masked only.
> 
> And none of that is an NMI. This is a completely SW-defined mechanism,
> and you can't rely on this to inject something that would behave as
> a NMI in in a guest. I thought the "pseudo" prefix would give it away :-(.
> 
>>>
>>> It's unclear about the purpose of "nmi" QMP/HMP command. It's why I
>>> put a RFC tag. The command has been supported by multiple architects
>>> including x86/ppc. However, they are having different behaviors. The
>>> system will be restarted on ppc with this command, but a NMI is injected
>>> through LAPIC on x86. So I'm not sure what architect (system reset on
>>> ppc or injecting NMI on x86) aarch64 should follow.
> 
> The x86 NMI has no equivalent on ARM, full stop. And the only thing that
> the ARM implementation should follow is the letter of the architecture,
> without added concepts.
> 
>> arm_pmu driver was reworked to use pseudo-NMIs. I don't know the exact
>> status of this work though
>> (https://patchwork.kernel.org/cover/11047407/). So we cannot use any
>> random NMI for guest injection.
>>
>> I wonder whether we should implement the KVM_NMI vcpu ioctl once we have
>> agreed on which behavior is expected upon NMI injection. However the
>> kernel doc says this ioctl only is well defined if "KVM_CREATE_IRQCHIP
>> has not been called" (?).
> 
> But what architectural concept would you map your KVM_NMI to? The number
> of things you can do is pretty limited:
> 
> - Reset: we already have this
> - Interrupt: you don't get to decide the priority or the group
> - SError: Pretty much fatal in all cases
> 
> You *could* try something like SDEI [1], but that's a pretty terrible
> interface too.

Thank you for the pointer.

So Gavin, not sure the QEMU QMP/HMP NMI command is relevant on ARM (at
least at this point)?

Thanks

Eric


> 
>         M.
> 
> [1]
> https://static.docs.arm.com/den0054/a/ARM_DEN0054A_Software_Delegated_Exception_Interface.pdf
>