Re: [PATCH v2] KVM: s390: Improve floating IRQ injection behavior

[Christian Borntraeger <borntraeger@linux.ibm.com>]

Am 19.08.26 um 14:54 schrieb Janosch Frank:
> On 8/18/26 6:58 PM, Halil Pasic wrote:
>> On Tue, 18 Aug 2026 17:14:52 +0200
>> Christian Borntraeger <borntraeger@linux.ibm.com> wrote:
>>
>>> Am 18.08.26 um 16:32 schrieb Halil Pasic:
>>>> On Mon, 17 Aug 2026 15:22:41 +0200
>>>> Christian Borntraeger <borntraeger@linux.ibm.com> wrote:
>>>>>>> +    irq_pend_mask = inti_to_irq_pend_mask(inti);
>>>>>>>         for (sigcpu = kvm->arch.float_int.last_sleep_cpu; ; sigcpu++) {
>>>>>>>             sigcpu %= online_vcpus;
>>>>>>>             dst_vcpu = kvm_get_vcpu(kvm, sigcpu);
>>>>>>> -        if (!is_vcpu_stopped(dst_vcpu))
>>>>>>> +        if (!is_vcpu_stopped(dst_vcpu) &&
>>>>>>> +            deliverable_irqs(dst_vcpu) & irq_pend_mask)
>>>>>>>                 break;
>>>>>>>             /* avoid endless loops if all vcpus are stopped */
>>>>>>>             if (nr_tries++ >= online_vcpus)
>>>>>>>                 return;
>>>>>>
>>>>>> [Severity: High]
>>>>>> Does this code drop the interrupt kick entirely if all vCPUs currently
>>>>>> have their interrupt masks closed?
>>>>>
>>>>> I think this is a corner case but still a valid finding. We can probably consider this
>>>>> slowpath and wakeup/set cpuflags for ALL cpus? maybe after doing 2 rounds instead of one?
>>>>
>>>> With GISA, I think the FW is supposed to deliver the floating interrupts
>>>> without dropping the vCPU out of SIE. I'm not 100% sure but I think we
>>>> can rely on that mechanism for the CPUs that are in SIE. Without GISA,
>>>> I think, it is reasonable to assume that vCPUs don't keep running for
>>>> ever. It has been a while since I have looked at this code, but I think
>>>> the SIE exit path would catch this. If that is true we would not really
>>>> lose initiative, but just see delayed interrupt delivery.
>>>>
>>>> Sleeping vCPUs on the other hand are not of interest in this context, I
>>>> think.
>>> This is all corner case handling. Imagine one CPU running with IO disabled
>>> and all other CPUs sleeping. If now the "opportunistic" wakeup fails the
>>> GISA IO interrupt will never be delivered unless there is another wakeup.
>>> In reality this is a will not happen, but see the latest kvm unit test
>>> patch from Janosch and it might also create latencies, the "pick one CPU
>>> to deliver and wake it up if normal processing does not work" obviously
>>> has a hole in specific cases.
>>
>> Right, but that is not the "if all vCPUs currently have their interrupt
>> masks closed" case that Sashiko is talking about. Or did I misunderstand
>> that?
>>
>> Yes, I agree there are holes, and I was hoping to contribute to a better
>> understanding on where the holes actually are and what are the
>> implications of those holes.
>>
> Yesterday I managed to find the actual problem behind this code for PV guests:

Thanks for digging through that.> 
> Non-ev Service IRQs are not allowed to be injected on re-entry when SCLP emulation has finished. They can only be injected when we receive the instruction notification for SCLP.
> 
> So having wakeups for service IRQs in the flic is useless. At the time they are injected (after SCLP processing and before SIE re-entry) there's no way to make one pending for a PV cpu.
> 
> I've since created a patch to kick cpus in the handling of the sclp instruction notification and that "fixes" the problems with the firq test.
> 
> 
> 
> For Linux we'll see delayed delivery at most since the masks are open most of the time. My guess is that the IRQ injection actually happens pretty fast. There are other OSs which might benefit though and we should generally ask ourselves if good enough for Linux is the right answer.
> 
> The stat counter did actually trigger a couple hundred times on an IPL so we're hitting the redistribution code path. Let's see what insights we can get from the performance measurements.

there are also ideas about disabling IO interrupts on some CPUs (e.g. vertical low)
and other OSes like zOS have the concept of CPUs excplictely disabled for IO
interrupts. So we better have a least a KVM implementation that works in those
situations.