Re: [Linux PPC] Disable PREEMPT

[Shrikanth Hegde <sshegde@linux.ibm.com>]

Hi Christian, Michal,


>>>>>>
>>>>>> I see it is mostly busy with printing
>>>>>> WARNING: at .enable_kernel_fp+0x30/0x78, CPU#3: qemu-system-ppc/4884
>>>>>>
>>>>>> The path seems to be same also.
>>>>>>
>>>>>> [c000000085ca7700] [c00000000005de84] .kvmppc_load_guest_fp+0x30/0x80
>>>>>> [c000000085ca7780] [c00000000005f2a0] .kvmppc_handle_exit+0x5bc/0x5cc
>>>>>> [c000000085ca7830] [c00000000006204c] .kvmppc_resume_host+0xb8/0x10c
>>>>>>
>>>>>>
>>>>>> Does below help to boot or make it stable?
>>>>>> ---
>>>>>>  arch/powerpc/kvm/booke.c | 9 ++++++++-
>>>>>>  1 file changed, 8 insertions(+), 1 deletion(-)
>>>>>>
>>>>>> diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c
>>>>>> index 13ad4cf5fa71..d4be7a306bfc 100644
>>>>>> --- a/arch/powerpc/kvm/booke.c
>>>>>> +++ b/arch/powerpc/kvm/booke.c
>>>>>> @@ -1404,10 +1404,17 @@ int kvmppc_handle_exit(struct kvm_vcpu *vcpu, unsigned int exit_nr)
>>>>>>          if (s <= 0)
>>>>>>              r = (s << 2) | RESUME_HOST | (r & RESUME_FLAG_NV);
>>>>>>          else {
>>>>>> -            /* interrupts now hard-disabled */
>>>>>> +            /*
>>>>>> +             * interrupts now hard-disabled. i.e EE=0
>>>>>> +             * But irqs_disabled relies on PACA entries
>>>>>> +             * and fixup below enables it.
>>>>>> +             * So irqs_disabled == false
>>>>>> +             */
>>>>>>              kvmppc_fix_ee_before_entry();
>>>>>> +            preempt_disable();
>>>>>>              kvmppc_load_guest_fp(vcpu);
>>>>>>              kvmppc_load_guest_altivec(vcpu);
>>>>>> +            preempt_enable_no_resched();
>>>>>>          }
>>>>>>      }
>>>>>>
>>>>>>
>>>>> Hi,
>>>>>
>>>>> Thanks a lot for your patch!
>>>>>
>>>>> I patched the 7.3-rc4 kernel sources with your new patch and compiled the kernel again yesterday. [1]
>>>>>
>>>>
>>>> Thanks for trying.
>>>>
>>>>> Unfortunately, the issue still exists.
>>>>>
>>>>
>>>> Can you share the console log of the kernel with the patch applied?
>>>> We need to see where it is stuck now. Has it made forward progress?
>>>>
>>>>> I tested it with a guest kernel with and without preempt support.
>>>>>
>>>>> Cheers,
>>>>> Christian
>>>>>
>>>>> [1] https://github.com/chzigotzky/kernels/releases/tag/v7.3.0-rc4-2-preempt
>>>>>
>>>>
>>> Here is a new serial log from the patched kernel: https://github.com/user-attachments/files/32570808/e5500_preempt_pcie_ports_compat_patched_putty.log
>>>
>>
>> That paca mismatch is independent of preemption modes.
>>
>> It shows a few warnings and even after that a console message has come through.
>> Do warning continue forever?
>>
>> To fix those warnings, can you try this diff on the top of the earlier ones?
>> (Igonore whitespace mangling if any, Also patch is UNTESTED. It is more of my
>> speculation.).
>>
>> Sorry, I don't have any way to test it out locally.
>>
>> ---
>>
>> diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
>> index 9194cf492d1c..0a014fa45b20 100644
>> --- a/arch/powerpc/kvm/powerpc.c
>> +++ b/arch/powerpc/kvm/powerpc.c
>> @@ -147,6 +147,19 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu)
>>                         continue;
>>                 }
>>
>> +#ifdef CONFIG_PPC64
>> +               if (unlikely(local_paca->irq_happened != PACA_IRQ_HARD_DIS)) {
>> +               /*
>> +                * An interrupt arrived between the soft and hard
>> +                * disable. Replay it and retry guest entry.
>> +                */
>> +                       local_irq_enable();
>> +                       local_irq_disable();
>> +                       hard_irq_disable();
>> +                       continue;
>> +               }
>> +#endif
>> +
>>                 guest_enter_irqoff();
>>                 return 1;
>>         }
>>
> 
> Hi,
> 
> I patched the RC4 of kernel 7.3 with the two patches and compiled it again today.
> 
> I'm happy to report that the patched PREEMPT kernel with KVM HV works on the X5000/40. (e5500)
> 

That's very nice to hear. I will write changelog for both and send it across soon.
Give me sometime.

> Serial log: https://github.com/user-attachments/files/32619259/e5500_preempt_pcie_ports_compat_patched-2_putty.log
> 
> I also successfully tested it with KVM PR on the X1000 today. (PA6T)
> 

Nice.

> Further information: https://github.com/chzigotzky/kernels/issues/19#issuecomment-5818925408
> 
> Many thanks for your help,
> 

Happy to help.
Thanks for reporting and testing. We can finally close this thread :)
Thanks to michal for helping in getting the logs.

> Christian
> 

Also, Michal, can you report that kvm specific issue you are seeing with the
ftrace fix applied in separate thread. This thread has become long already.