[Linux PPC] KVM PR and KVM HV do not work if the kernel was compiled with PREEMPT

[Linux PPC] KVM PR and KVM HV do not work if the kernel was compiled with PREEMPT

[Christian Zigotzky]

Hello,

KVM PR and KVM HV do not work if the kernel was compiled with PREEMPT.

The entire FSL Cyrus+ board freezes when using KVM HV with PREEMPT.

The guest kernel doesn't boot if we use KVM PR with a PREEMPT kernel on 
the PA Semi Nemo board.

We were previously able to disable PREEMPT in the kernel configuration, 
but the latest git kernels now enable it by default and it is no longer 
possible to disable it.

I created a patch for disabling PREEMPT today. [1]

Is it possible to let us decide whether to activate PREEMPT or not?

Thanks in advance,

Christian

[1] 
https://raw.githubusercontent.com/chzigotzky/kernels/a74fa6179eaeafcea7ad89f0e61c30ace038daf2/patches/X1000/Kconfig.preempt.patch
[2] Bug report: https://github.com/chzigotzky/kernels/issues/19

Re: [Linux PPC] KVM PR and KVM HV do not work if the kernel was compiled with PREEMPT

[Shrikanth Hegde]

On 2/14/26 7:32 PM, Christian Zigotzky wrote:
> Hello,
> 
> KVM PR and KVM HV do not work if the kernel was compiled with PREEMPT.
> 
> The entire FSL Cyrus+ board freezes when using KVM HV with PREEMPT.
> 
> The guest kernel doesn't boot if we use KVM PR with a PREEMPT kernel on 
> the PA Semi Nemo board.
> 
> We were previously able to disable PREEMPT in the kernel configuration, 
> but the latest git kernels now enable it by default and it is no longer 
> possible to disable it.
> 
> I created a patch for disabling PREEMPT today. [1]
> 
> Is it possible to let us decide whether to activate PREEMPT or not?
> 
> Thanks in advance,
> 
> Christian
> 
> [1] https://raw.githubusercontent.com/chzigotzky/kernels/ 
> a74fa6179eaeafcea7ad89f0e61c30ace038daf2/patches/X1000/ 
> Kconfig.preempt.patch
> [2] Bug report: https://github.com/chzigotzky/kernels/issues/19


Hi.

Do you have any trace why it is stuck? That would be useful.



My initial take is cond_resched is a nop. So we might be stuck there.
Eventually it should have come out though.

Could you please give the below patch a try and let me know?
Note: This likely still needs lazy bit handling. So keep in preempt=full.
(Not tested)


diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index 7667563fb9ff..fe215d1177fe 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -4901,7 +4901,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,
         }
  
         if (need_resched())
-               cond_resched();
+               schedule();
  
         kvmppc_update_vpas(vcpu);
  
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 9a89a6d98f97..54963c1d8b58 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -86,7 +86,7 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu)
         while (true) {
                 if (need_resched()) {
                         local_irq_enable();
-                       cond_resched();
+                       schedule();
                         hard_irq_disable();
                         continue;
                 }

[Linux PPC] KVM PR and KVM HV do not work if the kernel was compiled with PREEMPT

[Christian Zigotzky]

Hi,

Thank you for your patch. I tested the latest git kernel with your patch 
today. Unfortunately the system freezes while booting the VM with KVM HV.

Cheers,
Christian


On 17/02/26 06:02, Shrikanth Hegde wrote:
>
>
> On 2/14/26 7:32 PM, Christian Zigotzky wrote:
>> Hello,
>>
>> KVM PR and KVM HV do not work if the kernel was compiled with PREEMPT.
>>
>> The entire FSL Cyrus+ board freezes when using KVM HV with PREEMPT.
>>
>> The guest kernel doesn't boot if we use KVM PR with a PREEMPT kernel 
>> on the PA Semi Nemo board.
>>
>> We were previously able to disable PREEMPT in the kernel 
>> configuration, but the latest git kernels now enable it by default 
>> and it is no longer possible to disable it.
>>
>> I created a patch for disabling PREEMPT today. [1]
>>
>> Is it possible to let us decide whether to activate PREEMPT or not?
>>
>> Thanks in advance,
>>
>> Christian
>>
>> [1] https://raw.githubusercontent.com/chzigotzky/kernels/ 
>> a74fa6179eaeafcea7ad89f0e61c30ace038daf2/patches/X1000/ 
>> Kconfig.preempt.patch
>> [2] Bug report: https://github.com/chzigotzky/kernels/issues/19
>
>
> Hi.
>
> Do you have any trace why it is stuck? That would be useful.
>
>
>
> My initial take is cond_resched is a nop. So we might be stuck there.
> Eventually it should have come out though.
>
> Could you please give the below patch a try and let me know?
> Note: This likely still needs lazy bit handling. So keep in preempt=full.
> (Not tested)
>
>
> diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
> index 7667563fb9ff..fe215d1177fe 100644
> --- a/arch/powerpc/kvm/book3s_hv.c
> +++ b/arch/powerpc/kvm/book3s_hv.c
> @@ -4901,7 +4901,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, 
> u64 time_limit,
>         }
>
>         if (need_resched())
> -               cond_resched();
> +               schedule();
>
>         kvmppc_update_vpas(vcpu);
>
> diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
> index 9a89a6d98f97..54963c1d8b58 100644
> --- a/arch/powerpc/kvm/powerpc.c
> +++ b/arch/powerpc/kvm/powerpc.c
> @@ -86,7 +86,7 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu)
>         while (true) {
>                 if (need_resched()) {
>                         local_irq_enable();
> -                       cond_resched();
> +                       schedule();
>                         hard_irq_disable();
>                         continue;
>                 }


-- 
Sent with BrassMonkey 33.9.1 (https://github.com/chzigotzky/Web-Browsers-and-Suites-for-Linux-PPC/releases/tag/BrassMonkey_33.9.1)

Re: [Linux PPC] KVM PR and KVM HV do not work if the kernel was compiled with PREEMPT

[Shrikanth Hegde]

On 2/17/26 2:22 PM, Christian Zigotzky wrote:
> Hi,
> 
> Thank you for your patch. I tested the latest git kernel with your patch 
> today. Unfortunately the system freezes while booting the VM with KVM HV.
> 
> Cheers,
> Christian
> 
> 
Would it be possible to share the console log or boot log of KVM to the
point it freezes?

[Linux PPC] KVM PR and KVM HV do not work if the kernel was compiled with PREEMPT

[Christian Zigotzky]

On 17/02/26 10:03, Shrikanth Hegde wrote:
>
>
> On 2/17/26 2:22 PM, Christian Zigotzky wrote:
>> Hi,
>>
>> Thank you for your patch. I tested the latest git kernel with your 
>> patch today. Unfortunately the system freezes while booting the VM 
>> with KVM HV.
>>
>> Cheers,
>> Christian
>>
>>
> Would it be possible to share the console log or boot log of KVM to the
> point it freezes?

Unfortunately there aren't any error messages.

I also tried it with:

gdb --args qemu-system-ppc64 -M ppce500 -cpu e5500 -m 1024 -kernel 
uImage-7.0-preempt -drive 
format=raw,file=void-live-powerpc-20230317-updated2.img,index=0,if=virtio 
-netdev user,id=mynet0 -device virtio-net,netdev=mynet0 -append "rw 
root=/dev/vda" -device virtio-gpu-pci -device virtio-mouse-pci -device 
virtio-keyboard-pci -enable-kvm -smp 4

and with gdb in the guest:

qemu-system-ppc64 -s -S -M ppce500 -cpu e5500 -m 1024 -kernel 
uImage-7.0-preempt -drive 
format=raw,file=void-live-powerpc-20230317-updated2.img,index=0,if=virtio 
-netdev user,id=mynet0 -device virtio-net,netdev=mynet0 -append "rw 
root=/dev/vda" -device virtio-gpu-pci -device virtio-mouse-pci -device 
virtio-keyboard-pci -enable-kvm -smp 4

The system freezed without any error messages.

-- 
Sent with BrassMonkey 33.9.1 (https://github.com/chzigotzky/Web-Browsers-and-Suites-for-Linux-PPC/releases/tag/BrassMonkey_33.9.1)

Re: [Linux PPC] KVM PR and KVM HV do not work if the kernel was compiled with PREEMPT

[Madhavan Srinivasan]

On 2/17/26 3:14 PM, Christian Zigotzky wrote:
> On 17/02/26 10:03, Shrikanth Hegde wrote:
>>
>>
>> On 2/17/26 2:22 PM, Christian Zigotzky wrote:
>>> Hi,
>>>
>>> Thank you for your patch. I tested the latest git kernel with your 
>>> patch today. Unfortunately the system freezes while booting the VM 
>>> with KVM HV.
>>>
>>> Cheers,
>>> Christian
>>>
>>>
>> Would it be possible to share the console log or boot log of KVM to the
>> point it freezes?
>
> Unfortunately there aren't any error messages.
>
> I also tried it with:
>
> gdb --args qemu-system-ppc64 -M ppce500 -cpu e5500 -m 1024 -kernel 
> uImage-7.0-preempt -drive 
> format=raw,file=void-live-powerpc-20230317-updated2.img,index=0,if=virtio 
> -netdev user,id=mynet0 -device virtio-net,netdev=mynet0 -append "rw 
> root=/dev/vda" -device virtio-gpu-pci -device virtio-mouse-pci -device 
> virtio-keyboard-pci -enable-kvm -smp 4
>
> and with gdb in the guest:
>
> qemu-system-ppc64 -s -S -M ppce500 -cpu e5500 -m 1024 -kernel 
> uImage-7.0-preempt -drive 
> format=raw,file=void-live-powerpc-20230317-updated2.img,index=0,if=virtio 
> -netdev user,id=mynet0 -device virtio-net,netdev=mynet0 -append "rw 
> root=/dev/vda" -device virtio-gpu-pci -device virtio-mouse-pci -device 
> virtio-keyboard-pci -enable-kvm -smp 4
>
> The system freezed without any error messages.
can you please share the config.
I tried kvm-hv in my power8/power9 and seems to boot fine without any 
issues with today's upstream
Maddy

[Linux PPC] KVM PR and KVM HV do not work if the kernel was compiled with PREEMPT

[Christian Zigotzky]

On 17/02/26 11:17 Madhavan Srinivasan wrote:
>
> On 2/17/26 3:14 PM, Christian Zigotzky wrote:
>> On 17/02/26 10:03, Shrikanth Hegde wrote:
>>>
>>>
>>> On 2/17/26 2:22 PM, Christian Zigotzky wrote:
>>>> Hi,
>>>>
>>>> Thank you for your patch. I tested the latest git kernel with your 
>>>> patch today. Unfortunately the system freezes while booting the VM 
>>>> with KVM HV.
>>>>
>>>> Cheers,
>>>> Christian
>>>>
>>>>
>>> Would it be possible to share the console log or boot log of KVM to the
>>> point it freezes?
>>
>> Unfortunately there aren't any error messages.
>>
>> I also tried it with:
>>
>> gdb --args qemu-system-ppc64 -M ppce500 -cpu e5500 -m 1024 -kernel 
>> uImage-7.0-preempt -drive 
>> format=raw,file=void-live-powerpc-20230317-updated2.img,index=0,if=virtio 
>> -netdev user,id=mynet0 -device virtio-net,netdev=mynet0 -append "rw 
>> root=/dev/vda" -device virtio-gpu-pci -device virtio-mouse-pci 
>> -device virtio-keyboard-pci -enable-kvm -smp 4
>>
>> and with gdb in the guest:
>>
>> qemu-system-ppc64 -s -S -M ppce500 -cpu e5500 -m 1024 -kernel 
>> uImage-7.0-preempt -drive 
>> format=raw,file=void-live-powerpc-20230317-updated2.img,index=0,if=virtio 
>> -netdev user,id=mynet0 -device virtio-net,netdev=mynet0 -append "rw 
>> root=/dev/vda" -device virtio-gpu-pci -device virtio-mouse-pci 
>> -device virtio-keyboard-pci -enable-kvm -smp 4
>>
>> The system freezed without any error messages.
> can you please share the config.
> I tried kvm-hv in my power8/power9 and seems to boot fine without any 
> issues with today's upstream
> Maddy
Here are the configs.

With PREEMPT: 
https://github.com/chzigotzky/kernels/blob/d50d259ca63dd2b3b9a9eaabc82e5cc69c8cc9f0/configs/e5500_defconfig

Without PREEMPT: 
https://github.com/chzigotzky/kernels/blob/main/configs/e5500_defconfig

- Christian

Re: [Linux PPC] KVM PR and KVM HV do not work if the kernel was compiled with PREEMPT

[Shrikanth Hegde]

On 2/17/26 3:56 PM, Christian Zigotzky wrote:
> On 17/02/26 11:17 Madhavan Srinivasan wrote:
>>
>> On 2/17/26 3:14 PM, Christian Zigotzky wrote:
>>> On 17/02/26 10:03, Shrikanth Hegde wrote:
>>>>
>>>>
>>>> On 2/17/26 2:22 PM, Christian Zigotzky wrote:
>>>>> Hi,
>>>>>
>>>>> Thank you for your patch. I tested the latest git kernel with your 
>>>>> patch today. Unfortunately the system freezes while booting the VM 
>>>>> with KVM HV.
>>>>>
>>>>> Cheers,
>>>>> Christian
>>>>>
>>>>>
>>>> Would it be possible to share the console log or boot log of KVM to the
>>>> point it freezes?
>>>
>>> Unfortunately there aren't any error messages.
>>>
>>> I also tried it with:
>>>
>>> gdb --args qemu-system-ppc64 -M ppce500 -cpu e5500 -m 1024 -kernel 
>>> uImage-7.0-preempt -drive format=raw,file=void-live-powerpc-20230317- 
>>> updated2.img,index=0,if=virtio -netdev user,id=mynet0 -device virtio- 
>>> net,netdev=mynet0 -append "rw root=/dev/vda" -device virtio-gpu-pci - 
>>> device virtio-mouse-pci -device virtio-keyboard-pci -enable-kvm -smp 4
>>>
>>> and with gdb in the guest:
>>>
>>> qemu-system-ppc64 -s -S -M ppce500 -cpu e5500 -m 1024 -kernel 
>>> uImage-7.0-preempt -drive format=raw,file=void-live-powerpc-20230317- 
>>> updated2.img,index=0,if=virtio -netdev user,id=mynet0 -device virtio- 
>>> net,netdev=mynet0 -append "rw root=/dev/vda" -device virtio-gpu-pci - 
>>> device virtio-mouse-pci -device virtio-keyboard-pci -enable-kvm -smp 4
>>>
>>> The system freezed without any error messages.

Anything on host console?

>> can you please share the config.
>> I tried kvm-hv in my power8/power9 and seems to boot fine without any 
>> issues with today's upstream
>> Maddy
> Here are the configs.
> 
> With PREEMPT: https://github.com/chzigotzky/kernels/blob/ 
> d50d259ca63dd2b3b9a9eaabc82e5cc69c8cc9f0/configs/e5500_defconfig
> 
> Without PREEMPT: https://github.com/chzigotzky/kernels/blob/main/ 
> configs/e5500_defconfig
> 
> - Christian

I looked at the code. there was one more place where cond_resched was 
there.

If possible give the below try.
(Note: Didn;t replicate your issue)

---

diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index 7667563fb9ff..bf0ca49c9915 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -4825,10 +4825,13 @@ static int kvmppc_run_vcpu(struct kvm_vcpu *vcpu)
  		vc->runner = vcpu;
  		if (n_ceded == vc->n_runnable) {
  			kvmppc_vcore_blocked(vc);
-		} else if (need_resched()) {
+		} else if (need_resched() || spin_needbreak(&vc->lock)) {
  			kvmppc_vcore_preempt(vc);
  			/* Let something else run */
-			cond_resched_lock(&vc->lock);
+			spin_unlock(&vc->lock);
+			schedule();
+			//cond_resched_lock(&vc->lock);
+			spin_lock(&vc->lock);
  			if (vc->vcore_state == VCORE_PREEMPT)
  				kvmppc_vcore_end_preempt(vc);
  		} else {
@@ -4901,7 +4904,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, 
u64 time_limit,
  	}

  	if (need_resched())
-		cond_resched();
+		schedule();

  	kvmppc_update_vpas(vcpu);

diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 9a89a6d98f97..54963c1d8b58 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -86,7 +86,7 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu)
  	while (true) {
  		if (need_resched()) {
  			local_irq_enable();
-			cond_resched();
+			schedule();
  			hard_irq_disable();
  			continue;
  		}

[Linux PPC] KVM PR and KVM HV do not work if the kernel was compiled with PREEMPT

[Christian Zigotzky]

On 17/02/26 12:34, Shrikanth Hegde wrote:
>
>
> On 2/17/26 3:56 PM, Christian Zigotzky wrote:
>> On 17/02/26 11:17 Madhavan Srinivasan wrote:
>>>
>>> On 2/17/26 3:14 PM, Christian Zigotzky wrote:
>>>> On 17/02/26 10:03, Shrikanth Hegde wrote:
>>>>>
>>>>>
>>>>> On 2/17/26 2:22 PM, Christian Zigotzky wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Thank you for your patch. I tested the latest git kernel with 
>>>>>> your patch today. Unfortunately the system freezes while booting 
>>>>>> the VM with KVM HV.
>>>>>>
>>>>>> Cheers,
>>>>>> Christian
>>>>>>
>>>>>>
>>>>> Would it be possible to share the console log or boot log of KVM 
>>>>> to the
>>>>> point it freezes?
>>>>
>>>> Unfortunately there aren't any error messages.
>>>>
>>>> I also tried it with:
>>>>
>>>> gdb --args qemu-system-ppc64 -M ppce500 -cpu e5500 -m 1024 -kernel 
>>>> uImage-7.0-preempt -drive 
>>>> format=raw,file=void-live-powerpc-20230317- 
>>>> updated2.img,index=0,if=virtio -netdev user,id=mynet0 -device 
>>>> virtio- net,netdev=mynet0 -append "rw root=/dev/vda" -device 
>>>> virtio-gpu-pci - device virtio-mouse-pci -device 
>>>> virtio-keyboard-pci -enable-kvm -smp 4
>>>>
>>>> and with gdb in the guest:
>>>>
>>>> qemu-system-ppc64 -s -S -M ppce500 -cpu e5500 -m 1024 -kernel 
>>>> uImage-7.0-preempt -drive 
>>>> format=raw,file=void-live-powerpc-20230317- 
>>>> updated2.img,index=0,if=virtio -netdev user,id=mynet0 -device 
>>>> virtio- net,netdev=mynet0 -append "rw root=/dev/vda" -device 
>>>> virtio-gpu-pci - device virtio-mouse-pci -device 
>>>> virtio-keyboard-pci -enable-kvm -smp 4
>>>>
>>>> The system freezed without any error messages.
>
> Anything on host console?
>
>>> can you please share the config.
>>> I tried kvm-hv in my power8/power9 and seems to boot fine without 
>>> any issues with today's upstream
>>> Maddy
>> Here are the configs.
>>
>> With PREEMPT: https://github.com/chzigotzky/kernels/blob/ 
>> d50d259ca63dd2b3b9a9eaabc82e5cc69c8cc9f0/configs/e5500_defconfig
>>
>> Without PREEMPT: https://github.com/chzigotzky/kernels/blob/main/ 
>> configs/e5500_defconfig
>>
>> - Christian
>
> I looked at the code. there was one more place where cond_resched was 
> there.
>
> If possible give the below try.
> (Note: Didn;t replicate your issue)
>
> ---
>
> diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
> index 7667563fb9ff..bf0ca49c9915 100644
> --- a/arch/powerpc/kvm/book3s_hv.c
> +++ b/arch/powerpc/kvm/book3s_hv.c
> @@ -4825,10 +4825,13 @@ static int kvmppc_run_vcpu(struct kvm_vcpu *vcpu)
>          vc->runner = vcpu;
>          if (n_ceded == vc->n_runnable) {
>              kvmppc_vcore_blocked(vc);
> -        } else if (need_resched()) {
> +        } else if (need_resched() || spin_needbreak(&vc->lock)) {
>              kvmppc_vcore_preempt(vc);
>              /* Let something else run */
> -            cond_resched_lock(&vc->lock);
> +            spin_unlock(&vc->lock);
> +            schedule();
> +            //cond_resched_lock(&vc->lock);
> +            spin_lock(&vc->lock);
>              if (vc->vcore_state == VCORE_PREEMPT)
>                  kvmppc_vcore_end_preempt(vc);
>          } else {
> @@ -4901,7 +4904,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, 
> u64 time_limit,
>      }
>
>      if (need_resched())
> -        cond_resched();
> +        schedule();
>
>      kvmppc_update_vpas(vcpu);
>
> diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
> index 9a89a6d98f97..54963c1d8b58 100644
> --- a/arch/powerpc/kvm/powerpc.c
> +++ b/arch/powerpc/kvm/powerpc.c
> @@ -86,7 +86,7 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu)
>      while (true) {
>          if (need_resched()) {
>              local_irq_enable();
> -            cond_resched();
> +            schedule();
>              hard_irq_disable();
>              continue;
>          }
>
Hello Shrikanth,

I tested your patch today. The patched preempt kernel boots on my e5500 
host and in an e5500 VM. Unfortunately my guest and host freeze after 
loading the VirtIO-GPU.

Thanks for your help,

Christian

-- 
Sent with BrassMonkey 33.9.1 (https://github.com/chzigotzky/Web-Browsers-and-Suites-for-Linux-PPC/releases/tag/BrassMonkey_33.9.1)

Re: [Linux PPC] KVM PR and KVM HV do not work if the kernel was compiled with PREEMPT

[Shrikanth Hegde]

Hi Christian.


>> If possible give the below try.
>> (Note: Didn;t replicate your issue)
>>
>> ---
>>
>> diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
>> index 7667563fb9ff..bf0ca49c9915 100644
>> --- a/arch/powerpc/kvm/book3s_hv.c
>> +++ b/arch/powerpc/kvm/book3s_hv.c
>> @@ -4825,10 +4825,13 @@ static int kvmppc_run_vcpu(struct kvm_vcpu *vcpu)
>>          vc->runner = vcpu;
>>          if (n_ceded == vc->n_runnable) {
>>              kvmppc_vcore_blocked(vc);
>> -        } else if (need_resched()) {
>> +        } else if (need_resched() || spin_needbreak(&vc->lock)) {
>>              kvmppc_vcore_preempt(vc);
>>              /* Let something else run */
>> -            cond_resched_lock(&vc->lock);
>> +            spin_unlock(&vc->lock);
>> +            schedule();
>> +            //cond_resched_lock(&vc->lock);
>> +            spin_lock(&vc->lock);
>>              if (vc->vcore_state == VCORE_PREEMPT)
>>                  kvmppc_vcore_end_preempt(vc);
>>          } else {
>> @@ -4901,7 +4904,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, 
>> u64 time_limit,
>>      }
>>
>>      if (need_resched())
>> -        cond_resched();
>> +        schedule();
>>
>>      kvmppc_update_vpas(vcpu);
>>
>> diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
>> index 9a89a6d98f97..54963c1d8b58 100644
>> --- a/arch/powerpc/kvm/powerpc.c
>> +++ b/arch/powerpc/kvm/powerpc.c
>> @@ -86,7 +86,7 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu)
>>      while (true) {
>>          if (need_resched()) {
>>              local_irq_enable();
>> -            cond_resched();
>> +            schedule();
>>              hard_irq_disable();
>>              continue;
>>          }
>>
> Hello Shrikanth,
> t
> I tested your patch today. The patched preempt kernel boots on my e5500 
> host and in an e5500 VM. Unfortunately my guest and host freeze after 
> loading the VirtIO-GPU.
> 
> Thanks for your help,
> 
> Christian

Glad it helps you to boot into.

Since you said even host freezes, host maybe crashing.
Do you have the console log when you load the VirtIO-GPU?

[Linux PPC] 7.0-rc1: e500_mmu.c:923:26: error: assignment to 'struct tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *'

[Christian Zigotzky]

Hello,

Compiling issue for e5500 machines for the RC1 of kernel 7.0:

arch/powerpc/kvm/e500_mmu.c: In function 'kvmppc_e500_tlb_init':
arch/powerpc/kvm/e500_mmu.c:923:26: error: assignment to 'struct 
tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *' 
[-Werror=incompatible-pointer-types]
   923 |  vcpu_e500->gtlb_priv[0] = kzalloc_objs(struct tlbe_ref,
       |                          ^
arch/powerpc/kvm/e500_mmu.c:928:26: error: assignment to 'struct 
tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *' 
[-Werror=incompatible-pointer-types]
   928 |  vcpu_e500->gtlb_priv[1] = kzalloc_objs(struct tlbe_ref,

Temporary solution: # CONFIG_VIRTUALIZATION is not set

Please solve this issue.

Thanks,
Christian

Re: [Linux PPC] 7.0-rc1: e500_mmu.c:923:26: error: assignment to 'struct tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *'

[Christophe Leroy (CS GROUP)]

+Kees

Hello,

Le 23/02/2026 à 17:48, Christian Zigotzky a écrit :
> Hello,
> 
> Compiling issue for e5500 machines for the RC1 of kernel 7.0:
> 
> arch/powerpc/kvm/e500_mmu.c: In function 'kvmppc_e500_tlb_init':
> arch/powerpc/kvm/e500_mmu.c:923:26: error: assignment to 'struct 
> tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *' [- 
> Werror=incompatible-pointer-types]
>    923 |  vcpu_e500->gtlb_priv[0] = kzalloc_objs(struct tlbe_ref,
>        |                          ^
> arch/powerpc/kvm/e500_mmu.c:928:26: error: assignment to 'struct 
> tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *' [- 
> Werror=incompatible-pointer-types]
>    928 |  vcpu_e500->gtlb_priv[1] = kzalloc_objs(struct tlbe_ref,
> 
> Temporary solution: # CONFIG_VIRTUALIZATION is not set
> 
> Please solve this issue.
> 

Must be one of:

189f164e573e Convert remaining multi-line kmalloc_obj/flex GFP_KERNEL uses
bf4afc53b77a Convert 'alloc_obj' family to use the new default 
GFP_KERNEL argument
69050f8d6d07 treewide: Replace kmalloc with kmalloc_obj for non-scalar types


Christophe

Re: [Linux PPC] 7.0-rc1: e500_mmu.c:923:26: error: assignment to 'struct tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *'

[Christophe Leroy (CS GROUP)]

Le 23/02/2026 à 19:07, Christophe Leroy (CS GROUP) a écrit :
> +Kees
> 
> Hello,
> 
> Le 23/02/2026 à 17:48, Christian Zigotzky a écrit :
>> Hello,
>>
>> Compiling issue for e5500 machines for the RC1 of kernel 7.0:
>>
>> arch/powerpc/kvm/e500_mmu.c: In function 'kvmppc_e500_tlb_init':
>> arch/powerpc/kvm/e500_mmu.c:923:26: error: assignment to 'struct 
>> tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *' [- 
>> Werror=incompatible-pointer-types]
>>    923 |  vcpu_e500->gtlb_priv[0] = kzalloc_objs(struct tlbe_ref,
>>        |                          ^
>> arch/powerpc/kvm/e500_mmu.c:928:26: error: assignment to 'struct 
>> tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *' [- 
>> Werror=incompatible-pointer-types]
>>    928 |  vcpu_e500->gtlb_priv[1] = kzalloc_objs(struct tlbe_ref,
>>
>> Temporary solution: # CONFIG_VIRTUALIZATION is not set
>>
>> Please solve this issue.
>>
> 
> Must be one of:
> 
> 189f164e573e Convert remaining multi-line kmalloc_obj/flex GFP_KERNEL uses
> bf4afc53b77a Convert 'alloc_obj' family to use the new default 
> GFP_KERNEL argument
> 69050f8d6d07 treewide: Replace kmalloc with kmalloc_obj for non-scalar 
> types
> 

Can you try following change:

diff --git a/arch/powerpc/kvm/e500_mmu.c b/arch/powerpc/kvm/e500_mmu.c
index 48580c85f23b..75ed1496ead5 100644
--- a/arch/powerpc/kvm/e500_mmu.c
+++ b/arch/powerpc/kvm/e500_mmu.c
@@ -920,12 +920,12 @@ int kvmppc_e500_tlb_init(struct kvmppc_vcpu_e500 
*vcpu_e500)
  	vcpu_e500->gtlb_offset[0] = 0;
  	vcpu_e500->gtlb_offset[1] = KVM_E500_TLB0_SIZE;

-	vcpu_e500->gtlb_priv[0] = kzalloc_objs(struct tlbe_ref,
+	vcpu_e500->gtlb_priv[0] = kzalloc_objs(struct tlbe_priv,
  					       vcpu_e500->gtlb_params[0].entries);
  	if (!vcpu_e500->gtlb_priv[0])
  		goto free_vcpu;

-	vcpu_e500->gtlb_priv[1] = kzalloc_objs(struct tlbe_ref,
+	vcpu_e500->gtlb_priv[1] = kzalloc_objs(struct tlbe_priv,
  					       vcpu_e500->gtlb_params[1].entries);
  	if (!vcpu_e500->gtlb_priv[1])
  		goto free_vcpu;

---
Christophe

Re: [Linux PPC] 7.0-rc1: e500_mmu.c:923:26: error: assignment to 'struct tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *'

[Kees Cook]

On Mon, Feb 23, 2026 at 08:38:29PM +0100, Christophe Leroy (CS GROUP) wrote:
> 
> 
> Le 23/02/2026 à 19:07, Christophe Leroy (CS GROUP) a écrit :
> > +Kees
> > 
> > Hello,
> > 
> > Le 23/02/2026 à 17:48, Christian Zigotzky a écrit :
> > > Hello,
> > > 
> > > Compiling issue for e5500 machines for the RC1 of kernel 7.0:
> > > 
> > > arch/powerpc/kvm/e500_mmu.c: In function 'kvmppc_e500_tlb_init':
> > > arch/powerpc/kvm/e500_mmu.c:923:26: error: assignment to 'struct
> > > tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *' [-
> > > Werror=incompatible-pointer-types]
> > >    923 |  vcpu_e500->gtlb_priv[0] = kzalloc_objs(struct tlbe_ref,
> > >        |                          ^
> > > arch/powerpc/kvm/e500_mmu.c:928:26: error: assignment to 'struct
> > > tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *' [-
> > > Werror=incompatible-pointer-types]
> > >    928 |  vcpu_e500->gtlb_priv[1] = kzalloc_objs(struct tlbe_ref,
> > > 
> > > Temporary solution: # CONFIG_VIRTUALIZATION is not set
> > > 
> > > Please solve this issue.
> > > 
> > 
> > Must be one of:
> > 
> > 189f164e573e Convert remaining multi-line kmalloc_obj/flex GFP_KERNEL uses
> > bf4afc53b77a Convert 'alloc_obj' family to use the new default
> > GFP_KERNEL argument
> > 69050f8d6d07 treewide: Replace kmalloc with kmalloc_obj for non-scalar
> > types
> > 
> 
> Can you try following change:
> 
> diff --git a/arch/powerpc/kvm/e500_mmu.c b/arch/powerpc/kvm/e500_mmu.c
> index 48580c85f23b..75ed1496ead5 100644
> --- a/arch/powerpc/kvm/e500_mmu.c
> +++ b/arch/powerpc/kvm/e500_mmu.c
> @@ -920,12 +920,12 @@ int kvmppc_e500_tlb_init(struct kvmppc_vcpu_e500
> *vcpu_e500)
>  	vcpu_e500->gtlb_offset[0] = 0;
>  	vcpu_e500->gtlb_offset[1] = KVM_E500_TLB0_SIZE;
> 
> -	vcpu_e500->gtlb_priv[0] = kzalloc_objs(struct tlbe_ref,
> +	vcpu_e500->gtlb_priv[0] = kzalloc_objs(struct tlbe_priv,
>  					       vcpu_e500->gtlb_params[0].entries);
>  	if (!vcpu_e500->gtlb_priv[0])
>  		goto free_vcpu;
> 
> -	vcpu_e500->gtlb_priv[1] = kzalloc_objs(struct tlbe_ref,
> +	vcpu_e500->gtlb_priv[1] = kzalloc_objs(struct tlbe_priv,
>  					       vcpu_e500->gtlb_params[1].entries);
>  	if (!vcpu_e500->gtlb_priv[1])
>  		goto free_vcpu;

That looks like the correct fix to me. The old type was the correct size,
but was just the wrong type. Sorry I missed it when doing my tree-wide
multi-arch builds! What build target (or Kconfig) was needed to reach
this code?

Fixes: 69050f8d6d07 ("treewide: Replace kmalloc with kmalloc_obj for non-scalar types")
Reviewed-by: Kees Cook <kees@kernel.org>

Thanks!

-- 
Kees Cook

Re: [Linux PPC] 7.0-rc1: e500_mmu.c:923:26: error: assignment to 'struct tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *'

[Christian Zigotzky]

On 23/02/26 23:40, Kees Cook wrote:
> On Mon, Feb 23, 2026 at 08:38:29PM +0100, Christophe Leroy (CS GROUP) wrote:
>>
>> Le 23/02/2026 à 19:07, Christophe Leroy (CS GROUP) a écrit :
>>> +Kees
>>>
>>> Hello,
>>>
>>> Le 23/02/2026 à 17:48, Christian Zigotzky a écrit :
>>>> Hello,
>>>>
>>>> Compiling issue for e5500 machines for the RC1 of kernel 7.0:
>>>>
>>>> arch/powerpc/kvm/e500_mmu.c: In function 'kvmppc_e500_tlb_init':
>>>> arch/powerpc/kvm/e500_mmu.c:923:26: error: assignment to 'struct
>>>> tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *' [-
>>>> Werror=incompatible-pointer-types]
>>>>     923 |  vcpu_e500->gtlb_priv[0] = kzalloc_objs(struct tlbe_ref,
>>>>         |                          ^
>>>> arch/powerpc/kvm/e500_mmu.c:928:26: error: assignment to 'struct
>>>> tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *' [-
>>>> Werror=incompatible-pointer-types]
>>>>     928 |  vcpu_e500->gtlb_priv[1] = kzalloc_objs(struct tlbe_ref,
>>>>
>>>> Temporary solution: # CONFIG_VIRTUALIZATION is not set
>>>>
>>>> Please solve this issue.
>>>>
>>> Must be one of:
>>>
>>> 189f164e573e Convert remaining multi-line kmalloc_obj/flex GFP_KERNEL uses
>>> bf4afc53b77a Convert 'alloc_obj' family to use the new default
>>> GFP_KERNEL argument
>>> 69050f8d6d07 treewide: Replace kmalloc with kmalloc_obj for non-scalar
>>> types
>>>
>> Can you try following change:
>>
>> diff --git a/arch/powerpc/kvm/e500_mmu.c b/arch/powerpc/kvm/e500_mmu.c
>> index 48580c85f23b..75ed1496ead5 100644
>> --- a/arch/powerpc/kvm/e500_mmu.c
>> +++ b/arch/powerpc/kvm/e500_mmu.c
>> @@ -920,12 +920,12 @@ int kvmppc_e500_tlb_init(struct kvmppc_vcpu_e500
>> *vcpu_e500)
>>   	vcpu_e500->gtlb_offset[0] = 0;
>>   	vcpu_e500->gtlb_offset[1] = KVM_E500_TLB0_SIZE;
>>
>> -	vcpu_e500->gtlb_priv[0] = kzalloc_objs(struct tlbe_ref,
>> +	vcpu_e500->gtlb_priv[0] = kzalloc_objs(struct tlbe_priv,
>>   					       vcpu_e500->gtlb_params[0].entries);
>>   	if (!vcpu_e500->gtlb_priv[0])
>>   		goto free_vcpu;
>>
>> -	vcpu_e500->gtlb_priv[1] = kzalloc_objs(struct tlbe_ref,
>> +	vcpu_e500->gtlb_priv[1] = kzalloc_objs(struct tlbe_priv,
>>   					       vcpu_e500->gtlb_params[1].entries);
>>   	if (!vcpu_e500->gtlb_priv[1])
>>   		goto free_vcpu;
> That looks like the correct fix to me. The old type was the correct size,
> but was just the wrong type. Sorry I missed it when doing my tree-wide
> multi-arch builds! What build target (or Kconfig) was needed to reach
> this code?
>
> Fixes: 69050f8d6d07 ("treewide: Replace kmalloc with kmalloc_obj for non-scalar types")
> Reviewed-by: Kees Cook <kees@kernel.org>
>
> Thanks!
>

It works. Thank you. Tested-by: Christian Zigotzky <chzigotzky@xenosoft.de>

-- 
Sent with BrassMonkey 33.9.1 (https://github.com/chzigotzky/Web-Browsers-and-Suites-for-Linux-PPC/releases/tag/BrassMonkey_33.9.1)

Re: [Linux PPC] 7.0-rc1: e500_mmu.c:923:26: error: assignment to 'struct tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *'

[Christophe Leroy (CS GROUP)]

Le 23/02/2026 à 23:40, Kees Cook a écrit :
> On Mon, Feb 23, 2026 at 08:38:29PM +0100, Christophe Leroy (CS GROUP) wrote:
>>
>>
>> Le 23/02/2026 à 19:07, Christophe Leroy (CS GROUP) a écrit :
>>> +Kees
>>>
>>> Hello,
>>>
>>> Le 23/02/2026 à 17:48, Christian Zigotzky a écrit :
>>>> Hello,
>>>>
>>>> Compiling issue for e5500 machines for the RC1 of kernel 7.0:
>>>>
>>>> arch/powerpc/kvm/e500_mmu.c: In function 'kvmppc_e500_tlb_init':
>>>> arch/powerpc/kvm/e500_mmu.c:923:26: error: assignment to 'struct
>>>> tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *' [-
>>>> Werror=incompatible-pointer-types]
>>>>     923 |  vcpu_e500->gtlb_priv[0] = kzalloc_objs(struct tlbe_ref,
>>>>         |                          ^
>>>> arch/powerpc/kvm/e500_mmu.c:928:26: error: assignment to 'struct
>>>> tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *' [-
>>>> Werror=incompatible-pointer-types]
>>>>     928 |  vcpu_e500->gtlb_priv[1] = kzalloc_objs(struct tlbe_ref,
>>>>
>>>> Temporary solution: # CONFIG_VIRTUALIZATION is not set
>>>>
>>>> Please solve this issue.
>>>>
>>>
>>> Must be one of:
>>>
>>> 189f164e573e Convert remaining multi-line kmalloc_obj/flex GFP_KERNEL uses
>>> bf4afc53b77a Convert 'alloc_obj' family to use the new default
>>> GFP_KERNEL argument
>>> 69050f8d6d07 treewide: Replace kmalloc with kmalloc_obj for non-scalar
>>> types
>>>
>>
>> Can you try following change:
>>
>> diff --git a/arch/powerpc/kvm/e500_mmu.c b/arch/powerpc/kvm/e500_mmu.c
>> index 48580c85f23b..75ed1496ead5 100644
>> --- a/arch/powerpc/kvm/e500_mmu.c
>> +++ b/arch/powerpc/kvm/e500_mmu.c
>> @@ -920,12 +920,12 @@ int kvmppc_e500_tlb_init(struct kvmppc_vcpu_e500
>> *vcpu_e500)
>>   	vcpu_e500->gtlb_offset[0] = 0;
>>   	vcpu_e500->gtlb_offset[1] = KVM_E500_TLB0_SIZE;
>>
>> -	vcpu_e500->gtlb_priv[0] = kzalloc_objs(struct tlbe_ref,
>> +	vcpu_e500->gtlb_priv[0] = kzalloc_objs(struct tlbe_priv,
>>   					       vcpu_e500->gtlb_params[0].entries);
>>   	if (!vcpu_e500->gtlb_priv[0])
>>   		goto free_vcpu;
>>
>> -	vcpu_e500->gtlb_priv[1] = kzalloc_objs(struct tlbe_ref,
>> +	vcpu_e500->gtlb_priv[1] = kzalloc_objs(struct tlbe_priv,
>>   					       vcpu_e500->gtlb_params[1].entries);
>>   	if (!vcpu_e500->gtlb_priv[1])
>>   		goto free_vcpu;
> 
> That looks like the correct fix to me. The old type was the correct size,
> but was just the wrong type. Sorry I missed it when doing my tree-wide
> multi-arch builds! What build target (or Kconfig) was needed to reach
> this code?

I used ppc64e_defconfig + CONFIG_VIRTUALIZATION=y
+ CONFIG_KVM_E500MC=y


> 
> Fixes: 69050f8d6d07 ("treewide: Replace kmalloc with kmalloc_obj for non-scalar types")
> Reviewed-by: Kees Cook <kees@kernel.org>
> 
> Thanks!
> 

[Linux PPC] Disable PREEMPT

[Christian Zigotzky]

Hello,

We were previously able to disable PREEMPT in the kernel configuration, 
but the latest kernels now enable it by default and it is no longer 
possible to disable it.

Is it possible to let us decide whether to activate PREEMPT or not?

Currently we use the following revert patch for our non-preempt kernels.

--- a/kernel/Kconfig.preempt    2026-02-14 10:06:32.000000000 +0000
+++ b/kernel/Kconfig.preempt    2026-02-08 21:03:27.000000000 +0000
@@ -16,13 +16,11 @@ config ARCH_HAS_PREEMPT_LAZY

  choice
      prompt "Preemption Model"
-    default PREEMPT_LAZY if ARCH_HAS_PREEMPT_LAZY
      default PREEMPT_NONE

  config PREEMPT_NONE
      bool "No Forced Preemption (Server)"
      depends on !PREEMPT_RT
-    depends on ARCH_NO_PREEMPT
      select PREEMPT_NONE_BUILD if !PREEMPT_DYNAMIC
      help
        This is the traditional Linux preemption model, geared towards
@@ -37,7 +35,6 @@ config PREEMPT_NONE

  config PREEMPT_VOLUNTARY
      bool "Voluntary Kernel Preemption (Desktop)"
-    depends on !ARCH_HAS_PREEMPT_LAZY
      depends on !ARCH_NO_PREEMPT
      depends on !PREEMPT_RT
      select PREEMPT_VOLUNTARY_BUILD if !PREEMPT_DYNAMIC

---

Thanks in advance,

Christian

Re: [Linux PPC] Disable PREEMPT

[Shrikanth Hegde]

Hi Christian.


Is this sent as a patch or just a reply to the thread?

If it is sent as patch, Please follow these guidelines.
https://docs.kernel.org/process/submitting-patches.html

On 2/25/26 11:24 AM, Christian Zigotzky wrote:
> Hello,
> 
> We were previously able to disable PREEMPT in the kernel configuration, 
> but the latest kernels now enable it by default and it is no longer 
> possible to disable it.
> 
> Is it possible to let us decide whether to activate PREEMPT or not?
> 
> Currently we use the following revert patch for our non-preempt kernels.
> 
> --- a/kernel/Kconfig.preempt    2026-02-14 10:06:32.000000000 +0000
> +++ b/kernel/Kconfig.preempt    2026-02-08 21:03:27.000000000 +0000
> @@ -16,13 +16,11 @@ config ARCH_HAS_PREEMPT_LAZY
> 
>   choice
>       prompt "Preemption Model"
> -    default PREEMPT_LAZY if ARCH_HAS_PREEMPT_LAZY
>       default PREEMPT_NONE
> 
>   config PREEMPT_NONE
>       bool "No Forced Preemption (Server)"
>       depends on !PREEMPT_RT
> -    depends on ARCH_NO_PREEMPT
>       select PREEMPT_NONE_BUILD if !PREEMPT_DYNAMIC
>       help
>         This is the traditional Linux preemption model, geared towards
> @@ -37,7 +35,6 @@ config PREEMPT_NONE
> 
>   config PREEMPT_VOLUNTARY
>       bool "Voluntary Kernel Preemption (Desktop)"
> -    depends on !ARCH_HAS_PREEMPT_LAZY
>       depends on !ARCH_NO_PREEMPT
>       depends on !PREEMPT_RT
>       select PREEMPT_VOLUNTARY_BUILD if !PREEMPT_DYNAMIC
> 

I would instead fix any issue we have in PREEMPT FULL/LAZY rather than falling back to
old methods.

btw, did the previous patch sent, which you said helps you to boot, can you confirm
it did help you? If yes, i can send it as patch, since cond_resched if called from boot cpu
might be stuck in the loop forever.

IIRC, after boot, you had some driver causing crash. If so, share backtrace of the crash?
If you don't use that driver, is the guest stable?

Re: [Linux PPC] Disable PREEMPT

[Christophe Leroy (CS GROUP)]

+Peter

Hi Christian,

Le 25/02/2026 à 06:54, Christian Zigotzky a écrit :
> Hello,
> 
> We were previously able to disable PREEMPT in the kernel configuration, 
> but the latest kernels now enable it by default and it is no longer 
> possible to disable it.
> 
> Is it possible to let us decide whether to activate PREEMPT or not?

Can you give more details on why you want to be able to deactivate PREEMPT ?

Read https://github.com/torvalds/linux/commit/7dadeaa6e851e


> 
> Currently we use the following revert patch for our non-preempt kernels.
> 
> --- a/kernel/Kconfig.preempt    2026-02-14 10:06:32.000000000 +0000
> +++ b/kernel/Kconfig.preempt    2026-02-08 21:03:27.000000000 +0000
> @@ -16,13 +16,11 @@ config ARCH_HAS_PREEMPT_LAZY
> 
>   choice
>       prompt "Preemption Model"
> -    default PREEMPT_LAZY if ARCH_HAS_PREEMPT_LAZY
>       default PREEMPT_NONE
> 
>   config PREEMPT_NONE
>       bool "No Forced Preemption (Server)"
>       depends on !PREEMPT_RT
> -    depends on ARCH_NO_PREEMPT
>       select PREEMPT_NONE_BUILD if !PREEMPT_DYNAMIC
>       help
>         This is the traditional Linux preemption model, geared towards
> @@ -37,7 +35,6 @@ config PREEMPT_NONE
> 
>   config PREEMPT_VOLUNTARY
>       bool "Voluntary Kernel Preemption (Desktop)"
> -    depends on !ARCH_HAS_PREEMPT_LAZY
>       depends on !ARCH_NO_PREEMPT
>       depends on !PREEMPT_RT
>       select PREEMPT_VOLUNTARY_BUILD if !PREEMPT_DYNAMIC
> 
> ---
> 
> Thanks in advance,
> 
> Christian
> 

Re: [Linux PPC] Disable PREEMPT

[Peter Zijlstra]

On Wed, Feb 25, 2026 at 08:16:46AM +0100, Christophe Leroy (CS GROUP) wrote:
> +Peter
> 
> Hi Christian,
> 
> Le 25/02/2026 à 06:54, Christian Zigotzky a écrit :
> > Hello,
> > 
> > We were previously able to disable PREEMPT in the kernel configuration,
> > but the latest kernels now enable it by default and it is no longer
> > possible to disable it.
> > 
> > Is it possible to let us decide whether to activate PREEMPT or not?
> 
> Can you give more details on why you want to be able to deactivate PREEMPT ?
> 
> Read https://github.com/torvalds/linux/commit/7dadeaa6e851e

Right. So to recap, there are and will be more cases where relying on
preemption is mandatory. 

The typical example is huge page memset/memcpy. Currently that is done
in single page chunks with cond_resched() in between, but it is much
better (and faster) to do a full huge page in one go.

Combine this with the endless stream of mostly random cond_resched()
placements due to $reason, and it is all a bit of a maintenance
nightmare.

I suppose the question is; why do you think you require non-preempt
kernels?

[Linux PPC] Disable PREEMPT

[Christian Zigotzky]

On 25/02/26 08:16, Christophe Leroy (CS GROUP) wrote:
> +Peter
>
> Hi Christian,
>
> Le 25/02/2026 à 06:54, Christian Zigotzky a écrit :
>> Hello,
>>
>> We were previously able to disable PREEMPT in the kernel 
>> configuration, but the latest kernels now enable it by default and it 
>> is no longer possible to disable it.
>>
>> Is it possible to let us decide whether to activate PREEMPT or not?
>
> Can you give more details on why you want to be able to deactivate 
> PREEMPT ?
>
> Read https://github.com/torvalds/linux/commit/7dadeaa6e851e
I use PREEMPT_NONE kernels on my Linux PPC servers. (Without desktop)
>
>
>>
>> Currently we use the following revert patch for our non-preempt kernels.
>>
>> --- a/kernel/Kconfig.preempt    2026-02-14 10:06:32.000000000 +0000
>> +++ b/kernel/Kconfig.preempt    2026-02-08 21:03:27.000000000 +0000
>> @@ -16,13 +16,11 @@ config ARCH_HAS_PREEMPT_LAZY
>>
>>   choice
>>       prompt "Preemption Model"
>> -    default PREEMPT_LAZY if ARCH_HAS_PREEMPT_LAZY
>>       default PREEMPT_NONE
>>
>>   config PREEMPT_NONE
>>       bool "No Forced Preemption (Server)"
>>       depends on !PREEMPT_RT
>> -    depends on ARCH_NO_PREEMPT
>>       select PREEMPT_NONE_BUILD if !PREEMPT_DYNAMIC
>>       help
>>         This is the traditional Linux preemption model, geared towards
>> @@ -37,7 +35,6 @@ config PREEMPT_NONE
>>
>>   config PREEMPT_VOLUNTARY
>>       bool "Voluntary Kernel Preemption (Desktop)"
>> -    depends on !ARCH_HAS_PREEMPT_LAZY
>>       depends on !ARCH_NO_PREEMPT
>>       depends on !PREEMPT_RT
>>       select PREEMPT_VOLUNTARY_BUILD if !PREEMPT_DYNAMIC
>>
>> ---
>>
>> Thanks in advance,
>>
>> Christian
>>
>


-- 
Sent with BrassMonkey 33.9.1 (https://github.com/chzigotzky/Web-Browsers-and-Suites-for-Linux-PPC/releases/tag/BrassMonkey_33.9.1)

Re: [Linux PPC] Disable PREEMPT

[Christophe Leroy (CS GROUP)]

Hi Christian,

Le 26/02/2026 à 06:37, Christian Zigotzky a écrit :
> On 25/02/26 08:16, Christophe Leroy (CS GROUP) wrote:
>> +Peter
>>
>> Hi Christian,
>>
>> Le 25/02/2026 à 06:54, Christian Zigotzky a écrit :
>>> Hello,
>>>
>>> We were previously able to disable PREEMPT in the kernel 
>>> configuration, but the latest kernels now enable it by default and it 
>>> is no longer possible to disable it.
>>>
>>> Is it possible to let us decide whether to activate PREEMPT or not?
>>
>> Can you give more details on why you want to be able to deactivate 
>> PREEMPT ?
>>
>> Read https://eur01.safelinks.protection.outlook.com/? 
>> url=https%3A%2F%2Fgithub.com%2Ftorvalds%2Flinux%2Fcommit%2F7dadeaa6e851e&data=05%7C02%7Cchristophe.leroy2%40cs-soprasteria.com%7C7006c8b4b98c4315a7e508de74f9388a%7C8b87af7d86474dc78df45f69a2011bb5%7C0%7C0%7C639076810956939768%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=6xLB%2FOVSkmgZRamaBM9zaxm%2FUTk9ssf4409QJdiXkDg%3D&reserved=0
> I use PREEMPT_NONE kernels on my Linux PPC servers. (Without desktop)

You are doing that today because it is possible. Tomorrow it will not be 
possible anymore, for the reasons explained by Peter.

My question was not to know what you are doing but why, and why you 
think you still need PREEMPT_NONE and why the modern PREEMPT_LAZY is not 
good enough for you.

Christophe

>>
>>
>>>
>>> Currently we use the following revert patch for our non-preempt kernels.
>>>
>>> --- a/kernel/Kconfig.preempt    2026-02-14 10:06:32.000000000 +0000
>>> +++ b/kernel/Kconfig.preempt    2026-02-08 21:03:27.000000000 +0000
>>> @@ -16,13 +16,11 @@ config ARCH_HAS_PREEMPT_LAZY
>>>
>>>   choice
>>>       prompt "Preemption Model"
>>> -    default PREEMPT_LAZY if ARCH_HAS_PREEMPT_LAZY
>>>       default PREEMPT_NONE
>>>
>>>   config PREEMPT_NONE
>>>       bool "No Forced Preemption (Server)"
>>>       depends on !PREEMPT_RT
>>> -    depends on ARCH_NO_PREEMPT
>>>       select PREEMPT_NONE_BUILD if !PREEMPT_DYNAMIC
>>>       help
>>>         This is the traditional Linux preemption model, geared towards
>>> @@ -37,7 +35,6 @@ config PREEMPT_NONE
>>>
>>>   config PREEMPT_VOLUNTARY
>>>       bool "Voluntary Kernel Preemption (Desktop)"
>>> -    depends on !ARCH_HAS_PREEMPT_LAZY
>>>       depends on !ARCH_NO_PREEMPT
>>>       depends on !PREEMPT_RT
>>>       select PREEMPT_VOLUNTARY_BUILD if !PREEMPT_DYNAMIC
>>>
>>> ---
>>>
>>> Thanks in advance,
>>>
>>> Christian
>>>
>>
> 
> 

Re: [Linux PPC] Disable PREEMPT

[Michal Suchánek]

On Thu, Feb 26, 2026 at 02:24:50PM +0100, Christophe Leroy (CS GROUP) wrote:
> Hi Christian,
> 
> Le 26/02/2026 à 06:37, Christian Zigotzky a écrit :
> > On 25/02/26 08:16, Christophe Leroy (CS GROUP) wrote:
> > > +Peter
> > > 
> > > Hi Christian,
> > > 
> > > Le 25/02/2026 à 06:54, Christian Zigotzky a écrit :
> > > > Hello,
> > > > 
> > > > We were previously able to disable PREEMPT in the kernel
> > > > configuration, but the latest kernels now enable it by default
> > > > and it is no longer possible to disable it.
> > > > 
> > > > Is it possible to let us decide whether to activate PREEMPT or not?
> > > 
> > > Can you give more details on why you want to be able to deactivate
> > > PREEMPT ?
> > > 
> > > Read https://eur01.safelinks.protection.outlook.com/? url=https%3A%2F%2Fgithub.com%2Ftorvalds%2Flinux%2Fcommit%2F7dadeaa6e851e&data=05%7C02%7Cchristophe.leroy2%40cs-soprasteria.com%7C7006c8b4b98c4315a7e508de74f9388a%7C8b87af7d86474dc78df45f69a2011bb5%7C0%7C0%7C639076810956939768%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=6xLB%2FOVSkmgZRamaBM9zaxm%2FUTk9ssf4409QJdiXkDg%3D&reserved=0
> > I use PREEMPT_NONE kernels on my Linux PPC servers. (Without desktop)
> 
> You are doing that today because it is possible. Tomorrow it will not be
> possible anymore, for the reasons explained by Peter.
> 
> My question was not to know what you are doing but why, and why you think
> you still need PREEMPT_NONE and why the modern PREEMPT_LAZY is not good
> enough for you.

Hello,

I tried to compile a kernel with preemption for pSeries, and while
PREEMPT_VOLUNTARY works for me PREEMPT_LAZY crashes.

The kernel crashing is certainly a reason why a preemtion model would
not be good enough for me.

This is not the latest kernel so it might have been fixed in the
meantime for the platform in question. Will need to check that.

Nonethless, there are quite a few platforms. Originally no preemption
was the only option, and that's the reason why many people run that.
It's the conservative, known working option. And that's the reason a lot
of platfrom code does not get tested with more aggressive preemtion
models, and never gets fixed to work with them.

Simply disabling the no preemtion option does not make the platform code
ready.

Thanks

Michal

> 
> Christophe
> 
> > > 
> > > 
> > > > 
> > > > Currently we use the following revert patch for our non-preempt kernels.
> > > > 
> > > > --- a/kernel/Kconfig.preempt    2026-02-14 10:06:32.000000000 +0000
> > > > +++ b/kernel/Kconfig.preempt    2026-02-08 21:03:27.000000000 +0000
> > > > @@ -16,13 +16,11 @@ config ARCH_HAS_PREEMPT_LAZY
> > > > 
> > > >   choice
> > > >       prompt "Preemption Model"
> > > > -    default PREEMPT_LAZY if ARCH_HAS_PREEMPT_LAZY
> > > >       default PREEMPT_NONE
> > > > 
> > > >   config PREEMPT_NONE
> > > >       bool "No Forced Preemption (Server)"
> > > >       depends on !PREEMPT_RT
> > > > -    depends on ARCH_NO_PREEMPT
> > > >       select PREEMPT_NONE_BUILD if !PREEMPT_DYNAMIC
> > > >       help
> > > >         This is the traditional Linux preemption model, geared towards
> > > > @@ -37,7 +35,6 @@ config PREEMPT_NONE
> > > > 
> > > >   config PREEMPT_VOLUNTARY
> > > >       bool "Voluntary Kernel Preemption (Desktop)"
> > > > -    depends on !ARCH_HAS_PREEMPT_LAZY
> > > >       depends on !ARCH_NO_PREEMPT
> > > >       depends on !PREEMPT_RT
> > > >       select PREEMPT_VOLUNTARY_BUILD if !PREEMPT_DYNAMIC
> > > > 
> > > > ---
> > > > 
> > > > Thanks in advance,
> > > > 
> > > > Christian
> > > > 
> > > 
> > 
> > 
> 

Re: [Linux PPC] Disable PREEMPT

[Shrikanth Hegde]

Hi Michal.

On 9/3/26 3:23 PM, Michal Suchánek wrote:
> On Thu, Feb 26, 2026 at 02:24:50PM +0100, Christophe Leroy (CS GROUP) wrote:
>> Hi Christian,
>>
>> Le 26/02/2026 à 06:37, Christian Zigotzky a écrit :
>>> On 25/02/26 08:16, Christophe Leroy (CS GROUP) wrote:
>>>> +Peter
>>>>
>>>> Hi Christian,
>>>>
>>>> Le 25/02/2026 à 06:54, Christian Zigotzky a écrit :
>>>>> Hello,
>>>>>
>>>>> We were previously able to disable PREEMPT in the kernel
>>>>> configuration, but the latest kernels now enable it by default
>>>>> and it is no longer possible to disable it.
>>>>>
>>>>> Is it possible to let us decide whether to activate PREEMPT or not?
>>>>
>>>> Can you give more details on why you want to be able to deactivate
>>>> PREEMPT ?
>>>>
>>>> Read https://eur01.safelinks.protection.outlook.com/? url=https%3A%2F%2Fgithub.com%2Ftorvalds%2Flinux%2Fcommit%2F7dadeaa6e851e&data=05%7C02%7Cchristophe.leroy2%40cs-soprasteria.com%7C7006c8b4b98c4315a7e508de74f9388a%7C8b87af7d86474dc78df45f69a2011bb5%7C0%7C0%7C639076810956939768%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=6xLB%2FOVSkmgZRamaBM9zaxm%2FUTk9ssf4409QJdiXkDg%3D&reserved=0
>>> I use PREEMPT_NONE kernels on my Linux PPC servers. (Without desktop)
>>
>> You are doing that today because it is possible. Tomorrow it will not be
>> possible anymore, for the reasons explained by Peter.
>>
>> My question was not to know what you are doing but why, and why you think
>> you still need PREEMPT_NONE and why the modern PREEMPT_LAZY is not good
>> enough for you.
> 
> Hello,
> 
> I tried to compile a kernel with preemption for pSeries, and while
> PREEMPT_VOLUNTARY works for me PREEMPT_LAZY crashes.
> 
> The kernel crashing is certainly a reason why a preemtion model would
> not be good enough for me.
> 

Is it the same issue that you said the bug which was discussed internally?
i.e around kvmhv_run_single_vcpu ? or is it different one?

> This is not the latest kernel so it might have been fixed in the
> meantime for the platform in question. Will need to check that.
> 

If possible run against current upstream and share the results.

> Nonethless, there are quite a few platforms. Originally no preemption
> was the only option, and that's the reason why many people run that.
> It's the conservative, known working option. And that's the reason a lot
> of platfrom code does not get tested with more aggressive preemtion
> models, and never gets fixed to work with them.
> 

Full preemption has been there for many years!.
Lazy is not that aggressive compared to that.

> Simply disabling the no preemtion option does not make the platform code
> ready.
> 

Let's understand your crash case. Let's see where it is going wrong. I am suspecting
it is some wrong usage of preemption api rather than arch can't support preemption.

> Thanks
> 
> Michal
> 
>>
>> Christophe
>>
>>>>
>>>>
>>>>>
>>>>> Currently we use the following revert patch for our non-preempt kernels.
>>>>>
>>>>> --- a/kernel/Kconfig.preempt    2026-02-14 10:06:32.000000000 +0000
>>>>> +++ b/kernel/Kconfig.preempt    2026-02-08 21:03:27.000000000 +0000
>>>>> @@ -16,13 +16,11 @@ config ARCH_HAS_PREEMPT_LAZY
>>>>>
>>>>>    choice
>>>>>        prompt "Preemption Model"
>>>>> -    default PREEMPT_LAZY if ARCH_HAS_PREEMPT_LAZY
>>>>>        default PREEMPT_NONE
>>>>>
>>>>>    config PREEMPT_NONE
>>>>>        bool "No Forced Preemption (Server)"
>>>>>        depends on !PREEMPT_RT
>>>>> -    depends on ARCH_NO_PREEMPT
>>>>>        select PREEMPT_NONE_BUILD if !PREEMPT_DYNAMIC
>>>>>        help
>>>>>          This is the traditional Linux preemption model, geared towards
>>>>> @@ -37,7 +35,6 @@ config PREEMPT_NONE
>>>>>
>>>>>    config PREEMPT_VOLUNTARY
>>>>>        bool "Voluntary Kernel Preemption (Desktop)"
>>>>> -    depends on !ARCH_HAS_PREEMPT_LAZY
>>>>>        depends on !ARCH_NO_PREEMPT
>>>>>        depends on !PREEMPT_RT
>>>>>        select PREEMPT_VOLUNTARY_BUILD if !PREEMPT_DYNAMIC
>>>>>
>>>>> ---
>>>>>
>>>>> Thanks in advance,
>>>>>
>>>>> Christian
>>>>>
>>>>
>>>
>>>
>>

Re: [Linux PPC] Disable PREEMPT

[Michal Suchánek]

On Thu, Sep 03, 2026 at 04:26:21PM +0530, Shrikanth Hegde wrote:
> Hi Michal.
> 
> On 9/3/26 3:23 PM, Michal Suchánek wrote:
> > On Thu, Feb 26, 2026 at 02:24:50PM +0100, Christophe Leroy (CS GROUP) wrote:
> > > Hi Christian,
> > > 
> > > Le 26/02/2026 à 06:37, Christian Zigotzky a écrit :
> > > > On 25/02/26 08:16, Christophe Leroy (CS GROUP) wrote:
> > > > > +Peter
> > > > > 
> > > > > Hi Christian,
> > > > > 
> > > > > Le 25/02/2026 à 06:54, Christian Zigotzky a écrit :
> > > > > > Hello,
> > > > > > 
> > > > > > We were previously able to disable PREEMPT in the kernel
> > > > > > configuration, but the latest kernels now enable it by default
> > > > > > and it is no longer possible to disable it.
> > > > > > 
> > > > > > Is it possible to let us decide whether to activate PREEMPT or not?
> > > > > 
> > > > > Can you give more details on why you want to be able to deactivate
> > > > > PREEMPT ?
> > > > > 
> > > > > Read https://eur01.safelinks.protection.outlook.com/? url=https%3A%2F%2Fgithub.com%2Ftorvalds%2Flinux%2Fcommit%2F7dadeaa6e851e&data=05%7C02%7Cchristophe.leroy2%40cs-soprasteria.com%7C7006c8b4b98c4315a7e508de74f9388a%7C8b87af7d86474dc78df45f69a2011bb5%7C0%7C0%7C639076810956939768%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=6xLB%2FOVSkmgZRamaBM9zaxm%2FUTk9ssf4409QJdiXkDg%3D&reserved=0
> > > > I use PREEMPT_NONE kernels on my Linux PPC servers. (Without desktop)
> > > 
> > > You are doing that today because it is possible. Tomorrow it will not be
> > > possible anymore, for the reasons explained by Peter.
> > > 
> > > My question was not to know what you are doing but why, and why you think
> > > you still need PREEMPT_NONE and why the modern PREEMPT_LAZY is not good
> > > enough for you.
> > 
> > Hello,
> > 
> > I tried to compile a kernel with preemption for pSeries, and while
> > PREEMPT_VOLUNTARY works for me PREEMPT_LAZY crashes.
> > 
> > The kernel crashing is certainly a reason why a preemtion model would
> > not be good enough for me.
> > 
> 
> Is it the same issue that you said the bug which was discussed internally?
> i.e around kvmhv_run_single_vcpu ? or is it different one?

Right, the very same

> 
> > This is not the latest kernel so it might have been fixed in the
> > meantime for the platform in question. Will need to check that.
> > 
> 
> If possible run against current upstream and share the results.

https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default

[  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
[  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
[  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
[  472.091624][ T6181] Call Trace:
[  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
[  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
[  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
[  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
[  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
[  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
[  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
[  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
[  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
[  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
[  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
[  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
[  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
[  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
[  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
[  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
[  472.091938][ T6181] IRQMASK: 0 
[  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa 
[  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000 
[  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000 
[  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0 
[  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7 
[  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000 
[  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70 
[  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000 
[  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
[  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
[  472.092033][ T6181] ---- interrupt: 3000
[  472.098256][ T6181] pstore: backend (nvram) writing error (-1)

This is the host, cannot run the kernel as guest because it fails to boot most
of the time inside KVM.

> 
> > Nonethless, there are quite a few platforms. Originally no preemption
> > was the only option, and that's the reason why many people run that.
> > It's the conservative, known working option. And that's the reason a lot
> > of platfrom code does not get tested with more aggressive preemtion
> > models, and never gets fixed to work with them.
> > 
> 
> Full preemption has been there for many years!.

Possible for years, forced only recently.

> Lazy is not that aggressive compared to that.
> 
> > Simply disabling the no preemtion option does not make the platform code
> > ready.
> > 
> 
> Let's understand your crash case. Let's see where it is going wrong. I am suspecting
> it is some wrong usage of preemption api rather than arch can't support preemption.

Very likely some wrong use of the preemption API by the arch code, or no
use where it should have been used. It did not matter so long as people
could run their no preempt configs and ignore the problem.

Thanks

Michal

Re: [Linux PPC] Disable PREEMPT

[Shrikanth Hegde]

Hi Michal.

>> Is it the same issue that you said the bug which was discussed internally?
>> i.e around kvmhv_run_single_vcpu ? or is it different one?
> 
> Right, the very same
> 

Thanks.

>>
>>> This is not the latest kernel so it might have been fixed in the
>>> meantime for the platform in question. Will need to check that.
>>>
>>
>> If possible run against current upstream and share the results.
> 
> https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default
> 
> [  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> [  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> [  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> [  472.091624][ T6181] Call Trace:
> [  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
> [  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
> [  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
> [  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
> [  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> [  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
> [  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
> [  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
> [  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
> [  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
> [  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
> [  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
> [  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
> [  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
> [  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
> [  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
> [  472.091938][ T6181] IRQMASK: 0
> [  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa
> [  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000
> [  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000
> [  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0
> [  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7
> [  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000
> [  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70
> [  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000
> [  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
> [  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
> [  472.092033][ T6181] ---- interrupt: 3000
> [  472.098256][ T6181] pstore: backend (nvram) writing error (-1)
> 
> This is the host, cannot run the kernel as guest because it fails to boot most
> of the time inside KVM.
> 

Will look into. btw the patch shared i.e below patch fixes the boot hanf issue?

https://lore.kernel.org/all/1d79ec6f-ce7f-4432-80e0-a54fa1668642@linux.ibm.com/

>>
>>> Nonethless, there are quite a few platforms. Originally no preemption
>>> was the only option, and that's the reason why many people run that.
>>> It's the conservative, known working option. And that's the reason a lot
>>> of platfrom code does not get tested with more aggressive preemtion
>>> models, and never gets fixed to work with them.
>>>
>>
>> Full preemption has been there for many years!.
> 
> Possible for years, forced only recently.
> 
>> Lazy is not that aggressive compared to that.
>>
>>> Simply disabling the no preemtion option does not make the platform code
>>> ready.
>>>
>>
>> Let's understand your crash case. Let's see where it is going wrong. I am suspecting
>> it is some wrong usage of preemption api rather than arch can't support preemption.
> 
> Very likely some wrong use of the preemption API by the arch code, or no
> use where it should have been used. It did not matter so long as people
> could run their no preempt configs and ignore the problem.
> 
> Thanks
> 
> Michal

Re: [Linux PPC] Disable PREEMPT

[Michal Suchánek]

On Thu, Sep 03, 2026 at 05:01:13PM +0530, Shrikanth Hegde wrote:

> 
> > > 
> > > > This is not the latest kernel so it might have been fixed in the
> > > > meantime for the platform in question. Will need to check that.
> > > > 
> > > 
> > > If possible run against current upstream and share the results.
> > 
> > https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default
> > 
> > [  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > [  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> > [  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> > [  472.091624][ T6181] Call Trace:
> > [  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
> > [  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
> > [  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
> > [  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
> > [  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > [  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
> > [  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
> > [  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
> > [  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
> > [  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
> > [  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
> > [  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
> > [  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
> > [  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
> > [  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
> > [  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
> > [  472.091938][ T6181] IRQMASK: 0
> > [  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa
> > [  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000
> > [  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000
> > [  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0
> > [  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7
> > [  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000
> > [  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70
> > [  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000
> > [  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
> > [  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
> > [  472.092033][ T6181] ---- interrupt: 3000
> > [  472.098256][ T6181] pstore: backend (nvram) writing error (-1)
> > 
> > This is the host, cannot run the kernel as guest because it fails to boot most
> > of the time inside KVM.
> > 
> 
> Will look into. btw the patch shared i.e below patch fixes the boot hanf issue?
> 
> https://lore.kernel.org/all/1d79ec6f-ce7f-4432-80e0-a54fa1668642@linux.ibm.com/

I would not bet on that. It looks like host side patch, and the problem
is with booting 7.2.2 as guest. Some 6.12 frankenkernel can be booted on
the same host, and used to produce the crash above by running a kernel
build inside the guest.

Thanks

Michal

Re: [Linux PPC] Disable PREEMPT

[Shrikanth Hegde]

Hi Michal.

On 9/3/26 5:10 PM, Michal Suchánek wrote:
> On Thu, Sep 03, 2026 at 05:01:13PM +0530, Shrikanth Hegde wrote:
> 
>>
>>>>
>>>>> This is not the latest kernel so it might have been fixed in the
>>>>> meantime for the platform in question. Will need to check that.
>>>>>
>>>>
>>>> If possible run against current upstream and share the results.
>>>
>>> https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default
>>>
>>> [  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
>>> [  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
>>> [  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
>>> [  472.091624][ T6181] Call Trace:
>>> [  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
>>> [  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
>>> [  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
>>> [  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
>>> [  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
>>> [  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
>>> [  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
>>> [  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
>>> [  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
>>> [  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
>>> [  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
>>> [  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
>>> [  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
>>> [  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
>>> [  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
>>> [  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
>>> [  472.091938][ T6181] IRQMASK: 0
>>> [  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa
>>> [  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000
>>> [  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000
>>> [  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0
>>> [  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7
>>> [  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000
>>> [  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70
>>> [  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000
>>> [  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
>>> [  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
>>> [  472.092033][ T6181] ---- interrupt: 3000
>>> [  472.098256][ T6181] pstore: backend (nvram) writing error (-1)
>>>
>>> This is the host, cannot run the kernel as guest because it fails to boot most
>>> of the time inside KVM.
>>>
>>
>> Will look into. btw the patch shared i.e below patch fixes the boot hanf issue?
>>
>> https://lore.kernel.org/all/1d79ec6f-ce7f-4432-80e0-a54fa1668642@linux.ibm.com/
> 

I see code has changed a bit. So this patch is not correct anymore.
Sorry, i should have checked.

> I would not bet on that. It looks like host side patch, and the problem
> is with booting 7.2.2 as guest. Some 6.12 frankenkernel can be booted on
> the same host, and used to produce the crash above by running a kernel
> build inside the guest.
> 

I didn't get what you are saying. Could you see if it regression only in 7.2?
Does 7.1 works fine? If it is regression could you see if it is bisectable?
Also, could you share the repro steps?

Let me add harsh. He would know more on this.

> Thanks
> 
> Michal

Re: [Linux PPC] Disable PREEMPT

[Michal Suchánek]

On Thu, Sep 03, 2026 at 05:34:31PM +0530, Shrikanth Hegde wrote:
> Hi Michal.
> 
> On 9/3/26 5:10 PM, Michal Suchánek wrote:
> > On Thu, Sep 03, 2026 at 05:01:13PM +0530, Shrikanth Hegde wrote:
> > 
> > > 
> > > > > 
> > > > > > This is not the latest kernel so it might have been fixed in the
> > > > > > meantime for the platform in question. Will need to check that.
> > > > > > 
> > > > > 
> > > > > If possible run against current upstream and share the results.
> > > > 
> > > > https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default
> > > > 
> > > > [  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > > > [  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> > > > [  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> > > > [  472.091624][ T6181] Call Trace:
> > > > [  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
> > > > [  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
> > > > [  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
> > > > [  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
> > > > [  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > > > [  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
> > > > [  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
> > > > [  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
> > > > [  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
> > > > [  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
> > > > [  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
> > > > [  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
> > > > [  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
> > > > [  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
> > > > [  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
> > > > [  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
> > > > [  472.091938][ T6181] IRQMASK: 0
> > > > [  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa
> > > > [  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000
> > > > [  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000
> > > > [  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0
> > > > [  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7
> > > > [  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000
> > > > [  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70
> > > > [  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000
> > > > [  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
> > > > [  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
> > > > [  472.092033][ T6181] ---- interrupt: 3000
> > > > [  472.098256][ T6181] pstore: backend (nvram) writing error (-1)
> > > > 
> > > > This is the host, cannot run the kernel as guest because it fails to boot most
> > > > of the time inside KVM.
> > > > 
> > > 
> > > Will look into. btw the patch shared i.e below patch fixes the boot hanf issue?
> > > 
> > > https://lore.kernel.org/all/1d79ec6f-ce7f-4432-80e0-a54fa1668642@linux.ibm.com/
> > 
> 
> I see code has changed a bit. So this patch is not correct anymore.
> Sorry, i should have checked.
> 
> > I would not bet on that. It looks like host side patch, and the problem
> > is with booting 7.2.2 as guest. Some 6.12 frankenkernel can be booted on
> > the same host, and used to produce the crash above by running a kernel
> > build inside the guest.
> > 
> 
> I didn't get what you are saying. Could you see if it regression only in 7.2?
> Does 7.1 works fine? If it is regression could you see if it is bisectable?
> Also, could you share the repro steps?

The repro steps are: on a mid-size LPAR (120 CPU threads, 256G RAM)
create a KVM VM that uses all host resources. Run kernel build inside
KVM VM, using all CPUs. In about a minute the host crashes with the
above a or similar trace. I don't have any kernel version with aggressive
preemption enabled that does not suffer from this problem.

There is another problem that Linux 7.2.2 often locks up on boot inside
the guest, and as a result a different kernel is needed in the guest to
reproduce the host crash. This is more recent, linux 6.12 does not have
this problem. Will try some different kernel combinations to see if I
can narrow this down. For this the repro is merely booting the kernel in
the VM, it typically fails early before userspace produces any messages.

Thanks

Michal

Re: [Linux PPC] Disable PREEMPT

[Harsh Prateek Bora]

+ Amit, Gautam - since they recently came across a similar regression 
which is seen with non KVM workload as well.
+ Venkat - for CI improvement suggestion below.

Hi Michal,

On 03/09/26 5:55 pm, Michal Suchánek wrote:
> On Thu, Sep 03, 2026 at 05:34:31PM +0530, Shrikanth Hegde wrote:
>> Hi Michal.
>>
>> On 9/3/26 5:10 PM, Michal Suchánek wrote:
>>> On Thu, Sep 03, 2026 at 05:01:13PM +0530, Shrikanth Hegde wrote:
>>>
>>>>
>>>>>>
>>>>>>> This is not the latest kernel so it might have been fixed in the
>>>>>>> meantime for the platform in question. Will need to check that.
>>>>>>>
>>>>>>
>>>>>> If possible run against current upstream and share the results.
>>>>>
>>>>> https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default
>>>>>
>>>>> [  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
>>>>> [  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
>>>>> [  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
>>>>> [  472.091624][ T6181] Call Trace:
>>>>> [  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
>>>>> [  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
>>>>> [  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
>>>>> [  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
>>>>> [  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
>>>>> [  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
>>>>> [  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
>>>>> [  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
>>>>> [  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
>>>>> [  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
>>>>> [  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
>>>>> [  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
>>>>> [  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
>>>>> [  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
>>>>> [  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
>>>>> [  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
>>>>> [  472.091938][ T6181] IRQMASK: 0
>>>>> [  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa
>>>>> [  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000
>>>>> [  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000
>>>>> [  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0
>>>>> [  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7
>>>>> [  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000
>>>>> [  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70
>>>>> [  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000
>>>>> [  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
>>>>> [  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
>>>>> [  472.092033][ T6181] ---- interrupt: 3000
>>>>> [  472.098256][ T6181] pstore: backend (nvram) writing error (-1)
>>>>>
>>>>> This is the host, cannot run the kernel as guest because it fails to boot most
>>>>> of the time inside KVM.
>>>>>
>>>>
>>>> Will look into. btw the patch shared i.e below patch fixes the boot hanf issue?
>>>>
>>>> https://lore.kernel.org/all/1d79ec6f-ce7f-4432-80e0-a54fa1668642@linux.ibm.com/
>>>
>>
>> I see code has changed a bit. So this patch is not correct anymore.
>> Sorry, i should have checked.
>>
>>> I would not bet on that. It looks like host side patch, and the problem
>>> is with booting 7.2.2 as guest. Some 6.12 frankenkernel can be booted on
>>> the same host, and used to produce the crash above by running a kernel
>>> build inside the guest.
>>>
>>
>> I didn't get what you are saying. Could you see if it regression only in 7.2?
>> Does 7.1 works fine? If it is regression could you see if it is bisectable?
>> Also, could you share the repro steps?
> 
> The repro steps are: on a mid-size LPAR (120 CPU threads, 256G RAM)
> create a KVM VM that uses all host resources. Run kernel build inside
> KVM VM, using all CPUs. In about a minute the host crashes with the
> above a or similar trace. I don't have any kernel version with aggressive
> preemption enabled that does not suffer from this problem.

Looks like seen under heavy workloads when you say "KVM VM that uses all 
host rsources". Could you please confirm if the issue is seen with any 
other heavy workload also which consumes all host resources? I do not 
think we recently had any PPC KVM specific changes which would cause 
this. Also, if it's a regression, git bisect should point out the real 
culprit.

Venkat,
Can we include such a test case in upstream CI as well?

regards,
Harsh

> 
> There is another problem that Linux 7.2.2 often locks up on boot inside
> the guest, and as a result a different kernel is needed in the guest to
> reproduce the host crash. This is more recent, linux 6.12 does not have
> this problem. Will try some different kernel combinations to see if I
> can narrow this down. For this the repro is merely booting the kernel in
> the VM, it typically fails early before userspace produces any messages.
> 
> Thanks
> 
> Michal

Re: [Linux PPC] Disable PREEMPT

[Michal Suchánek]

On Thu, Sep 03, 2026 at 06:06:16PM +0530, Harsh Prateek Bora wrote:
> + Amit, Gautam - since they recently came across a similar regression which
> is seen with non KVM workload as well.
> + Venkat - for CI improvement suggestion below.
> 
> Hi Michal,
> 
> On 03/09/26 5:55 pm, Michal Suchánek wrote:
> > On Thu, Sep 03, 2026 at 05:34:31PM +0530, Shrikanth Hegde wrote:
> > > Hi Michal.
> > > 
> > > On 9/3/26 5:10 PM, Michal Suchánek wrote:
> > > > On Thu, Sep 03, 2026 at 05:01:13PM +0530, Shrikanth Hegde wrote:
> > > > 
> > > > > 
> > > > > > > 
> > > > > > > > This is not the latest kernel so it might have been fixed in the
> > > > > > > > meantime for the platform in question. Will need to check that.
> > > > > > > > 
> > > > > > > 
> > > > > > > If possible run against current upstream and share the results.
> > > > > > 
> > > > > > https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default
> > > > > > 
> > > > > > [  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > > > > > [  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> > > > > > [  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> > > > > > [  472.091624][ T6181] Call Trace:
> > > > > > [  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
> > > > > > [  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
> > > > > > [  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
> > > > > > [  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
> > > > > > [  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > > > > > [  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
> > > > > > [  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
> > > > > > [  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
> > > > > > [  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
> > > > > > [  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
> > > > > > [  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
> > > > > > [  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
> > > > > > [  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
> > > > > > [  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
> > > > > > [  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
> > > > > > [  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
> > > > > > [  472.091938][ T6181] IRQMASK: 0
> > > > > > [  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa
> > > > > > [  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000
> > > > > > [  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000
> > > > > > [  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0
> > > > > > [  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7
> > > > > > [  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000
> > > > > > [  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70
> > > > > > [  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000
> > > > > > [  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
> > > > > > [  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
> > > > > > [  472.092033][ T6181] ---- interrupt: 3000
> > > > > > [  472.098256][ T6181] pstore: backend (nvram) writing error (-1)
> > > > > > 
> > > > > > This is the host, cannot run the kernel as guest because it fails to boot most
> > > > > > of the time inside KVM.
> > > > > > 
> > > > > 
> > > > > Will look into. btw the patch shared i.e below patch fixes the boot hanf issue?
> > > > > 
> > > > > https://lore.kernel.org/all/1d79ec6f-ce7f-4432-80e0-a54fa1668642@linux.ibm.com/
> > > > 
> > > 
> > > I see code has changed a bit. So this patch is not correct anymore.
> > > Sorry, i should have checked.
> > > 
> > > > I would not bet on that. It looks like host side patch, and the problem
> > > > is with booting 7.2.2 as guest. Some 6.12 frankenkernel can be booted on
> > > > the same host, and used to produce the crash above by running a kernel
> > > > build inside the guest.
> > > > 
> > > 
> > > I didn't get what you are saying. Could you see if it regression only in 7.2?
> > > Does 7.1 works fine? If it is regression could you see if it is bisectable?
> > > Also, could you share the repro steps?
> > 
> > The repro steps are: on a mid-size LPAR (120 CPU threads, 256G RAM)
> > create a KVM VM that uses all host resources. Run kernel build inside
> > KVM VM, using all CPUs. In about a minute the host crashes with the
> > above a or similar trace. I don't have any kernel version with aggressive
> > preemption enabled that does not suffer from this problem.
> 
> Looks like seen under heavy workloads when you say "KVM VM that uses all
> host rsources". Could you please confirm if the issue is seen with any other
> heavy workload also which consumes all host resources? I do not think we
> recently had any PPC KVM specific changes which would cause this. Also, if
> it's a regression, git bisect should point out the real culprit.

It's a regression from switching to PREEMPT_LAZY (from
PREEMPT_VOLUNTARY). Very likely the code was buggy to start with but
because less aggressive preemption model was used the bug was never
triggered. I cannot reproduce this under load without KVM. The other
problem that is much harder to reproduce is machine locking up while
idling. A dump then also shows stack corruption with completely
but different stack trace, no KVM involved in that one. Did not see this
with recent mainline yet.

I do get a problem with 7.2.2 building a kernel outside of kVM but
that's probably because it's missing some of the recent entry fixes:

[ 5451.821931] [  T77619] as[77619]: segfault (11) at 3abd0 nip 7fff8fe56434 lr 7fff8fe55348 code 1 in ld64.so.2[36434,7fff8fe20000+60000]
[ 5451.821967] [  T77619] as[77619]: code: 60000000 60000000 60000000 f9490008 f9490010 39290020 f949fff8 f9490000 
[ 5451.821973] [  T77619] as[77619]: code: 4200ffec 89210376 3d42ffff 394a7ea0 <e9028000> 7d081050 f9410030 f9010020 
[ 5454.239768] [  T78922] awk[78922]: segfault (11) at 3abd0 nip 7fff8c386434 lr 7fff8c385348 code 1 in ld64.so.2[36434,7fff8c350000+60000]
[ 5454.239809] [  T78922] awk[78922]: code: 60000000 60000000 60000000 f9490008 f9490010 39290020 f949fff8 f9490000 
[ 5454.239815] [  T78922] awk[78922]: code: 4200ffec 89210376 3d42ffff 394a7ea0 <e9028000> 7d081050 f9410030 f9010020 
[ 5465.056566] [  T85518] sh[85518]: segfault (11) at 3abd0 nip 7fffb79a6434 lr 7fffb79a5348 code 1 in ld64.so.2[36434,7fffb7970000+60000]
[ 5465.056611] [  T85518] sh[85518]: code: 60000000 60000000 60000000 f9490008 f9490010 39290020 f949fff8 f9490000 
[ 5465.056616] [  T85518] sh[85518]: code: 4200ffec 89210376 3d42ffff 394a7ea0 <e9028000> 7d081050 f9410030 f9010020 
[ 5474.995476] [  T91034] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: sys_ioctl+0xfe0/0x1060
[ 5474.995495] [  T91037] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: vprintk_store+0x6a4/0x6c0
[ 5474.995508] [  T91037] ------------[ cut here ]------------
[ 5474.995513] [  T91037] kernel BUG at [regs_irqs_disabled(regs)] arch/powerpc/include/asm/entry-common.h:401!
[ 5474.995521] [  T91037] Oops: Exception in kernel mode, sig: 5 [#1]
[ 5474.995524] [  T91037] LE PAGE_SIZE=64K MMU=Radix  SMP NR_CPUS=8192 NUMA pSeries
[ 5474.995528] [  T91037] Modules linked in: rpadlpar_io rpaphp kvm_hv kvm mptcp_diag xsk_diag tcp_diag udp_diag raw_diag inet_diag unix_diag af_packet_diag netlink_diag xt_conntrack xt_MASQUERADE xfrm_user xt_set ip_set xt_addrtype nft_compat nf_conntrack_netbios_ns nf_conntrack_broadcast nft_masq nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct bonding tls nft_chain_nat nf_nat nf_conntrack tun nf_defrag_ipv6 nf_defrag_ipv4 rfkill af_packet binfmt_misc xfs aes_gcm_p10_crypto ibmveth crypto_simd cryptd xts vmx_crypto rtc_generic fuse nvme_fabrics ip_tables x_tables ibmvscsi scsi_transport_srp sd_mod scsi_dh_emc scsi_dh_rdac scsi_dh_alua sg scsi_mod scsi_common nvme nvme_core nvme_keyring pseries_wdt nvme_auth btrfs xor libblake2b raid6_pq dm_mirror dm_region_hash dm_log dm_mod br_netfilter bridge stp llc nf_tables
[ 5474.995641] [  T91037] CPU: 110 UID: 399 PID: 91037 Comm: cc1 Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
[ 5474.995649] [  T91037] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
[ 5474.995653] [  T91037] NIP:  c0000000013c2114 LR: c000000000148c88 CTR: c000000000008e40
[ 5474.995656] [  T91037] REGS: c000001f965d3b50 TRAP: 0700   Not tainted  (7.2.2-5.g6824496-default)
[ 5474.995660] [  T91037] MSR:  8000000000029033 <SF,EE,ME,IR,DR,RI,LE>  CR: 42004208  XER: 20040145
[ 5474.995670] [  T91037] CFAR: c0000000013c20b4 IRQMASK: 3 
                          GPR00: c00000000000904c c000001f965d3df0 c00000000205ad00 c000001f965d3e80 
                          GPR04: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
                          GPR08: 0000000000000000 0000000000000001 0000000000004000 0000000053474552 
                          GPR12: 0000000000000000 c000003ffef56300 0000000000000000 0000000000000000 
                          GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
                          GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
                          GPR24: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
                          GPR28: 0000000000000000 0000000000000000 0000000000000000 c000001f965d3e80 
[ 5474.995708] [  T91037] NIP [c0000000013c2114] irqentry_enter+0x74/0x1a0
[ 5474.995719] [  T91037] LR [c000000000148c88] do_page_fault+0x68/0x140
[ 5474.995727] [  T91037] Call Trace:
[ 5474.995731] [  T91037] [c000001f965d3e10] [c000001f965d3e50] 0xc000001f965d3e50
[ 5474.995739] [  T91037] [c000001f965d3e50] [c00000000000904c] instruction_access_common_virt+0x20c/0x210
[ 5474.995751] [  T91037] ---- interrupt: 400 at 0x1043f158
[ 5474.995754] [  T91037] NIP:  000000001043f158 LR: 00000000101218e0 CTR: 00007fff7aef4f00
[ 5474.995756] [  T91037] REGS: c000001f965d3e80 TRAP: 0400   Not tainted  (7.2.2-5.g6824496-default)
[ 5474.995759] [  T91037] MSR:  800000004280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 82004208  XER: 20040145
[ 5474.995770] [  T91037] CFAR: 00000000101218dc IRQMASK: 1 
                          GPR00: 00000000101218cc 00007ffffd284ce0 0000000011de7a00 0000000012c2c370 
                          GPR04: 0000000000000001 0000000000000000 0000000000000001 0000000000000000 
                          GPR08: 00007fff7b070d60 0000000000000000 0000000012c2c380 0000000000004000 
                          GPR12: 00007fff7aef4f00 00007fff7ba8da00 0000000000000000 0000000000000000 
                          GPR16: 0000000000000000 00007ffffd28c2ac 0000000000000000 0000000000000001 
                          GPR20: 0000000000000197 0000000000c03cff 0000000011e0d210 0000000000000008 
                          GPR24: 0000000000000000 00007ffffd284f58 0000000000000000 0000000000000000 
                          GPR28: 0000000000000001 00007ffffd28c2ac 0000000000000000 0000000000000197 
[ 5474.995809] [  T91037] NIP [000000001043f158] 0x1043f158
[ 5474.995812] [  T91037] LR [00000000101218e0] 0x101218e0
[ 5474.995814] [  T91037] ---- interrupt: 400
[ 5474.995816] [  T91037] Code: f8010030 4af8b211 60000000 e8010030 7c0803a6 4bffffd4 60420000 7929ffe2 69290001 0b090000 e9230138 792907e0 <0b090000> 4800001c e9230140 792907a4 
[ 5474.995835] [  T91037] ---[ end trace 0000000000000000 ]---
[ 5492.821144] [    C110] watchdog: CPU 110 Hard LOCKUP
[ 5492.821159] [    C110] watchdog: CPU 110 TB:3408898268781396, last heartbeat TB:3408888039246087 (19979ms ago)
[ 5492.821162] [    C110] Modules linked in: rpadlpar_io rpaphp kvm_hv kvm mptcp_diag xsk_diag tcp_diag udp_diag raw_diag inet_diag unix_diag af_packet_diag netlink_diag xt_conntrack xt_MASQUERADE xfrm_user xt_set ip_set xt_addrtype nft_compat nf_conntrack_netbios_ns nf_conntrack_broadcast nft_masq nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct bonding tls nft_chain_nat nf_nat nf_conntrack tun nf_defrag_ipv6 nf_defrag_ipv4 rfkill af_packet binfmt_misc xfs aes_gcm_p10_crypto ibmveth crypto_simd cryptd xts vmx_crypto rtc_generic fuse nvme_fabrics ip_tables x_tables ibmvscsi scsi_transport_srp sd_mod scsi_dh_emc scsi_dh_rdac scsi_dh_alua sg scsi_mod scsi_common nvme nvme_core nvme_keyring pseries_wdt nvme_auth btrfs xor libblake2b raid6_pq dm_mirror dm_region_hash dm_log dm_mod br_netfilter bridge stp llc nf_tables
[ 5492.821225] [    C110] CPU: 110 UID: 399 PID: 91037 Comm: cc1 Tainted: G      D             7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
[ 5492.821234] [    C110] Tainted: [D]=DIE
[ 5492.821235] [    C110] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
[ 5492.821237] [    C110] NIP:  c0000000013d2c90 LR: c0000000013d4564 CTR: c000000000919460
[ 5492.821241] [    C110] REGS: c000003ffeb93d60 TRAP: 0100   Tainted: G      D              (7.2.2-5.g6824496-default)
[ 5492.821244] [    C110] MSR:  8000000000081033 <SF,ME,IR,DR,RI,LE>  CR: 44004204  XER: 20040145
[ 5492.821252] [    C110] CFAR: 000000000000011c IRQMASK: 3 
                          GPR00: c0000000009194f0 c000001f965d3730 c00000000205ad00 c0000000037b6510 
                          GPR04: c000001f965d39d8 00000000000000dd fffffffffffe0000 0000000000010000 
                          GPR08: 0000000000000001 0000000000000001 000000000fe67ee1 0000000000004000 
                          GPR12: c000000000919460 c000003ffef56300 0000000000000000 0000000000000000 
                          GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
                          GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
                          GPR24: 0000000000002800 c0000000018c5670 c000001f965d39d8 0000003ff1ae0000 
                          GPR28: 0000000000de0000 c0000000032bbe80 c000003ff4d9be80 c0000000037b6510 
[ 5492.821283] [    C110] NIP [c0000000013d2c90] queued_spin_lock_slowpath+0x160/0x19a0
[ 5492.821296] [    C110] LR [c0000000013d4564] _raw_spin_lock_irqsave+0x94/0xc0
[ 5492.821299] [    C110] Call Trace:
[ 5492.821302] [    C110] [c000001f965d3830] [0000000000000263] 0x263
[ 5492.821309] [    C110] [c000001f965d3860] [c0000000009194f0] pstore_dump+0x90/0x430
[ 5492.821317] [    C110] [c000001f965d39b0] [c00000000031ff40] kmsg_dump_desc+0xb0/0x100
[ 5492.821325] [    C110] [c000001f965d3a10] [c00000000022e2a4] oops_exit+0x54/0x70
[ 5492.821331] [    C110] [c000001f965d3a70] [c00000000002760c] oops_end+0x6c/0x1a0
[ 5492.821340] [    C110] [c000001f965d3af0] [c00000000002a61c] program_check_exception+0x7c/0xe0
[ 5492.821344] [    C110] [c000001f965d3b20] [c000000000009b2c] program_check_common_virt+0x3bc/0x3c0
[ 5492.821352] [    C110] ---- interrupt: 700 at irqentry_enter+0x74/0x1a0
[ 5492.821358] [    C110] NIP:  c0000000013c2114 LR: c000000000148c88 CTR: c000000000008e40
[ 5492.821361] [    C110] REGS: c000001f965d3b50 TRAP: 0700   Tainted: G      D              (7.2.2-5.g6824496-default)
[ 5492.821363] [    C110] MSR:  8000000000029033 <SF,EE,ME,IR,DR,RI,LE>  CR: 42004208  XER: 20040145
[ 5492.821370] [    C110] CFAR: c0000000013c20b4 IRQMASK: 3 
                          GPR00: c00000000000904c c000001f965d3df0 c00000000205ad00 c000001f965d3e80 
                          GPR04: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
                          GPR08: 0000000000000000 0000000000000001 0000000000004000 0000000053474552 
                          GPR12: 0000000000000000 c000003ffef56300 0000000000000000 0000000000000000 
                          GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
                          GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
                          GPR24: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
                          GPR28: 0000000000000000 0000000000000000 0000000000000000 c000001f965d3e80 
[ 5492.821399] [    C110] NIP [c0000000013c2114] irqentry_enter+0x74/0x1a0
[ 5492.821403] [    C110] LR [c000000000148c88] do_page_fault+0x68/0x140
[ 5492.821409] [    C110] ---- interrupt: 700
[ 5492.821410] [    C110] [c000001f965d3df0] [0000000000000000] 0x0 (unreliable)
[ 5492.821414] [    C110] [c000001f965d3e10] [c000001f965d3e50] 0xc000001f965d3e50
[ 5492.821416] [    C110] [c000001f965d3e50] [c00000000000904c] instruction_access_common_virt+0x20c/0x210
[ 5492.821421] [    C110] ---- interrupt: 400 at 0x1043f158
[ 5492.821423] [    C110] NIP:  000000001043f158 LR: 00000000101218e0 CTR: 00007fff7aef4f00
[ 5492.821425] [    C110] REGS: c000001f965d3e80 TRAP: 0400   Tainted: G      D              (7.2.2-5.g6824496-default)
[ 5492.821428] [    C110] MSR:  800000004000f033 <SF,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 82004208  XER: 20040145
[ 5492.821436] [    C110] CFAR: 00000000101218dc IRQMASK: 1 
                          GPR00: 00000000101218cc 00007ffffd284ce0 0000000011de7a00 0000000012c2c370 
                          GPR04: 0000000000000001 0000000000000000 0000000000000001 0000000000000000 
                          GPR08: 00007fff7b070d60 0000000000000000 0000000012c2c380 0000000000004000 
                          GPR12: 00007fff7aef4f00 00007fff7ba8da00 0000000000000000 0000000000000000 
                          GPR16: 0000000000000000 00007ffffd28c2ac 0000000000000000 0000000000000001 
                          GPR20: 0000000000000197 0000000000c03cff 0000000011e0d210 0000000000000008 
                          GPR24: 0000000000000000 00007ffffd284f58 0000000000000000 0000000000000000 
                          GPR28: 0000000000000001 00007ffffd28c2ac 0000000000000000 0000000000000197 
[ 5492.821464] [    C110] NIP [000000001043f158] 0x1043f158
[ 5492.821466] [    C110] LR [00000000101218e0] 0x101218e0
[ 5492.821468] [    C110] ---- interrupt: 400
[ 5492.821470] [    C110] Code: 7c085040 41810e8c 39400000 39000000 3cc0fffe 60000000 813f0000 71290001 41820050 3ce00001 60000000 60420000 <7c40003c> 71090001 394a0001 40820028 

Thanks

Michal

Re: [Linux PPC] Disable PREEMPT

[Michal Suchánek]

On Thu, Sep 03, 2026 at 06:06:16PM +0530, Harsh Prateek Bora wrote:
> + Amit, Gautam - since they recently came across a similar regression which
> is seen with non KVM workload as well.
> + Venkat - for CI improvement suggestion below.
> 
> Hi Michal,
> 
> On 03/09/26 5:55 pm, Michal Suchánek wrote:

> > 
> > There is another problem that Linux 7.2.2 often locks up on boot inside
> > the guest, and as a result a different kernel is needed in the guest to
> > reproduce the host crash. This is more recent, linux 6.12 does not have
> > this problem. Will try some different kernel combinations to see if I
> > can narrow this down. For this the repro is merely booting the kernel in
> > the VM, it typically fails early before userspace produces any messages.

This one regressed between 6.19.12
https://github.com/openSUSE/kernel-source/blob/9f6830f/config/ppc64le/default
and 7.0.12
https://github.com/openSUSE/kernel-source/blob/2ebf0bc/config/ppc64le/default

Thanks

Michal

SLOF **********************************************************************
QEMU Starting
 Build Date = Aug 13 2026 14:47:36
 FW Version = abuild@OBS release 20230918
 Press "s" to enter Open Firmware.

Populating /vdevice methods
Populating /vdevice/vty@30000000
Populating /vdevice/nvram@71000000
Populating /pci@800000020000000
Loading Linux 7.0.12-1.g2ebf0bc-default ...
Loading initial ramdisk ...                
OF stdout device is: /vdevice/vty@30000000
Preparing to boot Linux version 7.0.12-1.g2ebf0bc-default (geeko@buildhost) (gcc (SUSE Linux) 15.3.0, GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.45.0.20251103-4) #1 SMP PREEMPT_DYNAMIC Mon Jun 15 08:39:32 UTC 2026 (2ebf0bc)
Detected machine type: 0000000000000101
command line: BOOT_IMAGE=/boot/vmlinux-7.0.12-1.g2ebf0bc-default root=UUID=304c7f07-efbb-1070-2f27-accd935ec088 rw quiet systemd.show_status=1 security=selinux selinux=1
Max number of cores passed to firmware: 8192 (NR_CPUS = 8192)
Calling ibm,client-architecture-support... done
memory layout at init:
  memory_limit : 0000000000000000 (16 MB aligned)
  alloc_bottom : 00000000066b0000
  alloc_top    : 0000000030000000
  alloc_top_hi : 0000003e00000000
  rmo_top      : 0000000030000000
  ram_top      : 0000003e00000000
instantiating rtas at 0x000000002fff0000... done
prom_hold_cpus: skipped
copying OF device tree...
Building dt strings...
Building dt structure...
Device tree strings 0x00000000066c0000 -> 0x00000000066c0bec
Device tree struct  0x00000000066d0000 -> 0x00000000066f0000
Quiescing Open Firmware ...
Booting Linux via __start() @ 0x0000000000250000 ...
[    0.000000][    T0] ERROR: Failed to allocate trace buffer
[    0.000000][    T0] ERROR: tracer: failed to allocate ring buffer!
Linux ppc64le
#1 SMP PREEMPT_D[    0.285758][  T673] BUG: Kernel NULL pointer dereference on read at 0x00000010
[    0.285877][  T673] Faulting instruction address: 0xc000000000485bd0
[    0.285903][    T1] VFS: Dquot-cache hash table entries: 8192 (order 0, 65536 bytes)
[    0.285954][  T673] Oops: Kernel access of bad area, sig: 7 [#1]
[    0.286133][  T673] LE PAGE_SIZE=64K MMU=Radix  SMP NR_CPUS=8192 NUMA pSeries
[    0.286240][  T673] Modules linked in:
[    0.286288][  T673] CPU: 15 UID: 0 PID: 673 Comm: kworker/u531:0 Not tainted 7.0.12-1.g2ebf0bc-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  cf0846124d7853fab338aeae87203b36cac18419
[    0.286527][  T673] Hardware name: IBM pSeries (emulated by qemu) Power11 (architected) 0x820200 0xf000007 of:SLOF,HEAD hv:linux,kvm pSeries
[    0.286739][  T673] Workqueue: trace_init_wq tracer_init_tracefs_work_func
[    0.286818][  T673] NIP:  c000000000485bd0 LR: c000000000448864 CTR: c0000000005a3420
[    0.286906][  T673] REGS: c00000000a7a7960 TRAP: 0300   Not tainted  (7.0.12-1.g2ebf0bc-default)
[    0.287014][  T673] MSR:  8000000000009033 <SF,EE,ME,IR,DR,RI,LE>  CR: 44088404  XER: 00000000
[    0.287122][  T673] CFAR: c000000000448860 DAR: 0000000000000010 DSISR: 00080000 IRQMASK: 0 
[    0.287122][  T673] GPR00: c000000000448864 c00000000a7a7c00 c000000001f38100 c000000002a5c098 
[    0.287122][  T673] GPR04: c0000000017dc5f0 c0000000017dc5e8 000000000000001f 0000000000000064 
[    0.287122][  T673] GPR08: c0000000017dc628 00000000000005f0 00000000000005e8 0000000084000404 
[    0.287122][  T673] GPR12: c0000000005a3420 c000003dfff43d80 c0000000018e5600 c0000000018e54f0 
[    0.287122][  T673] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
[    0.287122][  T673] GPR20: c0000000013ec418 c0000000013ef6d8 c0000000017dc5a0 c0000000017dc590 
[    0.287122][  T673] GPR24: c000000001825330 c0000000017dc628 c000000014b52a05 0000000000000000 
[    0.287122][  T673] GPR28: c0000000017dc5e8 0000000000000000 c0000000017dc5f0 c000000002a5e008 
[    0.287970][  T673] NIP [c000000000485bd0] __find_event_file+0x70/0x3c0
[    0.288046][  T673] LR [c000000000448864] init_tracer_tracefs+0x274/0xc80
[    0.288122][  T673] Call Trace:
[    0.288167][  T673] [c00000000a7a7c00] [c00000000a7a7c60] 0xc00000000a7a7c60 (unreliable)
[    0.288258][  T673] [c00000000a7a7c60] [c000000000448864] init_tracer_tracefs+0x274/0xc80
[    0.288348][  T673] [c00000000a7a7dc0] [c00000000203fcf0] tracer_init_tracefs_work_func+0x50/0x320
[    0.288452][  T673] [c00000000a7a7e50] [c0000000002620e8] process_one_work+0x1e8/0x5c0
[    0.288541][  T673] [c00000000a7a7f10] [c00000000026309c] worker_thread+0x1dc/0x3d0
[    0.288630][  T673] [c00000000a7a7f90] [c00000000026fa34] kthread+0x194/0x1b0
[    0.288721][  T673] [c00000000a7a7fe0] [c00000000000de58] start_kernel_thread+0x14/0x18
[    0.288810][  T673] Code: fb410030 fb810040 fba10048 7cbc2b78 3ba00000 fbc10050 7d194378 7c9e2378 2e2a0fc0 2da90fc0 f8010070 60420000 <e93b0010> 81490058 e8890018 714a0208 
[    0.289001][  T673] ---[ end trace 0000000000000000 ]---
[    0.290718][  T673] pstore: backend (nvram) writing error (-1)
[    0.290805][  T673] 
[    0.290839][  T673] note: kworker/u531:0[673] exited with irqs disabled
[    0.304261][    T1] NET: Registered PF_INET protocol family
[    0.304602][    T1] IP idents hash table entries: 262144 (order: 5, 2097152 bytes, linear)
[    0.315892][    T1] tcp_listen_portaddr_hash hash table entries: 65536 (order: 4, 1048576 bytes, linear)
[    0.316104][    T1] Table-perturb hash table entries: 65536 (order: 2, 262144 bytes, linear)
[    0.316224][    T1] TCP established hash table entries: 524288 (order: 6, 4194304 bytes, linear)
[    0.316873][    T1] TCP bind hash table entries: 65536 (order: 5, 2097152 bytes, linear)
[    0.317199][    T1] TCP: Hash tables configured (established 524288 bind 65536)
[    0.317748][    T1] MPTCP token hash table entries: 65536 (order: 5, 1572864 bytes, linear)
[    0.317970][    T1] UDP hash table entries: 65536 (order: 6, 4194304 bytes, linear)
[    0.318402][    T1] UDP-Lite hash table entries: 65536 (order: 6, 4194304 bytes, linear)
[    0.319148][    T1] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.319249][    T1] NET: Registered PF_XDP protocol family
[    0.320019][    T1] PCI: CLS 0 bytes, default 128
[    0.320281][    T1] rtas_flash: no firmware flash support
[    0.320967][  T770] Trying to unpack rootfs image as initramfs...
[    0.331385][    T1] Initialise system trusted keyrings
[    0.331739][    T1] Key type blacklist registered
[    0.332191][    T1] workingset: timestamp_bits=38 max_order=22 bucket_order=0
[    0.333804][    T1] integrity: Platform Keyring initialized
[    0.333875][    T1] integrity: Machine keyring initialized
[    0.333935][    T1] Allocating IMA blacklist keyring.
[    0.343039][    T1] Key type asymmetric registered
[    0.343109][    T1] Asymmetric key parser 'x509' registered
[    0.344215][    T1] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 246)
[    0.344623][    T1] io scheduler mq-deadline registered
[    0.344688][    T1] io scheduler kyber registered
[    0.344800][    T1] io scheduler bfq registered
[    0.380554][    T1] ledtrig-cpu: registered to indicate activity on CPUs
[    0.380794][    T1] virtio-pci 0000:00:01.0: enabling device (0100 -> 0103)
[    0.382384][    T1] virtio-pci 0000:00:01.0: ibm,query-pe-dma-windows(2026) 800 8000000 20000000 returned 0, lb=2000000000 ps=107 wn=1
[    0.383175][    T1] virtio-pci 0000:00:01.0: ibm,create-pe-dma-window(2027) 800 8000000 20000000 18 26 returned 0 (liobn = 0x80000001 starting addr = 8000000 0)
[    0.386395][    T1] virtio-pci 0000:00:01.0: lsa_required: 0, lsa_enabled: 0, direct mapping: 1
[    0.386509][    T1] virtio-pci 0000:00:01.0: iommu: 64-bit OK but direct DMA is limited by 800004000000000
[    0.386621][    T1] virtio-pci 0000:00:01.0: lsa_required: 0, lsa_enabled: 0, direct mapping: 1
[    0.386723][    T1] virtio-pci 0000:00:01.0: iommu: 64-bit OK but direct DMA is limited by 800004000000000
[    0.388040][    T1] virtio-pci 0000:00:03.0: enabling device (0100 -> 0103)
[    0.390005][    T1] virtio-pci 0000:00:03.0: lsa_required: 0, lsa_enabled: 0, direct mapping: 1
[    0.390111][    T1] virtio-pci 0000:00:03.0: iommu: 64-bit OK but direct DMA is limited by 800004000000000
[    0.390216][    T1] virtio-pci 0000:00:03.0: lsa_required: 0, lsa_enabled: 0, direct mapping: 1
[    0.390316][    T1] virtio-pci 0000:00:03.0: iommu: 64-bit OK but direct DMA is limited by 800004000000000
[    0.392116][    T1] virtio-pci 0000:00:04.0: enabling device (0100 -> 0103)
[    0.394076][    T1] virtio-pci 0000:00:04.0: lsa_required: 0, lsa_enabled: 0, direct mapping: 1
[    0.394184][    T1] virtio-pci 0000:00:04.0: iommu: 64-bit OK but direct DMA is limited by 800004000000000
[    0.394288][    T1] virtio-pci 0000:00:04.0: lsa_required: 0, lsa_enabled: 0, direct mapping: 1
[    0.394389][    T1] virtio-pci 0000:00:04.0: iommu: 64-bit OK but direct DMA is limited by 800004000000000
[    0.395729][    T1] virtio-pci 0000:00:05.0: enabling device (0100 -> 0103)
[    0.397705][    T1] virtio-pci 0000:00:05.0: lsa_required: 0, lsa_enabled: 0, direct mapping: 1
[    0.397814][    T1] virtio-pci 0000:00:05.0: iommu: 64-bit OK but direct DMA is limited by 800004000000000
[    0.397919][    T1] virtio-pci 0000:00:05.0: lsa_required: 0, lsa_enabled: 0, direct mapping: 1
[    0.398024][    T1] virtio-pci 0000:00:05.0: iommu: 64-bit OK but direct DMA is limited by 800004000000000
[    0.399426][    T1] virtio-pci 0000:00:06.0: enabling device (0100 -> 0103)
[    0.401288][    T1] virtio-pci 0000:00:06.0: lsa_required: 0, lsa_enabled: 0, direct mapping: 1
[    0.401396][    T1] virtio-pci 0000:00:06.0: iommu: 64-bit OK but direct DMA is limited by 800004000000000
[    0.401502][    T1] virtio-pci 0000:00:06.0: lsa_required: 0, lsa_enabled: 0, direct mapping: 1
[    0.401604][    T1] virtio-pci 0000:00:06.0: iommu: 64-bit OK but direct DMA is limited by 800004000000000
[    0.403644][    T1] virtio-pci 0000:00:07.0: enabling device (0100 -> 0103)
[    0.405723][    T1] virtio-pci 0000:00:07.0: lsa_required: 0, lsa_enabled: 0, direct mapping: 1
[    0.405830][    T1] virtio-pci 0000:00:07.0: iommu: 64-bit OK but direct DMA is limited by 800004000000000
[    0.405936][    T1] virtio-pci 0000:00:07.0: lsa_required: 0, lsa_enabled: 0, direct mapping: 1
[    0.406044][    T1] virtio-pci 0000:00:07.0: iommu: 64-bit OK but direct DMA is limited by 800004000000000
[    0.409460][    T1] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.427929][  T770] Freeing initrd memory: 21120K
[    0.431965][    T1] Non-volatile memory driver v1.3
[    0.432069][    T1] pseries_rng: Registering IBM pSeries RNG driver
[    0.433884][    T1] mousedev: PS/2 mouse device common for all mice
[    0.435769][    T1] pseries_idle_driver registered
[    0.435912][    T1] hid: raw HID events driver (C) Jiri Kosina
[    0.436062][    T1] drop_monitor: Initializing network drop monitor service
[    0.436412][    T1] NET: Registered PF_INET6 protocol family
[    0.437731][    T1] Segment Routing with IPv6
[    0.437819][    T1] RPL Segment Routing with IPv6
[    0.437897][    T1] In-situ OAM (IOAM) with IPv6
[    0.438002][    T1] PFKEY is deprecated and scheduled to be removed in 2027, please contact the netdev mailing list
[    0.438125][    T1] NET: Registered PF_KEY protocol family
[    0.438508][    T1] secvar-sysfs: Failed to retrieve secvar operations
[    0.441215][    T1] registered taskstats version 1
[    0.464617][    T1] Loading compiled-in X.509 certificates
[    0.481670][    T1] Loaded X.509 cert 'home:tiwai OBS Project: 466e10c6490242bdd3a9985914c1906197b53a81'
[    0.487739][    T1] Demotion targets for Node 0: null
[    0.487834][    T1] page_owner is disabled
[    0.518088][    T1] Key type .fscrypt registered
[    0.518163][    T1] Key type fscrypt-provisioning registered
[    0.518473][    T1] Key type big_key registered
[    0.536738][    T1] Key type encrypted registered
[    0.536916][    T1] Secure boot mode disabled
[    0.536994][    T1] ima: No TPM chip found, activating TPM-bypass!
[    0.537070][    T1] Loading compiled-in module X.509 certificates
[    0.537347][    T1] Loaded X.509 cert 'home:tiwai OBS Project: 466e10c6490242bdd3a9985914c1906197b53a81'
[    0.537451][    T1] ima: Allocated hash algorithm: sha256
[    0.538132][    T1] Secure boot mode disabled
[    0.538236][    T1] Trusted boot mode disabled
[    0.538303][    T1] ima: No architecture policies found
[    0.538448][    T1] evm: Initialising EVM extended attributes:
[    0.538523][    T1] evm: security.selinux
[    0.538569][    T1] evm: security.SMACK64 (disabled)
[    0.538629][    T1] evm: security.SMACK64EXEC (disabled)
[    0.538689][    T1] evm: security.SMACK64TRANSMUTE (disabled)
[    0.538762][    T1] evm: security.SMACK64MMAP (disabled)
[    0.538822][    T1] evm: security.apparmor
[    0.538868][    T1] evm: security.ima
[    0.538915][    T1] evm: security.capability
[    0.538983][    T1] evm: HMAC attrs: 0x1
[    0.553344][    T1] SED: plpks not available

Re: [Linux PPC] Disable PREEMPT

[Amit Machhiwal]

On 2026/09/03 02:55 PM, Michal Suchánek wrote:
> On Thu, Sep 03, 2026 at 06:06:16PM +0530, Harsh Prateek Bora wrote:
> > + Amit, Gautam - since they recently came across a similar regression which
> > is seen with non KVM workload as well.
> > + Venkat - for CI improvement suggestion below.
> > 
> > Hi Michal,
> > 
> > On 03/09/26 5:55 pm, Michal Suchánek wrote:
> > > On Thu, Sep 03, 2026 at 05:34:31PM +0530, Shrikanth Hegde wrote:
> > > > Hi Michal.
> > > > 
> > > > On 9/3/26 5:10 PM, Michal Suchánek wrote:
> > > > > On Thu, Sep 03, 2026 at 05:01:13PM +0530, Shrikanth Hegde wrote:
> > > > > 
> > > > > > 
> > > > > > > > 
> > > > > > > > > This is not the latest kernel so it might have been fixed in the
> > > > > > > > > meantime for the platform in question. Will need to check that.
> > > > > > > > > 
> > > > > > > > 
> > > > > > > > If possible run against current upstream and share the results.
> > > > > > > 
> > > > > > > https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default
> > > > > > > 
> > > > > > > [  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > > > > > > [  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> > > > > > > [  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> > > > > > > [  472.091624][ T6181] Call Trace:
> > > > > > > [  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
> > > > > > > [  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
> > > > > > > [  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
> > > > > > > [  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
> > > > > > > [  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > > > > > > [  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
> > > > > > > [  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
> > > > > > > [  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
> > > > > > > [  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
> > > > > > > [  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
> > > > > > > [  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
> > > > > > > [  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
> > > > > > > [  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
> > > > > > > [  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
> > > > > > > [  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
> > > > > > > [  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
> > > > > > > [  472.091938][ T6181] IRQMASK: 0
> > > > > > > [  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa
> > > > > > > [  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000
> > > > > > > [  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000
> > > > > > > [  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0
> > > > > > > [  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7
> > > > > > > [  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000
> > > > > > > [  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70
> > > > > > > [  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000
> > > > > > > [  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
> > > > > > > [  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
> > > > > > > [  472.092033][ T6181] ---- interrupt: 3000
> > > > > > > [  472.098256][ T6181] pstore: backend (nvram) writing error (-1)
> > > > > > > 
> > > > > > > This is the host, cannot run the kernel as guest because it fails to boot most
> > > > > > > of the time inside KVM.
> > > > > > > 
> > > > > > 
> > > > > > Will look into. btw the patch shared i.e below patch fixes the boot hanf issue?
> > > > > > 
> > > > > > https://lore.kernel.org/all/1d79ec6f-ce7f-4432-80e0-a54fa1668642@linux.ibm.com/
> > > > > 
> > > > 
> > > > I see code has changed a bit. So this patch is not correct anymore.
> > > > Sorry, i should have checked.
> > > > 
> > > > > I would not bet on that. It looks like host side patch, and the problem
> > > > > is with booting 7.2.2 as guest. Some 6.12 frankenkernel can be booted on
> > > > > the same host, and used to produce the crash above by running a kernel
> > > > > build inside the guest.
> > > > > 
> > > > 
> > > > I didn't get what you are saying. Could you see if it regression only in 7.2?
> > > > Does 7.1 works fine? If it is regression could you see if it is bisectable?
> > > > Also, could you share the repro steps?
> > > 
> > > The repro steps are: on a mid-size LPAR (120 CPU threads, 256G RAM)
> > > create a KVM VM that uses all host resources. Run kernel build inside
> > > KVM VM, using all CPUs. In about a minute the host crashes with the
> > > above a or similar trace. I don't have any kernel version with aggressive
> > > preemption enabled that does not suffer from this problem.
> > 
> > Looks like seen under heavy workloads when you say "KVM VM that uses all
> > host rsources". Could you please confirm if the issue is seen with any other
> > heavy workload also which consumes all host resources? I do not think we
> > recently had any PPC KVM specific changes which would cause this. Also, if
> > it's a regression, git bisect should point out the real culprit.
> 
> It's a regression from switching to PREEMPT_LAZY (from
> PREEMPT_VOLUNTARY). Very likely the code was buggy to start with but
> because less aggressive preemption model was used the bug was never
> triggered. I cannot reproduce this under load without KVM. The other

I think the problem here is the workload that may cause more aggressive
preemptions. Also, it may also not be specific to KVM if we could
trigger comparative number of preemptions on host too. Could we try
running a more aggressive kernel compilation like running make with 4x
or more parallel make jobs than the number of CPUs available on it. For
example:

  make -j $(( 4 * $(nproc)))

Thanks,
Amit

> problem that is much harder to reproduce is machine locking up while
> idling. A dump then also shows stack corruption with completely
> but different stack trace, no KVM involved in that one. Did not see this
> with recent mainline yet.
> 
> I do get a problem with 7.2.2 building a kernel outside of kVM but
> that's probably because it's missing some of the recent entry fixes:
> 
> [ 5451.821931] [  T77619] as[77619]: segfault (11) at 3abd0 nip 7fff8fe56434 lr 7fff8fe55348 code 1 in ld64.so.2[36434,7fff8fe20000+60000]
> [ 5451.821967] [  T77619] as[77619]: code: 60000000 60000000 60000000 f9490008 f9490010 39290020 f949fff8 f9490000 
> [ 5451.821973] [  T77619] as[77619]: code: 4200ffec 89210376 3d42ffff 394a7ea0 <e9028000> 7d081050 f9410030 f9010020 
> [ 5454.239768] [  T78922] awk[78922]: segfault (11) at 3abd0 nip 7fff8c386434 lr 7fff8c385348 code 1 in ld64.so.2[36434,7fff8c350000+60000]
> [ 5454.239809] [  T78922] awk[78922]: code: 60000000 60000000 60000000 f9490008 f9490010 39290020 f949fff8 f9490000 
> [ 5454.239815] [  T78922] awk[78922]: code: 4200ffec 89210376 3d42ffff 394a7ea0 <e9028000> 7d081050 f9410030 f9010020 
> [ 5465.056566] [  T85518] sh[85518]: segfault (11) at 3abd0 nip 7fffb79a6434 lr 7fffb79a5348 code 1 in ld64.so.2[36434,7fffb7970000+60000]
> [ 5465.056611] [  T85518] sh[85518]: code: 60000000 60000000 60000000 f9490008 f9490010 39290020 f949fff8 f9490000 
> [ 5465.056616] [  T85518] sh[85518]: code: 4200ffec 89210376 3d42ffff 394a7ea0 <e9028000> 7d081050 f9410030 f9010020 
> [ 5474.995476] [  T91034] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: sys_ioctl+0xfe0/0x1060
> [ 5474.995495] [  T91037] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: vprintk_store+0x6a4/0x6c0
> [ 5474.995508] [  T91037] ------------[ cut here ]------------
> [ 5474.995513] [  T91037] kernel BUG at [regs_irqs_disabled(regs)] arch/powerpc/include/asm/entry-common.h:401!
> [ 5474.995521] [  T91037] Oops: Exception in kernel mode, sig: 5 [#1]
> [ 5474.995524] [  T91037] LE PAGE_SIZE=64K MMU=Radix  SMP NR_CPUS=8192 NUMA pSeries
> [ 5474.995528] [  T91037] Modules linked in: rpadlpar_io rpaphp kvm_hv kvm mptcp_diag xsk_diag tcp_diag udp_diag raw_diag inet_diag unix_diag af_packet_diag netlink_diag xt_conntrack xt_MASQUERADE xfrm_user xt_set ip_set xt_addrtype nft_compat nf_conntrack_netbios_ns nf_conntrack_broadcast nft_masq nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct bonding tls nft_chain_nat nf_nat nf_conntrack tun nf_defrag_ipv6 nf_defrag_ipv4 rfkill af_packet binfmt_misc xfs aes_gcm_p10_crypto ibmveth crypto_simd cryptd xts vmx_crypto rtc_generic fuse nvme_fabrics ip_tables x_tables ibmvscsi scsi_transport_srp sd_mod scsi_dh_emc scsi_dh_rdac scsi_dh_alua sg scsi_mod scsi_common nvme nvme_core nvme_keyring pseries_wdt nvme_auth btrfs xor libblake2b raid6_pq dm_mirror dm_region_hash dm_log dm_mod br_netfilter bridge stp llc nf_tables
> [ 5474.995641] [  T91037] CPU: 110 UID: 399 PID: 91037 Comm: cc1 Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> [ 5474.995649] [  T91037] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> [ 5474.995653] [  T91037] NIP:  c0000000013c2114 LR: c000000000148c88 CTR: c000000000008e40
> [ 5474.995656] [  T91037] REGS: c000001f965d3b50 TRAP: 0700   Not tainted  (7.2.2-5.g6824496-default)
> [ 5474.995660] [  T91037] MSR:  8000000000029033 <SF,EE,ME,IR,DR,RI,LE>  CR: 42004208  XER: 20040145
> [ 5474.995670] [  T91037] CFAR: c0000000013c20b4 IRQMASK: 3 
>                           GPR00: c00000000000904c c000001f965d3df0 c00000000205ad00 c000001f965d3e80 
>                           GPR04: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
>                           GPR08: 0000000000000000 0000000000000001 0000000000004000 0000000053474552 
>                           GPR12: 0000000000000000 c000003ffef56300 0000000000000000 0000000000000000 
>                           GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
>                           GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
>                           GPR24: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
>                           GPR28: 0000000000000000 0000000000000000 0000000000000000 c000001f965d3e80 
> [ 5474.995708] [  T91037] NIP [c0000000013c2114] irqentry_enter+0x74/0x1a0
> [ 5474.995719] [  T91037] LR [c000000000148c88] do_page_fault+0x68/0x140
> [ 5474.995727] [  T91037] Call Trace:
> [ 5474.995731] [  T91037] [c000001f965d3e10] [c000001f965d3e50] 0xc000001f965d3e50
> [ 5474.995739] [  T91037] [c000001f965d3e50] [c00000000000904c] instruction_access_common_virt+0x20c/0x210
> [ 5474.995751] [  T91037] ---- interrupt: 400 at 0x1043f158
> [ 5474.995754] [  T91037] NIP:  000000001043f158 LR: 00000000101218e0 CTR: 00007fff7aef4f00
> [ 5474.995756] [  T91037] REGS: c000001f965d3e80 TRAP: 0400   Not tainted  (7.2.2-5.g6824496-default)
> [ 5474.995759] [  T91037] MSR:  800000004280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 82004208  XER: 20040145
> [ 5474.995770] [  T91037] CFAR: 00000000101218dc IRQMASK: 1 
>                           GPR00: 00000000101218cc 00007ffffd284ce0 0000000011de7a00 0000000012c2c370 
>                           GPR04: 0000000000000001 0000000000000000 0000000000000001 0000000000000000 
>                           GPR08: 00007fff7b070d60 0000000000000000 0000000012c2c380 0000000000004000 
>                           GPR12: 00007fff7aef4f00 00007fff7ba8da00 0000000000000000 0000000000000000 
>                           GPR16: 0000000000000000 00007ffffd28c2ac 0000000000000000 0000000000000001 
>                           GPR20: 0000000000000197 0000000000c03cff 0000000011e0d210 0000000000000008 
>                           GPR24: 0000000000000000 00007ffffd284f58 0000000000000000 0000000000000000 
>                           GPR28: 0000000000000001 00007ffffd28c2ac 0000000000000000 0000000000000197 
> [ 5474.995809] [  T91037] NIP [000000001043f158] 0x1043f158
> [ 5474.995812] [  T91037] LR [00000000101218e0] 0x101218e0
> [ 5474.995814] [  T91037] ---- interrupt: 400
> [ 5474.995816] [  T91037] Code: f8010030 4af8b211 60000000 e8010030 7c0803a6 4bffffd4 60420000 7929ffe2 69290001 0b090000 e9230138 792907e0 <0b090000> 4800001c e9230140 792907a4 
> [ 5474.995835] [  T91037] ---[ end trace 0000000000000000 ]---
> [ 5492.821144] [    C110] watchdog: CPU 110 Hard LOCKUP
> [ 5492.821159] [    C110] watchdog: CPU 110 TB:3408898268781396, last heartbeat TB:3408888039246087 (19979ms ago)
> [ 5492.821162] [    C110] Modules linked in: rpadlpar_io rpaphp kvm_hv kvm mptcp_diag xsk_diag tcp_diag udp_diag raw_diag inet_diag unix_diag af_packet_diag netlink_diag xt_conntrack xt_MASQUERADE xfrm_user xt_set ip_set xt_addrtype nft_compat nf_conntrack_netbios_ns nf_conntrack_broadcast nft_masq nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct bonding tls nft_chain_nat nf_nat nf_conntrack tun nf_defrag_ipv6 nf_defrag_ipv4 rfkill af_packet binfmt_misc xfs aes_gcm_p10_crypto ibmveth crypto_simd cryptd xts vmx_crypto rtc_generic fuse nvme_fabrics ip_tables x_tables ibmvscsi scsi_transport_srp sd_mod scsi_dh_emc scsi_dh_rdac scsi_dh_alua sg scsi_mod scsi_common nvme nvme_core nvme_keyring pseries_wdt nvme_auth btrfs xor libblake2b raid6_pq dm_mirror dm_region_hash dm_log dm_mod br_netfilter bridge stp llc nf_tables
> [ 5492.821225] [    C110] CPU: 110 UID: 399 PID: 91037 Comm: cc1 Tainted: G      D             7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> [ 5492.821234] [    C110] Tainted: [D]=DIE
> [ 5492.821235] [    C110] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> [ 5492.821237] [    C110] NIP:  c0000000013d2c90 LR: c0000000013d4564 CTR: c000000000919460
> [ 5492.821241] [    C110] REGS: c000003ffeb93d60 TRAP: 0100   Tainted: G      D              (7.2.2-5.g6824496-default)
> [ 5492.821244] [    C110] MSR:  8000000000081033 <SF,ME,IR,DR,RI,LE>  CR: 44004204  XER: 20040145
> [ 5492.821252] [    C110] CFAR: 000000000000011c IRQMASK: 3 
>                           GPR00: c0000000009194f0 c000001f965d3730 c00000000205ad00 c0000000037b6510 
>                           GPR04: c000001f965d39d8 00000000000000dd fffffffffffe0000 0000000000010000 
>                           GPR08: 0000000000000001 0000000000000001 000000000fe67ee1 0000000000004000 
>                           GPR12: c000000000919460 c000003ffef56300 0000000000000000 0000000000000000 
>                           GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
>                           GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
>                           GPR24: 0000000000002800 c0000000018c5670 c000001f965d39d8 0000003ff1ae0000 
>                           GPR28: 0000000000de0000 c0000000032bbe80 c000003ff4d9be80 c0000000037b6510 
> [ 5492.821283] [    C110] NIP [c0000000013d2c90] queued_spin_lock_slowpath+0x160/0x19a0
> [ 5492.821296] [    C110] LR [c0000000013d4564] _raw_spin_lock_irqsave+0x94/0xc0
> [ 5492.821299] [    C110] Call Trace:
> [ 5492.821302] [    C110] [c000001f965d3830] [0000000000000263] 0x263
> [ 5492.821309] [    C110] [c000001f965d3860] [c0000000009194f0] pstore_dump+0x90/0x430
> [ 5492.821317] [    C110] [c000001f965d39b0] [c00000000031ff40] kmsg_dump_desc+0xb0/0x100
> [ 5492.821325] [    C110] [c000001f965d3a10] [c00000000022e2a4] oops_exit+0x54/0x70
> [ 5492.821331] [    C110] [c000001f965d3a70] [c00000000002760c] oops_end+0x6c/0x1a0
> [ 5492.821340] [    C110] [c000001f965d3af0] [c00000000002a61c] program_check_exception+0x7c/0xe0
> [ 5492.821344] [    C110] [c000001f965d3b20] [c000000000009b2c] program_check_common_virt+0x3bc/0x3c0
> [ 5492.821352] [    C110] ---- interrupt: 700 at irqentry_enter+0x74/0x1a0
> [ 5492.821358] [    C110] NIP:  c0000000013c2114 LR: c000000000148c88 CTR: c000000000008e40
> [ 5492.821361] [    C110] REGS: c000001f965d3b50 TRAP: 0700   Tainted: G      D              (7.2.2-5.g6824496-default)
> [ 5492.821363] [    C110] MSR:  8000000000029033 <SF,EE,ME,IR,DR,RI,LE>  CR: 42004208  XER: 20040145
> [ 5492.821370] [    C110] CFAR: c0000000013c20b4 IRQMASK: 3 
>                           GPR00: c00000000000904c c000001f965d3df0 c00000000205ad00 c000001f965d3e80 
>                           GPR04: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
>                           GPR08: 0000000000000000 0000000000000001 0000000000004000 0000000053474552 
>                           GPR12: 0000000000000000 c000003ffef56300 0000000000000000 0000000000000000 
>                           GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
>                           GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
>                           GPR24: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
>                           GPR28: 0000000000000000 0000000000000000 0000000000000000 c000001f965d3e80 
> [ 5492.821399] [    C110] NIP [c0000000013c2114] irqentry_enter+0x74/0x1a0
> [ 5492.821403] [    C110] LR [c000000000148c88] do_page_fault+0x68/0x140
> [ 5492.821409] [    C110] ---- interrupt: 700
> [ 5492.821410] [    C110] [c000001f965d3df0] [0000000000000000] 0x0 (unreliable)
> [ 5492.821414] [    C110] [c000001f965d3e10] [c000001f965d3e50] 0xc000001f965d3e50
> [ 5492.821416] [    C110] [c000001f965d3e50] [c00000000000904c] instruction_access_common_virt+0x20c/0x210
> [ 5492.821421] [    C110] ---- interrupt: 400 at 0x1043f158
> [ 5492.821423] [    C110] NIP:  000000001043f158 LR: 00000000101218e0 CTR: 00007fff7aef4f00
> [ 5492.821425] [    C110] REGS: c000001f965d3e80 TRAP: 0400   Tainted: G      D              (7.2.2-5.g6824496-default)
> [ 5492.821428] [    C110] MSR:  800000004000f033 <SF,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 82004208  XER: 20040145
> [ 5492.821436] [    C110] CFAR: 00000000101218dc IRQMASK: 1 
>                           GPR00: 00000000101218cc 00007ffffd284ce0 0000000011de7a00 0000000012c2c370 
>                           GPR04: 0000000000000001 0000000000000000 0000000000000001 0000000000000000 
>                           GPR08: 00007fff7b070d60 0000000000000000 0000000012c2c380 0000000000004000 
>                           GPR12: 00007fff7aef4f00 00007fff7ba8da00 0000000000000000 0000000000000000 
>                           GPR16: 0000000000000000 00007ffffd28c2ac 0000000000000000 0000000000000001 
>                           GPR20: 0000000000000197 0000000000c03cff 0000000011e0d210 0000000000000008 
>                           GPR24: 0000000000000000 00007ffffd284f58 0000000000000000 0000000000000000 
>                           GPR28: 0000000000000001 00007ffffd28c2ac 0000000000000000 0000000000000197 
> [ 5492.821464] [    C110] NIP [000000001043f158] 0x1043f158
> [ 5492.821466] [    C110] LR [00000000101218e0] 0x101218e0
> [ 5492.821468] [    C110] ---- interrupt: 400
> [ 5492.821470] [    C110] Code: 7c085040 41810e8c 39400000 39000000 3cc0fffe 60000000 813f0000 71290001 41820050 3ce00001 60000000 60420000 <7c40003c> 71090001 394a0001 40820028 
> 
> Thanks
> 
> Michal

Re: [Linux PPC] Disable PREEMPT

[Shrikanth Hegde]

>> If possible run against current upstream and share the results.
> 
> https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default
> 
> [  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> [  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> [  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> [  472.091624][ T6181] Call Trace:
> [  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
> [  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
> [  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
> [  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
> [  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> [  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
> [  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
> [  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
> [  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
> [  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
> [  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
> [  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
> [  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
> [  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
> [  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
> [  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
> [  472.091938][ T6181] IRQMASK: 0
> [  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa
> [  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000
> [  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000
> [  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0
> [  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7
> [  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000
> [  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70
> [  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000
> [  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
> [  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
> [  472.092033][ T6181] ---- interrupt: 3000
> [  472.098256][ T6181] pstore: backend (nvram) writing error (-1)
> 
> This is the host, cannot run the kernel as guest because it fails to boot most
> of the time inside KVM.
> 
>>
>>> Nonethless, there are quite a few platforms. Originally no preemption
>>> was the only option, and that's the reason why many people run that.
>>> It's the conservative, known working option. And that's the reason a lot
>>> of platfrom code does not get tested with more aggressive preemtion
>>> models, and never gets fixed to work with them.
>>>
>>
>> Full preemption has been there for many years!.
> 
> Possible for years, forced only recently.
> 
>> Lazy is not that aggressive compared to that.
>>
>>> Simply disabling the no preemtion option does not make the platform code
>>> ready.
>>>
>>
>> Let's understand your crash case. Let's see where it is going wrong. I am suspecting
>> it is some wrong usage of preemption api rather than arch can't support preemption.
> 
> Very likely some wrong use of the preemption API by the arch code, or no
> use where it should have been used. It did not matter so long as people
> could run their no preempt configs and ignore the problem.
> 
> Thanks


Friendly LLM analysis says preemption is enabled too early and before
completing kvmppc_handle_exit_hv() / kvmppc_handle_nested_exit()

Below is ONLY a speculation and completely UNTESTED.
Maybe worth a try.

---
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index 0409ac9e7b31..78d2b0618887 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -5049,8 +5049,6 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,

         powerpc_local_irq_pmu_restore(flags);

-       preempt_enable();
-
         /*
          * cancel pending decrementer exception if DEC is now positive, or if
          * entering a nested guest in which case the decrementer is now owned
@@ -5072,6 +5070,8 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,
         }
         vcpu->arch.ret = r;

+       preempt_enable();
+
         if (is_kvmppc_resume_guest(r) && !kvmppc_vcpu_check_block(vcpu)) {
                 kvmppc_set_timer(vcpu);


> 
> Michal

Re: [Linux PPC] Disable PREEMPT

[Michal Suchánek]

On Thu, Sep 03, 2026 at 10:52:59PM +0530, Shrikanth Hegde wrote:
> 
> > > If possible run against current upstream and share the results.
> > 
> > https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default
> > 
> > [  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > [  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> > [  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> > [  472.091624][ T6181] Call Trace:
> > [  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
> > [  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
> > [  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
> > [  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
> > [  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > [  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
> > [  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
> > [  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
> > [  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
> > [  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
> > [  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
> > [  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
> > [  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
> > [  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
> > [  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
> > [  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
> > [  472.091938][ T6181] IRQMASK: 0
> > [  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa
> > [  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000
> > [  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000
> > [  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0
> > [  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7
> > [  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000
> > [  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70
> > [  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000
> > [  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
> > [  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
> > [  472.092033][ T6181] ---- interrupt: 3000
> > [  472.098256][ T6181] pstore: backend (nvram) writing error (-1)
> > 
> > This is the host, cannot run the kernel as guest because it fails to boot most
> > of the time inside KVM.
> > 
> > > 
> > > > Nonethless, there are quite a few platforms. Originally no preemption
> > > > was the only option, and that's the reason why many people run that.
> > > > It's the conservative, known working option. And that's the reason a lot
> > > > of platfrom code does not get tested with more aggressive preemtion
> > > > models, and never gets fixed to work with them.
> > > > 
> > > 
> > > Full preemption has been there for many years!.
> > 
> > Possible for years, forced only recently.
> > 
> > > Lazy is not that aggressive compared to that.
> > > 
> > > > Simply disabling the no preemtion option does not make the platform code
> > > > ready.
> > > > 
> > > 
> > > Let's understand your crash case. Let's see where it is going wrong. I am suspecting
> > > it is some wrong usage of preemption api rather than arch can't support preemption.
> > 
> > Very likely some wrong use of the preemption API by the arch code, or no
> > use where it should have been used. It did not matter so long as people
> > could run their no preempt configs and ignore the problem.
> > 
> > Thanks
> 
> 
> Friendly LLM analysis says preemption is enabled too early and before
> completing kvmppc_handle_exit_hv() / kvmppc_handle_nested_exit()
> 
> Below is ONLY a speculation and completely UNTESTED.
> Maybe worth a try.

The patch is munged by the e-mail client, and it causes immediate
voluntary preemprion in rcu critical section and hard lockup on starting
a KVM VM.

Also it would be sort of bad news if it worked because that would be
specific to book3s KVM HV and would not help with the KVM HV from the
original report which likely is not book3s, nor with KVM PR.

Thanks

Michal

Re: [Linux PPC] Disable PREEMPT

[Michal Suchánek]

On Thu, Sep 03, 2026 at 03:28:26PM +0200, Michal Suchánek wrote:
> On Thu, Sep 03, 2026 at 06:06:16PM +0530, Harsh Prateek Bora wrote:
> > + Amit, Gautam - since they recently came across a similar regression which
> > is seen with non KVM workload as well.
> > + Venkat - for CI improvement suggestion below.
> > 
> > Hi Michal,
> > 
> > On 03/09/26 5:55 pm, Michal Suchánek wrote:
> 
> > > 
> > > There is another problem that Linux 7.2.2 often locks up on boot inside
> > > the guest, and as a result a different kernel is needed in the guest to
> > > reproduce the host crash. This is more recent, linux 6.12 does not have
> > > this problem. Will try some different kernel combinations to see if I
> > > can narrow this down. For this the repro is merely booting the kernel in
> > > the VM, it typically fails early before userspace produces any messages.
> 
> This one regressed between 6.19.12
> https://github.com/openSUSE/kernel-source/blob/9f6830f/config/ppc64le/default
> and 7.0.12
> https://github.com/openSUSE/kernel-source/blob/2ebf0bc/config/ppc64le/default
> 
> Thanks
> 
> Michal
> 
> SLOF **********************************************************************
> QEMU Starting
>  Build Date = Aug 13 2026 14:47:36
>  FW Version = abuild@OBS release 20230918
>  Press "s" to enter Open Firmware.
> 
> Populating /vdevice methods
> Populating /vdevice/vty@30000000
> Populating /vdevice/nvram@71000000
> Populating /pci@800000020000000
> Loading Linux 7.0.12-1.g2ebf0bc-default ...
> Loading initial ramdisk ...                
> OF stdout device is: /vdevice/vty@30000000
> Preparing to boot Linux version 7.0.12-1.g2ebf0bc-default (geeko@buildhost) (gcc (SUSE Linux) 15.3.0, GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.45.0.20251103-4) #1 SMP PREEMPT_DYNAMIC Mon Jun 15 08:39:32 UTC 2026 (2ebf0bc)
> Detected machine type: 0000000000000101
> command line: BOOT_IMAGE=/boot/vmlinux-7.0.12-1.g2ebf0bc-default root=UUID=304c7f07-efbb-1070-2f27-accd935ec088 rw quiet systemd.show_status=1 security=selinux selinux=1
> Max number of cores passed to firmware: 8192 (NR_CPUS = 8192)
> Calling ibm,client-architecture-support... done
> memory layout at init:
>   memory_limit : 0000000000000000 (16 MB aligned)
>   alloc_bottom : 00000000066b0000
>   alloc_top    : 0000000030000000
>   alloc_top_hi : 0000003e00000000
>   rmo_top      : 0000000030000000
>   ram_top      : 0000003e00000000
> instantiating rtas at 0x000000002fff0000... done
> prom_hold_cpus: skipped
> copying OF device tree...
> Building dt strings...
> Building dt structure...
> Device tree strings 0x00000000066c0000 -> 0x00000000066c0bec
> Device tree struct  0x00000000066d0000 -> 0x00000000066f0000
> Quiescing Open Firmware ...
> Booting Linux via __start() @ 0x0000000000250000 ...
> [    0.000000][    T0] ERROR: Failed to allocate trace buffer
> [    0.000000][    T0] ERROR: tracer: failed to allocate ring buffer!
> Linux ppc64le
> #1 SMP PREEMPT_D[    0.285758][  T673] BUG: Kernel NULL pointer dereference on read at 0x00000010
> [    0.285877][  T673] Faulting instruction address: 0xc000000000485bd0
> [    0.285903][    T1] VFS: Dquot-cache hash table entries: 8192 (order 0, 65536 bytes)
> [    0.285954][  T673] Oops: Kernel access of bad area, sig: 7 [#1]
> [    0.286133][  T673] LE PAGE_SIZE=64K MMU=Radix  SMP NR_CPUS=8192 NUMA pSeries
> [    0.286240][  T673] Modules linked in:
> [    0.286288][  T673] CPU: 15 UID: 0 PID: 673 Comm: kworker/u531:0 Not tainted 7.0.12-1.g2ebf0bc-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  cf0846124d7853fab338aeae87203b36cac18419
> [    0.286527][  T673] Hardware name: IBM pSeries (emulated by qemu) Power11 (architected) 0x820200 0xf000007 of:SLOF,HEAD hv:linux,kvm pSeries
> [    0.286739][  T673] Workqueue: trace_init_wq tracer_init_tracefs_work_func
> [    0.286818][  T673] NIP:  c000000000485bd0 LR: c000000000448864 CTR: c0000000005a3420
> [    0.286906][  T673] REGS: c00000000a7a7960 TRAP: 0300   Not tainted  (7.0.12-1.g2ebf0bc-default)
> [    0.287014][  T673] MSR:  8000000000009033 <SF,EE,ME,IR,DR,RI,LE>  CR: 44088404  XER: 00000000
> [    0.287122][  T673] CFAR: c000000000448860 DAR: 0000000000000010 DSISR: 00080000 IRQMASK: 0 
> [    0.287122][  T673] GPR00: c000000000448864 c00000000a7a7c00 c000000001f38100 c000000002a5c098 
> [    0.287122][  T673] GPR04: c0000000017dc5f0 c0000000017dc5e8 000000000000001f 0000000000000064 
> [    0.287122][  T673] GPR08: c0000000017dc628 00000000000005f0 00000000000005e8 0000000084000404 
> [    0.287122][  T673] GPR12: c0000000005a3420 c000003dfff43d80 c0000000018e5600 c0000000018e54f0 
> [    0.287122][  T673] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
> [    0.287122][  T673] GPR20: c0000000013ec418 c0000000013ef6d8 c0000000017dc5a0 c0000000017dc590 
> [    0.287122][  T673] GPR24: c000000001825330 c0000000017dc628 c000000014b52a05 0000000000000000 
> [    0.287122][  T673] GPR28: c0000000017dc5e8 0000000000000000 c0000000017dc5f0 c000000002a5e008 
> [    0.287970][  T673] NIP [c000000000485bd0] __find_event_file+0x70/0x3c0
> [    0.288046][  T673] LR [c000000000448864] init_tracer_tracefs+0x274/0xc80
> [    0.288122][  T673] Call Trace:
> [    0.288167][  T673] [c00000000a7a7c00] [c00000000a7a7c60] 0xc00000000a7a7c60 (unreliable)
> [    0.288258][  T673] [c00000000a7a7c60] [c000000000448864] init_tracer_tracefs+0x274/0xc80
> [    0.288348][  T673] [c00000000a7a7dc0] [c00000000203fcf0] tracer_init_tracefs_work_func+0x50/0x320
> [    0.288452][  T673] [c00000000a7a7e50] [c0000000002620e8] process_one_work+0x1e8/0x5c0
> [    0.288541][  T673] [c00000000a7a7f10] [c00000000026309c] worker_thread+0x1dc/0x3d0
> [    0.288630][  T673] [c00000000a7a7f90] [c00000000026fa34] kthread+0x194/0x1b0
> [    0.288721][  T673] [c00000000a7a7fe0] [c00000000000de58] start_kernel_thread+0x14/0x18
> [    0.288810][  T673] Code: fb410030 fb810040 fba10048 7cbc2b78 3ba00000 fbc10050 7d194378 7c9e2378 2e2a0fc0 2da90fc0 f8010070 60420000 <e93b0010> 81490058 e8890018 714a0208 
> [    0.289001][  T673] ---[ end trace 0000000000000000 ]---
> [    0.290718][  T673] pstore: backend (nvram) writing error (-1)

This crash might be relevant to the problem, and while qemu can dump the
memory content crash refuses to open it:

crash: incompatible arguments:
   /usr/lib/modules/7.2.2-5.g6824496-default/vmlinux is not SMP -- /scratch/vmcore is SMP

The problem is not 100% reproducible, bisecting it may be tricky.

Thanks

Michal

Re: [Linux PPC] Disable PREEMPT

[Christian Zigotzky]

On 03 September 2026 at 07:23 Shrikanth Hegde <sshegde@linux.ibm.com> wrote:

Friendly LLM analysis says preemption is enabled too early and before
completing kvmppc_handle_exit_hv() / kvmppc_handle_nested_exit()

Below is ONLY a speculation and completely UNTESTED.
Maybe worth a try.

---
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index 0409ac9e7b31..78d2b0618887 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -5049,8 +5049,6 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,

       powerpc_local_irq_pmu_restore(flags);

-       preempt_enable();
-
       /*
        * cancel pending decrementer exception if DEC is now positive, or if
        * entering a nested guest in which case the decrementer is now owned
@@ -5072,6 +5070,8 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,
       }
       vcpu->arch.ret = r;

+       preempt_enable();
+
       if (is_kvmppc_resume_guest(r) && !kvmppc_vcpu_check_block(vcpu)) {
               kvmppc_set_timer(vcpu);


Michal

- - -

Michal,

Is this patch also for the issues with KVM PR and KVM HV if PREEMPT is activated?

The entire FSL Cyrus+ board freezes when using KVM HV with PREEMPT.

The guest kernel doesn't boot if we use KVM PR with a PREEMPT kernel on
the PA Semi Nemo board.

Bug report: https://github.com/chzigotzky/kernels/issues/19

Thanks,
Christian

Re: [Linux PPC] Disable PREEMPT

[Shrikanth Hegde]

On 9/4/26 1:05 PM, Michal Suchánek wrote:
> On Thu, Sep 03, 2026 at 10:52:59PM +0530, Shrikanth Hegde wrote:
>>
>>>> If possible run against current upstream and share the results.
>>>
>>> https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default
>>>
>>> [  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
>>> [  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
>>> [  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
>>> [  472.091624][ T6181] Call Trace:
>>> [  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
>>> [  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
>>> [  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
>>> [  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
>>> [  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
>>> [  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
>>> [  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
>>> [  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
>>> [  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
>>> [  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
>>> [  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
>>> [  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
>>> [  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
>>> [  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
>>> [  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
>>> [  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
>>> [  472.091938][ T6181] IRQMASK: 0
>>> [  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa
>>> [  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000
>>> [  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000
>>> [  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0
>>> [  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7
>>> [  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000
>>> [  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70
>>> [  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000
>>> [  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
>>> [  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
>>> [  472.092033][ T6181] ---- interrupt: 3000
>>> [  472.098256][ T6181] pstore: backend (nvram) writing error (-1)
>>>
>>> This is the host, cannot run the kernel as guest because it fails to boot most
>>> of the time inside KVM.
>>>
>>>>
>>>>> Nonethless, there are quite a few platforms. Originally no preemption
>>>>> was the only option, and that's the reason why many people run that.
>>>>> It's the conservative, known working option. And that's the reason a lot
>>>>> of platfrom code does not get tested with more aggressive preemtion
>>>>> models, and never gets fixed to work with them.
>>>>>
>>>>
>>>> Full preemption has been there for many years!.
>>>
>>> Possible for years, forced only recently.
>>>
>>>> Lazy is not that aggressive compared to that.
>>>>
>>>>> Simply disabling the no preemtion option does not make the platform code
>>>>> ready.
>>>>>
>>>>
>>>> Let's understand your crash case. Let's see where it is going wrong. I am suspecting
>>>> it is some wrong usage of preemption api rather than arch can't support preemption.
>>>
>>> Very likely some wrong use of the preemption API by the arch code, or no
>>> use where it should have been used. It did not matter so long as people
>>> could run their no preempt configs and ignore the problem.
>>>
>>> Thanks
>>
>>
>> Friendly LLM analysis says preemption is enabled too early and before
>> completing kvmppc_handle_exit_hv() / kvmppc_handle_nested_exit()
>>
>> Below is ONLY a speculation and completely UNTESTED.
>> Maybe worth a try.
> 
> The patch is munged by the e-mail client, and it causes immediate
> voluntary preemprion in rcu critical section and hard lockup on starting
> a KVM VM.
> 
> Also it would be sort of bad news if it worked because that would be
> specific to book3s KVM HV and would not help with the KVM HV from the
> original report which likely is not book3s, nor with KVM PR.
> 

Thanks for trying. We will try a local repro and look into it why stack is
getting corrupted.

> Thanks
> 
> Michal

Re: [Linux PPC] Disable PREEMPT

[Christian Zigotzky]

> On 04 September 2026 at 10:56 Christian Zigotzky <chzigotzky@xenosoft.de> wrote:
> 
> On 03 September 2026 at 07:23 Shrikanth Hegde <sshegde@linux.ibm.com> wrote:
> 
> Friendly LLM analysis says preemption is enabled too early and before
> completing kvmppc_handle_exit_hv() / kvmppc_handle_nested_exit()
> 
> Below is ONLY a speculation and completely UNTESTED.
> Maybe worth a try.
> 
> ---
> diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
> index 0409ac9e7b31..78d2b0618887 100644
> --- a/arch/powerpc/kvm/book3s_hv.c
> +++ b/arch/powerpc/kvm/book3s_hv.c
> @@ -5049,8 +5049,6 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,
> 
>       powerpc_local_irq_pmu_restore(flags);
> 
> -       preempt_enable();
> -
>       /*
>        * cancel pending decrementer exception if DEC is now positive, or if
>        * entering a nested guest in which case the decrementer is now owned
> @@ -5072,6 +5070,8 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,
>       }
>       vcpu->arch.ret = r;
> 
> +       preempt_enable();
> +
>       if (is_kvmppc_resume_guest(r) && !kvmppc_vcpu_check_block(vcpu)) {
>               kvmppc_set_timer(vcpu);
> 
> 
> Michal
> 
> - - -
> 
> Michal,

Sorry, I meant Shrikanth.

> 
> Is this patch also for the issues with KVM PR and KVM HV if PREEMPT is activated?
> 
> The entire FSL Cyrus+ board freezes when using KVM HV with PREEMPT.
> 
> The guest kernel doesn't boot if we use KVM PR with a PREEMPT kernel on
> the PA Semi Nemo board.
> 
> Bug report: https://github.com/chzigotzky/kernels/issues/19
> 
> Thanks,
> Christian

Re: [Linux PPC] Disable PREEMPT

[Shrikanth Hegde]

On 9/4/26 2:50 PM, Christian Zigotzky wrote:
> 
>> On 04 September 2026 at 10:56 Christian Zigotzky <chzigotzky@xenosoft.de> wrote:
>>
>> On 03 September 2026 at 07:23 Shrikanth Hegde <sshegde@linux.ibm.com> wrote:
>>
>> Friendly LLM analysis says preemption is enabled too early and before
>> completing kvmppc_handle_exit_hv() / kvmppc_handle_nested_exit()
>>
>> Below is ONLY a speculation and completely UNTESTED.
>> Maybe worth a try.
>>
>> ---
>> diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
>> index 0409ac9e7b31..78d2b0618887 100644
>> --- a/arch/powerpc/kvm/book3s_hv.c
>> +++ b/arch/powerpc/kvm/book3s_hv.c
>> @@ -5049,8 +5049,6 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,
>>
>>        powerpc_local_irq_pmu_restore(flags);
>>
>> -       preempt_enable();
>> -
>>        /*
>>         * cancel pending decrementer exception if DEC is now positive, or if
>>         * entering a nested guest in which case the decrementer is now owned
>> @@ -5072,6 +5070,8 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,
>>        }
>>        vcpu->arch.ret = r;
>>
>> +       preempt_enable();
>> +
>>        if (is_kvmppc_resume_guest(r) && !kvmppc_vcpu_check_block(vcpu)) {
>>                kvmppc_set_timer(vcpu);
>>
>>
>> Michal
>>
>> - - -
>>
>> Michal,
> 
> Sorry, I meant Shrikanth.
> 

Leave that patch, it was only a speculation.

>>
>> Is this patch also for the issues with KVM PR and KVM HV if PREEMPT is activated?
>>
>> The entire FSL Cyrus+ board freezes when using KVM HV with PREEMPT.
>>

You have any logs? can you try with latest code? maybe 7.2? there is usage
of generic api to handle the work now which renders the patch shared long back
which you tried irrelevant.

>> The guest kernel doesn't boot if we use KVM PR with a PREEMPT kernel on
>> the PA Semi Nemo board.
>>
>> Bug report: https://github.com/chzigotzky/kernels/issues/19
>>
>> Thanks,
>> Christian
> 

Re: [Linux PPC] Disable PREEMPT

[Christian Zigotzky]

> On 04 September 2026 at 11:24 Shrikanth Hegde <sshegde@linux.ibm.com> wrote:
> 
> can you try with latest code? maybe 7.2? there is usage
> of generic api to handle the work now which renders the patch shared long back
> which you tried irrelevant.

Thanks. I will test the RC1 of kernel 7.3 with PREEMPT enabled.

- Christian

Re: [Linux PPC] Disable PREEMPT

[Christian Zigotzky]

On 04/09/26 11:34, Christian Zigotzky wrote:
>> On 04 September 2026 at 11:24 Shrikanth Hegde <sshegde@linux.ibm.com> wrote:
>>
>> can you try with latest code? maybe 7.2? there is usage
>> of generic api to handle the work now which renders the patch shared long back
>> which you tried irrelevant.
> Thanks. I will test the RC1 of kernel 7.3 with PREEMPT enabled.
>
> - Christian

I compiled and tested the RC1 of kernel 7.3 with PREEMPT support today 
[1]. Unfortunately the issue [2] still exists.

The entire FSL Cyrus+ board freezes when using KVM HV with PREEMPT.

I will disable PREEMPT with the Kconfig.preempt.patch [3] for the RC2.

- Christian

[1] https://github.com/chzigotzky/kernels/releases/tag/v7.3.0-rc1-preempt
[2] https://github.com/chzigotzky/kernels/issues/19
[3] 
https://github.com/chzigotzky/kernels/blob/main/patches/e5500/Kconfig.preempt.patch

-- 
Sent with BrassMonkey 34.3.2.1 (https://github.com/chzigotzky/Web-Browsers-and-Suites-for-Linux-PPC/releases/tag/BrassMonkey_34.3.2.1)

Re: [Linux PPC] Disable PREEMPT

[Michal Suchánek]

On Sat, Sep 05, 2026 at 03:07:40PM +0200, Christian Zigotzky wrote:
> On 04/09/26 11:34, Christian Zigotzky wrote:
> > > On 04 September 2026 at 11:24 Shrikanth Hegde <sshegde@linux.ibm.com> wrote:
> > > 
> > > can you try with latest code? maybe 7.2? there is usage
> > > of generic api to handle the work now which renders the patch shared long back
> > > which you tried irrelevant.
> > Thanks. I will test the RC1 of kernel 7.3 with PREEMPT enabled.
> > 
> > - Christian
> 
> I compiled and tested the RC1 of kernel 7.3 with PREEMPT support today [1].
> Unfortunately the issue [2] still exists.
> 
> The entire FSL Cyrus+ board freezes when using KVM HV with PREEMPT.
> 
> I will disable PREEMPT with the Kconfig.preempt.patch [3] for the RC2.

There is the obvious question which I did not notice answered in this
thread: Did you set up sysrq and/or kdump and find out where the kernel
is stuck?

Thanks

Michal

Re: [Linux PPC] Disable PREEMPT

[Christian Zigotzky]

On 07 September 2026 at 10:51 am, Michal Suchánek <msuchanek@suse.de> wrote:

There is the obvious question which I did not notice answered in this
thread: Did you set up sysrq and/or kdump and find out where the kernel
is stuck?

Thanks

Michal

- - -

I think I need kexec for kdump due to the system freezing after activating of KVM HV. Unfortunately kexec doesn’t work on my machines.

Further information: https://github.com/chzigotzky/kernels/releases/tag/Kexec

Re: [Linux PPC] Disable PREEMPT

[Michal Suchánek]

On Mon, Sep 07, 2026 at 01:32:18PM +0200, Christian Zigotzky wrote:
> On 07 September 2026 at 10:51 am, Michal Suchánek <msuchanek@suse.de> wrote:
> 
> There is the obvious question which I did not notice answered in this
> thread: Did you set up sysrq and/or kdump and find out where the kernel
> is stuck?
> 
> Thanks
> 
> Michal
> 
> - - -
> 
> I think I need kexec for kdump due to the system freezing after activating of KVM HV. Unfortunately kexec doesn’t work on my machines.
> 
> Further information: https://github.com/chzigotzky/kernels/releases/tag/Kexec

So there is some bug with kexec initialization on your platform.

Sysrg for dumping stacks to console might still work.

Thanks

Michal

Re: [Linux PPC] Disable PREEMPT

[Michal Suchánek]

On Fri, Sep 04, 2026 at 02:34:21PM +0530, Shrikanth Hegde wrote:
> 
> 
> On 9/4/26 1:05 PM, Michal Suchánek wrote:
> > On Thu, Sep 03, 2026 at 10:52:59PM +0530, Shrikanth Hegde wrote:
> > > 
> > > > > If possible run against current upstream and share the results.
> > > > 
> > > > https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default
> > > > 
> > > > [  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > > > [  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> > > > [  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> > > > [  472.091624][ T6181] Call Trace:
> > > > [  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
> > > > [  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
> > > > [  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
> > > > [  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
> > > > [  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > > > [  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
> > > > [  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
> > > > [  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
> > > > [  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
> > > > [  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
> > > > [  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
> > > > [  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
> > > > [  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
> > > > [  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
> > > > [  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
> > > > [  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
> > > > [  472.091938][ T6181] IRQMASK: 0
> > > > [  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa
> > > > [  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000
> > > > [  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000
> > > > [  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0
> > > > [  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7
> > > > [  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000
> > > > [  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70
> > > > [  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000
> > > > [  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
> > > > [  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
> > > > [  472.092033][ T6181] ---- interrupt: 3000
> > > > [  472.098256][ T6181] pstore: backend (nvram) writing error (-1)
> > > > 
> > > > This is the host, cannot run the kernel as guest because it fails to boot most
> > > > of the time inside KVM.
> > > > 
> > > > > 
> > > > > > Nonethless, there are quite a few platforms. Originally no preemption
> > > > > > was the only option, and that's the reason why many people run that.
> > > > > > It's the conservative, known working option. And that's the reason a lot
> > > > > > of platfrom code does not get tested with more aggressive preemtion
> > > > > > models, and never gets fixed to work with them.
> > > > > > 
> > > > > 
> > > > > Full preemption has been there for many years!.
> > > > 
> > > > Possible for years, forced only recently.
> > > > 
> > > > > Lazy is not that aggressive compared to that.
> > > > > 
> > > > > > Simply disabling the no preemtion option does not make the platform code
> > > > > > ready.
> > > > > > 
> > > > > 
> > > > > Let's understand your crash case. Let's see where it is going wrong. I am suspecting
> > > > > it is some wrong usage of preemption api rather than arch can't support preemption.
> > > > 
> > > > Very likely some wrong use of the preemption API by the arch code, or no
> > > > use where it should have been used. It did not matter so long as people
> > > > could run their no preempt configs and ignore the problem.
> > > > 
> > > > Thanks
> > > 
> > > 
> > > Friendly LLM analysis says preemption is enabled too early and before
> > > completing kvmppc_handle_exit_hv() / kvmppc_handle_nested_exit()
> > > 
> > > Below is ONLY a speculation and completely UNTESTED.
> > > Maybe worth a try.
> > 
> > The patch is munged by the e-mail client, and it causes immediate
> > voluntary preemprion in rcu critical section and hard lockup on starting
> > a KVM VM.
> > 
> > Also it would be sort of bad news if it worked because that would be
> > specific to book3s KVM HV and would not help with the KVM HV from the
> > original report which likely is not book3s, nor with KVM PR.
> > 
> 
> Thanks for trying. We will try a local repro and look into it why stack is
> getting corrupted.

There is also stack corruption when idling. At this point the machine is
stuck half-dead and would not go through with the panic, it requires
dump from the hyprvisor side to capture this.

Thanks

Michal

[542489.550547] [      T0] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: __sprint_symbol+0x1e4/0x1f0
[542489.557809] [      T0] ------------[ cut here ]------------
[542489.557814] [      T0] Voluntary context switch within RCU read-side critical section!
[542489.557822] [      T0] WARNING: [] kernel/rcu/tree_plugin.h:332 at rcu_note_context_switch+0x6a0/0x7b0, CPU#59: swapper/59/0
[542489.557837] [      T0] Modules linked in: vhost_net vhost vhost_iotlb tap rpadlpar_io rpaphp kvm_hv kvm mptcp_diag xsk_diag tcp_diag udp_diag raw_diag inet_diag unix_diag af_packet_diag netlink_diag xt_conntrack xt_MASQUERADE xfrm_user xt_set ip_set xt_addrtype nft_compat nf_conntrack_netbios_ns nf_conntrack_broadcast nft_masq nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct bonding tls nft_chain_nat nf_nat nf_conntrack tun nf_defrag_ipv6 nf_defrag_ipv4 rfkill af_packet binfmt_misc xfs aes_gcm_p10_crypto crypto_simd ibmveth cryptd xts vmx_crypto rtc_generic fuse nvme_fabrics ip_tables x_tables ibmvscsi scsi_transport_srp sd_mod scsi_dh_emc scsi_dh_rdac scsi_dh_alua sg scsi_mod scsi_common nvme nvme_core nvme_keyring nvme_auth pseries_wdt btrfs xor libblake2b raid6_pq dm_mirror dm_region_hash dm_log dm_mod br_netfilter bridge stp llc nf_tables
[542489.557944] [      T0] CPU: 59 UID: 0 PID: 0 Comm: swapper/59 Tainted: G      D W           7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
[542489.557957] [      T0] Tainted: [D]=DIE, [W]=WARN
[542489.557961] [      T0] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
[542489.557967] [      T0] NIP:  c00000000035d700 LR: c00000000035d6fc CTR: 0000000000000000
[542489.557974] [      T0] REGS: c000001de047f990 TRAP: 0700   Tainted: G      D W            (7.2.2-5.g6824496-default)
[542489.557980] [      T0] MSR:  800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE>  CR: 44002284  XER: 0000000e
[542489.557999] [      T0] CFAR: c00000000022d7a8 IRQMASK: 1 
                           GPR00: c00000000035d6fc c000001de047fc30 c00000000205ad00 000000000000003f 
                           GPR04: 3ffffffffffc02ef c000001de047f950 c000001de047f910 0000003ff0e20000 
                           GPR08: 0000000000000027 0000000000000001 c000001de0191300 0000000044002284 
                           GPR12: c000003ff1614890 c000003fffff9f00 0000000000000000 000000002eef9860 
                           GPR16: 0000000000000000 c000003ff40dcc80 0000003ff0e20000 c000000000058010 
                           GPR20: c000000000058160 c0000000032bcc80 c0000000032bcc80 c000000000057f80 
                           GPR24: ffffffffffffffff 000000000000003b c000001de0191300 c000000003bea6e0 
                           GPR28: c000000000058010 ffffffffffffffff 0000000000000000 c000003ff40de100 
[542489.558066] [      T0] NIP [c00000000035d700] rcu_note_context_switch+0x6a0/0x7b0
[542489.558072] [      T0] LR [c00000000035d6fc] rcu_note_context_switch+0x69c/0x7b0
[542489.558078] [      T0] Call Trace:
[542489.558082] [      T0] [c000001de047fc30] [c00000000035d6fc] rcu_note_context_switch+0x69c/0x7b0 (unreliable)
[542489.558092] [      T0] [c000001de047fcd0] [c0000000013c4fc0] __schedule+0xf0/0x1d10
[542489.558108] [      T0] [c000001de047fe20] [c0000000013c6eb0] schedule_idle+0x30/0x60
[542489.558121] [      T0] [c000001de047fe40] [c0000000002caed8] do_idle+0x258/0x470
[542489.558133] [      T0] [c000001de047feb0] [c0000000002cb3b4] cpu_startup_entry+0x54/0x60
[542489.558145] [      T0] [c000001de047fee0] [c00000000005a720] start_secondary+0x860/0xd70
[542489.558154] [      T0] [c000001de047ffe0] [c00000000000e158] start_secondary_prolog+0x10/0x14
[542489.558162] [      T0] Code: f8c900a8 4bfffe6c 60000000 60420000 3c62ff87 39200001 f9410068 3d4201b2 386329d8 992aa087 4becff91 60000000 <0fe00000> e9410068 4bfff9c4 60420000 
[542489.558184] [      T0] ---[ end trace 0000000000000000 ]---
[542521.228724] [     C59] watchdog: CPU 59 Hard LOCKUP
[542521.228726] [     C59] watchdog: CPU 59 TB:3697937465165330, last heartbeat TB:3697921085812082 (31990ms ago)
[542521.228728] [     C59] Modules linked in: vhost_net vhost vhost_iotlb tap rpadlpar_io rpaphp kvm_hv kvm mptcp_diag xsk_diag tcp_diag udp_diag raw_diag inet_diag unix_diag af_packet_diag netlink_diag xt_conntrack xt_MASQUERADE xfrm_user xt_set ip_set xt_addrtype nft_compat nf_conntrack_netbios_ns nf_conntrack_broadcast nft_masq nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct bonding tls nft_chain_nat nf_nat nf_conntrack tun nf_defrag_ipv6 nf_defrag_ipv4 rfkill af_packet binfmt_misc xfs aes_gcm_p10_crypto crypto_simd ibmveth cryptd xts vmx_crypto rtc_generic fuse nvme_fabrics ip_tables x_tables ibmvscsi scsi_transport_srp sd_mod scsi_dh_emc scsi_dh_rdac scsi_dh_alua sg scsi_mod scsi_common nvme nvme_core nvme_keyring nvme_auth pseries_wdt btrfs xor libblake2b raid6_pq dm_mirror dm_region_hash dm_log dm_mod br_netfilter bridge stp llc nf_tables
[542521.228776] [     C59] CPU: 59 UID: 0 PID: 0 Comm: swapper/59 Tainted: G      D W           7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
[542521.228780] [     C59] Tainted: [D]=DIE, [W]=WARN
[542521.228781] [     C59] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
[542521.228783] [     C59] NIP:  c0000000013d2c94 LR: c0000000013d5330 CTR: c000000003812be8
[542521.228785] [     C59] REGS: c000003ffed2bd60 TRAP: 0100   Tainted: G      D W            (7.2.2-5.g6824496-default)
[542521.228788] [     C59] MSR:  8000000000081033 <SF,ME,IR,DR,RI,LE>  CR: 42002282  XER: 2004000e
[542521.228794] [     C59] CFAR: 000000000000011c IRQMASK: 3 
                           GPR00: c000000000260a58 c000003ffef2fb60 c00000000205ad00 c000001deaab3c00 
                           GPR04: 00000000000001f8 0000000000000000 fffffffffffe0000 0000000000010000 
                           GPR08: 0000000000000001 0000000000000001 000000001c82ef77 fffffffffffffffd 
                           GPR12: c000000003812be0 c000003fffff9f00 0000000000000000 0000000000000000 
                           GPR16: 0000000000000000 0000000000000000 0000000000000001 c000000003bc2b00 
                           GPR20: c000003ff40c8de8 0000000000000001 c000003ff40c8da8 c000003ffef2fe08 
                           GPR24: 0000000000000000 c000000003bea6e0 000000000000003b 0000003ff0e20000 
                           GPR28: 0000000000780000 c0000000032bbe80 c000003ff40dbe80 c000001deaab3c00 
[542521.228820] [     C59] NIP [c0000000013d2c94] queued_spin_lock_slowpath+0x164/0x19a0
[542521.228824] [     C59] LR [c0000000013d5330] _raw_spin_lock+0x80/0xa0
[542521.228827] [     C59] Call Trace:
[542521.228827] [     C59] [c000003ffef2fb60] [0000000000001000] 0x1000 (unreliable)
[542521.228833] [     C59] [c000003ffef2fc60] [c000001de0191300] 0xc000001de0191300
[542521.228836] [     C59] [c000003ffef2fc80] [c000000000260a58] __queue_work+0x2f8/0x740
[542521.228841] [     C59] [c000003ffef2fd50] [c00000000038ceac] call_timer_fn+0x4c/0x260
[542521.228845] [     C59] [c000003ffef2fde0] [c00000000038d308] __run_timers+0x248/0x410
[542521.228849] [     C59] [c000003ffef2fec0] [c00000000038d56c] run_timer_softirq+0x9c/0x130
[542521.228852] [     C59] [c000003ffef2fef0] [c00000000023ab0c] handle_softirqs+0x14c/0x4a0
[542521.228855] [     C59] [c000003ffef2ffe0] [c000000000018418] do_softirq_own_stack+0x38/0x50
[542521.228860] [     C59] [c000001de047f910] [c00000000023a238] __irq_exit_rcu+0x198/0x1c0
[542521.228862] [     C59] [c000001de047f940] [c000000000026acc] timer_interrupt+0x1cc/0x4f0
[542521.228866] [     C59] [c000001de047f9b0] [c000000000009fdc] decrementer_common_virt+0x28c/0x290
[542521.228871] [     C59] ---- interrupt: 900 at plpar_hcall_norets_notrace+0x18/0x2c
[542521.228876] [     C59] NIP:  c0000000001b7634 LR: c0000000013c4898 CTR: 0000000000000000
[542521.228877] [     C59] REGS: c000001de047f9e0 TRAP: 0900   Tainted: G      D W            (7.2.2-5.g6824496-default)
[542521.228879] [     C59] MSR:  800000000280b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE>  CR: 22000282  XER: 2004000e
[542521.228886] [     C59] CFAR: 0000000000000000 IRQMASK: 0 
                           GPR00: 0000000000000000 c000001de047fc80 c00000000205ad00 0000000000000000 
                           GPR04: 0000000000000000 0000000000000039 0000000000000000 0000000000000000 
                           GPR08: ffffffffffffff7f 0000000000000000 ffffffffffffffff 0000000000000000 
                           GPR12: 0000000000000000 c000003fffff9f00 0000000000000000 000000002eef9860 
                           GPR16: 0000000000000000 0000000000000001 0000000000000001 c000000000058010 
                           GPR20: c000000000058160 0000000000000002 c000000000057fb0 c000000000057f80 
                           GPR24: ffffffffffffffff 0001ed6431fbb852 0000000000000000 0000000000000001 
                           GPR28: 0000000000000000 0000000000000000 c0000000032a1878 c0000000032a1870 
[542521.228911] [     C59] NIP [c0000000001b7634] plpar_hcall_norets_notrace+0x18/0x2c
[542521.228915] [     C59] LR [c0000000013c4898] check_and_cede_processor+0x48/0x60
[542521.228918] [     C59] ---- interrupt: 900
[542521.228919] [     C59] [c000001de047fc80] [ffffffffffffffff] 0xffffffffffffffff (unreliable)
[542521.228922] [     C59] [c000001de047fce0] [c0000000013c4944] dedicated_cede_loop+0x94/0x1a0
[542521.228926] [     C59] [c000001de047fd30] [c0000000013c3f00] cpuidle_enter_state+0x300/0x748
[542521.228930] [     C59] [c000001de047fde0] [c000000000f5ff10] cpuidle_enter+0x50/0x80
[542521.228935] [     C59] [c000001de047fe20] [c0000000002ba358] call_cpuidle+0x48/0x90
[542521.228939] [     C59] [c000001de047fe40] [c0000000002caf5c] do_idle+0x2dc/0x470
[542521.228942] [     C59] [c000001de047feb0] [c0000000002cb3b8] cpu_startup_entry+0x58/0x60
[542521.228945] [     C59] [c000001de047fee0] [c00000000005a720] start_secondary+0x860/0xd70
[542521.228948] [     C59] [c000001de047ffe0] [c00000000000e158] start_secondary_prolog+0x10/0x14
[542521.228951] [     C59] Code: 41810e8c 39400000 39000000 3cc0fffe 60000000 813f0000 71290001 41820050 3ce00001 60000000 60420000 7c40003c <71090001> 394a0001 40820028 3d2201ba 
[568119.679467] [     C59] pstore: dump skipped in NMI path because of concurrent dump

Re: [Linux PPC] Disable PREEMPT

[Michal Suchánek]

On Thu, Sep 10, 2026 at 10:54:07AM +0200, Michal Suchánek wrote:
> On Mon, Sep 07, 2026 at 01:32:18PM +0200, Christian Zigotzky wrote:
> > On 07 September 2026 at 10:51 am, Michal Suchánek <msuchanek@suse.de> wrote:
> > 
> > There is the obvious question which I did not notice answered in this
> > thread: Did you set up sysrq and/or kdump and find out where the kernel
> > is stuck?
> > 
> > Thanks
> > 
> > Michal
> > 
> > - - -
> > 
> > I think I need kexec for kdump due to the system freezing after activating of KVM HV. Unfortunately kexec doesn’t work on my machines.
> > 
> > Further information: https://github.com/chzigotzky/kernels/releases/tag/Kexec
> 
> So there is some bug with kexec initialization on your platform.

Right, you are missing this part:
https://github.com/torvalds/linux/blob/50d05c7c76c96b90462f24debacca971d2e86713/arch/powerpc/kexec/core.c#L205

Maybe it needs to be called from your platform code or something.

Thanks

Michal

> 
> Sysrg for dumping stacks to console might still work.
> 
> Thanks
> 
> Michal
> 

Re: [Linux PPC] Disable PREEMPT

[Shrikanth Hegde]

On 9/10/26 2:29 PM, Michal Suchánek wrote:
> On Fri, Sep 04, 2026 at 02:34:21PM +0530, Shrikanth Hegde wrote:
>>
>>
>> On 9/4/26 1:05 PM, Michal Suchánek wrote:
>>> On Thu, Sep 03, 2026 at 10:52:59PM +0530, Shrikanth Hegde wrote:
>>>>
>>>>>> If possible run against current upstream and share the results.
>>>>>
>>>>> https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default
>>>>>
>>>>> [  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
>>>>> [  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
>>>>> [  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
>>>>> [  472.091624][ T6181] Call Trace:
>>>>> [  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
>>>>> [  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
>>>>> [  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
>>>>> [  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
>>>>> [  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
>>>>> [  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
>>>>> [  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
>>>>> [  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
>>>>> [  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
>>>>> [  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
>>>>> [  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
>>>>> [  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
>>>>> [  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
>>>>> [  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
>>>>> [  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
>>>>> [  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
>>>>> [  472.091938][ T6181] IRQMASK: 0
>>>>> [  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa
>>>>> [  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000
>>>>> [  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000
>>>>> [  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0
>>>>> [  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7
>>>>> [  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000
>>>>> [  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70
>>>>> [  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000
>>>>> [  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
>>>>> [  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
>>>>> [  472.092033][ T6181] ---- interrupt: 3000
>>>>> [  472.098256][ T6181] pstore: backend (nvram) writing error (-1)
>>>>>
>>>>> This is the host, cannot run the kernel as guest because it fails to boot most
>>>>> of the time inside KVM.
>>>>>
>>>>>>
>>>>>>> Nonethless, there are quite a few platforms. Originally no preemption
>>>>>>> was the only option, and that's the reason why many people run that.
>>>>>>> It's the conservative, known working option. And that's the reason a lot
>>>>>>> of platfrom code does not get tested with more aggressive preemtion
>>>>>>> models, and never gets fixed to work with them.
>>>>>>>
>>>>>>
>>>>>> Full preemption has been there for many years!.
>>>>>
>>>>> Possible for years, forced only recently.
>>>>>
>>>>>> Lazy is not that aggressive compared to that.
>>>>>>
>>>>>>> Simply disabling the no preemtion option does not make the platform code
>>>>>>> ready.
>>>>>>>
>>>>>>
>>>>>> Let's understand your crash case. Let's see where it is going wrong. I am suspecting
>>>>>> it is some wrong usage of preemption api rather than arch can't support preemption.
>>>>>
>>>>> Very likely some wrong use of the preemption API by the arch code, or no
>>>>> use where it should have been used. It did not matter so long as people
>>>>> could run their no preempt configs and ignore the problem.
>>>>>
>>>>> Thanks
>>>>
>>>>
>>>> Friendly LLM analysis says preemption is enabled too early and before
>>>> completing kvmppc_handle_exit_hv() / kvmppc_handle_nested_exit()
>>>>
>>>> Below is ONLY a speculation and completely UNTESTED.
>>>> Maybe worth a try.
>>>
>>> The patch is munged by the e-mail client, and it causes immediate
>>> voluntary preemprion in rcu critical section and hard lockup on starting
>>> a KVM VM.
>>>
>>> Also it would be sort of bad news if it worked because that would be
>>> specific to book3s KVM HV and would not help with the KVM HV from the
>>> original report which likely is not book3s, nor with KVM PR.
>>>
>>
>> Thanks for trying. We will try a local repro and look into it why stack is
>> getting corrupted.
> 
> There is also stack corruption when idling. At this point the machine is
> stuck half-dead and would not go through with the panic, it requires
> dump from the hyprvisor side to capture this.
> 
> Thanks
> 
> Michal
> 


is this on host with kvm running or this is without kvm running anything?

Do you see corruption if you don't have kvm module loaded?

These maybe two independent issue.
Our KVM team is trying local repro for the kvm issue you have reported.

If possible, could you try vmcore capture (maybe with fadump)?

> [542489.550547] [      T0] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: __sprint_symbol+0x1e4/0x1f0
> [542489.557809] [      T0] ------------[ cut here ]------------
> [542489.557814] [      T0] Voluntary context switch within RCU read-side critical section!
> [542489.557822] [      T0] WARNING: [] kernel/rcu/tree_plugin.h:332 at rcu_note_context_switch+0x6a0/0x7b0, CPU#59: swapper/59/0
> [542489.557837] [      T0] Modules linked in: vhost_net vhost vhost_iotlb tap rpadlpar_io rpaphp kvm_hv kvm mptcp_diag xsk_diag tcp_diag udp_diag raw_diag inet_diag unix_diag af_packet_diag netlink_diag xt_conntrack xt_MASQUERADE xfrm_user xt_set ip_set xt_addrtype nft_compat nf_conntrack_netbios_ns nf_conntrack_broadcast nft_masq nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct bonding tls nft_chain_nat nf_nat nf_conntrack tun nf_defrag_ipv6 nf_defrag_ipv4 rfkill af_packet binfmt_misc xfs aes_gcm_p10_crypto crypto_simd ibmveth cryptd xts vmx_crypto rtc_generic fuse nvme_fabrics ip_tables x_tables ibmvscsi scsi_transport_srp sd_mod scsi_dh_emc scsi_dh_rdac scsi_dh_alua sg scsi_mod scsi_common nvme nvme_core nvme_keyring nvme_auth pseries_wdt btrfs xor libblake2b raid6_pq dm_mirror dm_region_hash dm_log dm_mod br_netfilter bridge stp llc nf_tables
> [542489.557944] [      T0] CPU: 59 UID: 0 PID: 0 Comm: swapper/59 Tainted: G      D W           7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> [542489.557957] [      T0] Tainted: [D]=DIE, [W]=WARN
> [542489.557961] [      T0] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> [542489.557967] [      T0] NIP:  c00000000035d700 LR: c00000000035d6fc CTR: 0000000000000000
> [542489.557974] [      T0] REGS: c000001de047f990 TRAP: 0700   Tainted: G      D W            (7.2.2-5.g6824496-default)
> [542489.557980] [      T0] MSR:  800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE>  CR: 44002284  XER: 0000000e
> [542489.557999] [      T0] CFAR: c00000000022d7a8 IRQMASK: 1
>                             GPR00: c00000000035d6fc c000001de047fc30 c00000000205ad00 000000000000003f
>                             GPR04: 3ffffffffffc02ef c000001de047f950 c000001de047f910 0000003ff0e20000
>                             GPR08: 0000000000000027 0000000000000001 c000001de0191300 0000000044002284
>                             GPR12: c000003ff1614890 c000003fffff9f00 0000000000000000 000000002eef9860
>                             GPR16: 0000000000000000 c000003ff40dcc80 0000003ff0e20000 c000000000058010
>                             GPR20: c000000000058160 c0000000032bcc80 c0000000032bcc80 c000000000057f80
>                             GPR24: ffffffffffffffff 000000000000003b c000001de0191300 c000000003bea6e0
>                             GPR28: c000000000058010 ffffffffffffffff 0000000000000000 c000003ff40de100
> [542489.558066] [      T0] NIP [c00000000035d700] rcu_note_context_switch+0x6a0/0x7b0
> [542489.558072] [      T0] LR [c00000000035d6fc] rcu_note_context_switch+0x69c/0x7b0
> [542489.558078] [      T0] Call Trace:
> [542489.558082] [      T0] [c000001de047fc30] [c00000000035d6fc] rcu_note_context_switch+0x69c/0x7b0 (unreliable)
> [542489.558092] [      T0] [c000001de047fcd0] [c0000000013c4fc0] __schedule+0xf0/0x1d10
> [542489.558108] [      T0] [c000001de047fe20] [c0000000013c6eb0] schedule_idle+0x30/0x60
> [542489.558121] [      T0] [c000001de047fe40] [c0000000002caed8] do_idle+0x258/0x470
> [542489.558133] [      T0] [c000001de047feb0] [c0000000002cb3b4] cpu_startup_entry+0x54/0x60
> [542489.558145] [      T0] [c000001de047fee0] [c00000000005a720] start_secondary+0x860/0xd70
> [542489.558154] [      T0] [c000001de047ffe0] [c00000000000e158] start_secondary_prolog+0x10/0x14
> [542489.558162] [      T0] Code: f8c900a8 4bfffe6c 60000000 60420000 3c62ff87 39200001 f9410068 3d4201b2 386329d8 992aa087 4becff91 60000000 <0fe00000> e9410068 4bfff9c4 60420000
> [542489.558184] [      T0] ---[ end trace 0000000000000000 ]---
> [542521.228724] [     C59] watchdog: CPU 59 Hard LOCKUP
> [542521.228726] [     C59] watchdog: CPU 59 TB:3697937465165330, last heartbeat TB:3697921085812082 (31990ms ago)
> [542521.228728] [     C59] Modules linked in: vhost_net vhost vhost_iotlb tap rpadlpar_io rpaphp kvm_hv kvm mptcp_diag xsk_diag tcp_diag udp_diag raw_diag inet_diag unix_diag af_packet_diag netlink_diag xt_conntrack xt_MASQUERADE xfrm_user xt_set ip_set xt_addrtype nft_compat nf_conntrack_netbios_ns nf_conntrack_broadcast nft_masq nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct bonding tls nft_chain_nat nf_nat nf_conntrack tun nf_defrag_ipv6 nf_defrag_ipv4 rfkill af_packet binfmt_misc xfs aes_gcm_p10_crypto crypto_simd ibmveth cryptd xts vmx_crypto rtc_generic fuse nvme_fabrics ip_tables x_tables ibmvscsi scsi_transport_srp sd_mod scsi_dh_emc scsi_dh_rdac scsi_dh_alua sg scsi_mod scsi_common nvme nvme_core nvme_keyring nvme_auth pseries_wdt btrfs xor libblake2b raid6_pq dm_mirror dm_region_hash dm_log dm_mod br_netfilter bridge stp llc nf_tables
> [542521.228776] [     C59] CPU: 59 UID: 0 PID: 0 Comm: swapper/59 Tainted: G      D W           7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> [542521.228780] [     C59] Tainted: [D]=DIE, [W]=WARN
> [542521.228781] [     C59] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> [542521.228783] [     C59] NIP:  c0000000013d2c94 LR: c0000000013d5330 CTR: c000000003812be8
> [542521.228785] [     C59] REGS: c000003ffed2bd60 TRAP: 0100   Tainted: G      D W            (7.2.2-5.g6824496-default)
> [542521.228788] [     C59] MSR:  8000000000081033 <SF,ME,IR,DR,RI,LE>  CR: 42002282  XER: 2004000e
> [542521.228794] [     C59] CFAR: 000000000000011c IRQMASK: 3
>                             GPR00: c000000000260a58 c000003ffef2fb60 c00000000205ad00 c000001deaab3c00
>                             GPR04: 00000000000001f8 0000000000000000 fffffffffffe0000 0000000000010000
>                             GPR08: 0000000000000001 0000000000000001 000000001c82ef77 fffffffffffffffd
>                             GPR12: c000000003812be0 c000003fffff9f00 0000000000000000 0000000000000000
>                             GPR16: 0000000000000000 0000000000000000 0000000000000001 c000000003bc2b00
>                             GPR20: c000003ff40c8de8 0000000000000001 c000003ff40c8da8 c000003ffef2fe08
>                             GPR24: 0000000000000000 c000000003bea6e0 000000000000003b 0000003ff0e20000
>                             GPR28: 0000000000780000 c0000000032bbe80 c000003ff40dbe80 c000001deaab3c00
> [542521.228820] [     C59] NIP [c0000000013d2c94] queued_spin_lock_slowpath+0x164/0x19a0
> [542521.228824] [     C59] LR [c0000000013d5330] _raw_spin_lock+0x80/0xa0
> [542521.228827] [     C59] Call Trace:
> [542521.228827] [     C59] [c000003ffef2fb60] [0000000000001000] 0x1000 (unreliable)
> [542521.228833] [     C59] [c000003ffef2fc60] [c000001de0191300] 0xc000001de0191300
> [542521.228836] [     C59] [c000003ffef2fc80] [c000000000260a58] __queue_work+0x2f8/0x740
> [542521.228841] [     C59] [c000003ffef2fd50] [c00000000038ceac] call_timer_fn+0x4c/0x260
> [542521.228845] [     C59] [c000003ffef2fde0] [c00000000038d308] __run_timers+0x248/0x410
> [542521.228849] [     C59] [c000003ffef2fec0] [c00000000038d56c] run_timer_softirq+0x9c/0x130
> [542521.228852] [     C59] [c000003ffef2fef0] [c00000000023ab0c] handle_softirqs+0x14c/0x4a0
> [542521.228855] [     C59] [c000003ffef2ffe0] [c000000000018418] do_softirq_own_stack+0x38/0x50
> [542521.228860] [     C59] [c000001de047f910] [c00000000023a238] __irq_exit_rcu+0x198/0x1c0
> [542521.228862] [     C59] [c000001de047f940] [c000000000026acc] timer_interrupt+0x1cc/0x4f0
> [542521.228866] [     C59] [c000001de047f9b0] [c000000000009fdc] decrementer_common_virt+0x28c/0x290
> [542521.228871] [     C59] ---- interrupt: 900 at plpar_hcall_norets_notrace+0x18/0x2c
> [542521.228876] [     C59] NIP:  c0000000001b7634 LR: c0000000013c4898 CTR: 0000000000000000
> [542521.228877] [     C59] REGS: c000001de047f9e0 TRAP: 0900   Tainted: G      D W            (7.2.2-5.g6824496-default)
> [542521.228879] [     C59] MSR:  800000000280b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE>  CR: 22000282  XER: 2004000e
> [542521.228886] [     C59] CFAR: 0000000000000000 IRQMASK: 0
>                             GPR00: 0000000000000000 c000001de047fc80 c00000000205ad00 0000000000000000
>                             GPR04: 0000000000000000 0000000000000039 0000000000000000 0000000000000000
>                             GPR08: ffffffffffffff7f 0000000000000000 ffffffffffffffff 0000000000000000
>                             GPR12: 0000000000000000 c000003fffff9f00 0000000000000000 000000002eef9860
>                             GPR16: 0000000000000000 0000000000000001 0000000000000001 c000000000058010
>                             GPR20: c000000000058160 0000000000000002 c000000000057fb0 c000000000057f80
>                             GPR24: ffffffffffffffff 0001ed6431fbb852 0000000000000000 0000000000000001
>                             GPR28: 0000000000000000 0000000000000000 c0000000032a1878 c0000000032a1870
> [542521.228911] [     C59] NIP [c0000000001b7634] plpar_hcall_norets_notrace+0x18/0x2c
> [542521.228915] [     C59] LR [c0000000013c4898] check_and_cede_processor+0x48/0x60
> [542521.228918] [     C59] ---- interrupt: 900
> [542521.228919] [     C59] [c000001de047fc80] [ffffffffffffffff] 0xffffffffffffffff (unreliable)
> [542521.228922] [     C59] [c000001de047fce0] [c0000000013c4944] dedicated_cede_loop+0x94/0x1a0
> [542521.228926] [     C59] [c000001de047fd30] [c0000000013c3f00] cpuidle_enter_state+0x300/0x748
> [542521.228930] [     C59] [c000001de047fde0] [c000000000f5ff10] cpuidle_enter+0x50/0x80
> [542521.228935] [     C59] [c000001de047fe20] [c0000000002ba358] call_cpuidle+0x48/0x90
> [542521.228939] [     C59] [c000001de047fe40] [c0000000002caf5c] do_idle+0x2dc/0x470
> [542521.228942] [     C59] [c000001de047feb0] [c0000000002cb3b8] cpu_startup_entry+0x58/0x60
> [542521.228945] [     C59] [c000001de047fee0] [c00000000005a720] start_secondary+0x860/0xd70
> [542521.228948] [     C59] [c000001de047ffe0] [c00000000000e158] start_secondary_prolog+0x10/0x14
> [542521.228951] [     C59] Code: 41810e8c 39400000 39000000 3cc0fffe 60000000 813f0000 71290001 41820050 3ce00001 60000000 60420000 7c40003c <71090001> 394a0001 40820028 3d2201ba
> [568119.679467] [     C59] pstore: dump skipped in NMI path because of concurrent dump
> 

Re: [Linux PPC] Disable PREEMPT

[Michal Suchánek]

On Thu, Sep 10, 2026 at 04:11:20PM +0530, Shrikanth Hegde wrote:
> 
> 
> On 9/10/26 2:29 PM, Michal Suchánek wrote:
> > On Fri, Sep 04, 2026 at 02:34:21PM +0530, Shrikanth Hegde wrote:
> > > 
> > > 
> > > On 9/4/26 1:05 PM, Michal Suchánek wrote:
> > > > On Thu, Sep 03, 2026 at 10:52:59PM +0530, Shrikanth Hegde wrote:
> > > > > 
> > > > > > > If possible run against current upstream and share the results.
> > > > > > 
> > > > > > https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default
> > > > > > 
> > > > > > [  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > > > > > [  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> > > > > > [  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> > > > > > [  472.091624][ T6181] Call Trace:
> > > > > > [  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
> > > > > > [  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
> > > > > > [  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
> > > > > > [  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
> > > > > > [  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > > > > > [  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
> > > > > > [  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
> > > > > > [  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
> > > > > > [  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
> > > > > > [  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
> > > > > > [  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
> > > > > > [  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
> > > > > > [  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
> > > > > > [  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
> > > > > > [  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
> > > > > > [  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
> > > > > > [  472.091938][ T6181] IRQMASK: 0
> > > > > > [  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa
> > > > > > [  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000
> > > > > > [  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000
> > > > > > [  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0
> > > > > > [  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7
> > > > > > [  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000
> > > > > > [  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70
> > > > > > [  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000
> > > > > > [  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
> > > > > > [  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
> > > > > > [  472.092033][ T6181] ---- interrupt: 3000
> > > > > > [  472.098256][ T6181] pstore: backend (nvram) writing error (-1)
> > > > > > 
> > > > > > This is the host, cannot run the kernel as guest because it fails to boot most
> > > > > > of the time inside KVM.
> > > > > > 
> > > > > > > 
> > > > > > > > Nonethless, there are quite a few platforms. Originally no preemption
> > > > > > > > was the only option, and that's the reason why many people run that.
> > > > > > > > It's the conservative, known working option. And that's the reason a lot
> > > > > > > > of platfrom code does not get tested with more aggressive preemtion
> > > > > > > > models, and never gets fixed to work with them.
> > > > > > > > 
> > > > > > > 
> > > > > > > Full preemption has been there for many years!.
> > > > > > 
> > > > > > Possible for years, forced only recently.
> > > > > > 
> > > > > > > Lazy is not that aggressive compared to that.
> > > > > > > 
> > > > > > > > Simply disabling the no preemtion option does not make the platform code
> > > > > > > > ready.
> > > > > > > > 
> > > > > > > 
> > > > > > > Let's understand your crash case. Let's see where it is going wrong. I am suspecting
> > > > > > > it is some wrong usage of preemption api rather than arch can't support preemption.
> > > > > > 
> > > > > > Very likely some wrong use of the preemption API by the arch code, or no
> > > > > > use where it should have been used. It did not matter so long as people
> > > > > > could run their no preempt configs and ignore the problem.
> > > > > > 
> > > > > > Thanks
> > > > > 
> > > > > 
> > > > > Friendly LLM analysis says preemption is enabled too early and before
> > > > > completing kvmppc_handle_exit_hv() / kvmppc_handle_nested_exit()
> > > > > 
> > > > > Below is ONLY a speculation and completely UNTESTED.
> > > > > Maybe worth a try.
> > > > 
> > > > The patch is munged by the e-mail client, and it causes immediate
> > > > voluntary preemprion in rcu critical section and hard lockup on starting
> > > > a KVM VM.
> > > > 
> > > > Also it would be sort of bad news if it worked because that would be
> > > > specific to book3s KVM HV and would not help with the KVM HV from the
> > > > original report which likely is not book3s, nor with KVM PR.
> > > > 
> > > 
> > > Thanks for trying. We will try a local repro and look into it why stack is
> > > getting corrupted.
> > 
> > There is also stack corruption when idling. At this point the machine is
> > stuck half-dead and would not go through with the panic, it requires
> > dump from the hyprvisor side to capture this.
> > 
> > Thanks
> > 
> > Michal
> > 
> 
> 
> is this on host with kvm running or this is without kvm running anything?

Difficult to say after the fact. Given running KVM VMs is crashing with
this kernel most likely no VM was running.

> 
> Do you see corruption if you don't have kvm module loaded?

Difficult to say, this takes hours to days to reproduce, and the module
is normally loaded by default. I can blacklist it to make sure it does
not get loaded.

> 
> These maybe two independent issue.
> Our KVM team is trying local repro for the kvm issue you have reported.
> 
> If possible, could you try vmcore capture (maybe with fadump)?

I have the vmcore, this dmesg is from the vmcore. I would not get
displayed otherwise, or maybe some content from the console is lost.

Unfortunately, this is a kernel from a rolling distribution, and may be
difficult to analyze bacause new kernel build is used by now.

Thanks

Michal

> 
> > [542489.550547] [      T0] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: __sprint_symbol+0x1e4/0x1f0
> > [542489.557809] [      T0] ------------[ cut here ]------------
> > [542489.557814] [      T0] Voluntary context switch within RCU read-side critical section!
> > [542489.557822] [      T0] WARNING: [] kernel/rcu/tree_plugin.h:332 at rcu_note_context_switch+0x6a0/0x7b0, CPU#59: swapper/59/0
> > [542489.557837] [      T0] Modules linked in: vhost_net vhost vhost_iotlb tap rpadlpar_io rpaphp kvm_hv kvm mptcp_diag xsk_diag tcp_diag udp_diag raw_diag inet_diag unix_diag af_packet_diag netlink_diag xt_conntrack xt_MASQUERADE xfrm_user xt_set ip_set xt_addrtype nft_compat nf_conntrack_netbios_ns nf_conntrack_broadcast nft_masq nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct bonding tls nft_chain_nat nf_nat nf_conntrack tun nf_defrag_ipv6 nf_defrag_ipv4 rfkill af_packet binfmt_misc xfs aes_gcm_p10_crypto crypto_simd ibmveth cryptd xts vmx_crypto rtc_generic fuse nvme_fabrics ip_tables x_tables ibmvscsi scsi_transport_srp sd_mod scsi_dh_emc scsi_dh_rdac scsi_dh_alua sg scsi_mod scsi_common nvme nvme_core nvme_keyring nvme_auth pseries_wdt btrfs xor libblake2b raid6_pq dm_mirror dm_region_hash dm_log dm_mod br_netfilter bridge stp llc nf_tables
> > [542489.557944] [      T0] CPU: 59 UID: 0 PID: 0 Comm: swapper/59 Tainted: G      D W           7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> > [542489.557957] [      T0] Tainted: [D]=DIE, [W]=WARN
> > [542489.557961] [      T0] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> > [542489.557967] [      T0] NIP:  c00000000035d700 LR: c00000000035d6fc CTR: 0000000000000000
> > [542489.557974] [      T0] REGS: c000001de047f990 TRAP: 0700   Tainted: G      D W            (7.2.2-5.g6824496-default)
> > [542489.557980] [      T0] MSR:  800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE>  CR: 44002284  XER: 0000000e
> > [542489.557999] [      T0] CFAR: c00000000022d7a8 IRQMASK: 1
> >                             GPR00: c00000000035d6fc c000001de047fc30 c00000000205ad00 000000000000003f
> >                             GPR04: 3ffffffffffc02ef c000001de047f950 c000001de047f910 0000003ff0e20000
> >                             GPR08: 0000000000000027 0000000000000001 c000001de0191300 0000000044002284
> >                             GPR12: c000003ff1614890 c000003fffff9f00 0000000000000000 000000002eef9860
> >                             GPR16: 0000000000000000 c000003ff40dcc80 0000003ff0e20000 c000000000058010
> >                             GPR20: c000000000058160 c0000000032bcc80 c0000000032bcc80 c000000000057f80
> >                             GPR24: ffffffffffffffff 000000000000003b c000001de0191300 c000000003bea6e0
> >                             GPR28: c000000000058010 ffffffffffffffff 0000000000000000 c000003ff40de100
> > [542489.558066] [      T0] NIP [c00000000035d700] rcu_note_context_switch+0x6a0/0x7b0
> > [542489.558072] [      T0] LR [c00000000035d6fc] rcu_note_context_switch+0x69c/0x7b0
> > [542489.558078] [      T0] Call Trace:
> > [542489.558082] [      T0] [c000001de047fc30] [c00000000035d6fc] rcu_note_context_switch+0x69c/0x7b0 (unreliable)
> > [542489.558092] [      T0] [c000001de047fcd0] [c0000000013c4fc0] __schedule+0xf0/0x1d10
> > [542489.558108] [      T0] [c000001de047fe20] [c0000000013c6eb0] schedule_idle+0x30/0x60
> > [542489.558121] [      T0] [c000001de047fe40] [c0000000002caed8] do_idle+0x258/0x470
> > [542489.558133] [      T0] [c000001de047feb0] [c0000000002cb3b4] cpu_startup_entry+0x54/0x60
> > [542489.558145] [      T0] [c000001de047fee0] [c00000000005a720] start_secondary+0x860/0xd70
> > [542489.558154] [      T0] [c000001de047ffe0] [c00000000000e158] start_secondary_prolog+0x10/0x14
> > [542489.558162] [      T0] Code: f8c900a8 4bfffe6c 60000000 60420000 3c62ff87 39200001 f9410068 3d4201b2 386329d8 992aa087 4becff91 60000000 <0fe00000> e9410068 4bfff9c4 60420000
> > [542489.558184] [      T0] ---[ end trace 0000000000000000 ]---
> > [542521.228724] [     C59] watchdog: CPU 59 Hard LOCKUP
> > [542521.228726] [     C59] watchdog: CPU 59 TB:3697937465165330, last heartbeat TB:3697921085812082 (31990ms ago)
> > [542521.228728] [     C59] Modules linked in: vhost_net vhost vhost_iotlb tap rpadlpar_io rpaphp kvm_hv kvm mptcp_diag xsk_diag tcp_diag udp_diag raw_diag inet_diag unix_diag af_packet_diag netlink_diag xt_conntrack xt_MASQUERADE xfrm_user xt_set ip_set xt_addrtype nft_compat nf_conntrack_netbios_ns nf_conntrack_broadcast nft_masq nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct bonding tls nft_chain_nat nf_nat nf_conntrack tun nf_defrag_ipv6 nf_defrag_ipv4 rfkill af_packet binfmt_misc xfs aes_gcm_p10_crypto crypto_simd ibmveth cryptd xts vmx_crypto rtc_generic fuse nvme_fabrics ip_tables x_tables ibmvscsi scsi_transport_srp sd_mod scsi_dh_emc scsi_dh_rdac scsi_dh_alua sg scsi_mod scsi_common nvme nvme_core nvme_keyring nvme_auth pseries_wdt btrfs xor libblake2b raid6_pq dm_mirror dm_region_hash dm_log dm_mod br_netfilter bridge stp llc nf_tables
> > [542521.228776] [     C59] CPU: 59 UID: 0 PID: 0 Comm: swapper/59 Tainted: G      D W           7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> > [542521.228780] [     C59] Tainted: [D]=DIE, [W]=WARN
> > [542521.228781] [     C59] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> > [542521.228783] [     C59] NIP:  c0000000013d2c94 LR: c0000000013d5330 CTR: c000000003812be8
> > [542521.228785] [     C59] REGS: c000003ffed2bd60 TRAP: 0100   Tainted: G      D W            (7.2.2-5.g6824496-default)
> > [542521.228788] [     C59] MSR:  8000000000081033 <SF,ME,IR,DR,RI,LE>  CR: 42002282  XER: 2004000e
> > [542521.228794] [     C59] CFAR: 000000000000011c IRQMASK: 3
> >                             GPR00: c000000000260a58 c000003ffef2fb60 c00000000205ad00 c000001deaab3c00
> >                             GPR04: 00000000000001f8 0000000000000000 fffffffffffe0000 0000000000010000
> >                             GPR08: 0000000000000001 0000000000000001 000000001c82ef77 fffffffffffffffd
> >                             GPR12: c000000003812be0 c000003fffff9f00 0000000000000000 0000000000000000
> >                             GPR16: 0000000000000000 0000000000000000 0000000000000001 c000000003bc2b00
> >                             GPR20: c000003ff40c8de8 0000000000000001 c000003ff40c8da8 c000003ffef2fe08
> >                             GPR24: 0000000000000000 c000000003bea6e0 000000000000003b 0000003ff0e20000
> >                             GPR28: 0000000000780000 c0000000032bbe80 c000003ff40dbe80 c000001deaab3c00
> > [542521.228820] [     C59] NIP [c0000000013d2c94] queued_spin_lock_slowpath+0x164/0x19a0
> > [542521.228824] [     C59] LR [c0000000013d5330] _raw_spin_lock+0x80/0xa0
> > [542521.228827] [     C59] Call Trace:
> > [542521.228827] [     C59] [c000003ffef2fb60] [0000000000001000] 0x1000 (unreliable)
> > [542521.228833] [     C59] [c000003ffef2fc60] [c000001de0191300] 0xc000001de0191300
> > [542521.228836] [     C59] [c000003ffef2fc80] [c000000000260a58] __queue_work+0x2f8/0x740
> > [542521.228841] [     C59] [c000003ffef2fd50] [c00000000038ceac] call_timer_fn+0x4c/0x260
> > [542521.228845] [     C59] [c000003ffef2fde0] [c00000000038d308] __run_timers+0x248/0x410
> > [542521.228849] [     C59] [c000003ffef2fec0] [c00000000038d56c] run_timer_softirq+0x9c/0x130
> > [542521.228852] [     C59] [c000003ffef2fef0] [c00000000023ab0c] handle_softirqs+0x14c/0x4a0
> > [542521.228855] [     C59] [c000003ffef2ffe0] [c000000000018418] do_softirq_own_stack+0x38/0x50
> > [542521.228860] [     C59] [c000001de047f910] [c00000000023a238] __irq_exit_rcu+0x198/0x1c0
> > [542521.228862] [     C59] [c000001de047f940] [c000000000026acc] timer_interrupt+0x1cc/0x4f0
> > [542521.228866] [     C59] [c000001de047f9b0] [c000000000009fdc] decrementer_common_virt+0x28c/0x290
> > [542521.228871] [     C59] ---- interrupt: 900 at plpar_hcall_norets_notrace+0x18/0x2c
> > [542521.228876] [     C59] NIP:  c0000000001b7634 LR: c0000000013c4898 CTR: 0000000000000000
> > [542521.228877] [     C59] REGS: c000001de047f9e0 TRAP: 0900   Tainted: G      D W            (7.2.2-5.g6824496-default)
> > [542521.228879] [     C59] MSR:  800000000280b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE>  CR: 22000282  XER: 2004000e
> > [542521.228886] [     C59] CFAR: 0000000000000000 IRQMASK: 0
> >                             GPR00: 0000000000000000 c000001de047fc80 c00000000205ad00 0000000000000000
> >                             GPR04: 0000000000000000 0000000000000039 0000000000000000 0000000000000000
> >                             GPR08: ffffffffffffff7f 0000000000000000 ffffffffffffffff 0000000000000000
> >                             GPR12: 0000000000000000 c000003fffff9f00 0000000000000000 000000002eef9860
> >                             GPR16: 0000000000000000 0000000000000001 0000000000000001 c000000000058010
> >                             GPR20: c000000000058160 0000000000000002 c000000000057fb0 c000000000057f80
> >                             GPR24: ffffffffffffffff 0001ed6431fbb852 0000000000000000 0000000000000001
> >                             GPR28: 0000000000000000 0000000000000000 c0000000032a1878 c0000000032a1870
> > [542521.228911] [     C59] NIP [c0000000001b7634] plpar_hcall_norets_notrace+0x18/0x2c
> > [542521.228915] [     C59] LR [c0000000013c4898] check_and_cede_processor+0x48/0x60
> > [542521.228918] [     C59] ---- interrupt: 900
> > [542521.228919] [     C59] [c000001de047fc80] [ffffffffffffffff] 0xffffffffffffffff (unreliable)
> > [542521.228922] [     C59] [c000001de047fce0] [c0000000013c4944] dedicated_cede_loop+0x94/0x1a0
> > [542521.228926] [     C59] [c000001de047fd30] [c0000000013c3f00] cpuidle_enter_state+0x300/0x748
> > [542521.228930] [     C59] [c000001de047fde0] [c000000000f5ff10] cpuidle_enter+0x50/0x80
> > [542521.228935] [     C59] [c000001de047fe20] [c0000000002ba358] call_cpuidle+0x48/0x90
> > [542521.228939] [     C59] [c000001de047fe40] [c0000000002caf5c] do_idle+0x2dc/0x470
> > [542521.228942] [     C59] [c000001de047feb0] [c0000000002cb3b8] cpu_startup_entry+0x58/0x60
> > [542521.228945] [     C59] [c000001de047fee0] [c00000000005a720] start_secondary+0x860/0xd70
> > [542521.228948] [     C59] [c000001de047ffe0] [c00000000000e158] start_secondary_prolog+0x10/0x14
> > [542521.228951] [     C59] Code: 41810e8c 39400000 39000000 3cc0fffe 60000000 813f0000 71290001 41820050 3ce00001 60000000 60420000 7c40003c <71090001> 394a0001 40820028 3d2201ba
> > [568119.679467] [     C59] pstore: dump skipped in NMI path because of concurrent dump
> > 
> 

Re: [Linux PPC] Disable PREEMPT

[Michal Suchánek]

On Thu, Sep 10, 2026 at 12:31:09PM +0200, Michal Suchánek wrote:
> On Thu, Sep 10, 2026 at 10:54:07AM +0200, Michal Suchánek wrote:
> > On Mon, Sep 07, 2026 at 01:32:18PM +0200, Christian Zigotzky wrote:
> > > On 07 September 2026 at 10:51 am, Michal Suchánek <msuchanek@suse.de> wrote:
> > > 
> > > There is the obvious question which I did not notice answered in this
> > > thread: Did you set up sysrq and/or kdump and find out where the kernel
> > > is stuck?
> > > 
> > > Thanks
> > > 
> > > Michal
> > > 
> > > - - -
> > > 
> > > I think I need kexec for kdump due to the system freezing after activating of KVM HV. Unfortunately kexec doesn’t work on my machines.
> > > 
> > > Further information: https://github.com/chzigotzky/kernels/releases/tag/Kexec
> > 
> > So there is some bug with kexec initialization on your platform.
> 
> Right, you are missing this part:
> https://github.com/torvalds/linux/blob/50d05c7c76c96b90462f24debacca971d2e86713/arch/powerpc/kexec/core.c#L205
> 
> Maybe it needs to be called from your platform code or something.

Lookig at the code it likely runs always but if your device tree does
not have a /chosen node it does not add the property that kexec needs to
load the kernel. Can you try adding a /chosen node to your device tree?

Thanks

Michal

> 
> Thanks
> 
> Michal
> 
> > 
> > Sysrg for dumping stacks to console might still work.
> > 
> > Thanks
> > 
> > Michal
> > 
> 

Re: [Linux PPC] Disable PREEMPT

[Michal Suchánek]

On Thu, Sep 10, 2026 at 01:29:08PM +0200, Michal Suchánek wrote:
> On Thu, Sep 10, 2026 at 04:11:20PM +0530, Shrikanth Hegde wrote:
> > 
> > 
> > On 9/10/26 2:29 PM, Michal Suchánek wrote:
> > > On Fri, Sep 04, 2026 at 02:34:21PM +0530, Shrikanth Hegde wrote:
> > > > 
> > > > 
> > > > On 9/4/26 1:05 PM, Michal Suchánek wrote:
> > > > > On Thu, Sep 03, 2026 at 10:52:59PM +0530, Shrikanth Hegde wrote:
> > > > > > 
> > > > > > > > If possible run against current upstream and share the results.
> > > > > > > 
> > > > > > > https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default
> > > > > > > 
> > > > > > > [  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > > > > > > [  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> > > > > > > [  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> > > > > > > [  472.091624][ T6181] Call Trace:
> > > > > > > [  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
> > > > > > > [  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
> > > > > > > [  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
> > > > > > > [  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
> > > > > > > [  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > > > > > > [  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
> > > > > > > [  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
> > > > > > > [  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
> > > > > > > [  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
> > > > > > > [  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
> > > > > > > [  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
> > > > > > > [  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
> > > > > > > [  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
> > > > > > > [  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
> > > > > > > [  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
> > > > > > > [  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
> > > > > > > [  472.091938][ T6181] IRQMASK: 0
> > > > > > > [  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa
> > > > > > > [  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000
> > > > > > > [  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000
> > > > > > > [  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0
> > > > > > > [  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7
> > > > > > > [  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000
> > > > > > > [  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70
> > > > > > > [  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000
> > > > > > > [  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
> > > > > > > [  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
> > > > > > > [  472.092033][ T6181] ---- interrupt: 3000
> > > > > > > [  472.098256][ T6181] pstore: backend (nvram) writing error (-1)
> > > > > > > 
> > > > > > > This is the host, cannot run the kernel as guest because it fails to boot most
> > > > > > > of the time inside KVM.
> > > > > > > 
> > > > > > > > 
> > > > > > > > > Nonethless, there are quite a few platforms. Originally no preemption
> > > > > > > > > was the only option, and that's the reason why many people run that.
> > > > > > > > > It's the conservative, known working option. And that's the reason a lot
> > > > > > > > > of platfrom code does not get tested with more aggressive preemtion
> > > > > > > > > models, and never gets fixed to work with them.
> > > > > > > > > 
> > > > > > > > 
> > > > > > > > Full preemption has been there for many years!.
> > > > > > > 
> > > > > > > Possible for years, forced only recently.
> > > > > > > 
> > > > > > > > Lazy is not that aggressive compared to that.
> > > > > > > > 
> > > > > > > > > Simply disabling the no preemtion option does not make the platform code
> > > > > > > > > ready.
> > > > > > > > > 
> > > > > > > > 
> > > > > > > > Let's understand your crash case. Let's see where it is going wrong. I am suspecting
> > > > > > > > it is some wrong usage of preemption api rather than arch can't support preemption.
> > > > > > > 
> > > > > > > Very likely some wrong use of the preemption API by the arch code, or no
> > > > > > > use where it should have been used. It did not matter so long as people
> > > > > > > could run their no preempt configs and ignore the problem.
> > > > > > > 
> > > > > > > Thanks
> > > > > > 
> > > > > > 
> > > > > > Friendly LLM analysis says preemption is enabled too early and before
> > > > > > completing kvmppc_handle_exit_hv() / kvmppc_handle_nested_exit()
> > > > > > 
> > > > > > Below is ONLY a speculation and completely UNTESTED.
> > > > > > Maybe worth a try.
> > > > > 
> > > > > The patch is munged by the e-mail client, and it causes immediate
> > > > > voluntary preemprion in rcu critical section and hard lockup on starting
> > > > > a KVM VM.
> > > > > 
> > > > > Also it would be sort of bad news if it worked because that would be
> > > > > specific to book3s KVM HV and would not help with the KVM HV from the
> > > > > original report which likely is not book3s, nor with KVM PR.
> > > > > 
> > > > 
> > > > Thanks for trying. We will try a local repro and look into it why stack is
> > > > getting corrupted.
> > > 
> > > There is also stack corruption when idling. At this point the machine is
> > > stuck half-dead and would not go through with the panic, it requires
> > > dump from the hyprvisor side to capture this.
> > > 
> > > Thanks
> > > 
> > > Michal
> > > 
> > 
> > 
> > is this on host with kvm running or this is without kvm running anything?
> 
> Difficult to say after the fact. Given running KVM VMs is crashing with
> this kernel most likely no VM was running.
> 
> > 
> > Do you see corruption if you don't have kvm module loaded?
> 
> Difficult to say, this takes hours to days to reproduce, and the module
> is normally loaded by default. I can blacklist it to make sure it does
> not get loaded.
> 
> > 
> > These maybe two independent issue.
> > Our KVM team is trying local repro for the kvm issue you have reported.
> > 
> > If possible, could you try vmcore capture (maybe with fadump)?
> 
> I have the vmcore, this dmesg is from the vmcore. I would not get
> displayed otherwise, or maybe some content from the console is lost.
> 
> Unfortunately, this is a kernel from a rolling distribution, and may be
> difficult to analyze bacause new kernel build is used by now.
> 
> Thanks
> 
> Michal
> 
> > 
> > > [542489.550547] [      T0] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: __sprint_symbol+0x1e4/0x1f0
> > > [542489.557809] [      T0] ------------[ cut here ]------------
> > > [542489.557814] [      T0] Voluntary context switch within RCU read-side critical section!
> > > [542489.557822] [      T0] WARNING: [] kernel/rcu/tree_plugin.h:332 at rcu_note_context_switch+0x6a0/0x7b0, CPU#59: swapper/59/0
> > > [542489.557837] [      T0] Modules linked in: vhost_net vhost vhost_iotlb tap rpadlpar_io rpaphp kvm_hv kvm mptcp_diag xsk_diag tcp_diag udp_diag raw_diag inet_diag unix_diag af_packet_diag netlink_diag xt_conntrack xt_MASQUERADE xfrm_user xt_set ip_set xt_addrtype nft_compat nf_conntrack_netbios_ns nf_conntrack_broadcast nft_masq nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct bonding tls nft_chain_nat nf_nat nf_conntrack tun nf_defrag_ipv6 nf_defrag_ipv4 rfkill af_packet binfmt_misc xfs aes_gcm_p10_crypto crypto_simd ibmveth cryptd xts vmx_crypto rtc_generic fuse nvme_fabrics ip_tables x_tables ibmvscsi scsi_transport_srp sd_mod scsi_dh_emc scsi_dh_rdac scsi_dh_alua sg scsi_mod scsi_common nvme nvme_core nvme_keyring nvme_auth pseries_wdt btrfs xor libblake2b raid6_pq dm_mirror dm_region_hash dm_log dm_mod br_netfilter bridge stp llc nf_tables
> > > [542489.557944] [      T0] CPU: 59 UID: 0 PID: 0 Comm: swapper/59 Tainted: G      D W           7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> > > [542489.557957] [      T0] Tainted: [D]=DIE, [W]=WARN
> > > [542489.557961] [      T0] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> > > [542489.557967] [      T0] NIP:  c00000000035d700 LR: c00000000035d6fc CTR: 0000000000000000
> > > [542489.557974] [      T0] REGS: c000001de047f990 TRAP: 0700   Tainted: G      D W            (7.2.2-5.g6824496-default)
> > > [542489.557980] [      T0] MSR:  800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE>  CR: 44002284  XER: 0000000e
> > > [542489.557999] [      T0] CFAR: c00000000022d7a8 IRQMASK: 1
> > >                             GPR00: c00000000035d6fc c000001de047fc30 c00000000205ad00 000000000000003f
> > >                             GPR04: 3ffffffffffc02ef c000001de047f950 c000001de047f910 0000003ff0e20000
> > >                             GPR08: 0000000000000027 0000000000000001 c000001de0191300 0000000044002284
> > >                             GPR12: c000003ff1614890 c000003fffff9f00 0000000000000000 000000002eef9860
> > >                             GPR16: 0000000000000000 c000003ff40dcc80 0000003ff0e20000 c000000000058010
> > >                             GPR20: c000000000058160 c0000000032bcc80 c0000000032bcc80 c000000000057f80
> > >                             GPR24: ffffffffffffffff 000000000000003b c000001de0191300 c000000003bea6e0
> > >                             GPR28: c000000000058010 ffffffffffffffff 0000000000000000 c000003ff40de100
> > > [542489.558066] [      T0] NIP [c00000000035d700] rcu_note_context_switch+0x6a0/0x7b0
> > > [542489.558072] [      T0] LR [c00000000035d6fc] rcu_note_context_switch+0x69c/0x7b0
> > > [542489.558078] [      T0] Call Trace:
> > > [542489.558082] [      T0] [c000001de047fc30] [c00000000035d6fc] rcu_note_context_switch+0x69c/0x7b0 (unreliable)
> > > [542489.558092] [      T0] [c000001de047fcd0] [c0000000013c4fc0] __schedule+0xf0/0x1d10
> > > [542489.558108] [      T0] [c000001de047fe20] [c0000000013c6eb0] schedule_idle+0x30/0x60
> > > [542489.558121] [      T0] [c000001de047fe40] [c0000000002caed8] do_idle+0x258/0x470
> > > [542489.558133] [      T0] [c000001de047feb0] [c0000000002cb3b4] cpu_startup_entry+0x54/0x60
> > > [542489.558145] [      T0] [c000001de047fee0] [c00000000005a720] start_secondary+0x860/0xd70
> > > [542489.558154] [      T0] [c000001de047ffe0] [c00000000000e158] start_secondary_prolog+0x10/0x14
> > > [542489.558162] [      T0] Code: f8c900a8 4bfffe6c 60000000 60420000 3c62ff87 39200001 f9410068 3d4201b2 386329d8 992aa087 4becff91 60000000 <0fe00000> e9410068 4bfff9c4 60420000
> > > [542489.558184] [      T0] ---[ end trace 0000000000000000 ]---
> > > [542521.228724] [     C59] watchdog: CPU 59 Hard LOCKUP
> > > [542521.228726] [     C59] watchdog: CPU 59 TB:3697937465165330, last heartbeat TB:3697921085812082 (31990ms ago)
> > > [542521.228728] [     C59] Modules linked in: vhost_net vhost vhost_iotlb tap rpadlpar_io rpaphp kvm_hv kvm mptcp_diag xsk_diag tcp_diag udp_diag raw_diag inet_diag unix_diag af_packet_diag netlink_diag xt_conntrack xt_MASQUERADE xfrm_user xt_set ip_set xt_addrtype nft_compat nf_conntrack_netbios_ns nf_conntrack_broadcast nft_masq nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct bonding tls nft_chain_nat nf_nat nf_conntrack tun nf_defrag_ipv6 nf_defrag_ipv4 rfkill af_packet binfmt_misc xfs aes_gcm_p10_crypto crypto_simd ibmveth cryptd xts vmx_crypto rtc_generic fuse nvme_fabrics ip_tables x_tables ibmvscsi scsi_transport_srp sd_mod scsi_dh_emc scsi_dh_rdac scsi_dh_alua sg scsi_mod scsi_common nvme nvme_core nvme_keyring nvme_auth pseries_wdt btrfs xor libblake2b raid6_pq dm_mirror dm_region_hash dm_log dm_mod br_netfilter bridge stp llc nf_tables
> > > [542521.228776] [     C59] CPU: 59 UID: 0 PID: 0 Comm: swapper/59 Tainted: G      D W           7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> > > [542521.228780] [     C59] Tainted: [D]=DIE, [W]=WARN
> > > [542521.228781] [     C59] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> > > [542521.228783] [     C59] NIP:  c0000000013d2c94 LR: c0000000013d5330 CTR: c000000003812be8
> > > [542521.228785] [     C59] REGS: c000003ffed2bd60 TRAP: 0100   Tainted: G      D W            (7.2.2-5.g6824496-default)
> > > [542521.228788] [     C59] MSR:  8000000000081033 <SF,ME,IR,DR,RI,LE>  CR: 42002282  XER: 2004000e
> > > [542521.228794] [     C59] CFAR: 000000000000011c IRQMASK: 3
> > >                             GPR00: c000000000260a58 c000003ffef2fb60 c00000000205ad00 c000001deaab3c00
> > >                             GPR04: 00000000000001f8 0000000000000000 fffffffffffe0000 0000000000010000
> > >                             GPR08: 0000000000000001 0000000000000001 000000001c82ef77 fffffffffffffffd
> > >                             GPR12: c000000003812be0 c000003fffff9f00 0000000000000000 0000000000000000
> > >                             GPR16: 0000000000000000 0000000000000000 0000000000000001 c000000003bc2b00
> > >                             GPR20: c000003ff40c8de8 0000000000000001 c000003ff40c8da8 c000003ffef2fe08
> > >                             GPR24: 0000000000000000 c000000003bea6e0 000000000000003b 0000003ff0e20000
> > >                             GPR28: 0000000000780000 c0000000032bbe80 c000003ff40dbe80 c000001deaab3c00
> > > [542521.228820] [     C59] NIP [c0000000013d2c94] queued_spin_lock_slowpath+0x164/0x19a0
> > > [542521.228824] [     C59] LR [c0000000013d5330] _raw_spin_lock+0x80/0xa0
> > > [542521.228827] [     C59] Call Trace:
> > > [542521.228827] [     C59] [c000003ffef2fb60] [0000000000001000] 0x1000 (unreliable)
> > > [542521.228833] [     C59] [c000003ffef2fc60] [c000001de0191300] 0xc000001de0191300
> > > [542521.228836] [     C59] [c000003ffef2fc80] [c000000000260a58] __queue_work+0x2f8/0x740
> > > [542521.228841] [     C59] [c000003ffef2fd50] [c00000000038ceac] call_timer_fn+0x4c/0x260
> > > [542521.228845] [     C59] [c000003ffef2fde0] [c00000000038d308] __run_timers+0x248/0x410
> > > [542521.228849] [     C59] [c000003ffef2fec0] [c00000000038d56c] run_timer_softirq+0x9c/0x130
> > > [542521.228852] [     C59] [c000003ffef2fef0] [c00000000023ab0c] handle_softirqs+0x14c/0x4a0
> > > [542521.228855] [     C59] [c000003ffef2ffe0] [c000000000018418] do_softirq_own_stack+0x38/0x50
> > > [542521.228860] [     C59] [c000001de047f910] [c00000000023a238] __irq_exit_rcu+0x198/0x1c0
> > > [542521.228862] [     C59] [c000001de047f940] [c000000000026acc] timer_interrupt+0x1cc/0x4f0
> > > [542521.228866] [     C59] [c000001de047f9b0] [c000000000009fdc] decrementer_common_virt+0x28c/0x290
> > > [542521.228871] [     C59] ---- interrupt: 900 at plpar_hcall_norets_notrace+0x18/0x2c
> > > [542521.228876] [     C59] NIP:  c0000000001b7634 LR: c0000000013c4898 CTR: 0000000000000000
> > > [542521.228877] [     C59] REGS: c000001de047f9e0 TRAP: 0900   Tainted: G      D W            (7.2.2-5.g6824496-default)
> > > [542521.228879] [     C59] MSR:  800000000280b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE>  CR: 22000282  XER: 2004000e
> > > [542521.228886] [     C59] CFAR: 0000000000000000 IRQMASK: 0
> > >                             GPR00: 0000000000000000 c000001de047fc80 c00000000205ad00 0000000000000000
> > >                             GPR04: 0000000000000000 0000000000000039 0000000000000000 0000000000000000
> > >                             GPR08: ffffffffffffff7f 0000000000000000 ffffffffffffffff 0000000000000000
> > >                             GPR12: 0000000000000000 c000003fffff9f00 0000000000000000 000000002eef9860
> > >                             GPR16: 0000000000000000 0000000000000001 0000000000000001 c000000000058010
> > >                             GPR20: c000000000058160 0000000000000002 c000000000057fb0 c000000000057f80
> > >                             GPR24: ffffffffffffffff 0001ed6431fbb852 0000000000000000 0000000000000001
> > >                             GPR28: 0000000000000000 0000000000000000 c0000000032a1878 c0000000032a1870
> > > [542521.228911] [     C59] NIP [c0000000001b7634] plpar_hcall_norets_notrace+0x18/0x2c
> > > [542521.228915] [     C59] LR [c0000000013c4898] check_and_cede_processor+0x48/0x60
> > > [542521.228918] [     C59] ---- interrupt: 900
> > > [542521.228919] [     C59] [c000001de047fc80] [ffffffffffffffff] 0xffffffffffffffff (unreliable)
> > > [542521.228922] [     C59] [c000001de047fce0] [c0000000013c4944] dedicated_cede_loop+0x94/0x1a0
> > > [542521.228926] [     C59] [c000001de047fd30] [c0000000013c3f00] cpuidle_enter_state+0x300/0x748
> > > [542521.228930] [     C59] [c000001de047fde0] [c000000000f5ff10] cpuidle_enter+0x50/0x80
> > > [542521.228935] [     C59] [c000001de047fe20] [c0000000002ba358] call_cpuidle+0x48/0x90
> > > [542521.228939] [     C59] [c000001de047fe40] [c0000000002caf5c] do_idle+0x2dc/0x470
> > > [542521.228942] [     C59] [c000001de047feb0] [c0000000002cb3b8] cpu_startup_entry+0x58/0x60
> > > [542521.228945] [     C59] [c000001de047fee0] [c00000000005a720] start_secondary+0x860/0xd70
> > > [542521.228948] [     C59] [c000001de047ffe0] [c00000000000e158] start_secondary_prolog+0x10/0x14
> > > [542521.228951] [     C59] Code: 41810e8c 39400000 39000000 3cc0fffe 60000000 813f0000 71290001 41820050 3ce00001 60000000 60420000 7c40003c <71090001> 394a0001 40820028 3d2201ba
> > > [568119.679467] [     C59] pstore: dump skipped in NMI path because of concurrent dump
> > > 
> > 
> 

And now with the entry fixes merged I can finally crash the kernel
by runnin kernel build on the host with no KVM involvement. It does lock
up during crash and requires fadump to collect vmcore.

Thanks

Michal

[ 1919.695922] [   T2102] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: sys_ioctl+0xfe0/0x1060
[ 1919.695966] [   T2102] CPU: 4 UID: 0 PID: 2102 Comm: velociraptor-cl Not tainted 7.2.4-4.gefe5825-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  d4e9af9c041454ba4e51c7aa59aeea774951e118
[ 1919.695981] [   T2102] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
[ 1919.695991] [   T2102] Call Trace:
[ 1919.696653] [   T2102] ------------[ cut here ]------------
[ 1919.696665] [   T2102] Voluntary context switch within RCU read-side critical section!
[ 1919.696681] [   T2102] WARNING: [] kernel/rcu/tree_plugin.h:332 at rcu_note_context_switch+0x6a0/0x7b0, CPU#4: velociraptor-cl/2102
[ 1919.696700] [   T2102] Modules linked in: rpadlpar_io rpaphp mptcp_diag xsk_diag tcp_diag udp_diag raw_diag inet_diag unix_diag af_packet_diag netlink_diag xt_conntrack xt_MASQUERADE xfrm_user xt_set ip_set xt_addrtype nft_compat nf_conntrack_netbios_ns nf_conntrack_broadcast nft_masq nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct bonding tls nft_chain_nat nf_nat nf_conntrack tun nf_defrag_ipv6 nf_defrag_ipv4 rfkill af_packet binfmt_misc xfs aes_gcm_p10_crypto crypto_simd ibmveth cryptd xts vmx_crypto rtc_generic fuse nvme_fabrics ip_tables x_tables ibmvscsi scsi_transport_srp sd_mod scsi_dh_emc scsi_dh_rdac scsi_dh_alua sg scsi_mod scsi_common nvme nvme_core nvme_keyring nvme_auth pseries_wdt btrfs xor libblake2b raid6_pq dm_mirror dm_region_hash dm_log dm_mod br_netfilter bridge stp llc nf_tables
[ 1919.696862] [   T2102] CPU: 4 UID: 0 PID: 2102 Comm: velociraptor-cl Not tainted 7.2.4-4.gefe5825-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  d4e9af9c041454ba4e51c7aa59aeea774951e118
[ 1919.696874] [   T2102] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
[ 1919.696882] [   T2102] NIP:  c00000000035e000 LR: c00000000035dffc CTR: 0000000000000001
[ 1919.696889] [   T2102] REGS: c00000001c36b6b0 TRAP: 0700   Not tainted  (7.2.4-4.gefe5825-default)
[ 1919.696897] [   T2102] MSR:  8000000000029033 <SF,EE,ME,IR,DR,RI,LE>  CR: 48222200  XER: 00000000
[ 1919.696917] [   T2102] CFAR: c00000000022da28 IRQMASK: 1 
                          GPR00: c00000000035dffc c00000001c36b950 c00000000205ad00 000000000000003f 
                          GPR04: 3ffffffffffc023a c00000001c36b670 c00000001c36b630 0000001dd4260000 
                          GPR08: 0000000000000027 0000000000000001 c00000002992a300 0000000048222200 
                          GPR12: c000003ff160f960 c000001ddfffb700 c000001dd751cc80 0000000000000000 
                          GPR16: 0000000000000000 c0000000032bcc80 0000000000000000 0000000000000000 
                          GPR20: 0000000000000000 c0000000032bcc80 0000000000000000 0000000000000000 
                          GPR24: c00000002992a300 0000000000000000 0000000000000000 c000000003bea6e0 
                          GPR28: 0000000000000000 c00000002992a300 0000000000000000 c000001dd751e180 
[ 1919.697002] [   T2102] NIP [c00000000035e000] rcu_note_context_switch+0x6a0/0x7b0
[ 1919.697010] [   T2102] LR [c00000000035dffc] rcu_note_context_switch+0x69c/0x7b0
[ 1919.697018] [   T2102] Call Trace:
[ 1919.697022] [   T2102] [c00000001c36b950] [c00000000035dffc] rcu_note_context_switch+0x69c/0x7b0 (unreliable)
[ 1919.697033] [   T2102] [c00000001c36b9f0] [c0000000013c7e64] __schedule+0xf4/0x1d90
[ 1919.697045] [   T2102] [c00000001c36bb40] [c0000000013c9b48] schedule+0x48/0x130
[ 1919.697059] [   T2102] [c00000001c36bb70] [c0000000002c28b8] do_sched_yield+0xb8/0x150
[ 1919.697069] [   T2102] [c00000001c36bba0] [c0000000002c296c] sys_sched_yield+0x1c/0x30
[ 1919.697077] [   T2102] [c00000001c36bbc0] [c00000000002fa08] system_call_exception+0x188/0x430
[ 1919.697089] [   T2102] [c00000001c36be50] [c00000000000d620] system_call_common+0x160/0x2c4
[ 1919.697104] [   T2102] ---- interrupt: c00 at 0x100aba48
[ 1919.697113] [   T2102] NIP:  00000000100aba48 LR: 00000000100280a4 CTR: 0000000000000000
[ 1919.697120] [   T2102] REGS: c00000001c36be80 TRAP: 0c00   Not tainted  (7.2.4-4.gefe5825-default)
[ 1919.697126] [   T2102] MSR:  800000000000f033 <SF,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 24228408  XER: 00000000
[ 1919.697147] [   T2102] IRQMASK: 0 
                          GPR00: 000000000000009e 0000162f3b39bcc0 0000162f3a97ac40 0000162f3a7cabc8 
                          GPR04: 0000000000000000 0000162f3a1d6e00 0000000000000004 0000000000000101 
                          GPR08: 0000000000000005 0000000000000000 0000000000000000 0000000000000000 
                          GPR12: 0000000000000000 00007ffdffed68c0 0000000000000001 0000000000000001 
                          GPR16: 0000000000000000 fffffffffffffffd 0000000000000000 000000000000000c 
                          GPR20: 0000000000800000 0000000000000001 0000162f3b39bba0 00007ffdffece148 
                          GPR24: 00000007bd628d91 00000008c8044b7c 00000007d72eb2ef 000000009e36057c 
                          GPR28: 0000000002c19fa3 000000006e9ed76c 0000162f3a1d6e00 0000000000000001 
[ 1919.697230] [   T2102] NIP [00000000100aba48] 0x100aba48
[ 1919.697236] [   T2102] LR [00000000100280a4] 0x100280a4
[ 1919.697242] [   T2102] ---- interrupt: c00
[ 1919.697247] [   T2102] Code: f8c900a8 4bfffe6c 60000000 60420000 3c62ff87 39200001 f9410068 3d4201b2 38632b78 992aa187 4becf911 60000000 <0fe00000> e9410068 4bfff9c4 60420000 
[ 1919.697279] [   T2102] ---[ end trace 0000000000000000 ]---
[ 1934.578776] [      C4] watchdog: CPU 4 self-detected hard LOCKUP @ queued_spin_lock_slowpath+0x1004/0x19a0
[ 1934.578791] [      C4] watchdog: CPU 4 TB:3717005501731461, last heartbeat TB:3716998337363492 (13992ms ago)
[ 1934.578794] [      C4] Modules linked in: rpadlpar_io rpaphp mptcp_diag xsk_diag tcp_diag udp_diag raw_diag inet_diag unix_diag af_packet_diag netlink_diag xt_conntrack xt_MASQUERADE xfrm_user xt_set ip_set xt_addrtype nft_compat nf_conntrack_netbios_ns nf_conntrack_broadcast nft_masq nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct bonding tls nft_chain_nat nf_nat nf_conntrack tun nf_defrag_ipv6 nf_defrag_ipv4 rfkill af_packet binfmt_misc xfs aes_gcm_p10_crypto crypto_simd ibmveth cryptd xts vmx_crypto rtc_generic fuse nvme_fabrics ip_tables x_tables ibmvscsi scsi_transport_srp sd_mod scsi_dh_emc scsi_dh_rdac scsi_dh_alua sg scsi_mod scsi_common nvme nvme_core nvme_keyring nvme_auth pseries_wdt btrfs xor libblake2b raid6_pq dm_mirror dm_region_hash dm_log dm_mod br_netfilter bridge stp llc nf_tables
[ 1934.578841] [      C4] CPU: 4 UID: 399 PID: 49635 Comm: cc1 Tainted: G      D W           7.2.4-4.gefe5825-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  d4e9af9c041454ba4e51c7aa59aeea774951e118
[ 1934.578846] [      C4] Tainted: [D]=DIE, [W]=WARN
[ 1934.578847] [      C4] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
[ 1934.578849] [      C4] NIP:  c0000000013d6a54 LR: c0000000013d6a38 CTR: 0000000000000004
[ 1934.578851] [      C4] REGS: c000001ddfdd7d60 TRAP: 0900   Tainted: G      D W            (7.2.4-4.gefe5825-default)
[ 1934.578854] [      C4] MSR:  8000000000009033 <SF,EE,ME,IR,DR,RI,LE>  CR: 28002822  XER: 20040004
[ 1934.578861] [      C4] CFAR: 0000000000000000 IRQMASK: 1 
                          GPR00: c0000000013d8870 c000000021d0baf0 c00000000205ad00 c000003ff449be88 
                          GPR04: 0000000000000250 c000001dd7507960 00000000000b0077 c000000003bc1084 
                          GPR08: 0000000000000000 0000000000000000 c000003ff449be80 c00000019f264400 
                          GPR12: c000000000776400 c000001ddfffb700 0000000000000000 0000000000000000 
                          GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
                          GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
                          GPR24: 0000000000000000 c000000003bc1080 c000001dd751be88 0000001dd4260000 
                          GPR28: 00000000000a0000 c0000000032bbe80 c000001dd751be80 c000000003bc1084 
[ 1934.578887] [      C4] NIP [c0000000013d6a54] queued_spin_lock_slowpath+0x1004/0x19a0
[ 1934.578890] [      C4] LR [c0000000013d6a38] queued_spin_lock_slowpath+0xfe8/0x19a0
[ 1934.578893] [      C4] Call Trace:
[ 1934.578894] [      C4] [c000000021d0bbf0] [c0000000013d8870] queued_write_lock_slowpath+0x250/0x270
[ 1934.578897] [      C4] [c000000021d0bc20] [c0000000013d81bc] _raw_write_lock_irq+0x5c/0x70
[ 1934.578901] [      C4] [c000000021d0bc40] [c000000000237df4] do_exit+0x3b4/0xd00
[ 1934.578905] [      C4] [c000000021d0bd00] [c0000000002389cc] do_group_exit+0x3c/0xe0
[ 1934.578909] [      C4] [c000000021d0bd50] [c000000000238a94] sys_exit_group+0x24/0x30
[ 1934.578913] [      C4] [c000000021d0bd70] [c00000000002fa08] system_call_exception+0x188/0x430
[ 1934.578917] [      C4] [c000000021d0be50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
[ 1934.578922] [      C4] ---- interrupt: 3000 at 0x7fff7fd34344
[ 1934.578926] [      C4] NIP:  00007fff7fd34344 LR: 00007fff7fd34344 CTR: 0000000000000000
[ 1934.578928] [      C4] REGS: c000000021d0be80 TRAP: 3000   Tainted: G      D W            (7.2.4-4.gefe5825-default)
[ 1934.578930] [      C4] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 44002422  XER: 00000000
[ 1934.578938] [      C4] IRQMASK: 0 
                          GPR00: 00000000000000ea 00007fffe2884cc0 00007fff7fe96d00 0000000000000000 
                          GPR04: 0000000000000000 0000000038ed25e0 0000000000000000 0000000000000008 
                          GPR08: 0000000000000008 0000000000000000 0000000000000000 0000000000000000 
                          GPR12: 0000000000000000 00007fff8097dc80 0000000000000000 0000000000000000 
                          GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
                          GPR20: 0000000000000000 0000000000000000 0000000000000001 0000000000000000 
                          GPR24: 00007fff7fe90828 0000000000000000 0000000000000000 0000000000000001 
                          GPR28: 00007fff7fe99028 00007fff80977030 fffffffffffff000 0000000000000000 
[ 1934.578962] [      C4] NIP [00007fff7fd34344] 0x7fff7fd34344
[ 1934.578964] [      C4] LR [00007fff7fd34344] 0x7fff7fd34344
[ 1934.578965] [      C4] ---- interrupt: 3000
[ 1934.578966] [      C4] Code: 3888ffff 7f474a14 f8010110 7c8407b4 4ad9b325 fb430000 60000000 893e001d 2c090000 40820018 60420000 7c40003c <893e001d> 2c090000 4182fff4 60000000 
[ 2710.413634] [      C4] pstore: dump skipped in NMI path because of concurrent dump

Re: [Linux PPC] Disable PREEMPT

[Shrikanth Hegde]

Hi Michal,

On 9/10/26 5:46 PM, Michal Suchánek wrote:
> On Thu, Sep 10, 2026 at 01:29:08PM +0200, Michal Suchánek wrote:
>> On Thu, Sep 10, 2026 at 04:11:20PM +0530, Shrikanth Hegde wrote:
>>>
>>>
>>> On 9/10/26 2:29 PM, Michal Suchánek wrote:
>>>> On Fri, Sep 04, 2026 at 02:34:21PM +0530, Shrikanth Hegde wrote:
>>>>>
>>>>>
>>>>> On 9/4/26 1:05 PM, Michal Suchánek wrote:
>>>>>> On Thu, Sep 03, 2026 at 10:52:59PM +0530, Shrikanth Hegde wrote:
>>>>>>>
>>>>>>>>> If possible run against current upstream and share the results.
>>>>>>>>
>>>>>>>> https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default
>>>>>>>>
>>>>>>>> [  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
>>>>>>>> [  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
>>>>>>>> [  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
>>>>>>>> [  472.091624][ T6181] Call Trace:
>>>>>>>> [  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
>>>>>>>> [  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
>>>>>>>> [  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
>>>>>>>> [  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
>>>>>>>> [  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
>>>>>>>> [  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
>>>>>>>> [  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
>>>>>>>> [  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
>>>>>>>> [  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
>>>>>>>> [  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
>>>>>>>> [  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
>>>>>>>> [  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
>>>>>>>> [  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
>>>>>>>> [  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
>>>>>>>> [  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
>>>>>>>> [  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
>>>>>>>> [  472.091938][ T6181] IRQMASK: 0
>>>>>>>> [  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa
>>>>>>>> [  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000
>>>>>>>> [  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000
>>>>>>>> [  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0
>>>>>>>> [  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7
>>>>>>>> [  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000
>>>>>>>> [  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70
>>>>>>>> [  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000
>>>>>>>> [  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
>>>>>>>> [  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
>>>>>>>> [  472.092033][ T6181] ---- interrupt: 3000
>>>>>>>> [  472.098256][ T6181] pstore: backend (nvram) writing error (-1)
>>>>>>>>
>>>>>>>> This is the host, cannot run the kernel as guest because it fails to boot most
>>>>>>>> of the time inside KVM.
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Nonethless, there are quite a few platforms. Originally no preemption
>>>>>>>>>> was the only option, and that's the reason why many people run that.
>>>>>>>>>> It's the conservative, known working option. And that's the reason a lot
>>>>>>>>>> of platfrom code does not get tested with more aggressive preemtion
>>>>>>>>>> models, and never gets fixed to work with them.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Full preemption has been there for many years!.
>>>>>>>>
>>>>>>>> Possible for years, forced only recently.
>>>>>>>>
>>>>>>>>> Lazy is not that aggressive compared to that.
>>>>>>>>>
>>>>>>>>>> Simply disabling the no preemtion option does not make the platform code
>>>>>>>>>> ready.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Let's understand your crash case. Let's see where it is going wrong. I am suspecting
>>>>>>>>> it is some wrong usage of preemption api rather than arch can't support preemption.
>>>>>>>>
>>>>>>>> Very likely some wrong use of the preemption API by the arch code, or no
>>>>>>>> use where it should have been used. It did not matter so long as people
>>>>>>>> could run their no preempt configs and ignore the problem.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>
>>>>>>>
>>>>>>> Friendly LLM analysis says preemption is enabled too early and before
>>>>>>> completing kvmppc_handle_exit_hv() / kvmppc_handle_nested_exit()
>>>>>>>
>>>>>>> Below is ONLY a speculation and completely UNTESTED.
>>>>>>> Maybe worth a try.
>>>>>>
>>>>>> The patch is munged by the e-mail client, and it causes immediate
>>>>>> voluntary preemprion in rcu critical section and hard lockup on starting
>>>>>> a KVM VM.
>>>>>>
>>>>>> Also it would be sort of bad news if it worked because that would be
>>>>>> specific to book3s KVM HV and would not help with the KVM HV from the
>>>>>> original report which likely is not book3s, nor with KVM PR.
>>>>>>
>>>>>
>>>>> Thanks for trying. We will try a local repro and look into it why stack is
>>>>> getting corrupted.
>>>>

As we discussed offlist, samir helped to run a similar test on his machine, and he didn't
run into issue so far. we will try more.

I am just wondering what different in your case?
By any chance we are running into below one? Can you check your gcc version?

https://lore.kernel.org/all/CAABZP2z=xu+07-y5fqFLidZz1VpSgrSwXa1mFHPb=b3Ezr3OtA@mail.gmail.com/

Maybe CONFIG_DEBUG_PREEMPT worth a try to if it shows up anything.

>>>> There is also stack corruption when idling. At this point the machine is
>>>> stuck half-dead and would not go through with the panic, it requires
>>>> dump from the hyprvisor side to capture this.
>>>>
>>>> Thanks
>>>>
>>>> Michal
>>>>
>>>
>>>
>>> is this on host with kvm running or this is without kvm running anything?
>>
>> Difficult to say after the fact. Given running KVM VMs is crashing with
>> this kernel most likely no VM was running.
>>
>>>
>>> Do you see corruption if you don't have kvm module loaded?
>>
>> Difficult to say, this takes hours to days to reproduce, and the module
>> is normally loaded by default. I can blacklist it to make sure it does
>> not get loaded.
>>
We might need exact vmlinux and if possible vmcore to see the paca saving into
specific register or why it could possibly go wrong.

Michael, Christophe, anything rings a bell? What might be going wrong here?

Re: [Linux PPC] Disable PREEMPT

[Michal Suchánek]

On Fri, Sep 11, 2026 at 10:39:10PM +0530, Shrikanth Hegde wrote:
> Hi Michal,
> 
> On 9/10/26 5:46 PM, Michal Suchánek wrote:
> > On Thu, Sep 10, 2026 at 01:29:08PM +0200, Michal Suchánek wrote:
> > > On Thu, Sep 10, 2026 at 04:11:20PM +0530, Shrikanth Hegde wrote:
> > > > 
> > > > 
> > > > On 9/10/26 2:29 PM, Michal Suchánek wrote:
> > > > > On Fri, Sep 04, 2026 at 02:34:21PM +0530, Shrikanth Hegde wrote:
> > > > > > 
> > > > > > 
> > > > > > On 9/4/26 1:05 PM, Michal Suchánek wrote:
> > > > > > > On Thu, Sep 03, 2026 at 10:52:59PM +0530, Shrikanth Hegde wrote:
> > > > > > > > 
> > > > > > > > > > If possible run against current upstream and share the results.
> > > > > > > > > 
> > > > > > > > > https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default
> > > > > > > > > 
> > > > > > > > > [  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > > > > > > > > [  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> > > > > > > > > [  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> > > > > > > > > [  472.091624][ T6181] Call Trace:
> > > > > > > > > [  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
> > > > > > > > > [  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
> > > > > > > > > [  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
> > > > > > > > > [  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
> > > > > > > > > [  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > > > > > > > > [  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
> > > > > > > > > [  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
> > > > > > > > > [  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
> > > > > > > > > [  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
> > > > > > > > > [  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
> > > > > > > > > [  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
> > > > > > > > > [  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
> > > > > > > > > [  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
> > > > > > > > > [  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
> > > > > > > > > [  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
> > > > > > > > > [  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
> > > > > > > > > [  472.091938][ T6181] IRQMASK: 0
> > > > > > > > > [  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa
> > > > > > > > > [  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000
> > > > > > > > > [  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000
> > > > > > > > > [  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0
> > > > > > > > > [  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7
> > > > > > > > > [  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000
> > > > > > > > > [  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70
> > > > > > > > > [  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000
> > > > > > > > > [  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
> > > > > > > > > [  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
> > > > > > > > > [  472.092033][ T6181] ---- interrupt: 3000
> > > > > > > > > [  472.098256][ T6181] pstore: backend (nvram) writing error (-1)
> > > > > > > > > 
> > > > > > > > > This is the host, cannot run the kernel as guest because it fails to boot most
> > > > > > > > > of the time inside KVM.
> > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > > Nonethless, there are quite a few platforms. Originally no preemption
> > > > > > > > > > > was the only option, and that's the reason why many people run that.
> > > > > > > > > > > It's the conservative, known working option. And that's the reason a lot
> > > > > > > > > > > of platfrom code does not get tested with more aggressive preemtion
> > > > > > > > > > > models, and never gets fixed to work with them.
> > > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > Full preemption has been there for many years!.
> > > > > > > > > 
> > > > > > > > > Possible for years, forced only recently.
> > > > > > > > > 
> > > > > > > > > > Lazy is not that aggressive compared to that.
> > > > > > > > > > 
> > > > > > > > > > > Simply disabling the no preemtion option does not make the platform code
> > > > > > > > > > > ready.
> > > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > Let's understand your crash case. Let's see where it is going wrong. I am suspecting
> > > > > > > > > > it is some wrong usage of preemption api rather than arch can't support preemption.
> > > > > > > > > 
> > > > > > > > > Very likely some wrong use of the preemption API by the arch code, or no
> > > > > > > > > use where it should have been used. It did not matter so long as people
> > > > > > > > > could run their no preempt configs and ignore the problem.
> > > > > > > > > 
> > > > > > > > > Thanks
> > > > > > > > 
> > > > > > > > 
> > > > > > > > Friendly LLM analysis says preemption is enabled too early and before
> > > > > > > > completing kvmppc_handle_exit_hv() / kvmppc_handle_nested_exit()
> > > > > > > > 
> > > > > > > > Below is ONLY a speculation and completely UNTESTED.
> > > > > > > > Maybe worth a try.
> > > > > > > 
> > > > > > > The patch is munged by the e-mail client, and it causes immediate
> > > > > > > voluntary preemprion in rcu critical section and hard lockup on starting
> > > > > > > a KVM VM.
> > > > > > > 
> > > > > > > Also it would be sort of bad news if it worked because that would be
> > > > > > > specific to book3s KVM HV and would not help with the KVM HV from the
> > > > > > > original report which likely is not book3s, nor with KVM PR.
> > > > > > > 
> > > > > > 
> > > > > > Thanks for trying. We will try a local repro and look into it why stack is
> > > > > > getting corrupted.
> > > > > 
> 
> As we discussed offlist, samir helped to run a similar test on his machine, and he didn't
> run into issue so far. we will try more.
> 
> I am just wondering what different in your case?
> By any chance we are running into below one? Can you check your gcc version?
> 
> https://lore.kernel.org/all/CAABZP2z=xu+07-y5fqFLidZz1VpSgrSwXa1mFHPb=b3Ezr3OtA@mail.gmail.com/
> 
> Maybe CONFIG_DEBUG_PREEMPT worth a try to if it shows up anything.

Does not change anything, the kernel crashes all the same.

Nonetheless, I noticed that a tool that does some BPF tracing is
running, and without it the problem is not reproducible.

Thanks

Michal

Re: [Linux PPC] Disable PREEMPT

[Narayana Murty N]

On 14/09/26 4:28 PM, Michal Suchánek wrote:
> On Fri, Sep 11, 2026 at 10:39:10PM +0530, Shrikanth Hegde wrote:
>> Hi Michal,
>>
>> On 9/10/26 5:46 PM, Michal Suchánek wrote:
>>> On Thu, Sep 10, 2026 at 01:29:08PM +0200, Michal Suchánek wrote:
>>>> On Thu, Sep 10, 2026 at 04:11:20PM +0530, Shrikanth Hegde wrote:
>>>>>
>>>>>
>>>>> On 9/10/26 2:29 PM, Michal Suchánek wrote:
>>>>>> On Fri, Sep 04, 2026 at 02:34:21PM +0530, Shrikanth Hegde wrote:
>>>>>>>
>>>>>>>
>>>>>>> On 9/4/26 1:05 PM, Michal Suchánek wrote:
>>>>>>>> On Thu, Sep 03, 2026 at 10:52:59PM +0530, Shrikanth Hegde wrote:
>>>>>>>>>
>>>>>>>>>>> If possible run against current upstream and share the results.
>>>>>>>>>>
>>>>>>>>>> https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default
>>>>>>>>>>
>>>>>>>>>> [  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
>>>>>>>>>> [  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
>>>>>>>>>> [  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
>>>>>>>>>> [  472.091624][ T6181] Call Trace:
>>>>>>>>>> [  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
>>>>>>>>>> [  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
>>>>>>>>>> [  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
>>>>>>>>>> [  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
>>>>>>>>>> [  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
>>>>>>>>>> [  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
>>>>>>>>>> [  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
>>>>>>>>>> [  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
>>>>>>>>>> [  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
>>>>>>>>>> [  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
>>>>>>>>>> [  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
>>>>>>>>>> [  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
>>>>>>>>>> [  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
>>>>>>>>>> [  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
>>>>>>>>>> [  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
>>>>>>>>>> [  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
>>>>>>>>>> [  472.091938][ T6181] IRQMASK: 0
>>>>>>>>>> [  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa
>>>>>>>>>> [  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000
>>>>>>>>>> [  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000
>>>>>>>>>> [  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0
>>>>>>>>>> [  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7
>>>>>>>>>> [  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000
>>>>>>>>>> [  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70
>>>>>>>>>> [  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000
>>>>>>>>>> [  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
>>>>>>>>>> [  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
>>>>>>>>>> [  472.092033][ T6181] ---- interrupt: 3000
>>>>>>>>>> [  472.098256][ T6181] pstore: backend (nvram) writing error (-1)
>>>>>>>>>>
>>>>>>>>>> This is the host, cannot run the kernel as guest because it fails to boot most
>>>>>>>>>> of the time inside KVM.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Nonethless, there are quite a few platforms. Originally no preemption
>>>>>>>>>>>> was the only option, and that's the reason why many people run that.
>>>>>>>>>>>> It's the conservative, known working option. And that's the reason a lot
>>>>>>>>>>>> of platfrom code does not get tested with more aggressive preemtion
>>>>>>>>>>>> models, and never gets fixed to work with them.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Full preemption has been there for many years!.
>>>>>>>>>>
>>>>>>>>>> Possible for years, forced only recently.
>>>>>>>>>>
>>>>>>>>>>> Lazy is not that aggressive compared to that.
>>>>>>>>>>>
>>>>>>>>>>>> Simply disabling the no preemtion option does not make the platform code
>>>>>>>>>>>> ready.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Let's understand your crash case. Let's see where it is going wrong. I am suspecting
>>>>>>>>>>> it is some wrong usage of preemption api rather than arch can't support preemption.
>>>>>>>>>>
>>>>>>>>>> Very likely some wrong use of the preemption API by the arch code, or no
>>>>>>>>>> use where it should have been used. It did not matter so long as people
>>>>>>>>>> could run their no preempt configs and ignore the problem.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Friendly LLM analysis says preemption is enabled too early and before
>>>>>>>>> completing kvmppc_handle_exit_hv() / kvmppc_handle_nested_exit()
>>>>>>>>>
>>>>>>>>> Below is ONLY a speculation and completely UNTESTED.
>>>>>>>>> Maybe worth a try.
>>>>>>>>
>>>>>>>> The patch is munged by the e-mail client, and it causes immediate
>>>>>>>> voluntary preemprion in rcu critical section and hard lockup on starting
>>>>>>>> a KVM VM.
>>>>>>>>
>>>>>>>> Also it would be sort of bad news if it worked because that would be
>>>>>>>> specific to book3s KVM HV and would not help with the KVM HV from the
>>>>>>>> original report which likely is not book3s, nor with KVM PR.
>>>>>>>>
>>>>>>>
>>>>>>> Thanks for trying. We will try a local repro and look into it why stack is
>>>>>>> getting corrupted.
>>>>>>
>>
>> As we discussed offlist, samir helped to run a similar test on his machine, and he didn't
>> run into issue so far. we will try more.
>>
>> I am just wondering what different in your case?
>> By any chance we are running into below one? Can you check your gcc version?
>>
>> https://lore.kernel.org/all/CAABZP2z=xu+07-y5fqFLidZz1VpSgrSwXa1mFHPb=b3Ezr3OtA@mail.gmail.com/
>>
>> Maybe CONFIG_DEBUG_PREEMPT worth a try to if it shows up anything.
> 
> Does not change anything, the kernel crashes all the same.
> 
> Nonetheless, I noticed that a tool that does some BPF tracing is
> running, and without it the problem is not reproducible.
> 
> Thanks
> 
> Michal
Thanks Michal for the update.

Interesting finding! Since the crash is only reproducible when the BPF 
tracing tool is running, could you share the following details?

Which BPF tracing tool are you using (e.g., bpftrace, perf, bcc-based 
tool, a custom tool)? What is the specific BPF program or filter being 
attached — particularly which kernel function(s) or tracepoints it hooks 
into (e.g., is it attaching a kprobe/uprobe/tracepoint anywhere inside 
kvmhv_run_single_vcpu or related KVM HV paths)?
What is the BPF program type — kprobe, tracepoint, perf_event, 
fentry/fexit, etc.?

Regards,
Narayana Murty.

Re: [Linux PPC] Disable PREEMPT

[Michal Suchánek]

On Tue, Sep 15, 2026 at 11:17:23AM +0530, Narayana Murty N wrote:
> 
> 
> On 14/09/26 4:28 PM, Michal Suchánek wrote:
> > On Fri, Sep 11, 2026 at 10:39:10PM +0530, Shrikanth Hegde wrote:
> > > Hi Michal,
> > > 
> > > On 9/10/26 5:46 PM, Michal Suchánek wrote:
> > > > On Thu, Sep 10, 2026 at 01:29:08PM +0200, Michal Suchánek wrote:
> > > > > On Thu, Sep 10, 2026 at 04:11:20PM +0530, Shrikanth Hegde wrote:
> > > > > > 
> > > > > > 
> > > > > > On 9/10/26 2:29 PM, Michal Suchánek wrote:
> > > > > > > On Fri, Sep 04, 2026 at 02:34:21PM +0530, Shrikanth Hegde wrote:
> > > > > > > > 
> > > > > > > > 
> > > > > > > > On 9/4/26 1:05 PM, Michal Suchánek wrote:
> > > > > > > > > On Thu, Sep 03, 2026 at 10:52:59PM +0530, Shrikanth Hegde wrote:
> > > > > > > > > > 
> > > > > > > > > > > > If possible run against current upstream and share the results.
> > > > > > > > > > > 
> > > > > > > > > > > https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default
> > > > > > > > > > > 
> > > > > > > > > > > [  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > > > > > > > > > > [  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
> > > > > > > > > > > [  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
> > > > > > > > > > > [  472.091624][ T6181] Call Trace:
> > > > > > > > > > > [  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
> > > > > > > > > > > [  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
> > > > > > > > > > > [  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
> > > > > > > > > > > [  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
> > > > > > > > > > > [  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
> > > > > > > > > > > [  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
> > > > > > > > > > > [  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
> > > > > > > > > > > [  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
> > > > > > > > > > > [  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
> > > > > > > > > > > [  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
> > > > > > > > > > > [  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
> > > > > > > > > > > [  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
> > > > > > > > > > > [  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
> > > > > > > > > > > [  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
> > > > > > > > > > > [  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
> > > > > > > > > > > [  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
> > > > > > > > > > > [  472.091938][ T6181] IRQMASK: 0
> > > > > > > > > > > [  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa
> > > > > > > > > > > [  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000
> > > > > > > > > > > [  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000
> > > > > > > > > > > [  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0
> > > > > > > > > > > [  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7
> > > > > > > > > > > [  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000
> > > > > > > > > > > [  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70
> > > > > > > > > > > [  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000
> > > > > > > > > > > [  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
> > > > > > > > > > > [  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
> > > > > > > > > > > [  472.092033][ T6181] ---- interrupt: 3000
> > > > > > > > > > > [  472.098256][ T6181] pstore: backend (nvram) writing error (-1)
> > > > > > > > > > > 
> > > > > > > > > > > This is the host, cannot run the kernel as guest because it fails to boot most
> > > > > > > > > > > of the time inside KVM.
> > > > > > > > > > > 
> > > > > > > > > > > > 
> > > > > > > > > > > > > Nonethless, there are quite a few platforms. Originally no preemption
> > > > > > > > > > > > > was the only option, and that's the reason why many people run that.
> > > > > > > > > > > > > It's the conservative, known working option. And that's the reason a lot
> > > > > > > > > > > > > of platfrom code does not get tested with more aggressive preemtion
> > > > > > > > > > > > > models, and never gets fixed to work with them.
> > > > > > > > > > > > > 
> > > > > > > > > > > > 
> > > > > > > > > > > > Full preemption has been there for many years!.
> > > > > > > > > > > 
> > > > > > > > > > > Possible for years, forced only recently.
> > > > > > > > > > > 
> > > > > > > > > > > > Lazy is not that aggressive compared to that.
> > > > > > > > > > > > 
> > > > > > > > > > > > > Simply disabling the no preemtion option does not make the platform code
> > > > > > > > > > > > > ready.
> > > > > > > > > > > > > 
> > > > > > > > > > > > 
> > > > > > > > > > > > Let's understand your crash case. Let's see where it is going wrong. I am suspecting
> > > > > > > > > > > > it is some wrong usage of preemption api rather than arch can't support preemption.
> > > > > > > > > > > 
> > > > > > > > > > > Very likely some wrong use of the preemption API by the arch code, or no
> > > > > > > > > > > use where it should have been used. It did not matter so long as people
> > > > > > > > > > > could run their no preempt configs and ignore the problem.
> > > > > > > > > > > 
> > > > > > > > > > > Thanks
> > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > Friendly LLM analysis says preemption is enabled too early and before
> > > > > > > > > > completing kvmppc_handle_exit_hv() / kvmppc_handle_nested_exit()
> > > > > > > > > > 
> > > > > > > > > > Below is ONLY a speculation and completely UNTESTED.
> > > > > > > > > > Maybe worth a try.
> > > > > > > > > 
> > > > > > > > > The patch is munged by the e-mail client, and it causes immediate
> > > > > > > > > voluntary preemprion in rcu critical section and hard lockup on starting
> > > > > > > > > a KVM VM.
> > > > > > > > > 
> > > > > > > > > Also it would be sort of bad news if it worked because that would be
> > > > > > > > > specific to book3s KVM HV and would not help with the KVM HV from the
> > > > > > > > > original report which likely is not book3s, nor with KVM PR.
> > > > > > > > > 
> > > > > > > > 
> > > > > > > > Thanks for trying. We will try a local repro and look into it why stack is
> > > > > > > > getting corrupted.
> > > > > > > 
> > > 
> > > As we discussed offlist, samir helped to run a similar test on his machine, and he didn't
> > > run into issue so far. we will try more.
> > > 
> > > I am just wondering what different in your case?
> > > By any chance we are running into below one? Can you check your gcc version?
> > > 
> > > https://lore.kernel.org/all/CAABZP2z=xu+07-y5fqFLidZz1VpSgrSwXa1mFHPb=b3Ezr3OtA@mail.gmail.com/
> > > 
> > > Maybe CONFIG_DEBUG_PREEMPT worth a try to if it shows up anything.
> > 
> > Does not change anything, the kernel crashes all the same.
> > 
> > Nonetheless, I noticed that a tool that does some BPF tracing is
> > running, and without it the problem is not reproducible.
> > 
> > Thanks
> > 
> > Michal
> Thanks Michal for the update.
> 
> Interesting finding! Since the crash is only reproducible when the BPF
> tracing tool is running, could you share the following details?
> 
> Which BPF tracing tool are you using (e.g., bpftrace, perf, bcc-based tool,
> a custom tool)?

A custom tool, likely a version of this:
https://build.opensuse.org/package/show/security:sensor/velociraptor

> What is the specific BPF program or filter being attached —
> particularly which kernel function(s) or tracepoints it hooks into (e.g., is
> it attaching a kprobe/uprobe/tracepoint anywhere inside
> kvmhv_run_single_vcpu or related KVM HV paths)?
> What is the BPF program type — kprobe, tracepoint, perf_event, fentry/fexit,
> etc.?

It would be likely these programs:
https://github.com/SUSE/linux-security-sensor/tree/sensor-base-0.76.7/vql/linux/bpf

Thanks

Michal

Re: [Linux PPC] Disable PREEMPT

[Shrikanth Hegde]

Hi.

On 9/15/26 1:05 PM, Michal Suchánek wrote:
> On Tue, Sep 15, 2026 at 11:17:23AM +0530, Narayana Murty N wrote:
>>
>>
>> On 14/09/26 4:28 PM, Michal Suchánek wrote:
>>> On Fri, Sep 11, 2026 at 10:39:10PM +0530, Shrikanth Hegde wrote:
>>>> Hi Michal,
>>>>
>>>> On 9/10/26 5:46 PM, Michal Suchánek wrote:
>>>>> On Thu, Sep 10, 2026 at 01:29:08PM +0200, Michal Suchánek wrote:
>>>>>> On Thu, Sep 10, 2026 at 04:11:20PM +0530, Shrikanth Hegde wrote:
>>>>>>>
>>>>>>>
>>>>>>> On 9/10/26 2:29 PM, Michal Suchánek wrote:
>>>>>>>> On Fri, Sep 04, 2026 at 02:34:21PM +0530, Shrikanth Hegde wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 9/4/26 1:05 PM, Michal Suchánek wrote:
>>>>>>>>>> On Thu, Sep 03, 2026 at 10:52:59PM +0530, Shrikanth Hegde wrote:
>>>>>>>>>>>
>>>>>>>>>>>>> If possible run against current upstream and share the results.
>>>>>>>>>>>>
>>>>>>>>>>>> https://github.com/openSUSE/kernel-source/blob/6824496d1801f73def615dca8794202eeb7b0d86/config/ppc64le/default
>>>>>>>>>>>>
>>>>>>>>>>>> [  472.091531][ T6181] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
>>>>>>>>>>>> [  472.091598][ T6181] CPU: 29 UID: 107 PID: 6181 Comm: CPU 112/KVM Not tainted 7.2.2-5.g6824496-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  61871a5f06863b4006f5ef27cd9f18e8a7a2edad
>>>>>>>>>>>> [  472.091612][ T6181] Hardware name: IBM,9824-42A Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.20 (OB1110_130) hv:phyp pSeries
>>>>>>>>>>>> [  472.091624][ T6181] Call Trace:
>>>>>>>>>>>> [  472.091630][ T6181] [c00000000fdfb680] [c00000000134ce90] dump_stack_lvl+0x84/0xc0 (unreliable)
>>>>>>>>>>>> [  472.091653][ T6181] [c00000000fdfb6b0] [c00000000022e7c8] vpanic+0x324/0x5e4
>>>>>>>>>>>> [  472.091666][ T6181] [c00000000fdfb760] [c00000000022eac4] do_panic_on_target_cpu+0x0/0x2c
>>>>>>>>>>>> [  472.091677][ T6181] [c00000000fdfb780] [c0000000013c1ff8] __stack_chk_fail+0x48/0x60
>>>>>>>>>>>> [  472.091689][ T6181] [c00000000fdfb7f0] [c00800001aae219c] kvmhv_run_single_vcpu+0x19d4/0x1b50 [kvm_hv]
>>>>>>>>>>>> [  472.091712][ T6181] [c00000000fdfb940] [c00800001aae24b4] kvmppc_vcpu_run_hv+0x19c/0x12f0 [kvm_hv]
>>>>>>>>>>>> [  472.091732][ T6181] [c00000000fdfba10] [c00800001aeeed18] kvmppc_vcpu_run+0x30/0x48 [kvm]
>>>>>>>>>>>> [  472.091779][ T6181] [c00000000fdfba30] [c00800001aee9ef4] kvm_arch_vcpu_ioctl_run+0x35c/0x4a0 [kvm]
>>>>>>>>>>>> [  472.091813][ T6181] [c00000000fdfbac0] [c00800001aedaac4] kvm_vcpu_ioctl+0x1ac/0xad8 [kvm]
>>>>>>>>>>>> [  472.091844][ T6181] [c00000000fdfbca0] [c0000000007f1244] sys_ioctl+0x374/0x1060
>>>>>>>>>>>> [  472.091857][ T6181] [c00000000fdfbdb0] [c00000000002f7f8] system_call_exception+0x188/0x430
>>>>>>>>>>>> [  472.091871][ T6181] [c00000000fdfbe50] [c00000000000cfdc] system_call_vectored_common+0x15c/0x2ec
>>>>>>>>>>>> [  472.091886][ T6181] ---- interrupt: 3000 at 0x7fffb5565fac
>>>>>>>>>>>> [  472.091896][ T6181] NIP:  00007fffb5565fac LR: 00007fffb5565fac CTR: 0000000000000000
>>>>>>>>>>>> [  472.091904][ T6181] REGS: c00000000fdfbe80 TRAP: 3000   Not tainted  (7.2.2-5.g6824496-default)
>>>>>>>>>>>> [  472.091911][ T6181] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 42044402  XER: 00000000
>>>>>>>>>>>> [  472.091938][ T6181] IRQMASK: 0
>>>>>>>>>>>> [  472.091938][ T6181] GPR00: 0000000000000036 00007fbfa77ed7a0 00007fffb5677100 00000000000000fa
>>>>>>>>>>>> [  472.091938][ T6181] GPR04: 000000002000ae80 0000000000000000 0000000000000000 0000000000000000
>>>>>>>>>>>> [  472.091938][ T6181] GPR08: 00000000000000fa 0000000000000000 0000000000000000 0000000000000000
>>>>>>>>>>>> [  472.091938][ T6181] GPR12: 0000000000000000 00007fbfa77f5ec0 000000014676f000 00007fbfa77ee7c0
>>>>>>>>>>>> [  472.091938][ T6181] GPR16: 000000014674e8d0 00007fbfa77eeec0 00007fbfa77eeec0 fffffffffffffff7
>>>>>>>>>>>> [  472.091938][ T6181] GPR20: 00007fffb71210d0 0000000000000001 00007fbfa77eeec0 0000000000000000
>>>>>>>>>>>> [  472.091938][ T6181] GPR24: 00007fbfa77ed8e8 0000000105971428 000000002000ae80 0000000105f77a70
>>>>>>>>>>>> [  472.091938][ T6181] GPR28: 0000000000000000 0000000000000000 000000002000ae80 000000014674f000
>>>>>>>>>>>> [  472.092020][ T6181] NIP [00007fffb5565fac] 0x7fffb5565fac
>>>>>>>>>>>> [  472.092027][ T6181] LR [00007fffb5565fac] 0x7fffb5565fac
>>>>>>>>>>>> [  472.092033][ T6181] ---- interrupt: 3000
>>>>>>>>>>>> [  472.098256][ T6181] pstore: backend (nvram) writing error (-1)
>>>>>>>>>>>>
>>>>>>>>>>>> This is the host, cannot run the kernel as guest because it fails to boot most
>>>>>>>>>>>> of the time inside KVM.
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Nonethless, there are quite a few platforms. Originally no preemption
>>>>>>>>>>>>>> was the only option, and that's the reason why many people run that.
>>>>>>>>>>>>>> It's the conservative, known working option. And that's the reason a lot
>>>>>>>>>>>>>> of platfrom code does not get tested with more aggressive preemtion
>>>>>>>>>>>>>> models, and never gets fixed to work with them.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Full preemption has been there for many years!.
>>>>>>>>>>>>
>>>>>>>>>>>> Possible for years, forced only recently.
>>>>>>>>>>>>
>>>>>>>>>>>>> Lazy is not that aggressive compared to that.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Simply disabling the no preemtion option does not make the platform code
>>>>>>>>>>>>>> ready.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Let's understand your crash case. Let's see where it is going wrong. I am suspecting
>>>>>>>>>>>>> it is some wrong usage of preemption api rather than arch can't support preemption.
>>>>>>>>>>>>
>>>>>>>>>>>> Very likely some wrong use of the preemption API by the arch code, or no
>>>>>>>>>>>> use where it should have been used. It did not matter so long as people
>>>>>>>>>>>> could run their no preempt configs and ignore the problem.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Friendly LLM analysis says preemption is enabled too early and before
>>>>>>>>>>> completing kvmppc_handle_exit_hv() / kvmppc_handle_nested_exit()
>>>>>>>>>>>
>>>>>>>>>>> Below is ONLY a speculation and completely UNTESTED.
>>>>>>>>>>> Maybe worth a try.
>>>>>>>>>>
>>>>>>>>>> The patch is munged by the e-mail client, and it causes immediate
>>>>>>>>>> voluntary preemprion in rcu critical section and hard lockup on starting
>>>>>>>>>> a KVM VM.
>>>>>>>>>>
>>>>>>>>>> Also it would be sort of bad news if it worked because that would be
>>>>>>>>>> specific to book3s KVM HV and would not help with the KVM HV from the
>>>>>>>>>> original report which likely is not book3s, nor with KVM PR.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks for trying. We will try a local repro and look into it why stack is
>>>>>>>>> getting corrupted.
>>>>>>>>
>>>>
>>>> As we discussed offlist, samir helped to run a similar test on his machine, and he didn't
>>>> run into issue so far. we will try more.
>>>>
>>>> I am just wondering what different in your case?
>>>> By any chance we are running into below one? Can you check your gcc version?
>>>>
>>>> https://lore.kernel.org/all/CAABZP2z=xu+07-y5fqFLidZz1VpSgrSwXa1mFHPb=b3Ezr3OtA@mail.gmail.com/
>>>>
>>>> Maybe CONFIG_DEBUG_PREEMPT worth a try to if it shows up anything.
>>>
>>> Does not change anything, the kernel crashes all the same.
>>>
>>> Nonetheless, I noticed that a tool that does some BPF tracing is
>>> running, and without it the problem is not reproducible.
>>>
>>> Thanks
>>>
>>> Michal
>> Thanks Michal for the update.
>>
>> Interesting finding! Since the crash is only reproducible when the BPF
>> tracing tool is running, could you share the following details?
>>
>> Which BPF tracing tool are you using (e.g., bpftrace, perf, bcc-based tool,
>> a custom tool)?
> 
> A custom tool, likely a version of this:
> https://build.opensuse.org/package/show/security:sensor/velociraptor
> 
>> What is the specific BPF program or filter being attached —
>> particularly which kernel function(s) or tracepoints it hooks into (e.g., is
>> it attaching a kprobe/uprobe/tracepoint anywhere inside
>> kvmhv_run_single_vcpu or related KVM HV paths)?
>> What is the BPF program type — kprobe, tracepoint, perf_event, fentry/fexit,
>> etc.?
> 
> It would be likely these programs:
> https://github.com/SUSE/linux-security-sensor/tree/sensor-base-0.76.7/vql/linux/bpf
> 
> Thanks
> 
> Michal

It seems easy to re-create.
The key is running bcc/bpf program in parallel to kernel build.

For example, these two in parallal, leads to stack corruption panic.
Stacktrace does differ sometimes.

./funccount  rcu* -d 100
make -j 32

Boom. Will try to debug this further.

[ 3979.381003][    T0] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: prb_reserve+0x498/0x4a0
[ 3979.385530][  C114] pstore: backend (nvram) writing error (-1)
[ 3980.385590][  C114] Oops: System Reset, sig: 6 [#1]
[ 3980.385605][  C114] LE PAGE_SIZE=64K MMU=Radix  SMP NR_CPUS=2048 NUMA pSeries
[ 3980.385611][  C114] Modules linked in: bonding pseries_rng rng_core vmx_crypto fuse ibmvfc ibmveth dm_mirror dm_region_hash dm_log autofs4
[ 3980.385630][  C114] CPU: 114 UID: 0 PID: 0 Comm: swapper/114 Not tainted 7.3.0-rc1-00026-gd3d9a20b4eb9 #23 PREEMPT(full)
[ 3980.385634][  C114] Hardware name: IBM,9043-MRU Power11 (architected) 0x820200 0xf000007 of:IBM,FW1120.00 (RF1120_183) hv:phyp pSeries
[ 3980.385636][  C114] NIP:  c0000000001ae714 LR: c000000001552b08 CTR: 0000000000000000
[ 3980.385638][  C114] REGS: c0000007ffc2fd60 TRAP: 0100   Not tainted  (7.3.0-rc1-00026-gd3d9a20b4eb9)
[ 3980.385641][  C114] MSR:  800000000298b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE>  CR: 22000202  XER: 20040006
[ 3980.385649][  C114] CFAR: 000000000000011c IRQMASK: 3
[ 3980.385649][  C114] GPR00: 0000000000000000 c00000040030fd50 c000000001be8100 0000000000000000
[ 3980.385649][  C114] GPR04: 0000000000000001 000000000000003a 0000000000000000 0000010000000000
[ 3980.385649][  C114] GPR08: ffffffffffffffbf 0000000000000000 ffffffffffffffff 0000000000000000
[ 3980.385649][  C114] GPR12: 0000000000000000 c0000007fffdeb00 0000000000000000 000000002ef01f40
[ 3980.385649][  C114] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 3980.385649][  C114] GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000001
[ 3980.385649][  C114] GPR24: 0000000000000001 0000000000000000 0000039e85974a0e c0000000029ab5a8
[ 3980.385649][  C114] GPR28: c0000007fe8f9a40 0000000000000001 c0000000022020a8 c0000000022020b0
[ 3980.385675][  C114] NIP [c0000000001ae714] plpar_hcall_norets_notrace+0x18/0x2c
[ 3980.385684][  C114] LR [c000000001552b08] check_and_cede_processor+0x48/0x60
[ 3980.385692][  C114] Call Trace:
[ 3980.385693][  C114] [c00000040030fd50] [0000000042000202] 0x42000202 (unreliable)
[ 3980.385703][  C114] [c00000040030fdb0] [c000000001552d30] shared_cede_loop+0x70/0x170
[ 3980.385708][  C114] [c00000040030fdf0] [c000000001552170] cpuidle_enter_state+0x300/0x748
[ 3980.385712][  C114] [c00000040030fe90] [c0000000011394f0] cpuidle_enter+0x50/0x80
[ 3980.385718][  C114] [c00000040030fed0] [c0000000002a5a78] call_cpuidle+0x48/0x90
[ 3980.385722][  C114] [c00000040030fef0] [c0000000002acb0c] do_idle+0x2cc/0x470
[ 3980.385725][  C114] [c00000040030ff60] [c0000000002acf7c] cpu_startup_entry+0x4c/0x50
[ 3980.385729][  C114] [c00000040030ff90] [c00000000005be80] start_secondary+0x290/0x2a0
[ 3980.385733][  C114] [c00000040030ffe0] [c00000000000e258] start_secondary_prolog+0x10/0x14
[ 3980.385737][  C114] Code: 3d220063 39295780 7c634a78 7c630074 7863d182 4e800020 3c4c01a4 38429a04 7c421378 7c000026 90010008 44000022 <38800000> 988d0931 80010008 7c0ff120

Re: [Linux PPC] Disable PREEMPT

[Shrikanth Hegde]

Hi Michal,

> 
> It seems easy to re-create.
> The key is running bcc/bpf program in parallel to kernel build.
> 
> For example, these two in parallal, leads to stack corruption panic.
> Stacktrace does differ sometimes.
> 
> ./funccount  rcu* -d 100
> make -j 32
> 
> Boom. Will try to debug this further.

A bit more debugging.

1. A few functions seems to crash, specially ones that call schedule.
    So picked schedule itself for tracing
2. Used plain ftrace. No crash.
3. Used kprobes. Boom. Crash appears. But this helped to avoid BPF JIT, or BCC part of it.
4. Did some preempt count validation with annotations. Seems to be holding fine.
5. Since unlock can call schedule while tracing schedule, there is possible looping.
    Need to see further why it happens to be crashing there. Maybe regs or paca
    handling needs a closer look.

For now, below patch helps to keep the system stable under kprobes, bcc tools.
I will look further into see why exactly it is crashing and below one is going to
be the proper fix or not.

Feel free to give it a try.

---
  arch/powerpc/kernel/kprobes-ftrace.c | 2 ++
  1 file changed, 2 insertions(+)

diff --git a/arch/powerpc/kernel/kprobes-ftrace.c b/arch/powerpc/kernel/kprobes-ftrace.c
index f8208c027148..bcb1e8a8995d 100644
--- a/arch/powerpc/kernel/kprobes-ftrace.c
+++ b/arch/powerpc/kernel/kprobes-ftrace.c
@@ -63,7 +63,9 @@ void kprobe_ftrace_handler(unsigned long nip, unsigned long parent_nip,
  		__this_cpu_write(current_kprobe, NULL);
  	}
  out:
+	preempt_disable_notrace();
  	ftrace_test_recursion_unlock(bit);
+	preempt_enable_no_resched_notrace();
  }
  NOKPROBE_SYMBOL(kprobe_ftrace_handler);
  
-- 
2.52.0

Re: [Linux PPC] Disable PREEMPT

[Michal Suchánek]

Hello,

On Wed, Sep 16, 2026 at 11:41:38PM +0530, Shrikanth Hegde wrote:
> Hi Michal,
> 
> > 
> > It seems easy to re-create.
> > The key is running bcc/bpf program in parallel to kernel build.
> > 
> > For example, these two in parallal, leads to stack corruption panic.
> > Stacktrace does differ sometimes.
> > 
> > ./funccount  rcu* -d 100
> > make -j 32
> > 
> > Boom. Will try to debug this further.
> 
> A bit more debugging.
> 
> 1. A few functions seems to crash, specially ones that call schedule.
>    So picked schedule itself for tracing
> 2. Used plain ftrace. No crash.
> 3. Used kprobes. Boom. Crash appears. But this helped to avoid BPF JIT, or BCC part of it.
> 4. Did some preempt count validation with annotations. Seems to be holding fine.
> 5. Since unlock can call schedule while tracing schedule, there is possible looping.
>    Need to see further why it happens to be crashing there. Maybe regs or paca
>    handling needs a closer look.
> 
> For now, below patch helps to keep the system stable under kprobes, bcc tools.
> I will look further into see why exactly it is crashing and below one is going to
> be the proper fix or not.
> 
> Feel free to give it a try.

thanks for the patch.

While it is mangled by the e-mail client in different way compared to
the previous patch it does seem to mitigate the issue.

As it is unlikely the board where KVM lockup was originally observed is
running kpbobes it will probably not address the original problem.

Thanks

Michal

> 
> ---
>  arch/powerpc/kernel/kprobes-ftrace.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/arch/powerpc/kernel/kprobes-ftrace.c b/arch/powerpc/kernel/kprobes-ftrace.c
> index f8208c027148..bcb1e8a8995d 100644
> --- a/arch/powerpc/kernel/kprobes-ftrace.c
> +++ b/arch/powerpc/kernel/kprobes-ftrace.c
> @@ -63,7 +63,9 @@ void kprobe_ftrace_handler(unsigned long nip, unsigned long parent_nip,
>  		__this_cpu_write(current_kprobe, NULL);
>  	}
>  out:
> +	preempt_disable_notrace();
>  	ftrace_test_recursion_unlock(bit);
> +	preempt_enable_no_resched_notrace();
>  }
>  NOKPROBE_SYMBOL(kprobe_ftrace_handler);
> -- 
> 2.52.0

Re: [Linux PPC] Disable PREEMPT

[Shrikanth Hegde]

Hi Michal,

On 9/17/26 3:44 PM, Michal Suchánek wrote:
> Hello,
> 
> On Wed, Sep 16, 2026 at 11:41:38PM +0530, Shrikanth Hegde wrote:
>> Hi Michal,
>>
>>>
>>> It seems easy to re-create.
>>> The key is running bcc/bpf program in parallel to kernel build.
>>>
>>> For example, these two in parallal, leads to stack corruption panic.
>>> Stacktrace does differ sometimes.
>>>
>>> ./funccount  rcu* -d 100
>>> make -j 32
>>>
>>> Boom. Will try to debug this further.
>>
>> A bit more debugging.
>>
>> 1. A few functions seems to crash, specially ones that call schedule.
>>     So picked schedule itself for tracing
>> 2. Used plain ftrace. No crash.
>> 3. Used kprobes. Boom. Crash appears. But this helped to avoid BPF JIT, or BCC part of it.
>> 4. Did some preempt count validation with annotations. Seems to be holding fine.
>> 5. Since unlock can call schedule while tracing schedule, there is possible looping.
>>     Need to see further why it happens to be crashing there. Maybe regs or paca
>>     handling needs a closer look.
>>
>> For now, below patch helps to keep the system stable under kprobes, bcc tools.
>> I will look further into see why exactly it is crashing and below one is going to
>> be the proper fix or not.
>>
>> Feel free to give it a try.
> 
> thanks for the patch.
> 
> While it is mangled by the e-mail client in different way compared to
> the previous patch it does seem to mitigate the issue.

Thanks for trying.

> 
> As it is unlikely the board where KVM lockup was originally observed is
> running kpbobes it will probably not address the original problem.
> 

Give it a try.

I am suspecting both lockup, stack corruptions are secondary symptoms here.
While recreate, I have different type of traces.

> Thanks
> 
> Michal
> 
>>
>> ---
>>   arch/powerpc/kernel/kprobes-ftrace.c | 2 ++
>>   1 file changed, 2 insertions(+)
>>
>> diff --git a/arch/powerpc/kernel/kprobes-ftrace.c b/arch/powerpc/kernel/kprobes-ftrace.c
>> index f8208c027148..bcb1e8a8995d 100644
>> --- a/arch/powerpc/kernel/kprobes-ftrace.c
>> +++ b/arch/powerpc/kernel/kprobes-ftrace.c
>> @@ -63,7 +63,9 @@ void kprobe_ftrace_handler(unsigned long nip, unsigned long parent_nip,
>>   		__this_cpu_write(current_kprobe, NULL);
>>   	}
>>   out:
>> +	preempt_disable_notrace();
>>   	ftrace_test_recursion_unlock(bit);
>> +	preempt_enable_no_resched_notrace();
>>   }
>>   NOKPROBE_SYMBOL(kprobe_ftrace_handler);
>> -- 
>> 2.52.0

Re: [Linux PPC] Disable PREEMPT

[Michal Suchánek]

On Thu, Sep 17, 2026 at 03:58:42PM +0530, Shrikanth Hegde wrote:
> Hi Michal,
> 
> On 9/17/26 3:44 PM, Michal Suchánek wrote:
> > Hello,
> > 
> > On Wed, Sep 16, 2026 at 11:41:38PM +0530, Shrikanth Hegde wrote:
> > > Hi Michal,
> > > 
> > > > 
> > > > It seems easy to re-create.
> > > > The key is running bcc/bpf program in parallel to kernel build.
> > > > 
> > > > For example, these two in parallal, leads to stack corruption panic.
> > > > Stacktrace does differ sometimes.
> > > > 
> > > > ./funccount  rcu* -d 100
> > > > make -j 32
> > > > 
> > > > Boom. Will try to debug this further.
> > > 
> > > A bit more debugging.
> > > 
> > > 1. A few functions seems to crash, specially ones that call schedule.
> > >     So picked schedule itself for tracing
> > > 2. Used plain ftrace. No crash.
> > > 3. Used kprobes. Boom. Crash appears. But this helped to avoid BPF JIT, or BCC part of it.
> > > 4. Did some preempt count validation with annotations. Seems to be holding fine.
> > > 5. Since unlock can call schedule while tracing schedule, there is possible looping.
> > >     Need to see further why it happens to be crashing there. Maybe regs or paca
> > >     handling needs a closer look.
> > > 
> > > For now, below patch helps to keep the system stable under kprobes, bcc tools.
> > > I will look further into see why exactly it is crashing and below one is going to
> > > be the proper fix or not.
> > > 
> > > Feel free to give it a try.
> > 
> > thanks for the patch.
> > 
> > While it is mangled by the e-mail client in different way compared to
> > the previous patch it does seem to mitigate the issue.
> 
> Thanks for trying.
> 
> > 
> > As it is unlikely the board where KVM lockup was originally observed is
> > running kpbobes it will probably not address the original problem.
> > 
> 
> Give it a try.
> 
> I am suspecting both lockup, stack corruptions are secondary symptoms here.
> While recreate, I have different type of traces.

Yes, it does work for the crash I see. However, we hijacked the report
of KVM locking up on FSL Cyrus+, and that is likely a different problem
still not addressed.

Thanks

Michal

> 
> > Thanks
> > 
> > Michal
> > 
> > > 
> > > ---
> > >   arch/powerpc/kernel/kprobes-ftrace.c | 2 ++
> > >   1 file changed, 2 insertions(+)
> > > 
> > > diff --git a/arch/powerpc/kernel/kprobes-ftrace.c b/arch/powerpc/kernel/kprobes-ftrace.c
> > > index f8208c027148..bcb1e8a8995d 100644
> > > --- a/arch/powerpc/kernel/kprobes-ftrace.c
> > > +++ b/arch/powerpc/kernel/kprobes-ftrace.c
> > > @@ -63,7 +63,9 @@ void kprobe_ftrace_handler(unsigned long nip, unsigned long parent_nip,
> > >   		__this_cpu_write(current_kprobe, NULL);
> > >   	}
> > >   out:
> > > +	preempt_disable_notrace();
> > >   	ftrace_test_recursion_unlock(bit);
> > > +	preempt_enable_no_resched_notrace();
> > >   }
> > >   NOKPROBE_SYMBOL(kprobe_ftrace_handler);
> > > -- 
> > > 2.52.0
> 

Re: [Linux PPC] Disable PREEMPT

[Shrikanth Hegde]

Hi Michal,

>>
>> Give it a try.
>>
>> I am suspecting both lockup, stack corruptions are secondary symptoms here.
>> While recreate, I have different type of traces.
> 
> Yes, it does work for the crash I see.

Thanks. Give me sometime to see further on that. Didn't get any time today.

> However, we hijacked the report
> of KVM locking up on FSL Cyrus+, and that is likely a different problem
> still not addressed.
> 

Ah, that's what you meant. Yes, i still don't know where FSL Cyrus+ is stuck.

I would need help from FSL Cyrus+ to get some logs or is there a way we can spin of
a QEMU guest of it? We should have the log to start somewhere.