Re: [Linux PPC] Disable PREEMPT

["Michal Suchánek" <msuchanek@suse.de>]

On Thu, Sep 03, 2026 at 03:28:26PM +0200, Michal Suchánek wrote:
> On Thu, Sep 03, 2026 at 06:06:16PM +0530, Harsh Prateek Bora wrote:
> > + Amit, Gautam - since they recently came across a similar regression which
> > is seen with non KVM workload as well.
> > + Venkat - for CI improvement suggestion below.
> > 
> > Hi Michal,
> > 
> > On 03/09/26 5:55 pm, Michal Suchánek wrote:
> 
> > > 
> > > There is another problem that Linux 7.2.2 often locks up on boot inside
> > > the guest, and as a result a different kernel is needed in the guest to
> > > reproduce the host crash. This is more recent, linux 6.12 does not have
> > > this problem. Will try some different kernel combinations to see if I
> > > can narrow this down. For this the repro is merely booting the kernel in
> > > the VM, it typically fails early before userspace produces any messages.
> 
> This one regressed between 6.19.12
> https://github.com/openSUSE/kernel-source/blob/9f6830f/config/ppc64le/default
> and 7.0.12
> https://github.com/openSUSE/kernel-source/blob/2ebf0bc/config/ppc64le/default
> 
> Thanks
> 
> Michal
> 
> SLOF **********************************************************************
> QEMU Starting
>  Build Date = Aug 13 2026 14:47:36
>  FW Version = abuild@OBS release 20230918
>  Press "s" to enter Open Firmware.
> 
> Populating /vdevice methods
> Populating /vdevice/vty@30000000
> Populating /vdevice/nvram@71000000
> Populating /pci@800000020000000
> Loading Linux 7.0.12-1.g2ebf0bc-default ...
> Loading initial ramdisk ...                
> OF stdout device is: /vdevice/vty@30000000
> Preparing to boot Linux version 7.0.12-1.g2ebf0bc-default (geeko@buildhost) (gcc (SUSE Linux) 15.3.0, GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.45.0.20251103-4) #1 SMP PREEMPT_DYNAMIC Mon Jun 15 08:39:32 UTC 2026 (2ebf0bc)
> Detected machine type: 0000000000000101
> command line: BOOT_IMAGE=/boot/vmlinux-7.0.12-1.g2ebf0bc-default root=UUID=304c7f07-efbb-1070-2f27-accd935ec088 rw quiet systemd.show_status=1 security=selinux selinux=1
> Max number of cores passed to firmware: 8192 (NR_CPUS = 8192)
> Calling ibm,client-architecture-support... done
> memory layout at init:
>   memory_limit : 0000000000000000 (16 MB aligned)
>   alloc_bottom : 00000000066b0000
>   alloc_top    : 0000000030000000
>   alloc_top_hi : 0000003e00000000
>   rmo_top      : 0000000030000000
>   ram_top      : 0000003e00000000
> instantiating rtas at 0x000000002fff0000... done
> prom_hold_cpus: skipped
> copying OF device tree...
> Building dt strings...
> Building dt structure...
> Device tree strings 0x00000000066c0000 -> 0x00000000066c0bec
> Device tree struct  0x00000000066d0000 -> 0x00000000066f0000
> Quiescing Open Firmware ...
> Booting Linux via __start() @ 0x0000000000250000 ...
> [    0.000000][    T0] ERROR: Failed to allocate trace buffer
> [    0.000000][    T0] ERROR: tracer: failed to allocate ring buffer!
> Linux ppc64le
> #1 SMP PREEMPT_D[    0.285758][  T673] BUG: Kernel NULL pointer dereference on read at 0x00000010
> [    0.285877][  T673] Faulting instruction address: 0xc000000000485bd0
> [    0.285903][    T1] VFS: Dquot-cache hash table entries: 8192 (order 0, 65536 bytes)
> [    0.285954][  T673] Oops: Kernel access of bad area, sig: 7 [#1]
> [    0.286133][  T673] LE PAGE_SIZE=64K MMU=Radix  SMP NR_CPUS=8192 NUMA pSeries
> [    0.286240][  T673] Modules linked in:
> [    0.286288][  T673] CPU: 15 UID: 0 PID: 673 Comm: kworker/u531:0 Not tainted 7.0.12-1.g2ebf0bc-default #1 PREEMPT(full) openSUSE Tumbleweed (unreleased)  cf0846124d7853fab338aeae87203b36cac18419
> [    0.286527][  T673] Hardware name: IBM pSeries (emulated by qemu) Power11 (architected) 0x820200 0xf000007 of:SLOF,HEAD hv:linux,kvm pSeries
> [    0.286739][  T673] Workqueue: trace_init_wq tracer_init_tracefs_work_func
> [    0.286818][  T673] NIP:  c000000000485bd0 LR: c000000000448864 CTR: c0000000005a3420
> [    0.286906][  T673] REGS: c00000000a7a7960 TRAP: 0300   Not tainted  (7.0.12-1.g2ebf0bc-default)
> [    0.287014][  T673] MSR:  8000000000009033 <SF,EE,ME,IR,DR,RI,LE>  CR: 44088404  XER: 00000000
> [    0.287122][  T673] CFAR: c000000000448860 DAR: 0000000000000010 DSISR: 00080000 IRQMASK: 0 
> [    0.287122][  T673] GPR00: c000000000448864 c00000000a7a7c00 c000000001f38100 c000000002a5c098 
> [    0.287122][  T673] GPR04: c0000000017dc5f0 c0000000017dc5e8 000000000000001f 0000000000000064 
> [    0.287122][  T673] GPR08: c0000000017dc628 00000000000005f0 00000000000005e8 0000000084000404 
> [    0.287122][  T673] GPR12: c0000000005a3420 c000003dfff43d80 c0000000018e5600 c0000000018e54f0 
> [    0.287122][  T673] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
> [    0.287122][  T673] GPR20: c0000000013ec418 c0000000013ef6d8 c0000000017dc5a0 c0000000017dc590 
> [    0.287122][  T673] GPR24: c000000001825330 c0000000017dc628 c000000014b52a05 0000000000000000 
> [    0.287122][  T673] GPR28: c0000000017dc5e8 0000000000000000 c0000000017dc5f0 c000000002a5e008 
> [    0.287970][  T673] NIP [c000000000485bd0] __find_event_file+0x70/0x3c0
> [    0.288046][  T673] LR [c000000000448864] init_tracer_tracefs+0x274/0xc80
> [    0.288122][  T673] Call Trace:
> [    0.288167][  T673] [c00000000a7a7c00] [c00000000a7a7c60] 0xc00000000a7a7c60 (unreliable)
> [    0.288258][  T673] [c00000000a7a7c60] [c000000000448864] init_tracer_tracefs+0x274/0xc80
> [    0.288348][  T673] [c00000000a7a7dc0] [c00000000203fcf0] tracer_init_tracefs_work_func+0x50/0x320
> [    0.288452][  T673] [c00000000a7a7e50] [c0000000002620e8] process_one_work+0x1e8/0x5c0
> [    0.288541][  T673] [c00000000a7a7f10] [c00000000026309c] worker_thread+0x1dc/0x3d0
> [    0.288630][  T673] [c00000000a7a7f90] [c00000000026fa34] kthread+0x194/0x1b0
> [    0.288721][  T673] [c00000000a7a7fe0] [c00000000000de58] start_kernel_thread+0x14/0x18
> [    0.288810][  T673] Code: fb410030 fb810040 fba10048 7cbc2b78 3ba00000 fbc10050 7d194378 7c9e2378 2e2a0fc0 2da90fc0 f8010070 60420000 <e93b0010> 81490058 e8890018 714a0208 
> [    0.289001][  T673] ---[ end trace 0000000000000000 ]---
> [    0.290718][  T673] pstore: backend (nvram) writing error (-1)

This crash might be relevant to the problem, and while qemu can dump the
memory content crash refuses to open it:

crash: incompatible arguments:
   /usr/lib/modules/7.2.2-5.g6824496-default/vmlinux is not SMP -- /scratch/vmcore is SMP

The problem is not 100% reproducible, bisecting it may be tricky.

Thanks

Michal