panic in cpufreq_online() in 6.14-rc1 on PowerNV

panic in cpufreq_online() in 6.14-rc1 on PowerNV

[Dan Horák]

Hi,

I am getting a kernel panic on my Raptor Talos Power9 system after
updating to the 6.14-rc1 kernel from 6.13. Seems reproducable every
time, but I haven't start bisecting yet. Does it sound familiar to
anyone?

...
[    4.226443] powernv-cpufreq: cpufreq pstate min 0x63 nominal 0x30 max 0x0
[    4.226464] powernv-cpufreq: Workload Optimized Frequency is enabled in the platform
[    4.226662] BUG: Unable to handle kernel instruction fetch (NULL pointer?)
[    4.226687] Faulting instruction address: 0x00000000
[    4.226700] Oops: Kernel access of bad area, sig: 7 [#1]
[    4.226734] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA PowerNV
[    4.226770] Modules linked in:
[    4.226795] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.14.0-0.rc1.15.fc42.ppc64le #1
[    4.226854] Hardware name: T2P9D01 REV 1.00 POWER9 0x4e1202 opal:skiboot-bc106a0 PowerNV
[    4.226912] NIP:  0000000000000000 LR: c0000000012a57f4 CTR: 0000000000000000
[    4.226970] REGS: c00000000cdcf610 TRAP: 0400   Not tainted  (6.14.0-0.rc1.15.fc42.ppc64le)
[    4.227007] MSR:  9000000002089033 <SF,HV,VEC,EE,ME,IR,DR,RI,LE>  CR: 84008848  XER: 00000000
[    4.227063] CFAR: c0000000012a57f0 IRQMASK: 0
[    4.227063] GPR00: c0000000012a57a0 c00000000cdcf8b0 c000000002474900 c000000023447000
[    4.227063] GPR04: 0000000000000001 0000000000000000 0000000000000080 0000000000000000
[    4.227063] GPR08: 0000000000000000 c000000003cf56c8 0000000000000001 0000000000008000
[    4.227063] GPR12: 0000000000000000 c000000003f70000 c000000003cb39a8 c000000001d80988
[    4.227063] GPR16: 0000000000000001 c000000001c6a4d8 c000000001c4f828 c0000000012adf70
[    4.227063] GPR20: c000000003c30ed8 0000000000000001 0000000000000000 c0000000234474b8
[    4.227063] GPR24: c000000023447448 c000000003c30000 c000000003f492a0 0000000000000000
[    4.227063] GPR28: 0000000000000000 c000000003d56d60 c0000000039d4bc0 c000000023447000
[    4.227407] NIP [0000000000000000] 0x0
[    4.227440] LR [c0000000012a57f4] cpufreq_online+0x474/0x1250
[    4.227478] Call Trace:
[    4.227506] [c00000000cdcf8b0] [c0000000012a57a0] cpufreq_online+0x420/0x1250 (unreliable)
[    4.227559] [c00000000cdcf980] [c0000000012a6710] cpufreq_add_dev+0x110/0x170
[    4.227597] [c00000000cdcfa00] [c00000000101f858] subsys_interface_register+0x188/0x1d0
[    4.227647] [c00000000cdcfa70] [c0000000012a0cfc] cpufreq_register_driver+0x23c/0x470
[    4.227690] [c00000000cdcfb00] [c00000000308e894] powernv_cpufreq_init+0x910/0xa10
[    4.227728] [c00000000cdcfc40] [c000000000010c3c] do_one_initcall+0x7c/0x3ac
[    4.227766] [c00000000cdcfd20] [c00000000300e574] kernel_init_freeable+0x3d0/0x480
[    4.227798] [c00000000cdcfdf0] [c000000000011320] kernel_init+0x2c/0x1bc
[    4.227815] [c00000000cdcfe50] [c00000000000debc] ret_from_kernel_user_thread+0x14/0x1c
[    4.227834] --- interrupt: 0 at 0x0
[    4.227845] Code: XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
[    4.227919] ---[ end trace 0000000000000000 ]---
[    4.430870] ata5: SATA link down (SStatus 0 SControl 300)
[    4.430912] ata3: SATA link down (SStatus 0 SControl 300)
[    4.430944] ata7: SATA link down (SStatus 0 SControl 300)
[    4.440855] ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[    4.440900] ata6: SATA link down (SStatus 0 SControl 300)
[    4.440943] ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[    4.440985] ata4: SATA link down (SStatus 0 SControl 300)
[    4.552442] ata8.00: ATAPI: MARVELL VIRTUALL, 1.09, max UDMA/66
[    4.552596] ata8.00: configured for UDMA/66
[    4.553499] ata2.00: ATAPI: DRW-24D5MT, 1.00, max UDMA/133
[    4.554908] ata2.00: configured for UDMA/133
[    4.569631] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[    4.569884] ata1.00: Model 'Samsung SSD 860 EVO 500GB', rev 'RVT02B6Q', applying quirks: noncqtrim zeroaftertrim noncqonati
[    4.569986] ata1.00: supports DRM functions and may not be fully accessible
[    4.570020] ata1.00: ATA-11: Samsung SSD 860 EVO 500GB, RVT02B6Q, max UDMA/133
[    4.570798] ata1.00: 976773168 sectors, multi 1: LBA48 NCQ (depth 32), AA
[    4.576809] ata1.00: Features: Trust Dev-Sleep NCQ-sndrcv
[    4.577253] ata1.00: supports DRM functions and may not be fully accessible
[    4.584748] ata1.00: configured for UDMA/133
[    4.585278] scsi 0:0:0:0: Direct-Access     ATA      Samsung SSD 860  2B6Q PQ: 0 ANSI: 5
[    4.586021] sd 0:0:0:0: Attached scsi generic sg0 type 0
[    4.586556] ata1.00: Enabling discard_zeroes_data
[    4.586602] sd 0:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/466 GiB)
[    4.586650] sd 0:0:0:0: [sda] Write Protect is off
[    4.586698] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    4.586742] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    4.586854] sd 0:0:0:0: [sda] Preferred minimum I/O size 512 bytes
[    4.587066] scsi 1:0:0:0: CD-ROM            ASUS     DRW-24D5MT       1.00 PQ: 0 ANSI: 5
[    4.588265] ata1.00: Enabling discard_zeroes_data
[    6.037014] pstore: backend (nvram) writing error (-1)
[    6.037032]
[    7.037040] note: swapper/0[1] exited with irqs disabled
[    7.037097] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000007


	Thanks,

		Dan

Re: panic in cpufreq_online() in 6.14-rc1 on PowerNV

[Gautam Menghani]

Hi Dan,

The fix for this was pulled yesterday[1], you can apply it

[1] - https://lore.kernel.org/all/20250205181347.2079272-1-aboorvad@linux.ibm.com

Thanks,
Gautam

Re: panic in cpufreq_online() in 6.14-rc1 on PowerNV

[Dan Horák]

On Thu, 6 Feb 2025 14:30:40 +0530
Gautam Menghani <gautam@linux.ibm.com> wrote:

> Hi Dan,
> 
> The fix for this was pulled yesterday[1], you can apply it
> 
> [1] - https://lore.kernel.org/all/20250205181347.2079272-1-aboorvad@linux.ibm.com

thanks for the pointer, will give it a try


		Dan

Re: panic in cpufreq_online() in 6.14-rc1 on PowerNV

[Nicholas Piggin]

On Thu Feb 6, 2025 at 6:41 PM AEST, Dan Horák wrote:
> Hi,
>
> I am getting a kernel panic on my Raptor Talos Power9 system after
> updating to the 6.14-rc1 kernel from 6.13. Seems reproducable every
> time, but I haven't start bisecting yet. Does it sound familiar to
> anyone?

No, but it's possible it could be skiboot changes in PM code.

Thanks,
Nick

Re: panic in cpufreq_online() in 6.14-rc1 on PowerNV

[Dan Horák]

On Tue, 18 Feb 2025 11:35:08 +1000
"Nicholas Piggin" <npiggin@gmail.com> wrote:

> On Thu Feb 6, 2025 at 6:41 PM AEST, Dan Horák wrote:
> > Hi,
> >
> > I am getting a kernel panic on my Raptor Talos Power9 system after
> > updating to the 6.14-rc1 kernel from 6.13. Seems reproducable every
> > time, but I haven't start bisecting yet. Does it sound familiar to
> > anyone?
> 
> No, but it's possible it could be skiboot changes in PM code.

it was the issue Gautam pointed me to, fixed in 6.14-rc2


		Dan