Re: [bpf] Observing Kernel Softlock up while running BPF self tests

[Hari Bathini <hbathini@linux.ibm.com>]

Hi Venkat,

Thanks for reporting..

On 20/01/26 2:40 pm, Venkat Rao Bagalkote wrote:
> Greetings!!!
> 
> 
> IBM CI has reported a kernel softlockup, while running BPF selftests on 
> PowerPC kernel.
> 
> 
> Traces:
> 
> [ 1632.509843] audit: type=1334 audit(1769127975.721:164430): prog- 
> id=82135 op=LOAD
> [ 1632.509852] audit: type=1334 audit(1769127975.721:164431): prog- 
> id=82135 op=UNLOAD
> [ 1637.016921] Mode = AA
> [ 1660.780274] watchdog: BUG: soft lockup - CPU#8 stuck for 23s! 
> [rqsl_w/8:51609]
> [ 1660.780283] Modules linked in: bpf_test_rqspinlock(OE+) 8021q(E) 
> garp(E) mrp(E) stp(E) llc(E) vrf(E) tun(E) bpf_testmod(OE) veth(E) 
> nft_fib_inet(E) nft_fib_ipv4(E) nft_fib_ipv6(E) nft_fib(E) 
> nft_reject_inet(E) nf_reject_ipv4(E) nf_reject_ipv6(E) nft_reject(E) 
> bonding(E) nft_ct(E) tls(E) nft_chain_nat(E) rfkill(E) sunrpc(E) 
> ibmveth(E) hvcs(E) hvcserver(E) pseries_rng(E) vmx_crypto(E) 
> dm_multipath(E) fuse(E) dm_mod(E) drm(E) drm_panel_orientation_quirks(E) 
> zram(E) ext4(E) crc16(E) mbcache(E) jbd2(E) sr_mod(E) sd_mod(E) cdrom(E) 
> ibmvscsi(E) scsi_transport_srp(E) [last unloaded: livepatch_sample(EK)]
> [ 1660.780352] CPU: 8 UID: 0 PID: 51609 Comm: rqsl_w/8 Tainted: G    OE 
> K     6.19.0-rc4-g960c1fd29055 #1 VOLUNTARY
> [ 1660.780359] Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [K]=LIVEPATCH
> [ 1660.780362] Hardware name: IBM,8375-42A POWER9 (architected) 0x4e0202 
> 0xf000005 of:IBM,FW950.80 (VL950_131) hv:phyp pSeries
> [ 1660.780365] NIP:  c0000000000399a8 LR: c000000000039c24 CTR: 
> c000000000039ca0
> [ 1660.780368] REGS: c000000bc19cfd28 TRAP: 0900   Tainted: G      OE K 
>      (6.19.0-rc4-g960c1fd29055)
> [ 1660.780372] MSR:  800000000280b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> 
> CR: 28000288  XER: 0000000a
> [ 1660.780386] CFAR: 0000000000000000 IRQMASK: 0
> [ 1660.780386] GPR00: c000000000039c24 c000000bc19cfd00 c000000001f58100 
> c000000bc19cfcf8
> [ 1660.780386] GPR04: c000000bc19cfea8 0000000000000000 4000000000000002 
> c0000013ff916e08
> [ 1660.780386] GPR08: 00000013fd6c0000 0000000000000049 fffffffffffffffc 
> c0080000f73e0f98
> [ 1660.780386] GPR12: c000000000039ca0 c00000002e9b6700 c000000000270808 
> c000000bad8fe980
> [ 1660.780386] GPR16: 0000000000000000 0000000000000000 0000000000000000 
> 0000000000000000
> [ 1660.780386] GPR20: 0000000000000000 0000000000000000 0000000000000000 
> 0000000000000000
> [ 1660.780386] GPR24: 0000000000000000 0000000000000000 c0080000f7360048 
> 0000000000000000
> [ 1660.780386] GPR28: 0000000000000008 0000000000000000 000001827223b155 
> 0000000000000003
> [ 1660.780433] NIP [c0000000000399a8] __replay_soft_interrupts+0x38/0x150
> [ 1660.780443] LR [c000000000039c24] 
> arch_local_irq_restore.part.0+0xe4/0x160
> [ 1660.780449] Call Trace:
> [ 1660.780452] [c000000bc19cfd00] [c000000000039a0c] 
> __replay_soft_interrupts+0x9c/0x150 (unreliable)
> [ 1660.780460] [c000000bc19cfeb0] [c000000000039c24] 
> arch_local_irq_restore.part.0+0xe4/0x160
> [ 1660.780468] [c000000bc19cfef0] [c0080000f73e042c] 
> rqspinlock_worker_fn+0x244/0x300 [bpf_test_rqspinlock]
> [ 1660.780476] [c000000bc19cff90] [c000000000270954] kthread+0x154/0x170
> [ 1660.780482] [c000000bc19cffe0] [c00000000000df78] 
> start_kernel_thread+0x14/0x18
> [ 1660.780487] Code: 60000000 7c0802a6 f8010010 f821fe51 e92d0c78 
> f92101a8 39200000 38610028 892d0933 61290040 992d0933 48102e15 
> <60000000> 39200000 e9410130 f9210160
> 

Does the system recover after this or is it unresponsive?

- Hari