Re: [PATCH RT v2 0/3] riscv: add PREEMPT_RT support

Conor Dooley
2023-11-03T08:14:20Z

[-- Attachment #1.1: Type: text/plain, Size: 4158 bytes --]

On Thu, Nov 02, 2023 at 04:54:23PM +0100, Sebastian Andrzej Siewior wrote:
> On 2023-11-02 12:31:15 [+0000], Conor Dooley wrote:
> > [ 1.130384] BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:48
> > [ 1.130418]
> > [ 1.130414] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 0, name: swapper/1
> > [ 1.130971] [<ffffffff80267d78>] __alloc_pages+0xac/0x1be
> > [ 1.130978] alloc_workqueue+0x96/0x6de
> > [ 1.131005] [<ffffffff8000412a>] check_unaligned_access+0x34/0x336
> > [ 1.131010] kmem_cache_init_late+0x1c/0x36
> > [ 1.131038] [<ffffffff80004660>] check_unaligned_access_nonboot_cpu+0x12/0x1a
> > [ 1.131046] start_kernel+0x204/0x7e6
> > [ 1.131082] irq event stamp: 822
> > [ 1.131071] [<ffffffff800fcba0>] __flush_smp_call_function_queue+0x1de/0x790
> > [ 1.131093] hardirqs last enabled at (821): [<ffffffff80bd75b0>] default_idle_call+0xfa/0x152
> > [ 1.131111] [<ffffffff800fd3a0>] generic_smp_call_function_single_interrupt+0xe/0x1a
> > [ 1.131124] hardirqs last disabled at (822): [<ffffffff80bd52a0>] do_irq+0x16/0xa6
>
> Without the patch check_unaligned_access_boot_cpu() ->
> check_unaligned_access() is okay. The SMP/IPI variant (the patch imho)
> is bad. There must be no memory allocation from IRQ-off region - the
> smp-function call in this case.
>
> Would it be okay if I delay it until this solved? If you have something,
> then I could apply the two patches plus the fixup for this.

I'm don't think that problems are restricted to a tree containing Evan's
patch for parallel checks. For example:

[ 0.057684] rcu: Hierarchical SRCU implementation.
[ 0.057708] rcu: Max phase no-delay instances is 1000.
[ 0.100085] EFI services will not be available.
[ 0.102978] smp: Bringing up secondary CPUs ...
[ 0.112532] BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:48
[ 0.112580] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 0, name: swapper/1
[ 0.112597] preempt_count: 1, expected: 0
[ 0.112617] RCU nest depth: 1, expected: 1
[ 0.112660] 3 locks held by swapper/1/0:
[ 0.112679] #0: ff60000ffef8b958 (&pcp->lock){+.+.}-{2:2}, at: get_page_from_freelist+0x338/0x143e
[ 0.113123] #1: ffffffff81aa6920 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x0/0x32
[ 0.113187] #2: ffffffff81c42cf8 (&zone->lock){+.+.}-{2:2}, at: __rmqueue_pcplist+0x12a/0xbc0
[ 0.113268] irq event stamp: 0
[ 0.113281] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[ 0.113349] hardirqs last disabled at (0): [<ffffffff8001536a>] copy_process+0x53a/0xdec
[ 0.113368] softirqs last enabled at (0): [<ffffffff80015378>] copy_process+0x548/0xdec
[ 0.113384] softirqs last disabled at (0): [<0000000000000000>] 0x0
[ 0.113497] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 6.6.0-rc6-rt10-00111-g399ad8863993-dirty #1
[ 0.113570] Hardware name: Microchip PolarFire-SoC Icicle Kit (DT)
[ 0.113641] Call Trace:
[ 0.113705] [<ffffffff80006f76>] show_stack+0x2c/0x3c
[ 0.113723] [<ffffffff80bd4ba4>] dump_stack_lvl+0x5e/0x84
[ 0.113738] [<ffffffff80bd4bde>] dump_stack+0x14/0x1c
[ 0.113751] [<ffffffff8005b9c0>] __might_resched+0x1b4/0x1c2
[ 0.113764] [<ffffffff80bdfd3e>] rt_spin_lock+0x42/0xdc
[ 0.113779] [<ffffffff8026717a>] __rmqueue_pcplist+0x12a/0xbc0
[ 0.113792] [<ffffffff80268148>] get_page_from_freelist+0x37a/0x143e
[ 0.113805] [<ffffffff80267cbc>] __alloc_pages+0xac/0x1be
[ 0.113816] [<ffffffff80004128>] check_unaligned_access+0x32/0x332
[ 0.113827] [<ffffffff80009b16>] smp_callin+0x42/0x6c
[ 0.138406] cpu1: Ratio of byte access time to unaligned word access is 4.00, unaligned accesses are fast
[ 0.180249] cpu2: Ratio of byte access time to unaligned word access is 4.50, unaligned accesses are fast
[ 0.213500] cpu3: Ratio of byte access time to unaligned word access is 4.00, unaligned accesses are fast
[ 0.214860] smp: Brought up 1 node, 4 CPUs

(this is qemu, hence the fast unaligned access, despite OpenSBI
emulating them)

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

[-- Attachment #2: Type: text/plain, Size: 161 bytes --]