Re: arm64: WARN_ON_ONCE issue when resuming from hibernation

[Will Deacon <will.deacon@arm.com>]

[+ James]

On Fri, Dec 07, 2018 at 10:40:50AM +0900, Kunihiko Hayashi wrote:
> I found that a WARN_ON_ONCE dump occured in the resuming sequence from
> hibernation on arm64 SoC (I use UniPhier LD20 environment).
> 
>     ...
>     Disabling non-boot CPUs ...
>     CPU1: shutdown
>     psci: CPU1 killed.
>     CPU2: shutdown
>     psci: CPU2 killed.
>     CPU3: shutdown
>     psci: CPU3 killed.
>     WARNING: CPU: 0 PID: 1 at ../kernel/smp.c:416 smp_call_function_many+0xd4/0x350
>     Modules linked in:
>     CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.20.0-rc4 #1
>     ...
> 
> I show the result of reading the code, however,
> I'm not sure that this issue occurs in other arm64 SoC.
> 
> In the resuming sequence, once all CPUs are stopped and local IRQs
> are disabled [1].
> 
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/power/hibernate.c?h=v4.20-rc4#n450
> 
> In case of arm64, flush_icache_range() will be called after that.
> This calls kick_all_cpus_sync() to sync all CPUs with IPI, and
> since local IRQs are disabled, WARN_ON_ONCE() will be called in
> smp_call_function_many() [2].
> 
> [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/smp.c?h=v4.20-rc4#n415
> 
> The following tree shows a part of the callgraph.
> 
>     resume_target_kernel()
>     +- local_irq_disable()
>     +- swsusp_arch_resume()				/* for arm64 */
>        +- create_safe_exec_page()			/* for arm64 */
>           +- flush_icache_range()			/* for arm64 */
>              +- kick_all_cpus_sync()
>                 +- smp_call_function()
>                    +- smp_call_function_many()
>                       +- WARN_ON_ONCE(irq_disabled())
> 
> What is the possible way to solve this issue?

Given that all secondary CPUs are hotplugged out at this point, we can
just use the non-IPI version of flush_icache_range(). Completely untested
diff below.

Will

--->8

diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c
index 6b2686d54411..29cdc99688f3 100644
--- a/arch/arm64/kernel/hibernate.c
+++ b/arch/arm64/kernel/hibernate.c
@@ -214,7 +214,7 @@ static int create_safe_exec_page(void *src_start, size_t length,
 	}
 
 	memcpy((void *)dst, src_start, length);
-	flush_icache_range(dst, dst + length);
+	__flush_icache_range(dst, dst + length);
 
 	pgdp = pgd_offset_raw(allocator(mask), dst_addr);
 	if (pgd_none(READ_ONCE(*pgdp))) {

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel