[PATCH 4.19 061/191] padata: always acquire cpu_hotplug_lock before pinst->lock

[PATCH 4.19 061/191] padata: always acquire cpu_hotplug_lock before pinst->lock

[Greg Kroah-Hartman]

From: Daniel Jordan <daniel.m.jordan@oracle.com>

[ Upstream commit 38228e8848cd7dd86ccb90406af32de0cad24be3 ]

lockdep complains when padata's paths to update cpumasks via CPU hotplug
and sysfs are both taken:

  # echo 0 > /sys/devices/system/cpu/cpu1/online
  # echo ff > /sys/kernel/pcrypt/pencrypt/parallel_cpumask

  ======================================================
  WARNING: possible circular locking dependency detected
  5.4.0-rc8-padata-cpuhp-v3+ #1 Not tainted
  ------------------------------------------------------
  bash/205 is trying to acquire lock:
  ffffffff8286bcd0 (cpu_hotplug_lock.rw_sem){++++}, at: padata_set_cpumask+0x2b/0x120

  but task is already holding lock:
  ffff8880001abfa0 (&pinst->lock){+.+.}, at: padata_set_cpumask+0x26/0x120

  which lock already depends on the new lock.

padata doesn't take cpu_hotplug_lock and pinst->lock in a consistent
order.  Which should be first?  CPU hotplug calls into padata with
cpu_hotplug_lock already held, so it should have priority.

Fixes: 6751fb3c0e0c ("padata: Use get_online_cpus/put_online_cpus")
Signed-off-by: Daniel Jordan <daniel.m.jordan@oracle.com>
Cc: Eric Biggers <ebiggers@kernel.org>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Steffen Klassert <steffen.klassert@secunet.com>
Cc: linux-crypto@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 kernel/padata.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/padata.c b/kernel/padata.c
index cfab62923c452..c280cb153915f 100644
--- a/kernel/padata.c
+++ b/kernel/padata.c
@@ -671,8 +671,8 @@ int padata_set_cpumask(struct padata_instance *pinst, int cpumask_type,
 	struct cpumask *serial_mask, *parallel_mask;
 	int err = -EINVAL;
 
-	mutex_lock(&pinst->lock);
 	get_online_cpus();
+	mutex_lock(&pinst->lock);
 
 	switch (cpumask_type) {
 	case PADATA_CPU_PARALLEL:
@@ -690,8 +690,8 @@ int padata_set_cpumask(struct padata_instance *pinst, int cpumask_type,
 	err =  __padata_set_cpumasks(pinst, parallel_mask, serial_mask);
 
 out:
-	put_online_cpus();
 	mutex_unlock(&pinst->lock);
+	put_online_cpus();
 
 	return err;
 }
-- 
2.20.1

Re: [PATCH 4.19 061/191] padata: always acquire cpu_hotplug_lock before pinst->lock

[Daniel Jordan]

On Fri, Feb 21, 2020 at 08:40:34AM +0100, Greg Kroah-Hartman wrote:
> From: Daniel Jordan <daniel.m.jordan@oracle.com>
> 
> [ Upstream commit 38228e8848cd7dd86ccb90406af32de0cad24be3 ]
> 
> lockdep complains when padata's paths to update cpumasks via CPU hotplug
> and sysfs are both taken:
> 
>   # echo 0 > /sys/devices/system/cpu/cpu1/online
>   # echo ff > /sys/kernel/pcrypt/pencrypt/parallel_cpumask
> 
>   ======================================================
>   WARNING: possible circular locking dependency detected
>   5.4.0-rc8-padata-cpuhp-v3+ #1 Not tainted
>   ------------------------------------------------------
>   bash/205 is trying to acquire lock:
>   ffffffff8286bcd0 (cpu_hotplug_lock.rw_sem){++++}, at: padata_set_cpumask+0x2b/0x120
> 
>   but task is already holding lock:
>   ffff8880001abfa0 (&pinst->lock){+.+.}, at: padata_set_cpumask+0x26/0x120
> 
>   which lock already depends on the new lock.

I think this patch should be dropped from all stable queues (4.4, 4.9, 4.14,
4.19, 5.4, and 5.5).

The main benefit is to un-break lockdep for testing with future padata changes,
and an actual deadlock seems unlikely.

These stable versions don't fix the ordering in padata_remove_cpu() either
(nothing calls it though).

I tried the other stable padata patch in this cycle ("padata: validate cpumask
without removed CPU during offline"), it passed my tests and should stay in.

thanks,
Daniel

Re: [PATCH 4.19 061/191] padata: always acquire cpu_hotplug_lock before pinst->lock

[Sasha Levin]

On Fri, Feb 21, 2020 at 07:00:45PM -0500, Daniel Jordan wrote:
>On Fri, Feb 21, 2020 at 08:40:34AM +0100, Greg Kroah-Hartman wrote:
>> From: Daniel Jordan <daniel.m.jordan@oracle.com>
>>
>> [ Upstream commit 38228e8848cd7dd86ccb90406af32de0cad24be3 ]
>>
>> lockdep complains when padata's paths to update cpumasks via CPU hotplug
>> and sysfs are both taken:
>>
>>   # echo 0 > /sys/devices/system/cpu/cpu1/online
>>   # echo ff > /sys/kernel/pcrypt/pencrypt/parallel_cpumask
>>
>>   ======================================================
>>   WARNING: possible circular locking dependency detected
>>   5.4.0-rc8-padata-cpuhp-v3+ #1 Not tainted
>>   ------------------------------------------------------
>>   bash/205 is trying to acquire lock:
>>   ffffffff8286bcd0 (cpu_hotplug_lock.rw_sem){++++}, at: padata_set_cpumask+0x2b/0x120
>>
>>   but task is already holding lock:
>>   ffff8880001abfa0 (&pinst->lock){+.+.}, at: padata_set_cpumask+0x26/0x120
>>
>>   which lock already depends on the new lock.
>
>I think this patch should be dropped from all stable queues (4.4, 4.9, 4.14,
>4.19, 5.4, and 5.5).
>
>The main benefit is to un-break lockdep for testing with future padata changes,
>and an actual deadlock seems unlikely.
>
>These stable versions don't fix the ordering in padata_remove_cpu() either
>(nothing calls it though).
>
>I tried the other stable padata patch in this cycle ("padata: validate cpumask
>without removed CPU during offline"), it passed my tests and should stay in.

I've dropped it, thanks.

-- 
Thanks,
Sasha