All of lore.kernel.org
 help / color / mirror / Atom feed
From: Paul Mackerras <paulus@samba.org>
To: linux-kernel@vger.kernel.org, Tejun Heo <tj@kernel.org>
Subject: [PATCH] workqueue: Don't spin forever in worker_maybe_bind_and_lock
Date: Wed, 4 May 2011 11:47:49 +1000	[thread overview]
Message-ID: <20110504014749.GA28337@drongo> (raw)

On a 48-thread POWER7 box, I often see the system hang when offlining
processors.  What happens is that we get a rescuer thread trying to
move to some processor at the same time that a cpu offline operation
is happening for that processor, and we end up with one cpu spinning in
worker_maybe_bind_and_lock() and all of the rest of the online cpus
spinning inside the stop_machine code.  The rescuer thread is
continually calling set_cpus_allowed_ptr() which is continually
failing because the cpu it is trying to move to is no longer in the
cpu_active_mask.  The result is a deadlock.

This fixes worker_maybe_bind_and_lock so that it stops trying to move
to a cpu if that cpu is no longer in the cpu_active_mask, and instead
returns to its caller.  With this I no longer see the deadlocks when
offlining cpus.

Signed-off-by: Paul Mackerras <paulus@samba.org>
---
 kernel/workqueue.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 8859a41..12faf78 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -1289,6 +1289,8 @@ __acquires(&gcwq->lock)
 		    cpumask_equal(&current->cpus_allowed,
 				  get_cpu_mask(gcwq->cpu)))
 			return true;
+		if (!cpumask_test_cpu(gcwq->cpu, cpu_active_mask))
+			return false;
 		spin_unlock_irq(&gcwq->lock);
 
 		/* CPU has come up in between, retry migration */
-- 
1.7.4.1


             reply	other threads:[~2011-05-04  1:47 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-05-04  1:47 Paul Mackerras [this message]
2011-05-04  8:26 ` [PATCH] workqueue: Don't spin forever in worker_maybe_bind_and_lock Tejun Heo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20110504014749.GA28337@drongo \
    --to=paulus@samba.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.