From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3D2F30B501 for ; Tue, 16 Jun 2026 13:34:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781616873; cv=none; b=qSOsvanH+4OL3PLVCwVfOS6DdkVU+lr7nyxX8BfAnUji2tIB0/tISor1bFls03v6mfwEcvUjT86dm2nc3c9u6uaX2m6WbdADDyZFmqv5tWfRY4xTgdr3uD+zpYLox/U/JQCbY3T4klcdbcKZTHNKRMorn5U0+gbCxWO60fHGo50= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781616873; c=relaxed/simple; bh=X04KPe37s99Rw1FUNUB0UVe/XNyzb7ODJHUjdjyOF20=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=DNOYIrX4kCTPnFB95isIALBKExsfdswt5fyptQHKYUUFuz+39r/o0AoCkcV13rXl3Oi7Zb8N/nFPR77bYG70zq9Bec2Y+wca9m6+cZ5g3HFeMtrSmQLvLrMvCDbbZINRbIHRMXwfHqSdi5sJoiqL7BXDrTC2Ruz4vxacsaY0VVE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org; spf=pass smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=XViD6Exx; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="XViD6Exx" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=L2uuIPdC3e/AVyJ/T4HG85gYlmdLs0pJ88zWTykAPpo=; b=XViD6ExxrGHhufHBGHhFPq6RmI P8jUB/EnUNJtJit6+Ya2zgqNils/dMMKey9prVSg2iBUxw5/DeW9q2w+W9mK2JitB3+8dDwAtCXVK 0fFXjlx9SDQ77a/DmVCdA/93FYDuLnDWruzZRwfssjNgH/q85Uun/0nk49kfxijQ1VmvBTswf0NQ6 uIfzMFHSKBK11wDDftOc0OgSs9ckFRK3n09vCWwTSTWCwtMmsMTxT1vc2d+8V1keNGvzBYKxnJxFZ kuyPmh1BCJY41GW5QAQYGZMn87vQP1TTDQPWxd75gdlJL5yTBng8y5PHBTj1dYIYVMfgoM1WHlNML 5b6fUKzg==; Received: from authenticated-user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.96) (envelope-from ) id 1wZTvs-00DsxY-32; Tue, 16 Jun 2026 13:34:25 +0000 From: Breno Leitao Date: Tue, 16 Jun 2026 06:33:33 -0700 Subject: [PATCH v3 3/3] workqueue: defer the worker wakeup outside pool->lock in process_one_work() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260616-fastwake-v3-3-79da19fcd08f@debian.org> References: <20260616-fastwake-v3-0-79da19fcd08f@debian.org> In-Reply-To: <20260616-fastwake-v3-0-79da19fcd08f@debian.org> To: Tejun Heo , Lai Jiangshan Cc: linux-kernel@vger.kernel.org, marco.crivellari@suse.com, frederic@kernel.org, bigeasy@linutronix.de, Hillf Danton , Breno Leitao , kernel-team@meta.com, kmagar@redhat.com, psuriset@redhat.com X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=1712; i=leitao@debian.org; h=from:subject:message-id; bh=X04KPe37s99Rw1FUNUB0UVe/XNyzb7ODJHUjdjyOF20=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBqMVDQFsP6812aokwbdxm0Es7MoXkjFj+gnnOyP JNWGVSRO5+JAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCajFQ0AAKCRA1o5Of/Hh3 baJLD/49wtRdl6tVWTc6TAnJm3Y+klohCBApGLcbvCzsaF80cXTv3SChydoAlIf0io6l0k3DkHc bMfKvTAwy6WDLuNr3kf0mpue8tfB1On3QkWqf7xfXcI6AKm1zPjUhdUApMrOGHXCkD1dAFuzj2M Iu2p/eRud/U6kl21AsTJSsSQC95ThwC9E+dfJdjUIKHDwP9dNQFyKI0Y1NF6ktbRcepJ9huhy0D Gkl7P8H/AndI/h4gdSC1kHfIwFjKH6vpTxGIZwNboPztb1TDy+WcHia2s9TxwdtSshNgU7k5xBN L49ZAT3fWakdTm/QUPYYr3VdicfVVpC+PlIwVsnA4bdA7Ci5oh5oOCAsK1X8pcxbmA5UsuqA6lR AlMH1DTHhJ1uePg8+mTMguNI3S9lKwldvgp6otXC8udbldKGeTa8pnZ9YwOZn9KZN34/Hxn9zQ/ PoZgZUdAeG44vhdBiXdSB8eMaaZMJDV7dw9qdK3i9dCq93EnpxSUPhSNpWAkmgLPFLoXmi4Rhs2 2QlKz3cfpbzMcBi2Rp74SGVlsswYvWYY4sGyep/hJHeH5pjqTWZhn/DcmDOU4MSaMT1PO/suaZK qSB93UF7FJZUQlMPI7zhVVxnNbnfsQtUIrq+sCfFWpWvLCKvQa/KgtEcWLLXc8IEubDBrJ0p7hq +vvH+L164bFzNmA== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao process_one_work() kicks the pool to chain execution of the remaining work items on WORKER_NOT_RUNNING pools (the UNBOUND and CPU_INTENSIVE ones), calling kick_pool() while holding pool->lock. As in the enqueue path, the wakeup pulls the target rq->lock in under pool->lock. Use kick_pool_pick() to select and claim the worker under pool->lock and issue the wakeup with wake_up_q() after the lock is dropped via raw_spin_unlock_irq_wake(). Signed-off-by: Breno Leitao --- kernel/workqueue.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 44ad3450ff77f..9fb57635968e1 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -3250,6 +3250,7 @@ __acquires(&pool->lock) { struct pool_workqueue *pwq = get_work_pwq(work); struct worker_pool *pool = worker->pool; + DEFINE_WAKE_Q(wakeq); unsigned long work_data; int lockdep_start_depth, rcu_start_depth; bool bh_draining = pool->flags & POOL_BH_DRAINING; @@ -3304,7 +3305,7 @@ __acquires(&pool->lock) * chain execution of the pending work items for WORKER_NOT_RUNNING * workers such as the UNBOUND and CPU_INTENSIVE ones. */ - kick_pool(pool); + kick_pool_pick(pool, &wakeq); /* * Record the last pool and clear PENDING which should be the last @@ -3315,7 +3316,7 @@ __acquires(&pool->lock) set_work_pool_and_clear_pending(work, pool->id, pool_offq_flags(pool)); pwq->stats[PWQ_STAT_STARTED]++; - raw_spin_unlock_irq(&pool->lock); + raw_spin_unlock_irq_wake(&pool->lock, &wakeq); rcu_start_depth = rcu_preempt_depth(); lockdep_start_depth = lockdep_depth(current); -- 2.53.0-Meta