From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2AE103EB0EE for ; Fri, 26 Jun 2026 09:58:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782467900; cv=none; b=sqHK7eu2q6FbJb+z+zASJ/MpXrf/P1CV+WW0EkzTUHu1It9hy4iNWChVBsGJmBYR3jPt8nw0+GvF+cF7ZdA1HhXgOccoFIbsD7BsYYUN/6TQ6Ti9bGUWK1M4u7iBiet/eUeupQFo1bQ7U0+6aX8tdOtVfSIzmFu2AyIsp6QLUIg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782467900; c=relaxed/simple; bh=eMx+uhD5dwhPEDNEahHUWX7BJo8BW0YF8/v/DOzwVI8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=EYPkMHxrHkC+lfo+042Ywz8BiZm12rDDATBnNoz0TMShBzZnsiioDQKtej+jWjO6dGPGYYLhrlQ0VTS2IWV83zwLgAL1HUJcN6S5kPLA5HoCT/VHu1GVgVA/kbpaouaMKK3wHF+imckEWalLLc1R4PmYl2qVGv5r2bHY0yGk1h8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org; spf=pass smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=ROM2tgTO; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="ROM2tgTO" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=ZGltrLPve9ANZWHm3hh8HdqN19Kfsvmj3OQXKWEVRYY=; b=ROM2tgTOTyf10Md5Uxa244CRri aUnbOaBNlsbqWWut/crIChG1dx7HUsDQZyQgqO2rU6ajVw4aG/KWrZbHeTeK/ISVO8kUK9vbmUVol z61fsNBKBCeQFx4qAL1iUolQ7eU2izRlThkHfJVP20rFDOPePwt+coyIsg1+jXUu7pX16U8Zyx/sU kpA81mRmm1WmilYBnH1o7Nq+mA1RGsxYJ2dchEsnQ6H+cGgajBrEeeUBc3Lh5ohhcdVp/uIR/uXlq pkzCgl1nOuSykTMpGZMf8kmRflw/qzpkZPiocswUrDAGoEp01xugfR2RqXZgSZuHh3JStfSTCAQH2 +QRIxlgA==; Received: from authenticated-user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.96) (envelope-from ) id 1wd3KA-003t4g-2j; Fri, 26 Jun 2026 09:58:15 +0000 From: Breno Leitao Date: Fri, 26 Jun 2026 02:57:55 -0700 Subject: [PATCH v5 3/3] workqueue: defer the worker wakeup outside pool->lock in process_one_work() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260626-fastwake-v5-3-9ae2f1867234@debian.org> References: <20260626-fastwake-v5-0-9ae2f1867234@debian.org> In-Reply-To: <20260626-fastwake-v5-0-9ae2f1867234@debian.org> To: Tejun Heo , Lai Jiangshan Cc: linux-kernel@vger.kernel.org, marco.crivellari@suse.com, frederic@kernel.org, bigeasy@linutronix.de, Hillf Danton , Breno Leitao , kernel-team@meta.com, kmagar@redhat.com, psuriset@redhat.com, david.dai@linux.dev X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=1934; i=leitao@debian.org; h=from:subject:message-id; bh=eMx+uhD5dwhPEDNEahHUWX7BJo8BW0YF8/v/DOzwVI8=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBqPk0n2FvEcZP9QxlsjCj1SmgYd58CBgj4B3rgG 6/WuMqDo8SJAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCaj5NJwAKCRA1o5Of/Hh3 bdgmD/4zKVV9tZ4lsvaW11D3CfR5FC1Vgs94eIk+kUr8Ue2LSwjVAgTLFjHrngfBOWBj2xrF1Mb 5My91tiKlgWxAj6Rbvf44aOEtj9eSYG9Egsy4KMwl3HxsIwOuDXskFkBoUA5wrEYTOmN2uHgWmY FkpzVKgoGgfZ2B3NcLywg2FOBsvBtN9cnXLl/ZDN6OAmV2ZCEIgCQKKlzjPRU54CoDb20y/TcZ0 QevrcqFghG3ooFZ3Ca9cQXfKlCO+JwHgi3U0ixpZiKB9I5+4/oFbfxTH5zmRtAYS7hENG854q6o /LjF5EDy1B/fmpSYzkQ0SlIViI75El05SapAmzjkrZnq0kGWrcql3V4AwWg/nvFy/gEutk5SFF0 XRpaI97maqsM3ZGV5WPXjbzoec8Pe8GEHqqHulxIqQTilt0HnYvWQfWQLjVWJHopP8ceuyZBEQR jXuLFz5cdg/cQxlsQTCYOI2AChgNxNtvraxATzKK1urGDp24rrIoYpuzvf5FleWXGYxMf4Ego+Y 0s3YrHFEoIxM3xPR5O8M+T+I87MlWxuI+79MULvuI02OpV8sCJxqy1E+zKXPq+nz1g2EMxcDFNn +2Hq1GgJaZdBBVHZSjgdB01Lwsd2OqJyssnpy4y/cvvvV/X/0tcF5OYG0Y/JX9UMSgORI/qQ0Wk V2GD8tN1l1JnAyg== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao Use kick_pool_pick() to select and claim the worker under pool->lock and issue the wakeup with wake_up_process() after the lock is dropped. Unlike __queue_work(), this path has no surrounding RCU section, so take rcu_read_lock() before dropping pool->lock to keep the picked worker's task_struct valid across the wakeup. Signed-off-by: Breno Leitao Tested-by: Krishna Magar --- kernel/workqueue.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 594592768ef10..640590d270ce5 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -3255,6 +3255,7 @@ __acquires(&pool->lock) { struct pool_workqueue *pwq = get_work_pwq(work); struct worker_pool *pool = worker->pool; + struct task_struct *wake_task = NULL; unsigned long work_data; int lockdep_start_depth, rcu_start_depth; bool bh_draining = pool->flags & POOL_BH_DRAINING; @@ -3308,8 +3309,11 @@ __acquires(&pool->lock) * since nr_running would always be >= 1 at this point. This is used to * chain execution of the pending work items for WORKER_NOT_RUNNING * workers such as the UNBOUND and CPU_INTENSIVE ones. + * + * Select the worker under pool->lock; the wakeup is deferred until + * after the lock is dropped, guarded by the rcu_read_lock() below. */ - kick_pool(pool); + kick_pool_pick(pool, &wake_task); /* * Record the last pool and clear PENDING which should be the last @@ -3320,7 +3324,12 @@ __acquires(&pool->lock) set_work_pool_and_clear_pending(work, pool->id, pool_offq_flags(pool)); pwq->stats[PWQ_STAT_STARTED]++; + + rcu_read_lock(); raw_spin_unlock_irq(&pool->lock); + if (wake_task) + wake_up_process(wake_task); + rcu_read_unlock(); rcu_start_depth = rcu_preempt_depth(); lockdep_start_depth = lockdep_depth(current); -- 2.53.0-Meta