From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 20DAFEB64DC for ; Tue, 11 Jul 2023 23:39:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: Mime-Version:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=G1rDh34C0iTCFUsGUjKcF/IOmhf2PnhkTaNNmTg8vRI=; b=Xdb R5E4j/TbGKUJDOuDP6dsB8yUG+tNn6vooRTPEXDyIv7Of2JK+kATPjXVPeZsn1dIAp28ASXbAYCv5 M3Duqe8sOoO9HS0UxSe5EB2oYNiYG/G0KSTvxJcp0vUoEVt5drmkXIfHPOEa5EeBvLHQdcqJj86Ay ikSVsRQmY4qJ0hbbzG69e/kHRNbDRlETf+1fXLoVuIEvQL/2aNVWwPgSQzDDMaJo5LoX65FLUNH/A Pyka0wSWm9cGtexGzIJHW0ybPMH52MpB3jtQbXl4sgCMtD6DL0KpQOcOP+neqmGiabO0c1dh7TmHJ MKMIDCRrIKhZJ65AQxaj1nvCkCJ128Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qJMwH-00G4QC-0c; Tue, 11 Jul 2023 23:38:37 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qJMwD-00G4PM-1z for linux-arm-kernel@lists.infradead.org; Tue, 11 Jul 2023 23:38:35 +0000 Received: by mail-pg1-x54a.google.com with SMTP id 41be03b00d2f7-53fa00ed93dso8737951a12.3 for ; Tue, 11 Jul 2023 16:38:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689118711; x=1691710711; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=VlZMY7B4OQtHfVwCl2fGa87UX0dm/HNDQvgsMerbs4E=; b=6Ki0rULefzkrV0uqZTDhEme49t7KY8Ddvh964y9JGpH0d7zaaMcI88h9tK+e2Ab0fT veY/0elFR+0TnQbsPGLKw4Jmp/EnYeFI/PSjM63/7WezRDGYWEvCUsGnyE0hYonrxZCh hKVd2xrVwmNoXCz5hL6oiCirMv1oj6iXffQR50NxR8dIj89owViyD6Cq1q+dG0hECTvQ Gn9H85LU56bBM8WIwaFDgjeirp8M6EJB2ZueqDx7tDCWYbtZHHg6+WL/fE6cziK/CLIM 9MHIOD6Pg4vMMojX+pEwY1B5ucjdNhQVR3rlKMYkiAAdKcNXGpYWjVQ1oLZ+HiqDtv/m dEsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689118711; x=1691710711; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=VlZMY7B4OQtHfVwCl2fGa87UX0dm/HNDQvgsMerbs4E=; b=fv7BxGEcL7lE4kfrcqkFQJ5Fe4GCeib57cLRDrE4N8Wyu8uq4nCpsatSEaLSR0AfFU N8Y0ElF69odB5G5H6dY+2HWLUIm2RSw26HoZpD07e/dJCp18Xrswh0zWsbArRufY7NOl HGshcY73V/xdkdrMrdplV/63ii64Iqw02H+qZhp8CbfD8aPjWHvtq9liSKTPAJBvnKeT qlp+U+43IbiXkUqYtN3LaZDwgDdOyZtvrKV7NUlWCZ+2OsBLjhg40wfKjloY8iKQ+F8Z bBhW0l16iCK1t6sxh/OyNfHVbg9oPK1wd1IJd+PXsSEq3ugt27AZZkmrBjA5EUtHokbV Yocg== X-Gm-Message-State: ABy/qLb6a2SlaF9hEoUs5Nqr0zWKSxuiuLwTxLYIpvzUUoU5VIyop0Ik XPuRPFZSOIwkz16vn+v7P82nmDnlVI3n X-Google-Smtp-Source: APBJJlEh1KFYgX73qa7oSh36gSCXi/QGqv/5l19pZiBs2u+PQkzFrZtGOa0EdwtmAe2WF+om3uaJkkHyZ7t9 X-Received: from dhavale-ctop.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:5e39]) (user=dhavale job=sendgmr) by 2002:a63:9511:0:b0:55c:2f9f:e427 with SMTP id p17-20020a639511000000b0055c2f9fe427mr4581555pgd.5.1689118711049; Tue, 11 Jul 2023 16:38:31 -0700 (PDT) Date: Tue, 11 Jul 2023 16:38:15 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230711233816.2187577-1-dhavale@google.com> Subject: [PATCH v1] rcu: Fix and improve RCU read lock checks when !CONFIG_DEBUG_LOCK_ALLOC From: Sandeep Dhavale To: "Paul E. McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Matthias Brugger , AngeloGioacchino Del Regno Cc: linux-erofs@lists.ozlabs.org, xiang@kernel.org, Sandeep Dhavale , Will Shiu , kernel-team@android.com, rcu@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mediatek@lists.infradead.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230711_163833_670685_ECECD6CD X-CRM114-Status: GOOD ( 18.38 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently if CONFIG_DEBUG_LOCK_ALLOC is not set - rcu_read_lock_held() always returns 1 - rcu_read_lock_any_held() may return 0 with CONFIG_PREEMPT_RCU This is inconsistent and it was discovered when trying a fix for problem reported [1] with CONFIG_DEBUG_LOCK_ALLOC is not set and CONFIG_PREEMPT_RCU is enabled. Gist of the problem is that EROFS wants to detect atomic context so it can do inline decompression whenever possible, this is important performance optimization. It turns out that z_erofs_decompressqueue_endio() can be called from blk_mq_flush_plug_list() with rcu lock held and hence fix uses rcu_read_lock_any_held() to decide to use sync/inline decompression vs async decompression. As per documentation, we should return lock is held if we aren't certain. But it seems we can improve the checks for if the lock is held even if CONFIG_DEBUG_LOCK_ALLOC is not set instead of hardcoding to always return true. * rcu_read_lock_held() - For CONFIG_PREEMPT_RCU using rcu_preempt_depth() - using preemptible() (indirectly preempt_count()) * rcu_read_lock_bh_held() - For CONFIG_PREEMPT_RT Using in_softirq() (indirectly softirq_cont()) - using preemptible() (indirectly preempt_count()) Lastly to fix the inconsistency, rcu_read_lock_any_held() is updated to use other rcu_read_lock_*_held() checks. Two of the improved checks are moved to kernel/rcu/update.c because rcupdate.h is included from the very low level headers which do not know what current (task_struct) is so the macro rcu_preempt_depth() cannot be expanded in the rcupdate.h. See the original comment for rcu_preempt_depth() in patch at [2] for more information. [1] https://lore.kernel.org/all/20230621220848.3379029-1-dhavale@google.com/ [2] https://lore.kernel.org/all/1281392111-25060-8-git-send-email-paulmck@linux.vnet.ibm.com/ Reported-by: Will Shiu Signed-off-by: Sandeep Dhavale --- include/linux/rcupdate.h | 12 +++--------- kernel/rcu/update.c | 21 ++++++++++++++++++++- 2 files changed, 23 insertions(+), 10 deletions(-) diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index 5e5f920ade90..0d1d1d8c2360 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -319,14 +319,11 @@ int rcu_read_lock_any_held(void); # define rcu_lock_acquire(a) do { } while (0) # define rcu_lock_release(a) do { } while (0) -static inline int rcu_read_lock_held(void) -{ - return 1; -} +int rcu_read_lock_held(void); static inline int rcu_read_lock_bh_held(void) { - return 1; + return !preemptible() || in_softirq(); } static inline int rcu_read_lock_sched_held(void) @@ -334,10 +331,7 @@ static inline int rcu_read_lock_sched_held(void) return !preemptible(); } -static inline int rcu_read_lock_any_held(void) -{ - return !preemptible(); -} +int rcu_read_lock_any_held(void); static inline int debug_lockdep_rcu_enabled(void) { diff --git a/kernel/rcu/update.c b/kernel/rcu/update.c index 19bf6fa3ee6a..b34fc5bb96cf 100644 --- a/kernel/rcu/update.c +++ b/kernel/rcu/update.c @@ -390,8 +390,27 @@ int rcu_read_lock_any_held(void) } EXPORT_SYMBOL_GPL(rcu_read_lock_any_held); -#endif /* #ifdef CONFIG_DEBUG_LOCK_ALLOC */ +#else /* #ifdef CONFIG_DEBUG_LOCK_ALLOC */ +int rcu_read_lock_held(void) +{ + if (IS_ENABLED(CONFIG_PREEMPT_RCU)) + return rcu_preempt_depth(); + return !preemptible(); +} +EXPORT_SYMBOL_GPL(rcu_read_lock_held); + +int rcu_read_lock_any_held(void) +{ + if (rcu_read_lock_held() || + rcu_read_lock_bh_held() || + rcu_read_lock_sched_held()) + return 1; + return !preemptible(); +} +EXPORT_SYMBOL_GPL(rcu_read_lock_any_held); + +#endif /* #else #ifdef CONFIG_DEBUG_LOCK_ALLOC */ /** * wakeme_after_rcu() - Callback function to awaken a task after grace period * @head: Pointer to rcu_head member within rcu_synchronize structure -- 2.41.0.255.g8b1d071c50-goog _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel