From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A7BBC76188 for ; Fri, 19 Jul 2019 04:27:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 354F820873 for ; Fri, 19 Jul 2019 04:27:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1563510445; bh=tyRPrOwKObReDHMdAgwbRdxfiTZLlY8WCnzQi9XBHBM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=ZYc4Ce2eg+IiR2rcN8QkoQdqePH9Tc5w3yWKYTS53/yVk9OrtK8RbhOIEem5MmEz+ f77mvrXUfwpC7bU0fQtOPJW7lDLGjl8hFw34cX24BLDEO01eGaOHE46PFMWwQAj80g I5vbyLCXdYQpNszIjFKGQzqFlKLcxEJlLfkqxdYg= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387482AbfGSE1Y (ORCPT ); Fri, 19 Jul 2019 00:27:24 -0400 Received: from mail.kernel.org ([198.145.29.99]:40434 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732445AbfGSEG6 (ORCPT ); Fri, 19 Jul 2019 00:06:58 -0400 Received: from sasha-vm.mshome.net (c-73-47-72-35.hsd1.nh.comcast.net [73.47.72.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id EB88721873; Fri, 19 Jul 2019 04:06:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1563509217; bh=tyRPrOwKObReDHMdAgwbRdxfiTZLlY8WCnzQi9XBHBM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UWyKZESYBXt4RUSvIhxCwUmx1BCFI1c3ZGbDVd/+CWN0SdqPZSIp3SrNNN00bJQfa MpFb9laELaoX+hX2mEoyYuP0IxQd71oDoHoNTTCFi83J/64CuE5LgGNRyLrikZfOSZ 96KqqjzHovzT6NheY2J/KVfQaYWW6sM6oNZfmzzU= From: Sasha Levin To: linux-kernel@vger.kernel.org, stable@vger.kernel.org Cc: Jean-Philippe Brucker , =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , Michal Hocko , Andrew Morton , Linus Torvalds , Sasha Levin , linux-mm@kvack.org Subject: [PATCH AUTOSEL 5.1 132/141] mm/mmu_notifier: use hlist_add_head_rcu() Date: Fri, 19 Jul 2019 00:02:37 -0400 Message-Id: <20190719040246.15945-132-sashal@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190719040246.15945-1-sashal@kernel.org> References: <20190719040246.15945-1-sashal@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-stable: review X-Patchwork-Hint: Ignore Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Jean-Philippe Brucker [ Upstream commit 543bdb2d825fe2400d6e951f1786d92139a16931 ] Make mmu_notifier_register() safer by issuing a memory barrier before registering a new notifier. This fixes a theoretical bug on weakly ordered CPUs. For example, take this simplified use of notifiers by a driver: my_struct->mn.ops = &my_ops; /* (1) */ mmu_notifier_register(&my_struct->mn, mm) ... hlist_add_head(&mn->hlist, &mm->mmu_notifiers); /* (2) */ ... Once mmu_notifier_register() releases the mm locks, another thread can invalidate a range: mmu_notifier_invalidate_range() ... hlist_for_each_entry_rcu(mn, &mm->mmu_notifiers, hlist) { if (mn->ops->invalidate_range) The read side relies on the data dependency between mn and ops to ensure that the pointer is properly initialized. But the write side doesn't have any dependency between (1) and (2), so they could be reordered and the readers could dereference an invalid mn->ops. mmu_notifier_register() does take all the mm locks before adding to the hlist, but those have acquire semantics which isn't sufficient. By calling hlist_add_head_rcu() instead of hlist_add_head() we update the hlist using a store-release, ensuring that readers see prior initialization of my_struct. This situation is better illustated by litmus test MP+onceassign+derefonce. Link: http://lkml.kernel.org/r/20190502133532.24981-1-jean-philippe.brucker@arm.com Fixes: cddb8a5c14aa ("mmu-notifiers: core") Signed-off-by: Jean-Philippe Brucker Cc: Jérôme Glisse Cc: Michal Hocko Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Sasha Levin --- mm/mmu_notifier.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index 9c884abc7850..9f246c960e65 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -276,7 +276,7 @@ static int do_mmu_notifier_register(struct mmu_notifier *mn, * thanks to mm_take_all_locks(). */ spin_lock(&mm->mmu_notifier_mm->lock); - hlist_add_head(&mn->hlist, &mm->mmu_notifier_mm->list); + hlist_add_head_rcu(&mn->hlist, &mm->mmu_notifier_mm->list); spin_unlock(&mm->mmu_notifier_mm->lock); mm_drop_all_locks(mm); -- 2.20.1