From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE744C433FE for ; Sat, 22 Oct 2022 11:48:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 69F458E0001; Sat, 22 Oct 2022 07:48:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 39DA98E000B; Sat, 22 Oct 2022 07:48:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 154F08E0008; Sat, 22 Oct 2022 07:48:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id D0FF18E000B for ; Sat, 22 Oct 2022 07:48:36 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id AEB28140791 for ; Sat, 22 Oct 2022 11:48:35 +0000 (UTC) X-FDA: 80048412990.13.466FD31 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) by imf03.hostedemail.com (Postfix) with ESMTP id 595E22001C for ; Sat, 22 Oct 2022 11:48:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=ZxLGyKaonAFGosmEJEXCJIuUc/p2pYnEnlwCFC//QMs=; b=d5RZJQiWmMamzGxaIKQnEOtUwD T9H/sWSomE2By/o8fhYFxcJnvZOZCOoyYsj4DPqrDTAEiVXHR6xyiOPOuV3LpQeQUbDuoAPcmVKGs DJSaXerbzdFXHI/CeMHGBW7cMxLNnTrKYaXjJZ3DmnuM+KA59Jo1ENMyp0WIE4NfWdcSJlN6hSA3v I0C+dfhtEKU3yRXt+wguStxjqcWSysKILzObipzuaBHkU3xB0q+bVaL3s37Udbo8ewcAHquPYv+98 QYAVXxD+sKkG/ZpZypq3uRMpr18Br1YlM8YBXYyqaiaLhGv+xeybW3Binc8LI7NIXuDe907gPs0vd qMBpHPRw==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1omCzM-005XdG-1V; Sat, 22 Oct 2022 11:48:28 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id BDB6F302D82; Sat, 22 Oct 2022 13:48:26 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 33F9C28B8E514; Sat, 22 Oct 2022 13:48:26 +0200 (CEST) Message-ID: <20221022114424.906110403@infradead.org> User-Agent: quilt/0.66 Date: Sat, 22 Oct 2022 13:14:10 +0200 From: Peter Zijlstra To: x86@kernel.org, willy@infradead.org, torvalds@linux-foundation.org, akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, linux-mm@kvack.org, aarcange@redhat.com, kirill.shutemov@linux.intel.com, jroedel@suse.de, ubizjak@gmail.com Subject: [PATCH 07/13] mm/gup: Fix the lockless PMD access References: <20221022111403.531902164@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666439315; a=rsa-sha256; cv=none; b=aNhdWmCdfYIb3/k0+J7QjGj2ZRhI/5ssIlOAqT6BQguch7iNK/F1TdIw5TCQ05B/AZnW4j WdIPwWS0rFduNaqlaCM5C23cyAgpJIY6jGp3hKVfYI1+1NNqfrTHvUoQe1EfyrGnIR6GMY yPsQ6ERPdG0z+dMxbzmwuKpNFcx18k8= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=d5RZJQiW; spf=none (imf03.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666439315; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:references:dkim-signature; bh=ZxLGyKaonAFGosmEJEXCJIuUc/p2pYnEnlwCFC//QMs=; b=7W5AM1QWokhltqV0tBVRPzqMrIaiA+LQ2aE8qo5LmEYqd5eoHH/aEfNIr58E2cQ1eikz20 jxNHyKoqxe47LpxUpCkdE2M75kD5zY3r1xiG+WoQYwMkDyeRYpyELzWiiJlQ4NASfSPz8q okEPEjDkqSFLnz1PTh/kOZqfI3N4tFE= X-Stat-Signature: arngc5t14decdrsbfx8p9cqzxmontxzs X-Rspamd-Queue-Id: 595E22001C X-Rspam-User: X-Rspamd-Server: rspam03 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=d5RZJQiW; spf=none (imf03.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org; dmarc=none X-HE-Tag: 1666439314-82807 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On architectures where the PTE/PMD is larger than the native word size (i386-PAE for example), READ_ONCE() can do the wrong thing. Use pmdp_get_lockless() just like we use ptep_get_lockless(). Signed-off-by: Peter Zijlstra (Intel) --- kernel/events/core.c | 2 +- mm/gup.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7186,7 +7186,7 @@ static u64 perf_get_pgtable_size(struct return pud_leaf_size(pud); pmdp = pmd_offset_lockless(pudp, pud, addr); - pmd = READ_ONCE(*pmdp); + pmd = pmdp_get_lockless(pmdp); if (!pmd_present(pmd)) return 0; --- a/mm/gup.c +++ b/mm/gup.c @@ -2507,7 +2507,7 @@ static int gup_pmd_range(pud_t *pudp, pu pmdp = pmd_offset_lockless(pudp, pud, addr); do { - pmd_t pmd = READ_ONCE(*pmdp); + pmd_t pmd = pmdp_get_lockless(pmdp); next = pmd_addr_end(addr, end); if (!pmd_present(pmd))