From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from shelob.surriel.com (shelob.surriel.com [96.67.55.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8110A470EA1 for ; Fri, 24 Jul 2026 22:30:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=96.67.55.147 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784932216; cv=none; b=VX0kbcaGGpfgIfxrzmBoFWGwdwW2ofmjgMYzmc2kTCkrSybQCCiLvm5MVD08rhHIWC9P7JhTYsK8HaFNWwQRigTiguYPyhSaOCnSonymuj9yPd9Vij/wm/5dM/yGix7IIIRBVRPHY7kOmQDZbe35UjD8CwQVHeeXqGPcSw7auhs= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784932216; c=relaxed/simple; bh=KDF8zmn8nnQ5o+HE9p0R2eQN+l5v8f/djAorIliGAN4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ses1pDHzhl0CeEBg2hpvrBk4NV/sS0fpJ4ubBia8Gv736PFEoVZiw65tuYiDJFTlgLO9T3sPkkcacbb2tn0kFbWoAHsdWwl4FNNb+a1gO5xacO7eCgb5e2xA9PsldHwOGzfgHSgfXkoZ8Kg+R4iAcvVpTKi0IvdVz5AlC/phug0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=surriel.com; spf=pass smtp.mailfrom=surriel.com; dkim=pass (2048-bit key) header.d=surriel.com header.i=@surriel.com header.b=KaYyL4Mb; arc=none smtp.client-ip=96.67.55.147 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=surriel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=surriel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=surriel.com header.i=@surriel.com header.b="KaYyL4Mb" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=surriel.com ; s=mail; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=C7NU1MfovW+7lx1xP2l6wfJPEmWK+PLGNyfdih7cuN8=; b=KaYyL4MbCf5uM4PBwpzJonMKl9 cyXNE0D+OJUbxfv34AEjHgCSvXMJ8SgkagJY/CwSHuKatkwmtyO+J0MZYTRJuRK35fyAfLK4C5loJ bgEanDnDhIa18xkaNlqkFxBqRPp1XE7YbGrjwSbNQ2RJafZt9rjXhfmE6t3q/CBf0yUcnf8kaLj4v FSxcISpKkWhv3uZi5KgsIhYZQHey7sxnKewn2AG300IBLTrCuw0WfcbGYJKGlCbGqqmZOfQ4tIH/6 aZL4Q/572xDEJjOBhMPboLFBSluS4UpJxEBa+UjlzkqKyazeVe7cVtI3JTaWXgCcIJtnRzKMvWX4q z6mERBQg==; Received: from fangorn.home.surriel.com ([10.0.13.7]) by shelob.surriel.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.97.1) (envelope-from ) id 1wnOOs-0000000027h-0xfn; Fri, 24 Jul 2026 18:29:50 -0400 From: Rik van Riel To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel-team@meta.com, Dave Hansen , Peter Zijlstra , Suren Baghdasaryan , Lorenzo Stoakes , Vlastimil Babka , David Hildenbrand , "Liam R. Howlett" , Mike Rapoport , Michal Hocko , Jason Gunthorpe , John Hubbard , Peter Xu , Matthew Wilcox , Usama Arif , Rik van Riel , Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, "H. Peter Anvin" , Usama Arif Subject: [PATCH RFC v4 01/12] x86/mm: add untagged_addr_remote_unlocked() Date: Fri, 24 Jul 2026 18:29:23 -0400 Message-ID: <20260724222934.1463812-2-riel@surriel.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260724222934.1463812-1-riel@surriel.com> References: <20260724222934.1463812-1-riel@surriel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit __access_remote_vm() reads another process's memory under the mmap lock. On large machines running big multi-threaded applications, that lock is contended between readers and writers: an mmap() or munmap() in one thread stalls readers like /proc/PID/cmdline, /proc/PID/environ, or process_vm_readv(), even though the memory they read is almost always resident and reachable under the per-VMA lock instead. Looking up the VMA first requires untagging the address. untagged_addr_remote() asserts the mmap lock only because it reads mm->context.untag_mask, which can race with the write in mm_enable_lam(). That mask changes only while the target is single-threaded, when it enables LAM or resets it at exec. The read need not be stable: it is already read locklessly elsewhere, and a remote untag is best-effort. Add untagged_addr_remote_unlocked() for callers that have not taken the mmap lock, and annotate access to mm->context.untag_mask with READ_ONCE() and WRITE_ONCE() so the existing lockless reads are explicit and KCSAN-clean. untagged_addr_remote() keeps its assertion and shares the same code. Assisted-by: Claude:claude-opus-4.8 Acked-by: Usama Arif Signed-off-by: Rik van Riel --- arch/x86/include/asm/mmu_context.h | 6 +++--- arch/x86/include/asm/uaccess_64.h | 15 ++++++++++++--- arch/x86/kernel/process_64.c | 4 ++-- include/linux/uaccess.h | 7 +++++++ 4 files changed, 24 insertions(+), 8 deletions(-) diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h index ef5b507de34e..cee710f64658 100644 --- a/arch/x86/include/asm/mmu_context.h +++ b/arch/x86/include/asm/mmu_context.h @@ -100,18 +100,18 @@ static inline unsigned long mm_lam_cr3_mask(struct mm_struct *mm) static inline void dup_lam(struct mm_struct *oldmm, struct mm_struct *mm) { mm->context.lam_cr3_mask = oldmm->context.lam_cr3_mask; - mm->context.untag_mask = oldmm->context.untag_mask; + WRITE_ONCE(mm->context.untag_mask, READ_ONCE(oldmm->context.untag_mask)); } #define mm_untag_mask mm_untag_mask static inline unsigned long mm_untag_mask(struct mm_struct *mm) { - return mm->context.untag_mask; + return READ_ONCE(mm->context.untag_mask); } static inline void mm_reset_untag_mask(struct mm_struct *mm) { - mm->context.untag_mask = -1UL; + WRITE_ONCE(mm->context.untag_mask, -1UL); } #define arch_pgtable_dma_compat arch_pgtable_dma_compat diff --git a/arch/x86/include/asm/uaccess_64.h b/arch/x86/include/asm/uaccess_64.h index 20de34cc9aa6..a095165eb648 100644 --- a/arch/x86/include/asm/uaccess_64.h +++ b/arch/x86/include/asm/uaccess_64.h @@ -39,18 +39,27 @@ static inline unsigned long __untagged_addr(unsigned long addr) (__force __typeof__(addr))__untagged_addr(__addr); \ }) +/* + * mm->context.untag_mask changes only when the target enables LAM or execs, + * always single-threaded. An unlocked remote reader can race that; READ_ONCE + * keeps the value whole and the untag is best-effort. + */ static inline unsigned long __untagged_addr_remote(struct mm_struct *mm, unsigned long addr) { - mmap_assert_locked(mm); - return addr & (mm)->context.untag_mask; + return addr & READ_ONCE(mm->context.untag_mask); } -#define untagged_addr_remote(mm, addr) ({ \ +#define untagged_addr_remote_unlocked(mm, addr) ({ \ unsigned long __addr = (__force unsigned long)(addr); \ (__force __typeof__(addr))__untagged_addr_remote(mm, __addr); \ }) +#define untagged_addr_remote(mm, addr) ({ \ + mmap_assert_locked(mm); \ + untagged_addr_remote_unlocked(mm, addr); \ +}) + #endif #define valid_user_address(x) \ diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index d44afbe005bb..9fa659117f38 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -814,7 +814,7 @@ static void enable_lam_func(void *__mm) static void mm_enable_lam(struct mm_struct *mm) { mm->context.lam_cr3_mask = X86_CR3_LAM_U57; - mm->context.untag_mask = ~GENMASK(62, 57); + WRITE_ONCE(mm->context.untag_mask, ~GENMASK(62, 57)); /* * Even though the process must still be single-threaded at this @@ -952,7 +952,7 @@ long do_arch_prctl_64(struct task_struct *task, int option, unsigned long arg2) #endif #ifdef CONFIG_ADDRESS_MASKING case ARCH_GET_UNTAG_MASK: - return put_user(task->mm->context.untag_mask, + return put_user(mm_untag_mask(task->mm), (unsigned long __user *)arg2); case ARCH_ENABLE_TAGGED_ADDR: return prctl_enable_tagged_addr(task->mm, arg2); diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h index eddbbb65ccc4..7e6e4c89184c 100644 --- a/include/linux/uaccess.h +++ b/include/linux/uaccess.h @@ -34,6 +34,13 @@ }) #endif +#ifndef untagged_addr_remote_unlocked +#define untagged_addr_remote_unlocked(mm, addr) ({ \ + (void)(mm); \ + untagged_addr(addr); \ +}) +#endif + #ifdef masked_user_access_begin #define can_do_masked_user_access() 1 # ifndef masked_user_write_access_begin base-commit: 248951ddc14de84de3910f9b13f51491a8cd91df -- 2.53.0-Meta