From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D19C1C433EF for ; Wed, 11 May 2022 02:29:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8E61E8D0005; Tue, 10 May 2022 22:29:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 83DC38D0008; Tue, 10 May 2022 22:29:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 41A9D8D0005; Tue, 10 May 2022 22:29:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id E983E8D0002 for ; Tue, 10 May 2022 22:29:45 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id C4AF321E53 for ; Wed, 11 May 2022 02:29:45 +0000 (UTC) X-FDA: 79451881530.23.0FEC080 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by imf01.hostedemail.com (Postfix) with ESMTP id 588F040003 for ; Wed, 11 May 2022 02:29:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652236185; x=1683772185; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/zG/vU7kvADc4pg2W2FY9O2Nfewu2j+VHOpdujenCaM=; b=UKYgYJxUA9fqGDMj1ovh/I1m14zv5lP9BipMEuTq377VQGBF1mfngEgP VQoFT3ewGpIN4Tb3huTrcZ8X31b7uj0OTDGVxJxlHx11lF2lpEm4cE0Fl qXHjygJfnvVvlqP4Z/35RDj2W4mF5nAA6HHKEH5Qj0K4tA4NislvuT5n5 NvmYe9Z1bYMB0L2Bwi9M3gmfLjk8CI/WbeoWgK0cBrRiJ/WEgEonpw0nU 17ec7j/fzVE7h2dNwV0nKWsnV+lu124N5E+74YcbvziT4bpIcTaI+CEaX YVgBjKQbHWD+aWOzLyZ5mlAADAABtKjXKW5+MmS/w6DiTm38FG10UhYzp g==; X-IronPort-AV: E=McAfee;i="6400,9594,10343"; a="269695078" X-IronPort-AV: E=Sophos;i="5.91,215,1647327600"; d="scan'208";a="269695078" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 May 2022 19:29:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,215,1647327600"; d="scan'208";a="670166554" Received: from black.fi.intel.com ([10.237.72.28]) by fmsmga002.fm.intel.com with ESMTP; 10 May 2022 19:29:40 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 53BBA5D0; Wed, 11 May 2022 05:28:01 +0300 (EEST) From: "Kirill A. Shutemov" To: Dave Hansen , Andy Lutomirski , Peter Zijlstra Cc: x86@kernel.org, Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , "H . J . Lu" , Andi Kleen , Rick Edgecombe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [RFCv2 07/10] x86/mm: Handle tagged memory accesses from kernel threads Date: Wed, 11 May 2022 05:27:48 +0300 Message-Id: <20220511022751.65540-9-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220511022751.65540-1-kirill.shutemov@linux.intel.com> References: <20220511022751.65540-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 588F040003 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=UKYgYJxU; spf=none (imf01.hostedemail.com: domain of kirill.shutemov@linux.intel.com has no SPF policy when checking 192.55.52.115) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=pass (policy=none) header.from=intel.com X-Rspam-User: X-Stat-Signature: zpt5x4bnmkekahq5ogb4upbyn8hz8tof X-HE-Tag: 1652236173-869155 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When a kernel thread performs memory access on behalf of a process (like in async I/O, io_uring, etc.) it has to respect tagging setup of the process as user addresses can include tags. Normally, LAM setup is per-thread and recorded in thread features, but for this use case kernel also tracks LAM setup per-mm. mm->context.lam would record LAM that allows the most tag bits among the threads of the mm. The info used by switch_mm_irqs_off() to construct CR3 if the task is kernel thread. Thread featrues of the kernel thread get updated according to mm->context.lam. It allows untagged_addr() to work correctly. Signed-off-by: Kirill A. Shutemov --- arch/x86/include/asm/mmu.h | 1 + arch/x86/mm/tlb.c | 28 ++++++++++++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/arch/x86/include/asm/mmu.h b/arch/x86/include/asm/mmu.h index 5d7494631ea9..52f3749f14e8 100644 --- a/arch/x86/include/asm/mmu.h +++ b/arch/x86/include/asm/mmu.h @@ -40,6 +40,7 @@ typedef struct { #ifdef CONFIG_X86_64 unsigned short flags; + u8 lam; #endif struct mutex lock; diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index f9fe71d1f42c..b320556e1c22 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -185,6 +185,34 @@ static u8 gen_lam(struct task_struct *tsk, struct mm_struct *mm) if (!tsk) return LAM_NONE; + if (tsk->flags & PF_KTHREAD) { + /* + * For kernel thread use the most permissive LAM + * used by the mm. It's required to handle kernel thread + * memory accesses on behalf of a process. + * + * Adjust thread flags accodringly, so untagged_addr() would + * work correctly. + */ + + tsk->thread.features &= ~(X86_THREAD_LAM_U48 | + X86_THREAD_LAM_U57); + + switch (mm->context.lam) { + case LAM_NONE: + return LAM_NONE; + case LAM_U57: + tsk->thread.features |= X86_THREAD_LAM_U57; + return LAM_U57; + case LAM_U48: + tsk->thread.features |= X86_THREAD_LAM_U48; + return LAM_U48; + default: + WARN_ON_ONCE(1); + return LAM_NONE; + } + } + if (tsk->thread.features & X86_THREAD_LAM_U57) return LAM_U57; if (tsk->thread.features & X86_THREAD_LAM_U48) -- 2.35.1