From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03B83C54798 for ; Thu, 7 Mar 2024 21:08:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 885156B02B9; Thu, 7 Mar 2024 16:08:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 835606B02BD; Thu, 7 Mar 2024 16:08:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6FCF56B02BE; Thu, 7 Mar 2024 16:08:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 606A86B02B9 for ; Thu, 7 Mar 2024 16:08:42 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 314721205EC for ; Thu, 7 Mar 2024 21:08:42 +0000 (UTC) X-FDA: 81871482084.17.9A60CEF Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf17.hostedemail.com (Postfix) with ESMTP id 22A5C40020 for ; Thu, 7 Mar 2024 21:08:39 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=HtwEP3OT; spf=pass (imf17.hostedemail.com: domain of 31yzqZQoKCBAE487Eqx2utw44w1u.s421y3AD-220Bqs0.47w@flex--yosryahmed.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=31yzqZQoKCBAE487Eqx2utw44w1u.s421y3AD-220Bqs0.47w@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709845720; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6XyYC8At6iwQoZ2KZhBPWiG6rvVOkTpXpe7DpTJpPr4=; b=4QB+m9NdZAvpq8cV1u8QPlzqrzbpleXQ83SmzpsXz+uFLBBasoMnfE4r5jX/2vpbfqZhZg YQdFrXjQtel2b3DpdctjtbtmaA6vj23rg45kxq7yjRers1Yo4YhqSs8Zc+zxFcrcG4P78X V8Yqn4AbFXgnNNdPKuG54s/ypcp7hHg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709845720; a=rsa-sha256; cv=none; b=6QH1H0PNHPEunrZtE4Lh8ho5Ao5bEK3h6li7SKlddTrbYjIwcttBpzgjXD62yunRhzN6T9 CjiocW+6H52CCmhrZby4QZz1mabLrZQhQPYBUmFyEgLrD7OsGbsGVBuu2x7J7RPVYvh6KQ gvqVjUCdnnS9Kkygb4XyB8KwBfm9VEA= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=HtwEP3OT; spf=pass (imf17.hostedemail.com: domain of 31yzqZQoKCBAE487Eqx2utw44w1u.s421y3AD-220Bqs0.47w@flex--yosryahmed.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=31yzqZQoKCBAE487Eqx2utw44w1u.s421y3AD-220Bqs0.47w@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-dbf216080f5so387356276.1 for ; Thu, 07 Mar 2024 13:08:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709845719; x=1710450519; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=6XyYC8At6iwQoZ2KZhBPWiG6rvVOkTpXpe7DpTJpPr4=; b=HtwEP3OThybxU5Cf8cZA8wFcDeHUIASCRXkVvyYuPzTmyIcam29TvgVNzh0Fe8ZChR i+Y662c2Jxf3qjtC/8DXrQexm4AGX9sslrNnm7CxvdSa6AnCfT0pHurg9j4yZvX0/cFE s0A+5swiSEegG5Gqb5X1tactv7dlrnqiW8MIca9TEbqyzD/J2YXYt9xbqvsaCLFP9WT7 N7KyfEpNgGrLgY1LlIpXDo3BrDmF6DJDRpotF9vSi+ZarUdOlsUCIlQZHbxZM0XnFm7d xp/XC8738QMp8v5YyfDrCvELvqWEhkyD4IerxplV6S/BQhhSHlD9S5Wgz98Nx0BjPO3y wpKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709845719; x=1710450519; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=6XyYC8At6iwQoZ2KZhBPWiG6rvVOkTpXpe7DpTJpPr4=; b=lQK4y+dwotv6ceKIu9b3bFFmdvIHVL1DZoY2fqotFwoSvFdkbpDvn9rA8THl05h1QO +l74P6EQ17VLHrJcYFTwPrbdaPEVXMUyvFDYfFcrP6nO08AR/+5JdW6eyAdhiiWdbhWo HPK8yqKFNJohgGsr93ba4datdMbzwzrqGVuzIkeq0oa33sCHcQgGeky8fOKNf6Id+mWK tLDSBFZmd6BZrTKLtg2yYP4/sNZVNeQ0MPnBN8eZfqsLJzWq6o/8i+o1trLM8lsmGylA M178xlU8vEbL/IftGWBTKnA1kJMa+xbzSkpCK+QkVeiHdvoI9I8hfCaGjtkU1ShIvMrK nctg== X-Forwarded-Encrypted: i=1; AJvYcCWUPZdOyJKy/IaRlcGmCTvJ1zh8sxxqkDTVICsYJ6pVW1EoNnI0682TfrVfpH8Hf57hU2pHS3OXLRxKEftZgnYMSYo= X-Gm-Message-State: AOJu0YwtNdAAzt28VibD7mzOj04vbp36qvTNq6F1QiTJNBoO4J1jm3ZX h36s34csTHpbfm99VBWN/QA4bF+PeyRSleMD0H/7LhSQvBPNzK8j3QGajPpbq1Ja5wxW1LbgaOy jkjopE6JlEMlCAcPVhw== X-Google-Smtp-Source: AGHT+IGEbtocHNRLjCCD7YOLDaguhaYk3Rz2sIBTQC6DEUXQQHBbgH/L4++AntHYM2n+UBcTHJ2Q0TjAK8fSrzN0 X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:29b4]) (user=yosryahmed job=sendgmr) by 2002:a25:260f:0:b0:dcc:f01f:65e1 with SMTP id m15-20020a25260f000000b00dccf01f65e1mr4901464ybm.8.1709845719696; Thu, 07 Mar 2024 13:08:39 -0800 (PST) Date: Thu, 7 Mar 2024 21:08:37 +0000 In-Reply-To: <034466a4-0917-47c4-934b-9549c3076624@intel.com> Mime-Version: 1.0 References: <20240307133916.3782068-1-yosryahmed@google.com> <20240307133916.3782068-3-yosryahmed@google.com> <034466a4-0917-47c4-934b-9549c3076624@intel.com> Message-ID: Subject: Re: [RFC PATCH 2/3] x86/mm: make sure LAM is up-to-date during context switching From: Yosry Ahmed To: Dave Hansen Cc: "Kirill A. Shutemov" , Andrew Morton , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Peter Zijlstra , Andy Lutomirski , x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="us-ascii" X-Rspamd-Queue-Id: 22A5C40020 X-Rspam-User: X-Stat-Signature: kroh5dhwmwccwqagof8bs9fb949gfkz1 X-Rspamd-Server: rspam03 X-HE-Tag: 1709845719-199140 X-HE-Meta: U2FsdGVkX1+eTVDVCbtN5BeCuuzV9zEiEiSLR1mcYnWhcCP/duaL8qNpIxSA8l+5X7Lxiq0OD76szYNThqQ4IKv59cn2gTyrTfxEVos1dk4jazhE3kVNlWEsel6cjOeiuD20xq8/2Lps71d3yPMbCW0NNTCfCx6oIRY8dXfX1XvQ639fnnFtDjvc2/khRGBh5EBYAEoiJJNN1Y9+teBf8n38OTOvDpI5HYEZOxdUL6IhPapmtHC3QGYmmBqzR4pZPfJJv6jXwKp9kjnbuhKxJOFrRV0SbpYddOoviJIgx0Ob0ka0MM5v/DiGvtZS9epjzttWX/L3ZNQJXNszfRVBdp1YFi/biULYwERI/P359xaLzPhX7sSP2Naz9NUGEM5k/wdYG8w2hBPQ5Dosn0ZFtmX0UlmSiLhGml/7bkqFKXmSMLuAP6+2HmeJ6Ui0GWhQ9pLT0tGh2BaLhC7UwdmaXBmuc0r1LZKZPfSN9Ox0OtSrIqO5R2O19DhS704DSh3lfpR+Nz1kT02jVep0CrncxPQ/6JfOz94cQRiQwKM9qRMwhz/+SDH4c3tMDhGYCPnCRTY495qRpXPeug9iOZOs1Fd8/uJXUrYs8RY1aZ9RN4KQ/Pwq186V38pMNv5RF8JgSk+6ZnoPT5T3iL/a864TTpG3sOyK3y/5eeUr1q2lPoo3HdHk/sdtQb3d2J7CuNFGN/xjHVxHMvbxJgexem+empEwK7QwlZOlkp5dhquf+BC/GDVAw57Y7fKmrYSdgcENDweD2LZsr7tKey7yXuL0R9I6h8TpL5I41YYURuasIbCrpSsGPjpH0m7Npzbxz5DAGGFx8zIU1Ne5lGCKuCmev8ORWgvPGoQQPH5FidpSz7zS+aKUzEIMSQ0OBx0goRVQ0YBWuu2lnTmTrrZR+9VEEPfeSWlEAMHBJT6g7AcIU90fTFAmsByOvORF3kfg+5BN+4wt6pUSu0l8+riEcLq VwwaoLZR xlrUNM2uWcTHwezmb7cRaUhhfKSmM9zzwgf+RaRGU5KTVXaaG+T4XG9mkIuVdG7WZtn0jb9Qy7hboDAXMuAzEoJyEvXz26e97TcRzvHFFA7az+mj9EnsBETa5Dq6u2O8/zhbvtNpCPBn/B+LZDxuYSkZhxrMITVI95Qimkjz04XWJBdXz+oSjpJFyu2dS7w+lE9Z93co+ZR7r/RDlgd22cFlNzdvCOPOr8JmG9UAXSGDLzoyES5xm+cRFVrfIxB2v+dBiTqDUisaVAvegO5vPUDTrbmZNKIp+yZc4d2IhgRG2qy1DVZWz3BmlwPUeOXvJkw5WQdD2w9x2nXZT/po+i8G+PckwLjgUQcLwH9qQSD37g7k3IWLwUysxG4FKgc1XXWCY X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Mar 07, 2024 at 09:56:07AM -0800, Dave Hansen wrote: > On 3/7/24 09:29, Kirill A. Shutemov wrote: > > On Thu, Mar 07, 2024 at 01:39:15PM +0000, Yosry Ahmed wrote: > >> During context switching, if we are not switching to new mm and no TLB > >> flush is needed, we do not write CR3. However, it is possible that a > >> user thread enables LAM while a kthread is running on a different CPU > >> with the old LAM CR3 mask. If the kthread context switches into any > >> thread of that user process, it may not write CR3 with the new LAM mask, > >> which would cause the user thread to run with a misconfigured CR3 that > >> disables LAM on the CPU. > > I don't think it is possible. As I said we can only enable LAM when the > > process has single thread. If it enables LAM concurrently with kernel > > thread and kernel thread gets control on the same CPU after the userspace > > thread of the same process LAM is already going to be enabled. No need in > > special handling. > > I think it's something logically like this: > > // main thread > kthread_use_mm() > cr3 |= mm->lam_cr3_mask; > mm->lam_cr3_mask = foo; > cpu_tlbstate.lam = mm->lam_cr3_mask; IIUC it doesn't have to be through kthread_use_mm(). If we context switch directly from the user thread to a kthread, the kthread will keep using the user thread's mm AFAICT. > > Obviously the kthread's LAM state is going to be random. It's > fundamentally racing with the enabling thread. That part is fine. > > The main pickle is the fact that CR3 and cpu_tlbstate.lam are out of > sync. That seems worth fixing. That's what is fixed by patch 1, specifically a race between switch_mm_irqs_off() and LAM being enabled. This patch is fixing a different problem: CPU 1 CPU 2 /* user thread running */ context_switch() /* to kthread */ /* user thread enables LAM */ context_switch() context_switch() /* to user thread */ In this case, there are no races, but the second context switch on CPU 1 may not write CR3 (if TLB is up-to-date), in which case we will run the user thread with CR3 having the wrong LAM mask. This could cause bigger problems, right? > > Or is there something else that keeps this whole thing from racing in > the first place? +1 that would be good to know, but I didn't find anything.