From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E0A2C54E4A for ; Thu, 7 Mar 2024 22:29:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CAECB6B02CF; Thu, 7 Mar 2024 17:29:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C5F616B02D0; Thu, 7 Mar 2024 17:29:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B26F76B02D1; Thu, 7 Mar 2024 17:29:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A23E96B02CF for ; Thu, 7 Mar 2024 17:29:34 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 7D2161C0F7D for ; Thu, 7 Mar 2024 22:29:34 +0000 (UTC) X-FDA: 81871685868.23.54D26D4 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf22.hostedemail.com (Postfix) with ESMTP id CC372C001D for ; Thu, 7 Mar 2024 22:29:31 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=idolHMAC; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of 3yz_qZQoKCCoeUYXeGNSKJMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--yosryahmed.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3yz_qZQoKCCoeUYXeGNSKJMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--yosryahmed.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709850572; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qDRTlWsJQzO9MrOpDvjMSk7xOJu//0tU48E07HkojKw=; b=388VRBq+Vc3w6I34N91P57YtmQkztEZpvkWgnPQSl2wGI9vOibRAHtAQYuLd4FHbX+eF6e 2HZd527cHXH6q3YD7ei5xfYEbbcsujVLxJeQEjOI8QpjJZXfwpGBnp8s6xbUvqgidxG/q0 QRuiQRgXjt/udF7an69IB7Fb8AVnXGM= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=idolHMAC; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of 3yz_qZQoKCCoeUYXeGNSKJMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--yosryahmed.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3yz_qZQoKCCoeUYXeGNSKJMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--yosryahmed.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709850572; a=rsa-sha256; cv=none; b=XvcMD3KlDuzEukQ555z3NWeP81gaXjSMpb//521uoMV9qae7lPFf0UeqdvVXKvN42/9Cq9 l5MaeiOfZQnvyA9dr1gLuMp5xINaHQVnLDqo8dY8dnMz+slshSmWQcunKOhKjjf7ZBuQLQ wm0DnQSP4cI85bGrhvsfIO1rbdq3fOQ= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-dc647f65573so2502300276.2 for ; Thu, 07 Mar 2024 14:29:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709850572; x=1710455372; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=qDRTlWsJQzO9MrOpDvjMSk7xOJu//0tU48E07HkojKw=; b=idolHMACSOpLmcVzGPMcARgd+mqdciuPJKiIOZh/qL41R/I/huCKObUvRHK5DatO4Q VDGFCiMYBYrmE+JV/Xo5LPW/plFjkgnmtSsxWk3C8TFKNtAnCeEL2fu6MKyv92gRguLB PVEh6Om5t8Wt1vQCntyUa1aDCNpA+61mpf1VugZ/N6dVHXF7t2zlsfj9ixC9HkEm4TRz H7Hb94/PaGlCaW6KBFGZQO5me5/xCb/I4vi1ilB5qrzKoycKI6f+Jc+GQ/giEVozT6gg agv2Nvg6//5FDihszcDh0r/we9pj9cTI9xEDyKCetZ9iZH3Hpe95wKGVEmcgV0jyTOtg 4l0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709850572; x=1710455372; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=qDRTlWsJQzO9MrOpDvjMSk7xOJu//0tU48E07HkojKw=; b=APabZWqtBTxfAUlEwunpjPobjoetvMgKrqKotHLis9qDr9iE5DjTGvRhmtuoTtVy6l 2rxJFH321R4F5fn46gcWcQ6mECP0eeCPqyqirzaUZPnOpsoUe33eURSGwADmOIGjB5Y/ Eq/G7BYcJQwtUqvolx9ajaw0PIo7w1wWtld2sMYEZdIV0qHXCJ+YW5R82xFmbA4auWFq f7txBhp0ddfliWBaRSocd7jhl7YDfik57yNntkqsZITXi9XkqWhpwJIU1+SyE4qPF5Sb kaNh5z+5/xyxkizb6FtriF+Bhv8RQOaUdFsmgc/t+ST6dcqJ77VhRP1qq17ZHHcxMeQt XHoQ== X-Forwarded-Encrypted: i=1; AJvYcCW6kLf+e1vQXZD9bwI3qlWbcft508BhQ9C/kMQQ90E4fpy00wTIf5KedaT5ZcBmpCTSOtIiJtnmPdDVU51yNwP+cpk= X-Gm-Message-State: AOJu0YyG1Hf7KS290+KQ9JsH4PBtzdWNENV3k2GDAWslrQAxx1P0y+cO nhUfIsO3BxAPj8YD3/io40nAACnHGmkdUbOy1BiLlTIq3xkR7d2HDcW5BQ9qCOF0OiwHWhjwv+H D1/cTgU9bVAi0iNUuTw== X-Google-Smtp-Source: AGHT+IHDwl1ajC/sq4yCjvIzRdBW2muKHXy32tYYhISNYzdVAaFxNZRfi+kW14+7gtR/JD5yznJEicQhSCKjcM/p X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:29b4]) (user=yosryahmed job=sendgmr) by 2002:a05:6902:1741:b0:dc7:5c0d:f177 with SMTP id bz1-20020a056902174100b00dc75c0df177mr4870186ybb.6.1709850571847; Thu, 07 Mar 2024 14:29:31 -0800 (PST) Date: Thu, 7 Mar 2024 22:29:29 +0000 In-Reply-To: Mime-Version: 1.0 References: <20240307133916.3782068-1-yosryahmed@google.com> <20240307133916.3782068-3-yosryahmed@google.com> <83b019e2-1b84-491a-b0b9-beb02e45d80c@intel.com> Message-ID: Subject: Re: [RFC PATCH 2/3] x86/mm: make sure LAM is up-to-date during context switching From: Yosry Ahmed To: Dave Hansen Cc: Andrew Morton , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Peter Zijlstra , Andy Lutomirski , "Kirill A. Shutemov" , x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="us-ascii" X-Rspam-User: X-Stat-Signature: sa14ypgg1fkyj3uadae3k39shohw6mfa X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: CC372C001D X-HE-Tag: 1709850571-978321 X-HE-Meta: U2FsdGVkX19xFi2+uIEoCtiVvdlQZYmOpwNS3H9VhPkzZHk3pfKsgjUTssjrp4AcdRl3eONINygaYOg4d2MNGxbOwIFB6Bvo3st5bmkrWvuBLmyh/LGeX7Bkyn9kyfp8wR1ws1zWLLrNgm2M+Jrqg/7JdGaJl7EkVUEDW47P+06JvHGQsEu4zsRbehcuLHlFCCWf+geWL3hGdRddJPuNqGJ44PPk1kRvLfNqm63CYZt6xFOqw3W4iwMaGukQXESC58ibIJEUgpk+c11CxJzI+A8jI0xhtD+tA4mRmP40ZlVfc+u13sxCeRmvwHxFrj3jpS6LZ9pv49L4x2QHKyGlTj74QUQoo9ehkuhm04lpy1P4LBIgAd0x1/Zyal5XumO8xkUsXLGS1F19ds+Pr6lZXNrXN3XqNECdj14Uh2nqezPUcrN+hUTU9aAUvbyxX2JY7ahzi9eXmDoi6Z3BWqtuxjAsuPtObeJHFJbeyPriGnKuoDHNr9QxsZ0ffJUu0yhj9wWlaZja0M1lHAqNqWIPZRrD56dfNYK6bRYZD6iUUTt0opcR2v3Cm1wn1wFapblASatWbUj/uJ0H/wWka63TYn5NdtHvfFUedVPrslZf5nT6cjsg/hiSC/r74COPGZejVZ2d3NtNM1QSFtW2Xl77CGDKvdPWWgDzak9SmD35Aa2VYTfgNWsf0pRrMjBZW7/dcFI0s6OMNxN4EzjDBJnMY1q3iMezzJZLdjyFTJsyfDzmJhQzltkXml29Xx02ejCSVkKVyuXdFRJ88gJ+02YFs896A1alWYLN96WaeOAB1MQKABN8WHK2dSwGG9sY+k8zGFd0I72gbxjhe2r/cmPy2nkk+7zBi9I90VdEnOzThhIQ5dHfMz63rwG39RcVfii5ltB73ceDODe+P5z3FiXTQmd/akPI6hNSVm4r6kP4FIcEUtT7CtAwypjsGikkHf0sob6P/Hhlg5GdO0Zu4a8 9pnQZs37 GreMoHRE0fsywev8tseBelInrinpZnic6ptN5fcotjDeoZ6XdkmilrbbKaHBgV5Vv1F3CVy7ChHgkzM/G8jOiOwpHE5RxbDHm2zZrDN+y5hStS752V5CV+EX8gMjMWjc7HPSy++Ysk6MgAKn/dKyJgd/LYQa9CXMAcdnAihvzzMoFv+y/o1AcwoKgqry0iZnIBLUQUvjHx2SaonurbVZb8GXGLHBTvBWWJ14RXM9/qruydcSNCdDBrtIrMFODY0wNQ4kRHrSakPUZ4/QwEPiYwOHcGoIxtRkkwBHxyZsmb1ZOZfblodPQGlFo9f+G+xzbfcR+bgaTSYi5tGv+lrt5sKA/ClocpsgyejlhOwBvHPqazSwRfl+lQJ2aE9UaSe5I576DUnRM2jJJzhHfy75umq9ZGr7372UMIpKuNw82VqtByy3FYzWNUh1nCDZok9E83tYsJKtsIT6ckjvvl5jwaMX2lUKxr9td0y1XLvqb+wSVQTnlQS87K5t54vb/tkWm0thEUfM0OPW1XOwr6CL6xtdM0nrZKxJdbITFm7BmCIgFwUTWDeiOdqPImaEaRqA6zy6TcdW5fO9/Ul5ZNEi39LBfwDqA0kqsVkT9rZMd5Jbeth7/PE9gvNZuXQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Mar 07, 2024 at 01:39:53PM -0800, Dave Hansen wrote: > On 3/7/24 13:04, Yosry Ahmed wrote: > > I thought about doing inc_mm_tlb_gen() when LAM is updated, but it felt > > hacky and more importantly doesn't make it clear in switch_mm_irqs_off() > > that we correctly handle LAM updates. We can certainly add a comment, > > but I think an explicit check for CPU LAM vs. mm LAM is much clearer. > > > > WDYT? > > The mm generations are literally there so that if the mm changes that > all the CPUs know they need an update. Changing LAM enabling is 100% > consistent with telling other CPUs that they need an update. > > I'd be curious of Andy feels differently though. The mm generations are TLB-specific and all the code using them implies as such (e.g. look at the checks in switch_mm_irqs_off() when prev == next). We can go around and update comments and/or function names to make them more generic, but this seems excessive. If we don't, the code becomes less clear imo. I agree that the use case here is essentially the same (let other CPUs know they need to write CR3), but I still think that since the LAM case is just a simple one-time enablement, an explicit check in switch_mm_irqs_off() would be clearer. Just my 2c, let me know what you prefer :) > > >> Considering how fun this code path is, a little effort at an actual > >> reproduction would be really appreciated. > > > > I tried reproducing it but gave up quickly. We need a certain sequence > > of events to happen: > > > > CPU 1 CPU 2 > > kthread_use_mm() > > /* user thread enables LAM */ > > context_switch() > > context_switch() /* to user thread */ > > First, it would be fine to either create a new kthread for reproduction > purposes or to hack an existing one. For instance, have have the LAM > prctl() take an extra ref on the mm and stick it in a global variable: > > mmgrab(current->mm); > global_mm = current->mm; > > Then in the kthread, grab the mm and use it: > > while (!global_mm); > kthread_use_mm(global_mm); > ... check for the race > mmdrop(global_mm); > > You can also hackily wait for thread to move with a stupid spin loop: > > while (smp_processor_id() != 1); > > and then actually move it with sched_setaffinity() from userspace. That > can make it easier to get that series of events to happen in lockstep. I will take a stab at doing something similar and let you know, thanks.