From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22EF4C77B73 for ; Sun, 28 May 2023 00:02:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6ECD8900003; Sat, 27 May 2023 20:02:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 69C3D900002; Sat, 27 May 2023 20:02:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 58BA7900003; Sat, 27 May 2023 20:02:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 48228900002 for ; Sat, 27 May 2023 20:02:20 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 0D541A02ED for ; Sun, 28 May 2023 00:02:20 +0000 (UTC) X-FDA: 80837711640.20.2B6CA14 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2056.outbound.protection.outlook.com [40.107.96.56]) by imf17.hostedemail.com (Postfix) with ESMTP id 4784D40008 for ; Sun, 28 May 2023 00:02:17 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=NoSL1hYS; dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf17.hostedemail.com: domain of jgg@nvidia.com designates 40.107.96.56 as permitted sender) smtp.mailfrom=jgg@nvidia.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685232137; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9nCkmOFxqAxobnMVv7hNzj6uZ+TE+ZyDSuZtkEUezN8=; b=Q4C8tv/qpvz8f6+QE4GmfAFIXeDd2nSUNShwfq0i9VemusaedQ6g7KK3iHXwwp6w6KnAYk kKATo7RC2bDZWFy2BjoiFH2CpfaiVYkoSkwZ3Upj0d2J7TvbdRe6uiP4m+usbFiQfXBggC w7W+cW3JUqbHHTm6b65274M8iNwmqyY= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=NoSL1hYS; dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf17.hostedemail.com: domain of jgg@nvidia.com designates 40.107.96.56 as permitted sender) smtp.mailfrom=jgg@nvidia.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685232137; a=rsa-sha256; cv=none; b=XdtyTTmvjKnWSxLkjpkcWko2kbp2JVtoa0mABu16+UvHBjBHP0IbNsWQeNTk8SAhuMvszp uep6lN9RDUTM+C0v+WRsOLM6oLGqz4Xzg++SUOA/URoqCgRIMaPVyzm+jUwUBC68AOLLMb yNKtRinLtbaL26KoDQQP4Mpc/kSbVOo= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9nCkmOFxqAxobnMVv7hNzj6uZ+TE+ZyDSuZtkEUezN8=; b=NoSL1hYSLH+hCjjvRCWtxRp25pim4IWD4morYIFSc/ApLJPDafcdYG0mojVyAUYQ/ZtSd4zoK79G/LpM9kUzMm4MZF3nnqBMXuDEmaNqc2RXxi8KjSP7OQVbXdiz+rgfCzWczuO0gZCpldEpSmuI9Gp7B6dopFYOXjE6nvZ+XGKB3FhSTiilMVeZnnklUmEb+j/0B6EE2gGV7Lf8gLtP7htsf4PPU23dFB6NH7i736/VLWZFirYz0W0lE+3O/OPecdGsLvlkxP2FbA4ouXrx8pZGQOV5UEnEnCmx3mMSSs+ORff5yEtiK2cjxQQoRb0xylUL5ova99C8uwDGXE+gUg== Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by SA1PR12MB7224.namprd12.prod.outlook.com (2603:10b6:806:2bb::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6433.19; Sun, 28 May 2023 00:02:14 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::f7a7:a561:87e9:5fab]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::f7a7:a561:87e9:5fab%6]) with mapi id 15.20.6433.017; Sun, 28 May 2023 00:02:13 +0000 Date: Sat, 27 May 2023 21:02:13 -0300 From: Jason Gunthorpe To: Alistair Popple Cc: Andrew Morton , will@kernel.org, catalin.marinas@arm.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, robin.murphy@arm.com, nicolinc@nvidia.com, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, John Hubbard , zhi.wang.linux@gmail.com, Sean Christopherson Subject: Re: [PATCH 2/2] arm64: Notify on pte permission upgrades Message-ID: References: <3cece716fc09724793aa832e755abfc9d70a8bb3.1684892404.git-series.apopple@nvidia.com> <5d8e1f752051173d2d1b5c3e14b54eb3506ed3ef.1684892404.git-series.apopple@nvidia.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5d8e1f752051173d2d1b5c3e14b54eb3506ed3ef.1684892404.git-series.apopple@nvidia.com> X-ClientProxiedBy: BL1PR13CA0427.namprd13.prod.outlook.com (2603:10b6:208:2c3::12) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|SA1PR12MB7224:EE_ X-MS-Office365-Filtering-Correlation-Id: 639034a2-e5c7-4949-2fd9-08db5f0eca99 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6kAZYL9mowSl+UVRAC5LA12XWZBraztQTfMbc5k1DgjPwImRto7l64Bpnx/O33zi9DJRIr6WGEQ84bLmTR5ww25EgFRrT9BXXi3KbcPjQ9ztZo2yCqVEeO2oclPLA7+F+EqcXyelrDQG54P2yW+DDNpdvL290CiMg+MKC9XmLv1SNlMDEQXP7uX5RPzU6B98b9R2RCZzgbld/v7SUtx0dTzyThG6HWiEZprdsDIAr6/CyCCWOWx3HD8GAD8XTFd0Ox4/kY4LeC6FswE0J0jZglua03pPEI8REZfYJ0jBRP17YXeJ8yiSqvNj6YgNH1vvrQctfDLWpY3MD9zHcHQVVqCvsE+6VjVvH8QvNqmZDoHFFfoTgvVQ/ti7KOoY3eRybCKuC5yABvcQEcF3n+oGoeB5Zy+Unbl2FcyW19OMScZ8L8J/HqXE21GbRqSfz/kHpaaJs+L3UV2gTSs0O/WhyHn9XZdBAFoahaFviPv/NZkS+H9wItf0+Two9tdlPw9cSy1X80BllQAbothbE3M4YXJxlpOzRq+hP86o5YqJ9vjFuauh41z/F27t7ZND5Cla7zjs1KM6j6h6RYOV1ffReYBCawiKnWtFm7TeVs436Xo= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(39860400002)(366004)(376002)(396003)(346002)(136003)(451199021)(316002)(7416002)(6506007)(26005)(6512007)(36756003)(66556008)(66476007)(66946007)(6636002)(4326008)(38100700002)(5660300002)(6486002)(41300700001)(8936002)(8676002)(6862004)(83380400001)(86362001)(478600001)(37006003)(54906003)(186003)(2906002)(2616005)(67856001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?lHpEOu+N+n8/huC67kU0jKmA3cv4uo5uMQ3O0mjD81HeufLLEeEVTh0JLIgW?= =?us-ascii?Q?MN42dkNB1PMzdVul+HyW0o/O+K7987wTiVjPk1N7FblCqX2dk6O00FmVAmMn?= =?us-ascii?Q?/BQwvEMoRQ4gqmiTwt86CEV6rLgwlEsz+Qi/VuZEj8aHFazXAzCwQQXfYUf+?= =?us-ascii?Q?auXODap/vSubioZLQCeTmzWNXIIu7qbFBecvGaSYM67gK0AkMhab5dDzkVs/?= =?us-ascii?Q?o6o8Sm5IB/adNZoOYM//bZsC43NZqruY/UlbfSwkRTPCcrOn9nhR16jCB2KY?= =?us-ascii?Q?8dS+f7xnpmqNBMEor/QAReLcp7sKJJ3FzPj1v8zieXJIWY8SdNJM19HRVcr6?= =?us-ascii?Q?lUBmGWyiXemwQyka1KIBfNM0UvA+H+Z8E3UGt5lsIpQbNWw+JhxRfiPBKk0a?= =?us-ascii?Q?fxrkgEuSNvyu/7QZ65lJRUuqJx6PfIrQRA/somegM3LnhehUEdYVlvAp/lD7?= =?us-ascii?Q?QOQyqH2+/ynGQ8gm7Kna1R5clWOxWGOi6F1TeScLtD4LTBryeYhp+jsgTiBJ?= =?us-ascii?Q?L4lqiUPO2rsy4HRfcmW5q9/pOlsmiIGhYo9DXBCZF1inkJvZT5xo2cctuysL?= =?us-ascii?Q?qyazeAggwSmhsVi4x3ylnKQNU7UL4P1Oft632IkqwWK44OvWtFVThtg0dI43?= =?us-ascii?Q?wkvdxF5ZOe/9MnlgmBNuoGt7/2bsmxLvkQf7V1rXUYQDVJNb38QFYZF7JjEU?= =?us-ascii?Q?Aaw3j9ekejMoLrTWkpdciaAf7qSLzAsLOXz6yfBSN/zEpYGze7BJl6c6n19X?= =?us-ascii?Q?ihLrnAkUxpuehTsqiP8cj4r/+Yq/bhXGyhnbxzYfb0Ci3uZEds7NAC/4Ymoz?= =?us-ascii?Q?sPdwCvK0Jpas9hLz1xvqA/x0hfYHrJ1P3igqQzUvu5im1t4wbB5WelSlz8an?= =?us-ascii?Q?NbWMiL25/juNTsojhfmiW30EsR0oDUj9Gc1Zn0v5x7re57np1Wt3MiEvdkau?= =?us-ascii?Q?kCSt77eRVMX4dWu1zKdHcwOEiUMSb6dWDNT7xQzcO483BCfBgux/v3li6U82?= =?us-ascii?Q?OaUBiZgB3W5RU6BLhbXcMK1p/gVBhXwmHFmzLxKQIAYeEOZ7xBvjgcoUFpjC?= =?us-ascii?Q?a8adABI/LiXIz7CE5dswb6UR7asmEQh+AemTQqcZrBb6T008u+HlmHXfcvrY?= =?us-ascii?Q?CH6iK05K8kANp+b6rwdq7iWWYNY0XZM92TDX8SJFzkv4hmpCnwZz7ZgyVbBP?= =?us-ascii?Q?Khgcx3c2sXA96YpKWL+Xh+NdGZFFoc4GJIIZJAko4TY4WipG6f4K580qy2es?= =?us-ascii?Q?hPL2nSGEqXKxLOfVnfB1rREXT9umugSIiEROiBn3Zqmhmmn0pf9lUAlqa+/b?= =?us-ascii?Q?fuO/BtW0A+NF5a/R0H/zos4zJCVl9JMuUx09gVxGWzAJAF4Hr3KucKOJScie?= =?us-ascii?Q?CRhvKLlasNb2FwQyppis2NiGlXXgs4CNgwhoCIUzQxolepl2ONLafO9D4iCw?= =?us-ascii?Q?/ks+Mc1NdoJ9VrANgLSKwor7StlDjnVrDmYE8N2iisGr6A0LGzkzZINP/hT8?= =?us-ascii?Q?+PDRpee9e7aZ+4QXjrr/G8IrifXAkAX2qzcoYyf2VM/wQAWRLcacMyveaLgd?= =?us-ascii?Q?yB7VL1g0KcFXgtxdvkUZ5Ab3KF9gX1SLi16uqY3K?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 639034a2-e5c7-4949-2fd9-08db5f0eca99 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 May 2023 00:02:13.7903 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ZljLCsotoGeRcSaOOG1Vj0gssEmVPYxI/nN44D8M0mvcA2ssoLzBJ0jLgMMF6EQx X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7224 X-Rspamd-Queue-Id: 4784D40008 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: u6hfhtzt5itua9tosd75hxwoybnrtjka X-HE-Tag: 1685232137-379829 X-HE-Meta: U2FsdGVkX191EMRgh6/UjnWM5J28r+s0GLO94ZEJguXUlQGwKXqqLZ2UvFfuHWRKuVynk8AM/rDlJOQm2IXxhRMdm8irzR8KqITtOHf/KAVYPsdYi1EDpKlNbMpFq6ENqyOJgCbxUlZdGCU4eqqaProa8Vedfp+N3ryyHlG9KkT/yG3G1VagVBQvSp4H17WxNHp2gpvJjBxPDnSvoCoJu3dYCp7ZHzjWJJ913soaZXsDRF/Zq0eaGJo/7l3bgJ53+gmdTrkMs7A3AnRy8VamgQZ8bvs+nrl0aRg2lK/INmWBKpc+ETZDc3UjFP1dj9s2uEzz2AZBkkcVAL3ErC/lesxAiLGl/A/3kzkj5Nr8HL308Yqe5k9120yEwhF7VxdTvksMar/EDZrWlGLLVNq1HilQN0xUSzY5pA4dra5qeV5Acwy87/U5URC5h9nklQiGratQBoxwN48JBME/nRJlYnWc9jptJNhd8YygwPc+TN5X2+akrA/cTXhb/mw1ld+LzKgcgbIcKKwbr2sGkw/N0dV6hOqryUaBkx2SdkGqes+7yPNaJt6SrDBmJRS3g02oFODqJ9LaGEaobjFyVMni2y1m80Wa/oEphlMLKMDouzULtXiv2E2O3TXQ5QzAe1G3J2pRO6SEUZ/ySZUwyDtM1jOcTyi14VJpLZoeLLeE6PZaOSkqnCVB0fNkLtQOUhP8G3/qmuyUEtbxC0XbT7wgS7sET/JQUpDBf1S3Wx1WMYRiTPhMxOilsr2JR6hhqMbEa0UCubV7dgY1frovrYGf/IzGpCDB4mOEkgzANah2m6bCozCBtn69nyJn+yj1eIhPFq+b+7hMP4ftuGHYFnycm8/eq/479Vu4ec1XR6KudBwxax/zpDiF/jQNj32JvXsRlLH0Y+qm+h0tVeJb1ScQMfm9neza5ppkR/5ZtwGyDD9NjYIkm2v+8Yf2itBtref181yqqZBP8Ixdf2rABRe kiku00sH +JQl3Erl9XNoTyaMP19yYr4vYwrqlpwILJTHou5cEG5cfAhJMgm4BWLg05boQMtMxRi8M/c7ZSB1PCzOIsdWYE0yDadevlJIwebUwua79K4yPgmSmkuc5hwWNJbjU5v6nP7GvXMnIbgs3EEuaqIQ68d/woICJms/5DdbBTrITHB2F1jcKly0y7pQie75KA8JwOA/FrJNpeCPy2O5zg0kkNqllWhvboxVB0XSUrGaQ5kawQQw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, May 24, 2023 at 11:47:29AM +1000, Alistair Popple wrote: > ARM64 requires TLB invalidates when upgrading pte permission from > read-only to read-write. However mmu_notifiers assume upgrades do not > need notifications and none are sent. This causes problems when a > secondary TLB such as implemented by an ARM SMMU doesn't support > broadcast TLB maintenance (BTM) and caches a read-only PTE. I don't really like this design, but I see how you get here.. mmu notifiers behavior should not be tied to the architecture, they are supposed to be generic reflections of what the MM is doing so that they can be hooked into by general purpose drivers. If you want to hardwire invalidate_range to be only for SVA cases that actually share the page table itself and rely on some arch-defined invalidation, then we should give the op a much better name and discourage anyone else from abusing the new ops variable behavior. > As no notification is sent and the SMMU does not snoop TLB invalidates > it will continue to return read-only entries to a device even though > the CPU page table contains a writable entry. This leads to a > continually faulting device and no way of handling the fault. Doesn't the fault generate a PRI/etc? If we get a PRI maybe we should just have the iommu driver push an iotlb invalidation command before it acks it? PRI is already really slow so I'm not sure a pipelined invalidation is going to be a problem? Does the SMMU architecture permit negative caching which would suggest we need it anyhow? Jason