From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24F1EC433EF for ; Fri, 7 Jan 2022 00:19:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6E3C16B0074; Thu, 6 Jan 2022 19:19:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 66C5E6B0075; Thu, 6 Jan 2022 19:19:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 497756B0078; Thu, 6 Jan 2022 19:19:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0145.hostedemail.com [216.40.44.145]) by kanga.kvack.org (Postfix) with ESMTP id 338EF6B0074 for ; Thu, 6 Jan 2022 19:19:51 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id E6F4E86E62 for ; Fri, 7 Jan 2022 00:19:50 +0000 (UTC) X-FDA: 79001582940.16.43C1421 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2059.outbound.protection.outlook.com [40.107.92.59]) by imf08.hostedemail.com (Postfix) with ESMTP id 4BF3616000D for ; Fri, 7 Jan 2022 00:19:50 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=K4xcWwc7Rf6lHSBVj1igRfLeZ5dgMuV35TU3QhxFKZy1PAUffH5OfgYOBp+C6X9AOS+9aJXreGYcGek8xShvxTCAnjn7Pa9eooXwBaCBP6QC+AkUCuf3wd7oDfkh7habB6VOJ2yyrBKb3ZhA7RNQW9gNE+VrffTRfnmsoDMhsnMMqnGsTpV3HmQRaDT393Uah+THpvRhQ0O8qJ3aqpz1rBu8gBRZdp0mYr0nAx6gWzpw5Af6t2tqCIIjESpKUqqPqzrYBG6REYYGnTTBKfsmeYNfnucvwrbu//kXkEnnZsImPE/4kcnraZTS21zXRckMKb0bw3D76211QANbEg+rMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QfzH5c807geQ2tRH0qnURQnxPFLXit/pfq3ANqQn5Wg=; b=AkdXgI/B+ADFRxHyu5qQLIX8cQDGjVR3n7oH/XdZglzWOZh1A8tD57K5Ur6hwCPKE87aVhexaNqqA6Je00tiu3F09Kci8CjwlDq157qxUgrQkXFgXM7/8lMupQM3aLsSgqQk1Z4E/0SmQv11HaA7HduC/OEUz5GXOLe3Oe9B3+m+NulGjGrs8VVplkQZc3fMrtYhi45sIFVb+/7dyxRQOK9vEPBzkfMvxPeFZokzALRQ5wbCly4vFuyc0oGbJsyAiyzu1ry1gH4P15e5SzQY43hvrolSwIFQHGzUabnniLN8T7xT7WAiPsOoqWxgDIMwgQDSnnAFANve6vuBiG7hvQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QfzH5c807geQ2tRH0qnURQnxPFLXit/pfq3ANqQn5Wg=; b=LCkw+TN++PjB61AbhuUiISxScp3IrTpGcFMdaYYKD/ksP3MiQHkOkyzDVbtSkRycr/bDuGR/lh9fY/KD+YZv1D6YnQJD71fV8UeOvCMXlM6USxPJPe8MGW+MG75VnnfkavercV0zDJMXJRib8and6d9GKvhW5nGRCmx6hupsRvlr/A0LCGRbxqpPkPV+pDMD6XgM0jFawRGzyqHFgIUHZEuVmy+st4asMGV/znI/SsmQ2fVT3NynVvksV3oBeSpa0Hsr9OpT7dFE/qPTspbhmKGIckZ2Xrdw/aFpRQ8vF9kYP8PUIcp+faxP5l0OEm2Rk4bt6pepUl0+J0FS/dgBqw== Received: from BL0PR12MB5506.namprd12.prod.outlook.com (2603:10b6:208:1cb::22) by BL1PR12MB5378.namprd12.prod.outlook.com (2603:10b6:208:31d::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4867.9; Fri, 7 Jan 2022 00:19:48 +0000 Received: from BL0PR12MB5506.namprd12.prod.outlook.com ([fe80::464:eb3d:1fde:e6af]) by BL0PR12MB5506.namprd12.prod.outlook.com ([fe80::464:eb3d:1fde:e6af%5]) with mapi id 15.20.4867.011; Fri, 7 Jan 2022 00:19:48 +0000 Date: Thu, 6 Jan 2022 20:19:45 -0400 From: Jason Gunthorpe To: Alex Williamson Cc: Daniel Jordan , Alexander Duyck , Andrew Morton , Ben Segall , Cornelia Huck , Dan Williams , Dave Hansen , Dietmar Eggemann , Herbert Xu , Ingo Molnar , Johannes Weiner , Josh Triplett , Michal Hocko , Nico Pache , Pasha Tatashin , Peter Zijlstra , Steffen Klassert , Steve Sistare , Tejun Heo , Tim Chen , Vincent Guittot , linux-mm@kvack.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org Subject: Re: [RFC 08/16] vfio/type1: Cache locked_vm to ease mmap_lock contention Message-ID: <20220107001945.GN2328285@nvidia.com> References: <20220106004656.126790-1-daniel.m.jordan@oracle.com> <20220106004656.126790-9-daniel.m.jordan@oracle.com> <20220106005339.GX2328285@nvidia.com> <20220106011708.6ajbhzgreevu62gl@oracle.com> <20220106123456.GZ2328285@nvidia.com> <20220106140527.5c292d34.alex.williamson@redhat.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20220106140527.5c292d34.alex.williamson@redhat.com> X-ClientProxiedBy: YT3PR01CA0102.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:85::21) To BL0PR12MB5506.namprd12.prod.outlook.com (2603:10b6:208:1cb::22) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 88785956-3994-4f2e-d4c0-08d9d1736a21 X-MS-TrafficTypeDiagnostic: BL1PR12MB5378:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Mg0cBB8rADGOSeTzZHvPOlydnIsxuzyIXnr+XPmUaE0RGUgvBQbIAx/tL7KP3xgGaCjmJdZ5mmHoKQ0Nqi09rfe8p+pHfRcsqD+m+Nw6kSfYUjj4G2OFOmNgvyyTfj/D0hlYKaSNzi5ApmRd4kUkQ6AGpHEUagNcnu6UzRmzw5AvNhc8bNqiC5jKvjLJVQQrpDFIe+JSdd0a/43OaFTRtfYEcNNGhRRySKv4BCTERyr1EVyCzxLR+FBzhkLo1FEb7uu+EyvlQ4FBfeXpddkDQ+KjUzajSJ4UA8ef2jWRM/+rxiB2ZmTgyKYCePDZ7BLgNXFpCjbcgPsVWhWv2BST2i0TcTQ70BTu//hH5u0qv7hvdjTr/FgCCeFe7u7st84/sIuFBmTW+flX30QJcWFqt7nKZ4pOxzzpp222mOYa5C8aJZG6I44U/kUhMORtE1S+wEGWZ09qvqCoBCpjY/eCvBUsGLGMSgdzXvaky/JCsdePHtD3h15hrau40r/yf4t0NBEZcdrNzTFHP3AggGtBYX4tvGaZId6qKZ62/F8AZsCkrDB1NOtteFQH63mVPvMzx1LIdXZDxKXwDSp/THuy9V6PXb689wkpSA9z7MqufeaFMYVnZ51BFOPJglaw9F/5cgL17tO3PaMElt+8smjcGA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL0PR12MB5506.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(7416002)(83380400001)(2616005)(8676002)(6666004)(38100700002)(316002)(4326008)(6512007)(86362001)(33656002)(26005)(36756003)(186003)(1076003)(2906002)(6916009)(54906003)(508600001)(6486002)(66556008)(8936002)(6506007)(66476007)(66946007)(5660300002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?zX9rpdTIlEZcmo5enejYD9zDml4cVHNe6uCjfLzCIbQ0RpkU2XeKU2XLyPge?= =?us-ascii?Q?9B82PsqHFNwi2lW9pgoNZ3AM5vnZxbLSsmDQ2HHLyMXaP4lfpBFN/Vbf49tM?= =?us-ascii?Q?KU0Ee0XS2RK+CSohND7bkeIEIe8riEn3DZ3PXhAxYrRSlDjpAzgxsQsAWLWN?= =?us-ascii?Q?EnfboKephntbX40osEhk6YY16WKCzjtEhfF7CTrvic0k3/z3Csbjck4qPOGT?= =?us-ascii?Q?3t/l0ZiC136klyGeXwvv/7Zlncgk+5KR8XfVyqabyoHbker984CIytGNLao3?= =?us-ascii?Q?uJ3LAaW0CEepF7owTtZtwzyVVAmPhieRolJ7eP5TkseYGm+cffvmsGDSsRud?= =?us-ascii?Q?tdwa2MB5RqM0bV9RHpkhJKc+jZfa8XlxPPdr078pCD7c/OTG/bvXAO2/H0B4?= =?us-ascii?Q?mh1+c4HuZObQF5PwWHcHDCGMd2pV3lavDK3ZRTvjwX4bCaG37iitDwNAcWd8?= =?us-ascii?Q?bcoLB0MdMMi74hOdaU9DiO03K6Oklh68vL6+R3aGcaN5FwRIADSrupTuGxQY?= =?us-ascii?Q?KbrALYsUmC8CKvOPfo1as0hbaFzUyPsQfAvWSvfbDqfKgx0j9xZ/rzs4YyLi?= =?us-ascii?Q?/IU8pbNXCPORRbiKkpTmMaIr31+FuLMxE5NwCilps2th30f2vgax0kiJThtY?= =?us-ascii?Q?KYvHO8BeDotKv1UwmlJNyJx9nn/ia68pMfmwra/wPzLaeHCQQbN3DLkGVqoX?= =?us-ascii?Q?NdBXcAYtUr5d6BBg5q8oylIFoA4/Lz29gcZHhmwzMDchoN2DhnFI7o3fpHK6?= =?us-ascii?Q?8plxkTj1Efriy50sPdhz8ZjcNOMgsXldXHE5/pPGhORWwpDhaGt87McXmgHr?= =?us-ascii?Q?yzTs1vMYhz9D3FFAGm36EOagKB2K5gyJ6wfjUAekbJtQWDO4mTvvkrU9IMJb?= =?us-ascii?Q?mQu5nsyu8UiCCD+cenDqd0QAOTaXzzcxZj/RNE5kkmdp+885hMlO8Kcyvfki?= =?us-ascii?Q?2xPLpu2wvUF8nqjIH6aInNJWpqQ4vh+f62dL+tepcS+NrR46H6LDBjbAZJ/I?= =?us-ascii?Q?qgOqyD2W0bI7BRk+7+sYorFC8jcZV/GxGr1nlAM29H90PYYRJj2O1jvlE8N5?= =?us-ascii?Q?5p+e+qM0wsBNr+GVyQnSF48u7SOF/pykCPnJ4DoTQmTDrUH2vQwut7OvgH9X?= =?us-ascii?Q?KGiLe4+EHuJrt9cUa/qduGu2uYbbxTBQiTmnFxBDlsvjM5f9SKuFAsfE3Qeb?= =?us-ascii?Q?pSZxCQ8l34wTZpbAD0mf3QDJYvcPc0BGqxeOfzxPPNku3hwpCCEaC7ziCx8Q?= =?us-ascii?Q?tPBhIeVEVZ1Db6TueChohzY10vUDuELF5yBYy0rHoz99DxIf9C1fSW5xiQus?= =?us-ascii?Q?ebnVqh+dy8MkmSgEBBBTrsC5Lze6C7hdnYebvgqUwOSKxRhGhZJb6f3X83cj?= =?us-ascii?Q?+S4IgA20k6YGa5qbFp4FtQKa1oS9+bHbKwmq9ooIvde8D7mpZ1wmLgarcubu?= =?us-ascii?Q?YpEr6vB7ajSMrDvza9keat7+S/Z0GohvqnVB4xoi74B+SDNtANsZUvjjbM+a?= =?us-ascii?Q?eIRDBQ2tT0p+vj3/QZqtaG3m5xUhTLp/BZV0voS3kAz2IItmE22LtUX9HNJb?= =?us-ascii?Q?dP3N/pHCe1OjkEshjv8=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 88785956-3994-4f2e-d4c0-08d9d1736a21 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB5506.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jan 2022 00:19:48.5132 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 5bLsXlVBGox73VGUMi2tt8syOAImiuNUsFuPRSUzzF87LyJYySCEUl0Lw0NDtWkm X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5378 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 4BF3616000D X-Stat-Signature: yg4mqm9sa8bqn7eox38u44fffir837du Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=LCkw+TN+; spf=none (imf08.hostedemail.com: domain of jgg@nvidia.com has no SPF policy when checking 40.107.92.59) smtp.mailfrom=jgg@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com X-HE-Tag: 1641514790-138368 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Jan 06, 2022 at 02:05:27PM -0700, Alex Williamson wrote: > > > Yeah, good question. I tried doing it that way recently and it did > > > improve performance a bit, but I thought it wasn't enough of a gain to > > > justify how it overaccounted by the size of the entire pin. > > > > Why would it over account? > > We'd be guessing that the entire virtual address mapping counts against > locked memory limits, but it might include PFNMAP pages or pages that > are already account via the page pinning interface that mdev devices > use. At that point we're risking that the user isn't concurrently > doing something else that could fail as a result of pre-accounting and > fixup later schemes like this. Thanks, At least in iommufd I'm planning to keep the P2P ranges seperated from the normal page ranges. For user space compat we'd have to scan over the VA range looking for special VMAs. I expect in most cases there are few VMAs.. Computing the # pages pinned by mdevs requires a interval tree scan, in Daniel's target scenario the intervals will be empty so this costs nothing. At least it seems like it is not an insurmountable problem if it makes an appreciable difference.. After seeing Daniels's patches I've been wondering if the pin step in iommufd's draft could be parallized on a per-map basis without too much trouble. It might give Daniel a way to do a quick approach comparison.. Jason