From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1A7C5C4828F for ; Fri, 9 Feb 2024 11:58:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=GbpgtcL350Zjim7v3ffJy97/pXODwIZk2h58RWBy/2Q=; b=47AshVbKLFhLc6 T/88jOtQyOXfP7oSUcBnHJrSccvggwhIo0V4J59XKGD+uqFNehVX40K5H0voUsJOPqZ0BOM7n3Arp bOrlq0p+YrKXXFIgG/VxMVaAADJbhdY6A5P5lkWpQN/YUV3JN8a7c6T27XGzIrywPl/0ajJ1vq5EA xOzktzFPC6dHRHtr7OCJJMraGkrW33mejF40wpZGHC9eqm3VtMpJpDyZ5o5AtWpbfYzccXgb7wO2V 3X+Zu57GUmgspNy0V0tXBiu45FYfSR/RUatNQfEwc7OLYpA5mrSmBwydrQ8YdLaYzqAi52sjTR9o7 +a7ePm4PnKgTpVCy3BDA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rYPWf-0000000Gv9f-3lIq; Fri, 09 Feb 2024 11:58:37 +0000 Received: from mail-wm1-x332.google.com ([2a00:1450:4864:20::332]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rYPWd-0000000Gv80-3Jt1 for linux-arm-kernel@lists.infradead.org; Fri, 09 Feb 2024 11:58:37 +0000 Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-410708db221so2933305e9.0 for ; Fri, 09 Feb 2024 03:58:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1707479913; x=1708084713; darn=lists.infradead.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=ZRwaXllpUiiGZN6hHGF+KZe1kKd23K1mjfQaoa29eGc=; b=OE5rBSzhGZUqvbHqPkP2DIEFW0EsyZe0vcPvqqDU33UQ1lQsQIwOtdNGgY2qZnME/P CxrptfScNs2S/qOj5WXIoiHE85Ly7cknb58YecPBScWy/z/fvH3AjHikbdK8P+HOwT55 1Nq1V9bf+sdgW4J1OoGevuDIDaIHkHr2i4LZH4jBQjI5zawhxfzKhTrH/m8YRKW/pU/0 i8xr1Dugqi6DnHkdNPMrDOmx7keeEAZZC9Ky0LDANtUHGK+xKru2sUsWFPVePnQs/2K6 fgk0tAgfsenMxPHBya2ybLxWhyAXA8QWXpYsUeV/feRES4ZG6TKfs0vafneheVf0NSDN ZzPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707479913; x=1708084713; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=ZRwaXllpUiiGZN6hHGF+KZe1kKd23K1mjfQaoa29eGc=; b=Vm+yx5KAehJ1Mj+UEoqJchW1X57L+x0OrVvDQXXmt2AGFOa140QeGWf/K0SPU7+nxT ZwnC9zDf0QWmf32bp6zhothPpMbr/8dIGovzimG23dSslmC9h1pyrdypmwrDbpuYSDIS 1if/V8SKUdKPxC0tMOKfDD3gi37Y2o4sUT9R1ZFplThF8gyJ9ObIaTora+CK5hhd6zDw qymvbVvqM6TEsO5urqVdwUm68sSx33n8YlTETl86na1bCf9Ol9ZSWKhBOdxVwNPHxW2U mN9In0Ls+I7HGH5mP0G82n2kKZtrZNVuwAjNWXbbhEaorzgdqOtFWyyG53spDxcArhu8 GXKA== X-Forwarded-Encrypted: i=1; AJvYcCUZ6+atzR1mNkDPpYOg7YrBCATF/ilLKjvf7RPyVDlKJH40m2CPgrPTLxoQmCgmiZUL9jbDsfePN5WJb33cD6MRcAjiNmP3eZSseGREFrUrwPH1f04= X-Gm-Message-State: AOJu0Yz/aqXWtZVFm4+nPtRmVHRIcG6A+TrbGTt3eRggwXzlFBtN9++e Q8lLffhp/S71JxMdCsfRvrQOqwlU9B4/FD2w/MyW+ZYwPWoOjsk3GFDQSck1xSo= X-Google-Smtp-Source: AGHT+IGvouDyT1S/q9yBMPyy37QzOHgMuxyrYD38phGx8Dc9ItCNIuOOaFGDMsnXDKAJsNkPQgu+mA== X-Received: by 2002:adf:e80b:0:b0:33b:49f6:cf89 with SMTP id o11-20020adfe80b000000b0033b49f6cf89mr1004637wrm.51.1707479912855; Fri, 09 Feb 2024 03:58:32 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCWfJPbY1ZXOjDhGDVDTfIfsNKg0YgtUMR/KFHggANeTQqZEgEdSQ8JUr+gOAaWCsU8tV/5X8V9HYdIjiYHgzVXDUQ5WC7s7hYFHWVh4Orn0lFB7dFzjYEYwhX4H/6nTr3n4ok8ZI2jGlDCDlnkXeJPG6H5jpYFnVeJvkFWfYzbR0808nIQNlFN5rN7NAlwNs1PnGTr7IBca4BK6Dq5EIoE/VNjIwFZ4JaYPl06dhVrHikaNnYp7IvOEicAqT8Sq6teH8vlQr6en/VAAKbfE/iHCwXa3cStqlEtGR5Hrw06oPEdz0YFENE6cTjy4a1Q6cUWPwqxUBLsusATY+3hnyUpN/htkTo6wLY9cGe6ySAWWzvUYSIPdPZJh31uCcr5IJ9NfOwlA9kI/I80WL3A= Received: from myrica ([2.221.137.100]) by smtp.gmail.com with ESMTPSA id i4-20020a5d55c4000000b0033aedaea1b2sm1636931wrw.30.2024.02.09.03.58.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Feb 2024 03:58:32 -0800 (PST) Date: Fri, 9 Feb 2024 11:58:24 +0000 From: Jean-Philippe Brucker To: Shameer Kolothum Cc: kvmarm@lists.linux.dev, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linuxarm@huawei.com, kevin.tian@intel.com, jgg@ziepe.ca, alex.williamson@redhat.com, maz@kernel.org, oliver.upton@linux.dev, will@kernel.org, robin.murphy@arm.com, jonathan.cameron@huawei.com Subject: Re: [RFC PATCH v2 0/7] iommu/arm-smmu-v3: Use pinned KVM VMID for stage 2 Message-ID: <20240209115824.GA2922446@myrica> References: <20240208151837.35068-1-shameerali.kolothum.thodi@huawei.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240208151837.35068-1-shameerali.kolothum.thodi@huawei.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240209_035836_021856_C74AA4E9 X-CRM114-Status: GOOD ( 34.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Shameer, On Thu, Feb 08, 2024 at 03:18:30PM +0000, Shameer Kolothum wrote: > Hi, > > On an ARM64 system with a SMMUv3 implementation that fully supports > Broadcast TLB Maintenance(BTM) feature as part of the Distributed > Virtual Memory(DVM) protocol, the CPU TLB invalidate instructions are > received by SMMUv3. This is very useful when the SMMUv3 shares the > page tables with the CPU(eg: Guest SVA use case). For this to work, > the SMMUv3 must use the same VMID that is allocated by KVM to configure > the nested stage 2(S2) translations. The series makes sense to me. Maybe a little more detail to help the KVM maintainers understand why we need something like this even though the s2 page tables aren't shared between CPU and SMMU: * When enabling BTM in the SMMU, all TLB invalidations to the inner-shareable domain issued by the CPU are taken into account by the SMMU. That includes for example the TLBI IPAS2E1IS from __kvm_tlb_flush_vmid_range(). * BTM is enabled globally in the SMMU CR2 register. If we enable BTM for host SVA, then it also affects KVM. * Stage-1 TLB entries in the SMMU have a bit (ASET) saying "this entry is private and does not participate in BTM", which we set for private SMMU address spaces. Annoyingly, the stage-2 TLB entries do not have it. With BTM all VMIDs are shared between CPU and SMMU. * So, if the SMMU driver allocates VMID privately and we enable BTM, then CPU invalidations will remove unrelated SMMU TLB entries. Instead, the SMMU driver needs to coordinate with KVM on VMID allocation. * Private stage-2 address spaces in the SMMU would need to allocate VMIDs that aren't used by KVM, but that's not a use-case at the moment: - For assigning devices to a host process or to a VM, we use private stage-1 mappings. stage-2 will be used to enable nesting translation, and will typically mirror the KVM stage-2 since it pins the guest address space. - If the SMMU doesn't support stage-1, the driver falls back to stage-2 for private address spaces. For such an implementation we disable BTM. - The old VFIO_TYPE1_NESTING_IOMMU lets userspace allocate a private stage-2, and has only been used for testing as far as I know. I don't think I ever found a program that used it in the wild, but haven't checked recently. The effects of using it with BTM enabled is performance degradation: TLB entries of that VFIO container get invalidated by unrelated KVM activity, and maybe that can be used in a side-channel attack. It needs to be deprecated over a few releases (starting with a warning maybe?), and the replacement API shouldn't allow creating a stage-2 without a KVM context. Thanks, Jean > > An earlier proposal sent out[1] a while back resulted in changing the > ARM64/KVM VMID allocator similar to the ASID allocator to make it > better suited for this. > > This RFC adds, > -Support for pinned KVM VMID. > -Support associating KVM pointer and iommufd ctx. > -Changes to domain_alloc_user() to receive a kvm pointer. > -Configure SMMUV3 S2 using KVM VMID > -Finally enable BTM only if SMMUV3 supports S1 translation. This > is to make sure that PAGING domains always use S1 and S2 is only > used for nested domains with a valid KVM. The idea is to make sure > when BTM is enabled in Guest, we use KVM VMID for S2. > > Not sure I miss any explicit TLB invalidations with any use case > that may configure a S2 with a private VMID that matches a KVM > one. > > This is based on Jason's ongoing SMMUv3 refactor series[2]. > > Please take a look and let me know. > > Thanks, > Shameer > > 1. https://lore.kernel.org/linux-arm-kernel/20210222155338.26132-1-shameerali.kolothum.thodi@huawei.com/ > 2. https://lore.kernel.org/linux-arm-kernel/0-v5-cd1be8dd9c71+3fa-smmuv3_newapi_p1_jgg@nvidia.com/ > > Jean-Philippe Brucker (1): > iommu/arm-smmu-v3: Enable broadcast TLB maintenance > > Shameer Kolothum (6): > KVM: Add generic infrastructure to support pinned VMIDs > KVM: arm64: Introduce support to pin VMIDs > KVM: arm64: Add interfaces for pinned VMID support > iommufd: Associate kvm pointer to iommufd ctx > iommu: Pass in kvm pointer to domain_alloc_user > iommu/arm-smmu-v3: Use KVM VMID for s2 stage > > arch/arm64/include/asm/kvm_host.h | 3 + > arch/arm64/kvm/Kconfig | 1 + > arch/arm64/kvm/arm.c | 14 ++++ > arch/arm64/kvm/vmid.c | 84 ++++++++++++++++++++- > drivers/iommu/amd/iommu.c | 1 + > drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 42 +++++++++-- > drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 3 + > drivers/iommu/intel/iommu.c | 1 + > drivers/iommu/iommufd/hw_pagetable.c | 5 +- > drivers/iommu/iommufd/iommufd_private.h | 3 + > drivers/iommu/iommufd/main.c | 14 ++++ > drivers/iommu/iommufd/selftest.c | 1 + > drivers/vfio/device_cdev.c | 3 + > include/linux/iommu.h | 3 +- > include/linux/iommufd.h | 7 ++ > include/linux/kvm_host.h | 18 +++++ > virt/kvm/Kconfig | 3 + > virt/kvm/kvm_main.c | 23 ++++++ > 18 files changed, 218 insertions(+), 11 deletions(-) > > -- > 2.34.1 > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel