From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 98BEDC7EE26 for ; Fri, 19 May 2023 15:33:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=u981dNo2rvn27rM3qnxYTH2phe4Rc2fX8If2Hj/mUKQ=; b=ZJu5DZgrIH4ovF T/5zT0gEuqV0W6ZhTF9czqaOnWn8M3/INWly+4pywPEmBsDhnmUdjhOfwlYIQ7rhpbpk0boPSAqsN 2FJpqRQEg1qFbz4PScwI8jua3pnkmFyjQUoWhRTgSVBKm9zjGjrm8A/H3mlSh/HRBIM0BNly0g+Hy Q5V25w6d0L3hqkPeoScmoUQPWqvO7iTEntqcpPsYbfNfhZd9xMcomKlYH7XXsoqGQIYlfz5fsNesr I9Mwx/TfMbilD04KJjqQp+n4EdU6woFFpzMjinpLNza/L2J9mujq+5f1rVoxyGuMkXdvCZPe437wj 18BxSrRSiru55DMRz1wQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q026d-00Gbub-1j; Fri, 19 May 2023 15:33:23 +0000 Received: from mail-wm1-x333.google.com ([2a00:1450:4864:20::333]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q026a-00Gbu0-2v for linux-arm-kernel@lists.infradead.org; Fri, 19 May 2023 15:33:22 +0000 Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-3f423c17bafso68385e9.0 for ; Fri, 19 May 2023 08:33:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1684510398; x=1687102398; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Z86CxvgDjR+XK0uf9yod5lX5QLSIaIX8FS/Ty9Oxyco=; b=X24fX+TqJfReB8Iq09t+a2WgYpxkcKcPyoTdANZwDXVObw3De/NVmgUnPwCujOcl46 CABGTuR28Vs7n+trdr346X8iOUoiygSp1+QCzIt6c5ToZHrdiAPBl1wstzBJ6tPg41zl 8ynHbqa4InnPx1D3esoawq+LR3FMusbAHK3kB52L1ewSV/Ixenat4oiri3OnWyhDWzri RhtPM366yNU50x1bbiWCatBsHqhH/QGe1s8qSugQsJY0kolfXlmg8/5ue4CCk3G1Eg68 ylqEqmBzJN8sau4gf7gv5V+NnJI+nodRNzyKyNX2g4DWIXpSYq18ZLLYfrI4dfOLZIJZ 9p1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684510398; x=1687102398; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Z86CxvgDjR+XK0uf9yod5lX5QLSIaIX8FS/Ty9Oxyco=; b=QuiSxtCDm9rvX1ruuVUUWZdPFZomKnqJ1+iWkmo9twG6OEcTXk7QofEr+ZBUt76Tdo LACI9vWtDYoae0/dsSe5jB47+0LizLapKw/8EMoS8z6LTA72XEJY9HkOoOXQGNIuPhUV Fs0p6rBxJiIkFhZhGTsE7X1mJPiHIo1cIvT1vh5d8sAwTXAgobv2DCqdIuQe2vF8/pQ7 RHBhbK3TIo8HIKm+ANEtzf16/lbjS1ts5aGma4IBYG7VMusmTfpjXXLKWgaIlXK0fvDl lIkdPYynPTlPJ+4lPpTwMDV/BFOWsgxMXpaZLJZaloHWeK+qKR0Q9jk4cIMksanEPX4H yODA== X-Gm-Message-State: AC+VfDyFRPGF8q3qktwW2QQnI1B4f7t3w3dRW472HqSh0p7IBriHyjXO Uny60PQfFK3JwjLo9+rq9yj+Bg== X-Google-Smtp-Source: ACHHUZ6qEKVEHmq9piPQVm6SKlYXBJCIKcsvG3FkugXOXvqVGo678gNN0J/WEnFpjTK4AcF0U0OV4A== X-Received: by 2002:a05:600c:4f4f:b0:3f4:2594:118a with SMTP id m15-20020a05600c4f4f00b003f42594118amr216324wmq.2.1684510397778; Fri, 19 May 2023 08:33:17 -0700 (PDT) Received: from google.com (44.232.78.34.bc.googleusercontent.com. [34.78.232.44]) by smtp.gmail.com with ESMTPSA id z19-20020a7bc7d3000000b003f18b942338sm2735332wmk.3.2023.05.19.08.33.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 May 2023 08:33:17 -0700 (PDT) Date: Fri, 19 May 2023 15:33:13 +0000 From: Mostafa Saleh To: Jean-Philippe Brucker Cc: maz@kernel.org, catalin.marinas@arm.com, will@kernel.org, joro@8bytes.org, robin.murphy@arm.com, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, dbrazdil@google.com, ryan.roberts@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Subject: Re: [RFC PATCH 19/45] KVM: arm64: iommu: Add domains Message-ID: References: <20230201125328.2186498-1-jean-philippe@linaro.org> <20230201125328.2186498-20-jean-philippe@linaro.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20230201125328.2186498-20-jean-philippe@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230519_083320_944642_29C3448E X-CRM114-Status: GOOD ( 16.88 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Jean, On Wed, Feb 01, 2023 at 12:53:03PM +0000, Jean-Philippe Brucker wrote: > +/* > + * Serialize access to domains and IOMMU driver internal structures (command > + * queue, device tables) > + */ > +static hyp_spinlock_t iommu_lock; > + I was looking more into this lock and I think we can make it per IOMMU instead of having one big lock to avoid congestion, as I see it is only used to protect per-IOMMU resources. Some special handling needed as hyp_spinlock_t is not exposed to EL1. Maybe something like this: diff --git a/arch/arm64/kvm/hyp/hyp-constants.c b/arch/arm64/kvm/hyp/hyp-constants.c index b257a3b4bfc5..96d30a37f9e6 100644 --- a/arch/arm64/kvm/hyp/hyp-constants.c +++ b/arch/arm64/kvm/hyp/hyp-constants.c @@ -3,11 +3,13 @@ #include #include #include +#include int main(void) { DEFINE(STRUCT_HYP_PAGE_SIZE, sizeof(struct hyp_page)); DEFINE(PKVM_HYP_VM_SIZE, sizeof(struct pkvm_hyp_vm)); DEFINE(PKVM_HYP_VCPU_SIZE, sizeof(struct pkvm_hyp_vcpu)); + DEFINE(HYP_SPINLOCK_SIZE, sizeof(hyp_spinlock_t)); return 0; } diff --git a/drivers/iommu/arm/arm-smmu-v3/Makefile b/drivers/iommu/arm/arm-smmu-v3/Makefile index a90b97d8bae3..cf9195e24a08 100644 --- a/drivers/iommu/arm/arm-smmu-v3/Makefile +++ b/drivers/iommu/arm/arm-smmu-v3/Makefile @@ -6,6 +6,7 @@ arm_smmu_v3-objs-$(CONFIG_ARM_SMMU_V3_SVA) += arm-smmu-v3-sva.o arm_smmu_v3-objs := $(arm_smmu_v3-objs-y) obj-$(CONFIG_ARM_SMMU_V3_PKVM) += arm_smmu_v3_kvm.o +ccflags-$(CONFIG_ARM_SMMU_V3_PKVM) += -Iarch/arm64/kvm/ arm_smmu_v3_kvm-objs-y += arm-smmu-v3-kvm.o arm_smmu_v3_kvm-objs-y += arm-smmu-v3-common.o arm_smmu_v3_kvm-objs := $(arm_smmu_v3_kvm-objs-y) diff --git a/include/kvm/iommu.h b/include/kvm/iommu.h index ab888da731bc..82827b99b1ed 100644 --- a/include/kvm/iommu.h +++ b/include/kvm/iommu.h @@ -5,6 +5,12 @@ #include #include #include +#ifdef __KVM_NVHE_HYPERVISOR__ +#include +#else +#include "hyp_constants.h" +#endif /* * Parameters from the trusted host: @@ -23,6 +29,11 @@ struct kvm_hyp_iommu { struct io_pgtable_params *pgtable; bool power_is_off; +#ifdef __KVM_NVHE_HYPERVISOR__ + hyp_spinlock_t iommu_lock; +#else + u8 unused[HYP_SPINLOCK_SIZE]; +#endif }; struct kvm_hyp_iommu_memcache { diff --git a/arch/arm64/kvm/hyp/nvhe/iommu/iommu.c b/arch/arm64/kvm/hyp/nvhe/iommu/iommu.c index 1f4d5fcc1386..afaf173e65ed 100644 --- a/arch/arm64/kvm/hyp/nvhe/iommu/iommu.c +++ b/arch/arm64/kvm/hyp/nvhe/iommu/iommu.c @@ -14,12 +14,6 @@ struct kvm_hyp_iommu_memcache __ro_after_init *kvm_hyp_iommu_memcaches; -/* - * Serialize access to domains and IOMMU driver internal structures (command - * queue, device tables) - */ -static hyp_spinlock_t iommu_lock; - #define domain_to_iopt(_iommu, _domain, _domain_id) \ (struct io_pgtable) { \ .ops = &(_iommu)->pgtable->ops, \ @@ -93,10 +87,10 @@ int kvm_iommu_alloc_domain(pkvm_handle_t iommu_id, pkvm_handle_t domain_id, { int ret = -EINVAL; struct io_pgtable iopt; - struct kvm_hyp_iommu *iommu; + struct kvm_hyp_iommu *iommu = kvm_iommu_ops.get_iommu_by_id(iommu_id); struct kvm_hyp_iommu_domain *domain; - hyp_spin_lock(&iommu_lock); + hyp_spin_lock(&iommu->iommu_lock); domain = handle_to_domain(iommu_id, domain_id, &iommu); if (!domain) goto out_unlock; @@ -112,7 +106,7 @@ int kvm_iommu_alloc_domain(pkvm_handle_t iommu_id, pkvm_handle_t domain_id, domain->refs = 1; domain->pgd = iopt.pgd; out_unlock: - hyp_spin_unlock(&iommu_lock); + hyp_spin_unlock(&iommu->iommu_lock); return ret; } @@ -120,10 +114,10 @@ int kvm_iommu_free_domain(pkvm_handle_t iommu_id, pkvm_handle_t domain_id) { int ret = -EINVAL; struct io_pgtable iopt; - struct kvm_hyp_iommu *iommu; + struct kvm_hyp_iommu *iommu = kvm_iommu_ops.get_iommu_by_id(iommu_id); struct kvm_hyp_iommu_domain *domain; - hyp_spin_lock(&iommu_lock); + hyp_spin_lock(&iommu->iommu_lock); domain = handle_to_domain(iommu_id, domain_id, &iommu); if (!domain) goto out_unlock; @@ -137,7 +131,7 @@ int kvm_iommu_free_domain(pkvm_handle_t iommu_id, pkvm_handle_t domain_id) memset(domain, 0, sizeof(*domain)); out_unlock: - hyp_spin_unlock(&iommu_lock); + hyp_spin_unlock(&iommu->iommu_lock); return ret; } -- (I didn't include full patch as it is too long, but mainly the rest is s/&iommu_lock/&iommu->iommu_lock) Please let me know what do you think? Thanks, Mostafa _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel