From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C13C94657C6 for ; Wed, 15 Jul 2026 11:59:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116767; cv=none; b=DB4tVM4rU+BS32wl7PLDVgTcRrdXL+0u8kKvzKYOZbOlApqiuDrL0mYCnzkt1tNMRMf4tFEK6gNYzgJKSBEwwb0qQee+Z6cN5hqfXVHL1YubRPMFAvN1jKreu9fDz8LKKuc+PgwTp93PtD4SNAIi+6EMR6n4m1NatIbA9UCfSkU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116767; c=relaxed/simple; bh=BKXOjPm4HibB0zKKPDg8mVqvFcn5okB02q1ktl64G/I=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Wdk7kzm2DvEViTuyOU1o1rnN1upUCKW7C/NUKvH5Q1BPHfg+2wPBhyxF38ZMULivBDAfxpErNFj6yaDT+LEcbrEorR4LnBFhoNW8z9jMbDbj14vHMauERrPf3R9EUBvCWhqolzZQpVp4b0Z7f9/p6pV433DxRaTUYAPfSqxwmBM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=n8BFiR7o; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="n8BFiR7o" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-492488f8583so77518285e9.2 for ; Wed, 15 Jul 2026 04:59:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116763; x=1784721563; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=4P0vhJSxO+G+Vc4nk48G6MnzTMo1FQyjg7u/hgjKJK4=; b=n8BFiR7ofZgzoDS3nrCZVyLoOVzGGOctj1n4SfZ75Vc7z/CHFf/wnMuNkPxO79VtHm imxwvK/IeSYea7FzBODlhG2lRosGCUnXYaNEb5drfYMLHzY8c6WIdSbkd1qFtGalk3b7 5hRn9wntdKb3Y0MNH9QJYPGPvHrNKVi+A9+e8H17ErDSZlwGf7VJoJN0l65tfG3Xk7Ti J1MXK/ghHmj/TYLXS3oOlRVVRT5WMpsgVyq4IjmQqoxI+UHYytwRI+/KiEhWwyXmSjGC ROTth7iTkqmmKYYgtT0fFUHks8LjOamFaHt7axktAU1KQp+fTnbrc0IDbrKe6uaWmzeJ 39lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116763; x=1784721563; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=4P0vhJSxO+G+Vc4nk48G6MnzTMo1FQyjg7u/hgjKJK4=; b=QfJWCGBhYURYGLgUp/HGxqDzpj003O11qIM5Vn6tfv8/9AxlOTAPvPoRe0KJa0CN09 55NGv1pkGHse/9vRbuikpzpEu+91UXI0lFO0DysT0mN4Ib5VDug9//eGHykomlziJM97 gqzq7bp3/KJc0NQWX9FTAdkX9u28E55QY5SNF/ClFeKyaEeUdr6KD9liy/c+tdfC6Ych zi7mbQ+wTJTGtNqp2tV7TtXOe7eo4VEnCByL6Ppmoa8g+uXUmjUrlbMflqjlvRXXXB2A dZXXSOKnTPFzomk8kQ3frh90Vvl6mWj5io+MCOhsMtpVupuKI9rcnProKCuBd7IiZ18E HqeQ== X-Forwarded-Encrypted: i=1; AHgh+RpnAa/oCJQPjmYAhwbpt8e3w2XRJRPRcXfsyxAY9/1CStStgVcG44pktXxGnuYe6IBOgVXiNzeCN5d4V4c=@vger.kernel.org X-Gm-Message-State: AOJu0YziaRXqHijMj6UdIJqmkLePXtwqS1GuDmHnuHVh7ZMhM6QGUqVu AUFEiE8G/QBGOLGFGwSgsdH3HpkVwfaHFcNroFuKwotZ5auL7WH3iUyPyniiYF1XSd0b8HbSLcH xZIwJhduaJy1kIg== X-Received: from wmga19.prod.google.com ([2002:a05:600c:2d53:b0:495:3dd3:e994]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:c16a:b0:493:b89b:9a27 with SMTP id 5b1f17b1804b1-495389d2a9cmr64618175e9.9.1784116762784; Wed, 15 Jul 2026 04:59:22 -0700 (PDT) Date: Wed, 15 Jul 2026 11:58:49 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-9-smostafa@google.com> Subject: [PATCH v7 08/24] KVM: arm64: iommu: Add memory pool From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Type: text/plain; charset="UTF-8" IOMMU drivers need to allocate memory for the shadow page table. Similar to the host stage-2 CPU page table, the IOMMU pool is allocated early from the carveout and it's memory is added to a pool which the IOMMU driver can allocate from and reclaim to at run time. As this is too early for drivers to use initcalls, the number of pages allocated is set from command line "kvm-arm.iommu_pgt_mem". Later when the driver registers, it will pass how many pages it needs, and if it was less than what was allocated, it will fail to register. Signed-off-by: Mostafa Saleh --- .../admin-guide/kernel-parameters.txt | 4 ++ arch/arm64/include/asm/kvm_host.h | 3 +- arch/arm64/kvm/hyp/include/nvhe/iommu.h | 8 +++- arch/arm64/kvm/hyp/nvhe/iommu.c | 21 +++++++++- arch/arm64/kvm/hyp/nvhe/setup.c | 11 ++++- arch/arm64/kvm/iommu.c | 42 ++++++++++++++++++- arch/arm64/kvm/pkvm.c | 1 + 7 files changed, 85 insertions(+), 5 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index b5493a7f8f22..5cbac46bc941 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -3253,6 +3253,10 @@ Kernel parameters trap: set WFI instruction trap notrap: clear WFI instruction trap + kvm-arm.iommu_pgt_mem=nn[KMG] + [KVM, ARM, EARLY] + Memory allocated for the IOMMU pool from the KVM carveout + when running in protected mode. kvm_cma_resv_ratio=n [PPC,EARLY] Reserves given percentage from system memory area for diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 43821767ccab..8d3b4500b360 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -1713,7 +1713,8 @@ long kvm_get_cap_for_kvm_ioctl(unsigned int ioctl, long *ext); #ifndef __KVM_NVHE_HYPERVISOR__ struct pkvm_iommu_ops; -int pkvm_iommu_register_driver(struct pkvm_iommu_ops *hyp_ops); +int pkvm_iommu_register_driver(struct pkvm_iommu_ops *hyp_ops, unsigned int nr_pages); +unsigned int pkvm_iommu_pages(void); #endif #endif /* __ARM64_KVM_HOST_H__ */ diff --git a/arch/arm64/kvm/hyp/include/nvhe/iommu.h b/arch/arm64/kvm/hyp/include/nvhe/iommu.h index 857d7dd2ebc3..028e89a3448d 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/iommu.h +++ b/arch/arm64/kvm/hyp/include/nvhe/iommu.h @@ -10,8 +10,14 @@ struct pkvm_iommu_ops { int (*host_stage2_idmap)(phys_addr_t start, phys_addr_t end, int prot); }; -int pkvm_iommu_init(void); +int pkvm_iommu_init(void *pool_base, unsigned int nr_pages); int pkvm_iommu_host_stage2_idmap(phys_addr_t start, phys_addr_t end, enum kvm_pgtable_prot prot); + +/* Allocate pages from the IOMMU carveout, returns zeroed memory. */ +void *pkvm_iommu_donate_pages(u8 order); +/* Free pages from pkvm_iommu_donate_pages(). Must use same size */ +void pkvm_iommu_reclaim_pages(void *ptr); + #endif /* __ARM64_KVM_NVHE_IOMMU_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/iommu.c b/arch/arm64/kvm/hyp/nvhe/iommu.c index 08009609ec59..8891cb7920c4 100644 --- a/arch/arm64/kvm/hyp/nvhe/iommu.c +++ b/arch/arm64/kvm/hyp/nvhe/iommu.c @@ -16,6 +16,7 @@ struct pkvm_iommu_ops *pkvm_iommu_ops; /* Protected by host_mmu.lock */ static bool pkvm_idmap_initialized; +static struct hyp_pool iommu_pages_pool; static inline int pkvm_to_iommu_prot(enum kvm_pgtable_prot prot) { @@ -112,7 +113,7 @@ static int pkvm_iommu_snapshot_host_stage2(void) return ret; } -int pkvm_iommu_init(void) +int pkvm_iommu_init(void *pool_base, unsigned int nr_pages) { int ret; @@ -121,6 +122,14 @@ int pkvm_iommu_init(void) !pkvm_iommu_ops->host_stage2_idmap) return 0; + if (!nr_pages) + return -ENOMEM; + + ret = hyp_pool_init(&iommu_pages_pool, hyp_virt_to_pfn(pool_base), + nr_pages, 0); + if (ret) + return ret; + ret = pkvm_iommu_ops->init(); if (ret) return ret; @@ -138,3 +147,13 @@ int pkvm_iommu_host_stage2_idmap(phys_addr_t start, phys_addr_t end, return pkvm_iommu_ops->host_stage2_idmap(start, end, pkvm_to_iommu_prot(prot)); } + +void *pkvm_iommu_donate_pages(u8 order) +{ + return hyp_alloc_pages(&iommu_pages_pool, order); +} + +void pkvm_iommu_reclaim_pages(void *ptr) +{ + hyp_put_page(&iommu_pages_pool, ptr); +} diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index c6698ecea4a2..39d59ff30f34 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -23,6 +23,8 @@ unsigned long hyp_nr_cpus; +unsigned int hyp_kvm_iommu_pages; + #define hyp_percpu_size ((unsigned long)__per_cpu_end - \ (unsigned long)__per_cpu_start) @@ -34,6 +36,7 @@ static void *selftest_base; static void *ffa_proxy_pages; static struct kvm_pgtable_mm_ops pkvm_pgtable_mm_ops; static struct hyp_pool hpool; +static void *iommu_base; static int divide_memory_pool(void *virt, unsigned long size) { @@ -71,6 +74,12 @@ static int divide_memory_pool(void *virt, unsigned long size) if (!ffa_proxy_pages) return -ENOMEM; + if (hyp_kvm_iommu_pages) { + iommu_base = hyp_early_alloc_contig(hyp_kvm_iommu_pages); + if (!iommu_base) + return -ENOMEM; + } + return 0; } @@ -339,7 +348,7 @@ void __noreturn __pkvm_init_finalise(void) * that would be leaked if the hypervisor fails after as there is no * remove_iommu_driver() ops at the moment, that can be added later if necessary. */ - ret = pkvm_iommu_init(); + ret = pkvm_iommu_init(iommu_base, hyp_kvm_iommu_pages); if (ret) goto out; diff --git a/arch/arm64/kvm/iommu.c b/arch/arm64/kvm/iommu.c index 6492285012a4..c80663b506c0 100644 --- a/arch/arm64/kvm/iommu.c +++ b/arch/arm64/kvm/iommu.c @@ -7,10 +7,11 @@ #include extern struct pkvm_iommu_ops *kvm_nvhe_sym(pkvm_iommu_ops); +extern unsigned int kvm_nvhe_sym(hyp_kvm_iommu_pages); static DEFINE_MUTEX(pkvm_iommu_reg_lock); -int __init pkvm_iommu_register_driver(struct pkvm_iommu_ops *hyp_ops) +int pkvm_iommu_register_driver(struct pkvm_iommu_ops *hyp_ops, unsigned int nr_pages) { guard(mutex)(&pkvm_iommu_reg_lock); @@ -20,6 +21,45 @@ int __init pkvm_iommu_register_driver(struct pkvm_iommu_ops *hyp_ops) if (kvm_nvhe_sym(pkvm_iommu_ops)) return -EBUSY; + /* See kvm_iommu_pages() */ + if (nr_pages > kvm_nvhe_sym(hyp_kvm_iommu_pages)) { + kvm_err("IOMMU pool needs 0x%x pages, check kvm-arm.iommu_pgt_mem", nr_pages); + return -ENOMEM; + } + kvm_nvhe_sym(pkvm_iommu_ops) = hyp_ops; return 0; } + +unsigned int pkvm_iommu_pages(void) +{ + /* + * This is used very early during setup_arch() before any initcalls + * or any drivers are registered. + * This value is set by a command line option. + * Later, when the driver is registered, it will pass the number + * pages needed for it's page tables, if it was more than what + * the system has already allocated, it will fail registration. + */ + return kvm_nvhe_sym(hyp_kvm_iommu_pages); +} + +/* Number of pages to reserve for iommu pool*/ +static int __init early_iommu_pgt_mem(char *arg) +{ + unsigned long long requested_size; + + if (!arg) + return -EINVAL; + + requested_size = memparse(arg, NULL); + + if (requested_size > UINT_MAX) { + kvm_err("kvm-arm.iommu_pgt_mem is too large\n"); + return -EINVAL; + } + + kvm_nvhe_sym(hyp_kvm_iommu_pages) = requested_size >> PAGE_SHIFT; + return 0; +} +early_param("kvm-arm.iommu_pgt_mem", early_iommu_pgt_mem); diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c index 053e4f733e4b..72ca1393c504 100644 --- a/arch/arm64/kvm/pkvm.c +++ b/arch/arm64/kvm/pkvm.c @@ -63,6 +63,7 @@ void __init kvm_hyp_reserve(void) hyp_mem_pages += hyp_vmemmap_pages(STRUCT_HYP_PAGE_SIZE); hyp_mem_pages += pkvm_selftest_pages(); hyp_mem_pages += hyp_ffa_proxy_pages(); + hyp_mem_pages += pkvm_iommu_pages(); /* * Try to allocate a PMD-aligned region to reduce TLB pressure once -- 2.55.0.141.g00534a21ce-goog