From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 02F14C25B76 for ; Wed, 5 Jun 2024 13:40:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=EGpX8YVeTT5BFUIIT3aBkXlmRfy9jdcaejH0uqd+EhI=; b=ZfIlC4zjJxTMgc r00k7ZBS5YocOzOKcka82P49QrwwPts1rwNBdUoIwINHkl2I61BQ0G+Pd/5JrwVZkiBAjq/GWm2vE XbQoSycufr7DQZ7x53jPBXE6YPup589R3m3WRFSDYOWwsw6wuHDlNayXfYmrF4M2lvbnmHacQgQUX D1DTSfNcWUpsmTIKW3o1UnrwIOFOYFI8mxrds8nuyCSoZbheAhgXgeNFDTEbCiuLpbv+5UKU4vfN1 RUl5lSR3cZQ5gO9W79MfpNcqpjXBEtIBGVaBiLyz8FZM27WMtrO45JHMk7Pjq4bfQY32piKnFvofk F/N2h4lkDLQKkxIOhnEw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sEqrl-00000006IJp-22E5; Wed, 05 Jun 2024 13:39:49 +0000 Received: from sin.source.kernel.org ([2604:1380:40e1:4800::1]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sEqrh-00000006IFG-1L1P for linux-arm-kernel@lists.infradead.org; Wed, 05 Jun 2024 13:39:48 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 450ECCE0E96; Wed, 5 Jun 2024 13:39:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B14AFC3277B; Wed, 5 Jun 2024 13:39:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1717594781; bh=RhsbuZVJInfZKJ146CqaCAq41JrtoARkqm29fDz9dZ4=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=nfAtTCtxIlc4xYcYU7jXbW1LvVi11nQrvwLpigNxFCsr9dh6bVkHYar02+guVHCPr xTaMYwGcMAWji01bc3DUCqjpZCm4YMn6pTNAZd39HXAGP1igw/uJT3wsydZtI7d22e BhneFmyvWTLuwUvbVCxhGuGsNwOXpwWGQRnFw2Gxlqxz1wRFfsH2kX0lTHZaSEujDV ugtg1itATs/7CSwh01muDgT6y/zssKQl+9It+QFeNJefuV9AF5C+QMyX9cBYqaagqo dWEBqkI//IdijEuk/AKcFhqILiAYt5zp9oyd9/nOQjnJIGNUfh3EHDY1lgvZ09KjTc oVdXkf1xau+1Q== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1sEqrb-000xii-HA; Wed, 05 Jun 2024 14:39:39 +0100 Date: Wed, 05 Jun 2024 14:39:39 +0100 Message-ID: <86a5jzld9g.wl-maz@kernel.org> From: Marc Zyngier To: Steven Price Cc: kvm@vger.kernel.org, kvmarm@lists.linux.dev, Catalin Marinas , Will Deacon , James Morse , Oliver Upton , Suzuki K Poulose , Zenghui Yu , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Joey Gouly , Alexandru Elisei , Christoffer Dall , Fuad Tabba , linux-coco@lists.linux.dev, Ganapatrao Kulkarni Subject: Re: [PATCH v3 12/14] arm64: realm: Support nonsecure ITS emulation shared In-Reply-To: <20240605093006.145492-13-steven.price@arm.com> References: <20240605093006.145492-1-steven.price@arm.com> <20240605093006.145492-13-steven.price@arm.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.2 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: steven.price@arm.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev, catalin.marinas@arm.com, will@kernel.org, james.morse@arm.com, oliver.upton@linux.dev, suzuki.poulose@arm.com, yuzenghui@huawei.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, joey.gouly@arm.com, alexandru.elisei@arm.com, christoffer.dall@arm.com, tabba@google.com, linux-coco@lists.linux.dev, gankulkarni@os.amperecomputing.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240605_063945_769651_67F0B4B1 X-CRM114-Status: GOOD ( 37.16 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The subject line is... odd. I'd expect something like: "irqchip/gic-v3-its: Share ITS tables with a non-trusted hypervisor" because nothing here should be CCA specific. On Wed, 05 Jun 2024 10:30:04 +0100, Steven Price wrote: > > Within a realm guest the ITS is emulated by the host. This means the > allocations must have been made available to the host by a call to > set_memory_decrypted(). Introduce an allocation function which performs > this extra call. This doesn't mention that this patch radically changes the allocation of some tables. > > Co-developed-by: Suzuki K Poulose > Signed-off-by: Suzuki K Poulose > Signed-off-by: Steven Price > --- > Changes since v2: > * Drop 'shared' from the new its_xxx function names as they are used > for non-realm guests too. > * Don't handle the NUMA_NO_NODE case specially - alloc_pages_node() > should do the right thing. > * Drop a pointless (void *) cast. > --- > drivers/irqchip/irq-gic-v3-its.c | 90 ++++++++++++++++++++++++-------- > 1 file changed, 67 insertions(+), 23 deletions(-) > > diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c > index 40ebf1726393..ca72f830f4cc 100644 > --- a/drivers/irqchip/irq-gic-v3-its.c > +++ b/drivers/irqchip/irq-gic-v3-its.c > @@ -18,6 +18,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -27,6 +28,7 @@ > #include > #include > #include > +#include > #include > #include > > @@ -163,6 +165,7 @@ struct its_device { > struct its_node *its; > struct event_lpi_map event_map; > void *itt; > + u32 itt_order; > u32 nr_ites; > u32 device_id; > bool shared; > @@ -198,6 +201,30 @@ static DEFINE_IDA(its_vpeid_ida); > #define gic_data_rdist_rd_base() (gic_data_rdist()->rd_base) > #define gic_data_rdist_vlpi_base() (gic_data_rdist_rd_base() + SZ_128K) > > +static struct page *its_alloc_pages_node(int node, gfp_t gfp, > + unsigned int order) > +{ > + struct page *page; > + > + page = alloc_pages_node(node, gfp, order); > + > + if (page) > + set_memory_decrypted((unsigned long)page_address(page), > + 1 << order); Please use BIT(order). > + return page; > +} > + > +static struct page *its_alloc_pages(gfp_t gfp, unsigned int order) > +{ > + return its_alloc_pages_node(NUMA_NO_NODE, gfp, order); > +} > + > +static void its_free_pages(void *addr, unsigned int order) > +{ > + set_memory_encrypted((unsigned long)addr, 1 << order); > + free_pages((unsigned long)addr, order); > +} > + > /* > * Skip ITSs that have no vLPIs mapped, unless we're on GICv4.1, as we > * always have vSGIs mapped. > @@ -2212,7 +2239,8 @@ static struct page *its_allocate_prop_table(gfp_t gfp_flags) > { > struct page *prop_page; > > - prop_page = alloc_pages(gfp_flags, get_order(LPI_PROPBASE_SZ)); > + prop_page = its_alloc_pages(gfp_flags, > + get_order(LPI_PROPBASE_SZ)); > if (!prop_page) > return NULL; > > @@ -2223,8 +2251,8 @@ static struct page *its_allocate_prop_table(gfp_t gfp_flags) > > static void its_free_prop_table(struct page *prop_page) > { > - free_pages((unsigned long)page_address(prop_page), > - get_order(LPI_PROPBASE_SZ)); > + its_free_pages(page_address(prop_page), > + get_order(LPI_PROPBASE_SZ)); > } > > static bool gic_check_reserved_range(phys_addr_t addr, unsigned long size) > @@ -2346,7 +2374,8 @@ static int its_setup_baser(struct its_node *its, struct its_baser *baser, > order = get_order(GITS_BASER_PAGES_MAX * psz); > } > > - page = alloc_pages_node(its->numa_node, GFP_KERNEL | __GFP_ZERO, order); > + page = its_alloc_pages_node(its->numa_node, > + GFP_KERNEL | __GFP_ZERO, order); > if (!page) > return -ENOMEM; > > @@ -2359,7 +2388,7 @@ static int its_setup_baser(struct its_node *its, struct its_baser *baser, > /* 52bit PA is supported only when PageSize=64K */ > if (psz != SZ_64K) { > pr_err("ITS: no 52bit PA support when psz=%d\n", psz); > - free_pages((unsigned long)base, order); > + its_free_pages(base, order); > return -ENXIO; > } > > @@ -2415,7 +2444,7 @@ static int its_setup_baser(struct its_node *its, struct its_baser *baser, > pr_err("ITS@%pa: %s doesn't stick: %llx %llx\n", > &its->phys_base, its_base_type_string[type], > val, tmp); > - free_pages((unsigned long)base, order); > + its_free_pages(base, order); > return -ENXIO; > } > > @@ -2554,8 +2583,8 @@ static void its_free_tables(struct its_node *its) > > for (i = 0; i < GITS_BASER_NR_REGS; i++) { > if (its->tables[i].base) { > - free_pages((unsigned long)its->tables[i].base, > - its->tables[i].order); > + its_free_pages(its->tables[i].base, > + its->tables[i].order); > its->tables[i].base = NULL; > } > } > @@ -2821,7 +2850,8 @@ static bool allocate_vpe_l2_table(int cpu, u32 id) > > /* Allocate memory for 2nd level table */ > if (!table[idx]) { > - page = alloc_pages(GFP_KERNEL | __GFP_ZERO, get_order(psz)); > + page = its_alloc_pages(GFP_KERNEL | __GFP_ZERO, > + get_order(psz)); > if (!page) > return false; > > @@ -2940,7 +2970,8 @@ static int allocate_vpe_l1_table(void) > > pr_debug("np = %d, npg = %lld, psz = %d, epp = %d, esz = %d\n", > np, npg, psz, epp, esz); > - page = alloc_pages(GFP_ATOMIC | __GFP_ZERO, get_order(np * PAGE_SIZE)); > + page = its_alloc_pages(GFP_ATOMIC | __GFP_ZERO, > + get_order(np * PAGE_SIZE)); > if (!page) > return -ENOMEM; > > @@ -2986,8 +3017,8 @@ static struct page *its_allocate_pending_table(gfp_t gfp_flags) > { > struct page *pend_page; > > - pend_page = alloc_pages(gfp_flags | __GFP_ZERO, > - get_order(LPI_PENDBASE_SZ)); > + pend_page = its_alloc_pages(gfp_flags | __GFP_ZERO, > + get_order(LPI_PENDBASE_SZ)); > if (!pend_page) > return NULL; > > @@ -2999,7 +3030,7 @@ static struct page *its_allocate_pending_table(gfp_t gfp_flags) > > static void its_free_pending_table(struct page *pt) > { > - free_pages((unsigned long)page_address(pt), get_order(LPI_PENDBASE_SZ)); > + its_free_pages(page_address(pt), get_order(LPI_PENDBASE_SZ)); > } > > /* > @@ -3334,8 +3365,9 @@ static bool its_alloc_table_entry(struct its_node *its, > > /* Allocate memory for 2nd level table */ > if (!table[idx]) { > - page = alloc_pages_node(its->numa_node, GFP_KERNEL | __GFP_ZERO, > - get_order(baser->psz)); > + page = its_alloc_pages_node(its->numa_node, > + GFP_KERNEL | __GFP_ZERO, > + get_order(baser->psz)); > if (!page) > return false; > > @@ -3418,7 +3450,9 @@ static struct its_device *its_create_device(struct its_node *its, u32 dev_id, > unsigned long *lpi_map = NULL; > unsigned long flags; > u16 *col_map = NULL; > + struct page *page; > void *itt; > + int itt_order; > int lpi_base; > int nr_lpis; > int nr_ites; > @@ -3430,7 +3464,6 @@ static struct its_device *its_create_device(struct its_node *its, u32 dev_id, > if (WARN_ON(!is_power_of_2(nvecs))) > nvecs = roundup_pow_of_two(nvecs); > > - dev = kzalloc(sizeof(*dev), GFP_KERNEL); > /* > * Even if the device wants a single LPI, the ITT must be > * sized as a power of two (and you need at least one bit...). > @@ -3438,7 +3471,16 @@ static struct its_device *its_create_device(struct its_node *its, u32 dev_id, > nr_ites = max(2, nvecs); > sz = nr_ites * (FIELD_GET(GITS_TYPER_ITT_ENTRY_SIZE, its->typer) + 1); > sz = max(sz, ITS_ITT_ALIGN) + ITS_ITT_ALIGN - 1; > - itt = kzalloc_node(sz, GFP_KERNEL, its->numa_node); > + itt_order = get_order(sz); > + page = its_alloc_pages_node(its->numa_node, > + GFP_KERNEL | __GFP_ZERO, > + itt_order); So we go from an allocation that was so far measured in *bytes* to something that is now at least a page. Per device. This seems a bit excessive to me, specially when it isn't conditioned on anything and is now imposed on all platforms, including the non-CCA systems (which are exactly 100% of the machines). Another thing is that if we go with page alignment, then the 256 byte alignment can obviously be removed everywhere (hint: MAPD needs to change). Thanks, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel