From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3616DC433EF for ; Thu, 9 Dec 2021 09:43:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=xcaIp+B9Dt/uwIVKjGsxBlBc866aWq+6B4XBmdRC3lE=; b=OZ1Ju1YvwT2y7a k0IZHSYlA9ru+puSAl2IiCmZU04jLz/lI0FF4ujSRWKr5wzbfWIF9RtGvSQw3YejwVKn31UlKSdUW ZGgFDYdXixK6cJsBUnZ1FnBTk6W4qgj/FkDoC0x/egcYwOT2mgYBRlHyO1LMAPJ4F+h3YGblqaWy3 B3qnBo/zsgf1O82jJb2zMQ0ss/k/S6f+odLeVirKxC/tUl0J8pJkcRI2PuLf9L3iHXxSE51CO1/bh NwU1XikdWtdly1ZqIXFVGdviwoaNWc+ZUo0YhL+Zi3CKgaNjbgSvrlXDL7NAKu7Nx6uUcojIQdpNx YmcqVV4CW29MyEfqZqgA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mvFvf-00FSnP-Lf; Thu, 09 Dec 2021 09:41:31 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mvFvc-00FSml-Bl for linux-arm-kernel@lists.infradead.org; Thu, 09 Dec 2021 09:41:30 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id C1FB6B82410; Thu, 9 Dec 2021 09:41:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9886CC004DD; Thu, 9 Dec 2021 09:41:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1639042885; bh=JQV1xRE97nLSlxPmaTanqHNuBvMOTbt/bOqT4iPFIQc=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=RvpIfzO0iU+in13N1sZHHx+SuSFXbskZKFkoQacEvMaXiAAXVVzVVtEAvyRzD0npB lXKLf55lNUsNqeRKQFtX8PwWRGZB+aJIoj0yfEm30LTdW1yzP9VCn0YpBaiQmDiM/6 RIIhKUFur0wvlT6ITMtbDM2JUTyCWztQE2r9KR7lo2eUN5Xb3315d8YpAHGY7GPPHC rfaBcwDgVCEaTrFguiR0DR9PNzuJANU24eyJ9p7U5cRAuA4jsbVozDezxMYrlElik8 cVdCH1pH+YEe06kGxizSwScZuLCDAr8bRJi0e5+BRb7C9Rl4+1ysGwoed99WAXjQmx AlR4IeHfgy/lA== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1mvFvX-00Az42-K7; Thu, 09 Dec 2021 09:41:23 +0000 Date: Thu, 09 Dec 2021 09:41:23 +0000 Message-ID: <877dce15fg.wl-maz@kernel.org> From: Marc Zyngier To: Will Deacon Cc: linux-arm-kernel@lists.infradead.org, Thomas Gleixner Subject: Re: [PATCH] irqchip/gic-v3-its: Mark some in-memory data structures as 'decrypted' In-Reply-To: <20211208155916.681-1-will@kernel.org> References: <20211208155916.681-1-will@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: will@kernel.org, linux-arm-kernel@lists.infradead.org, tglx@linutronix.de X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211209_014128_710440_7AAACC76 X-CRM114-Status: GOOD ( 38.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, 08 Dec 2021 15:59:16 +0000, Will Deacon wrote: > > The GICv3 ITS driver allocates memory for its tables using alloc_pages() > and performs explicit cache maintenance if necessary. On systems such > as those running pKVM, where the memory encryption API is implemented, > memory shared with the ITS must first be transitioned to the "decrypted" > state, as it would be if allocated via the DMA API. > > Allow pKVM guests to interact with an ITS emulation by ensuring that the > shared pages are decrypted at the point of allocation and encrypted > again upon free(). > > Cc: Thomas Gleixner > Cc: Marc Zyngier > Signed-off-by: Will Deacon > --- > > Although pKVM doesn't yet expose share/unshare hypercalls to the guest, > this change is agnostic of the hypervisor and could be queued > independently as it has no functional impact when the memory encryption > API is not implemented. > > drivers/irqchip/irq-gic-v3-its.c | 40 ++++++++++++++++++++++++++------ > 1 file changed, 33 insertions(+), 7 deletions(-) > > diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c > index eb0882d15366..4559b8dfb9bc 100644 > --- a/drivers/irqchip/irq-gic-v3-its.c > +++ b/drivers/irqchip/irq-gic-v3-its.c > @@ -27,6 +27,7 @@ > #include > #include > #include > +#include > #include > #include > > @@ -2166,6 +2167,7 @@ static void gic_reset_prop_table(void *va) > > /* Make sure the GIC will observe the written configuration */ > gic_flush_dcache_to_poc(va, LPI_PROPBASE_SZ); > + set_memory_decrypted((unsigned long)va, LPI_PROPBASE_SZ >> PAGE_SHIFT); > } > > static struct page *its_allocate_prop_table(gfp_t gfp_flags) > @@ -2183,8 +2185,10 @@ static struct page *its_allocate_prop_table(gfp_t gfp_flags) > > static void its_free_prop_table(struct page *prop_page) > { > - free_pages((unsigned long)page_address(prop_page), > - get_order(LPI_PROPBASE_SZ)); > + unsigned long va = (unsigned long)page_address(prop_page); > + > + set_memory_encrypted(va, LPI_PROPBASE_SZ >> PAGE_SHIFT); > + free_pages(va, get_order(LPI_PROPBASE_SZ)); > } > > static bool gic_check_reserved_range(phys_addr_t addr, unsigned long size) > @@ -2377,6 +2381,8 @@ static int its_setup_baser(struct its_node *its, struct its_baser *baser, > return -ENXIO; > } > > + set_memory_decrypted((unsigned long)base, > + PAGE_ORDER_TO_SIZE(order) >> PAGE_SHIFT); > baser->order = order; > baser->base = base; > baser->psz = psz; > @@ -2512,8 +2518,12 @@ static void its_free_tables(struct its_node *its) > > for (i = 0; i < GITS_BASER_NR_REGS; i++) { > if (its->tables[i].base) { > - free_pages((unsigned long)its->tables[i].base, > - its->tables[i].order); > + unsigned long base = (unsigned long)its->tables[i].base; > + u32 order = its->tables[i].order; > + u32 npages = PAGE_ORDER_TO_SIZE(order) >> PAGE_SHIFT; > + > + set_memory_encrypted(base, npages); > + free_pages(base, order); > its->tables[i].base = NULL; > } > } > @@ -2934,6 +2944,7 @@ static int its_alloc_collections(struct its_node *its) > static struct page *its_allocate_pending_table(gfp_t gfp_flags) > { > struct page *pend_page; > + void *va; > > pend_page = alloc_pages(gfp_flags | __GFP_ZERO, > get_order(LPI_PENDBASE_SZ)); > @@ -2941,14 +2952,19 @@ static struct page *its_allocate_pending_table(gfp_t gfp_flags) > return NULL; > > /* Make sure the GIC will observe the zero-ed page */ > - gic_flush_dcache_to_poc(page_address(pend_page), LPI_PENDBASE_SZ); > + va = page_address(pend_page); > + gic_flush_dcache_to_poc(va, LPI_PENDBASE_SZ); > + set_memory_decrypted((unsigned long)va, LPI_PENDBASE_SZ >> PAGE_SHIFT); > > return pend_page; > } > > static void its_free_pending_table(struct page *pt) > { > - free_pages((unsigned long)page_address(pt), get_order(LPI_PENDBASE_SZ)); > + unsigned long va = (unsigned long)page_address(pt); > + > + set_memory_encrypted(va, LPI_PENDBASE_SZ >> PAGE_SHIFT); > + free_pages(va, get_order(LPI_PENDBASE_SZ)); > } > > /* > @@ -3268,14 +3284,20 @@ static bool its_alloc_table_entry(struct its_node *its, > > /* Allocate memory for 2nd level table */ > if (!table[idx]) { > + void *l2addr; > + > page = alloc_pages_node(its->numa_node, GFP_KERNEL | __GFP_ZERO, > get_order(baser->psz)); > if (!page) > return false; > > + l2addr = page_address(page); > + set_memory_decrypted((unsigned long)l2addr, > + baser->psz >> PAGE_SHIFT); > + > /* Flush Lvl2 table to PoC if hw doesn't support coherency */ > if (!(baser->val & GITS_BASER_SHAREABILITY_MASK)) > - gic_flush_dcache_to_poc(page_address(page), baser->psz); > + gic_flush_dcache_to_poc(l2addr, baser->psz); > > table[idx] = cpu_to_le64(page_to_phys(page) | GITS_BASER_VALID); > > @@ -5043,6 +5065,8 @@ static int __init its_probe_one(struct resource *res, > its->fwnode_handle = handle; > its->get_msi_base = its_irq_get_msi_base; > its->msi_domain_flags = IRQ_DOMAIN_FLAG_MSI_REMAP; > + set_memory_decrypted((unsigned long)its->cmd_base, > + ITS_CMD_QUEUE_SZ >> PAGE_SHIFT); > > its_enable_quirks(its); > > @@ -5099,6 +5123,8 @@ static int __init its_probe_one(struct resource *res, > out_free_tables: > its_free_tables(its); > out_free_cmd: > + set_memory_encrypted((unsigned long)its->cmd_base, > + ITS_CMD_QUEUE_SZ >> PAGE_SHIFT); > free_pages((unsigned long)its->cmd_base, get_order(ITS_CMD_QUEUE_SZ)); > out_unmap_sgir: > if (its->sgir_base) This misses the subtly hidden per-endpoint ITT allocation (see its_create_device()), which is the only one that is not a full page allocation. You could of course upgrade it to a full page, but this is going to waste a lot of memory on systems with large page sizes, lots of devices, and only few interrupts per device. I guess that this would be acceptable on systems such as those currently targeted by pKVM, but you'd need to make it a runtime decision. Another possibility would be to build a custom allocator for this, but this feels like a massive undertaking. Maybe that's the push we needed to switch the ITS driver to be a full-blown platform driver instead of a side hack. M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel