From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9DA9EC27C55 for ; Mon, 10 Jun 2024 17:46:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=lMhxl7P+xbaRgo1LK+6FgrbY1qnkM9hXHSMPCR56QSY=; b=lyIIYHtpVQy0i0 BoMLfQGYtU9/lzJIuKLVK9keDqIQapslme5a/CCHDdWDafvJpB1AhodP6tduD/lUSsQtsJhrkwwnb /qYmt8uPiiQ9P68yrCLmlgTw5NPd3/UrOC2QKYsI5QlA7I+qdTENNF4h4YBRpz9TzeNiUBkXqEtJ9 NKjGs88a9IsXL/YDeKe6RIhzQl4vDe3ZrGScsvijpRbQ6KuOukOJBdoBrqaRyWIm+mTMTiqWU1TvB jFZCP8JZV/7Fxa3pQkybFwTsgAWmqlJYEmqddb/spOUxSPWnmobbk28wKvPXC2r/8YIBbiM1hgt0C 5oHaoenFB2IaHfeqanJg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sGj62-000000061YX-3Uv8; Mon, 10 Jun 2024 17:46:18 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sGj5x-000000061Xx-0B20 for linux-arm-kernel@lists.infradead.org; Mon, 10 Jun 2024 17:46:16 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 020E760B1B; Mon, 10 Jun 2024 17:46:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6CB9FC2BBFC; Mon, 10 Jun 2024 17:46:08 +0000 (UTC) Date: Mon, 10 Jun 2024 18:46:06 +0100 From: Catalin Marinas To: Michael Kelley Cc: Steven Price , "kvm@vger.kernel.org" , "kvmarm@lists.linux.dev" , Marc Zyngier , Will Deacon , James Morse , Oliver Upton , Suzuki K Poulose , Zenghui Yu , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , Joey Gouly , Alexandru Elisei , Christoffer Dall , Fuad Tabba , "linux-coco@lists.linux.dev" , Ganapatrao Kulkarni Subject: Re: [PATCH v3 00/14] arm64: Support for running as a guest in Arm CCA Message-ID: References: <20240605093006.145492-1-steven.price@arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240610_104614_636362_11ACC65D X-CRM114-Status: GOOD ( 29.89 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Jun 10, 2024 at 05:03:44PM +0000, Michael Kelley wrote: > From: Catalin Marinas Sent: Monday, June 10, 2024 3:34 AM > > I wonder whether something like __GFP_DECRYPTED could be used to get > > shared memory from the allocation time and avoid having to change the > > vmalloc() ranges. This way functions like netvsc_init_buf() would get > > decrypted memory from the start and vmbus_establish_gpadl() would not > > need to call set_memory_decrypted() on a vmalloc() address. > > I would not have any conceptual objections to such an approach. But I'm > certainly not an expert in that area so I'm not sure what it would take > to make that work for vmalloc(). I presume that __GFP_DECRYPTED > should also work for kmalloc()? > > I've seen the separate discussion about a designated pool of decrypted > memory, to avoid always allocating a new page and decrypting when a > smaller allocation is sufficient. If such a pool could also work for page size > or larger allocations, it would have the additional benefit of concentrating > decrypted allocations in fewer 2 Meg large pages vs. scattering wherever > and forcing the break-up of more large page mappings in the direct map. Yeah, my quick, not fully tested hack here: https://lore.kernel.org/linux-arm-kernel/ZmNJdSxSz-sYpVgI@arm.com/ It's the underlying page allocator that gives back decrypted pages when the flag is passed, so it should work for alloc_pages() and friends. The kmalloc() changes only ensure that we have separate caches for this memory and they are not merged. It needs some more work on kmem_cache, maybe introducing a SLAB_DECRYPTED flag as well as not to rely on the GFP flag. For vmalloc(), we'd need a pgprot_decrypted() macro to ensure the decrypted pages are marked with the appropriate attributes (arch specific), otherwise it's fairly easy to wire up if alloc_pages() gives back decrypted memory. > I'll note that netvsc devices can be added or removed from a running VM. > The vmalloc() memory allocated by netvsc_init_buf() can be freed, and/or > additional calls to netvsc_init_buf() can be made at any time -- they aren't > limited to initial Linux boot. So the mechanism for getting decrypted > memory at allocation time must be reasonably dynamic. I think the above should work. But, of course, we'd have to get this past the mm maintainers, it's likely that I missed something. > Rejecting vmalloc() addresses may work for the moment -- I don't know > when CCA guests might be tried on Hyper-V. The original SEV-SNP and TDX > work started that way as well. :-) Handling the vmalloc() case was added > later, though I think on x86 the machinery to also flip all the alias PTEs was > already mostly or completely in place, probably for other reasons. So > fixing the vmalloc() case was more about not assuming that the underlying > physical address range is contiguous. Instead, each page must be processed > independently, which was straightforward. There may be a slight performance impact but I guess that's not on a critical path. Walking the page tables and changing the vmalloc ptes should be fine but for each page, we'd have to break the linear map, flush the TLBs, re-create the linear map. Those TLBs may become a bottleneck, especially on hardware with lots of CPUs and the microarchitecture. Note that even with a __GFP_DECRYPTED attribute, we'd still need to go for individual pages in the linear map. -- Catalin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel