From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ej1-f44.google.com (mail-ej1-f44.google.com [209.85.218.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CAE4B1FAE for ; Fri, 28 Oct 2022 09:28:21 +0000 (UTC) Received: by mail-ej1-f44.google.com with SMTP id k2so11636590ejr.2 for ; Fri, 28 Oct 2022 02:28:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=o1dEA6QDjwPI5Lqqp4PnQXLAiiJ6pk7DSaf8g2e6LNg=; b=rpEWC7xCFV/pUAbhRF1OBflH1ReSXldbCEKcEiVXMZLJXp8WrR3mYW7CBPs9qGTxSM UGFff4Lu0d14TN77FyLd0S9EnSY0LwHdWlyjmpIw5Uh3bqrlOxRssh9m5P40Wz4xRloS OLJbpdgCIaF2TJ1p94MYswxOseYLaRD3oo1680naj5NFO8VNcKuVHpXns4R/T0kzSnSl 6sG2RtN/JJuOujfjwbAwgoeAyXZeOCk7QVT+a7GTgDbEMqsXhoFR3DCAWpYLwzhnx/3T 862fYsgfYJ1UwFXWPaEkP6iNe3jaIGVJaxoW84T8ZgKGeEwDHbbx+wTlIeFXqFzuv6Nl iVwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=o1dEA6QDjwPI5Lqqp4PnQXLAiiJ6pk7DSaf8g2e6LNg=; b=bJNoZ4kxzpmlWSawHJh5d1DzGr7d/hr7HpWuK33sClrMlGQ8riE0Ta0RMWQLAM7QCa v+0nkaQjCd5vuL7Ju/Km6/D0HVd6IkEunaqo1YU3/5AYt7MDl1nnp4A3raVsadiilLSh m33gKh+gqQ/uMJ66U5vtKnQmP4x2nQ1mEmJOEw6oIZ2fBi6DC6G3WCZe9SyaIHTjE2bE 4ghTS98NxEnvhKwsLiCcJkZvqVIElhfmnI0DDkfLxymGo2SmfiSQrvoEV7BZe2Q34KDf nKaJ3uHuUICgTK3PV77wKg00bXT9MNsE7wl8XjEUOKK7KWVmBS6bK5Scfg8rNnqXDykS 3XNg== X-Gm-Message-State: ACrzQf0kJf1mAcl1GIjRsKsX019FBAJn0cs4WZ/K7gRlKb6e8FWTsTYN 2TFqtlS26zuuii9RGnx2u2wG5w== X-Google-Smtp-Source: AMsMyM7L7iJ/b3nYLQTZha1zpDQVJblHC8G85NArgRqyqWymZq33WRAiEkkyQmt3H6qTWyPBQb1TDQ== X-Received: by 2002:a17:907:980e:b0:78d:b6ea:25b3 with SMTP id ji14-20020a170907980e00b0078db6ea25b3mr45999011ejc.98.1666949299844; Fri, 28 Oct 2022 02:28:19 -0700 (PDT) Received: from google.com (64.227.90.34.bc.googleusercontent.com. [34.90.227.64]) by smtp.gmail.com with ESMTPSA id 24-20020a170906311800b007933047f930sm1909751ejx.157.2022.10.28.02.28.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Oct 2022 02:28:19 -0700 (PDT) Date: Fri, 28 Oct 2022 09:28:15 +0000 From: Quentin Perret To: Oliver Upton Cc: Will Deacon , kvmarm@lists.linux.dev, Sean Christopherson , Vincent Donnefort , Alexandru Elisei , Catalin Marinas , Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= , James Morse , Chao Peng , Suzuki K Poulose , Mark Rutland , Fuad Tabba , Marc Zyngier , kernel-team@android.com, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH v5 02/25] KVM: arm64: Allow attaching of non-coalescable pages to a hyp pool Message-ID: References: <20221020133827.5541-1-will@kernel.org> <20221020133827.5541-3-will@kernel.org> Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Hey Oliver, On Friday 28 Oct 2022 at 00:17:40 (+0000), Oliver Upton wrote: > On Thu, Oct 20, 2022 at 02:38:04PM +0100, Will Deacon wrote: > > From: Quentin Perret > > > > All the contiguous pages used to initialize a 'struct hyp_pool' are > > considered coalescable, which means that the hyp page allocator will > > actively try to merge them with their buddies on the hyp_put_page() path. > > However, using hyp_put_page() on a page that is not part of the inital > > memory range given to a hyp_pool() is currently unsupported. > > > > In order to allow dynamically extending hyp pools at run-time, add a > > check to __hyp_attach_page() to allow inserting 'external' pages into > > the free-list of order 0. This will be necessary to allow lazy donation > > of pages from the host to the hypervisor when allocating guest stage-2 > > page-table pages at EL2. > > Is it ever going to be the case that we wind up mixing static and > dynamic memory within the same buddy allocator? Reading ahead a bit it > would seem pKVM uses separate allocators (i.e. pkvm_hyp_vm::pool for > donated memory) but just wanted to make sure. So, in the code we have that builds on top of this, yes, but to be frank it's a bit of a corner case. The per-guest pool is initially populated with a physically contiguous memory range that covers the need to allocate the guest's stage-2 PGD, which may be up to 16 contiguous pages IIRC. But aside from that, all subsequent allocations will be at order 0 granularity, and those pages are added to the pool dynamically. > I suppose what I'm getting at is the fact that the pool range makes > little sense in this case. Adding a field to hyp_pool describing the > type of pool that it is would make this more readable, such that we know > a pool contains only donated memory, and thus zero order pages should > never be coalesced. Right. In fact I think it would be fair to say that the buddy allocator is overkill for what we need so far. The only high-order allocations we do are for concatenated stage-2 PGDs (host and guest), but these could be easily special-cased, and we'd be left with only page-size allocs for which a simple free list would do just fine. I've considered removing the buddy allocator TBH, but it doesn't really hurt to have it, and it might turn out to be useful in the future (e.g. SMMU support might require high-order allocs IIUC, and the ability to coalesce would come handy then). From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 44383C38A02 for ; Fri, 28 Oct 2022 09:29:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=lukJ6wLaGMLBRt5v1bAfoCICgmBuOtlaprI4FunMvcY=; b=iPNE0xM+e6NPad ycyAZoexotbpPzmmTNczAAvNy6DeqthzHo6mvCjU93d7KnWxWPcQrLB3g30tUgyRazU8PUx+SA2X6 +c8oGr9rYw67h9RB61n2GHkmknP/ivipZ9WyUy5DjfOuzw8yD4O8U3hScGYa7N3kiiFIM9UhSRakd QB6nbTdL2e4naBmyeBZS/N4Zqz9/CoYOUeLWj2pe3EruV0w7tSm0hnYiagktHfIigD3ZP4EsHQnhV 1pG1cBpP8u+ySV9YAXTNxfnUFZG1sBxoIzLvlXfdT+TCNprfl31BlZfOtOax9KND8YMsn152lDucL DA1O9lVsVJlFLRzgXPGA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ooLf9-00GK57-Qi; Fri, 28 Oct 2022 09:28:27 +0000 Received: from mail-ej1-x62c.google.com ([2a00:1450:4864:20::62c]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ooLf4-00GK3P-7z for linux-arm-kernel@lists.infradead.org; Fri, 28 Oct 2022 09:28:26 +0000 Received: by mail-ej1-x62c.google.com with SMTP id f27so11619985eje.1 for ; Fri, 28 Oct 2022 02:28:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=o1dEA6QDjwPI5Lqqp4PnQXLAiiJ6pk7DSaf8g2e6LNg=; b=rpEWC7xCFV/pUAbhRF1OBflH1ReSXldbCEKcEiVXMZLJXp8WrR3mYW7CBPs9qGTxSM UGFff4Lu0d14TN77FyLd0S9EnSY0LwHdWlyjmpIw5Uh3bqrlOxRssh9m5P40Wz4xRloS OLJbpdgCIaF2TJ1p94MYswxOseYLaRD3oo1680naj5NFO8VNcKuVHpXns4R/T0kzSnSl 6sG2RtN/JJuOujfjwbAwgoeAyXZeOCk7QVT+a7GTgDbEMqsXhoFR3DCAWpYLwzhnx/3T 862fYsgfYJ1UwFXWPaEkP6iNe3jaIGVJaxoW84T8ZgKGeEwDHbbx+wTlIeFXqFzuv6Nl iVwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=o1dEA6QDjwPI5Lqqp4PnQXLAiiJ6pk7DSaf8g2e6LNg=; b=kTcfxv8hSkaw5408DPFWeXXE/+GOp1c5htUdOKz7yN4nLeXmnOxSSCVjOzgJ9wzXcq R0GRqgV8d427qptg7xS7p6eAq7BnuALHAXVWrOuFLjrsA3bXkHM+20rY1GXtcTEd7HgF 3uCZUQbAq3FChLAXiQhkuiX/LRKmMMfq2iXlJ/W94gTCL4mLlq9tzvpRPzCG1BcEDayU nuoPgzBJKqGXMvGqsX9aiB1FMFpv81xzJKKwn5DEvVkF8vfcNNKl3HfW0Wtn3iwy3Jef 2UmYM9j+HXZdB5Ffbl/T1E1CeateASe8CxnSeffyryKocsrd0VpdUOnA5eeBGKL5xP23 PWZA== X-Gm-Message-State: ACrzQf2EMNveLUf52EIArFIF0VV6PAlp5Xdj8BOZ0E02CDCNxla8NYkx a3gmi25KUKiKCOd+n0C6zYCLXw== X-Google-Smtp-Source: AMsMyM7L7iJ/b3nYLQTZha1zpDQVJblHC8G85NArgRqyqWymZq33WRAiEkkyQmt3H6qTWyPBQb1TDQ== X-Received: by 2002:a17:907:980e:b0:78d:b6ea:25b3 with SMTP id ji14-20020a170907980e00b0078db6ea25b3mr45999011ejc.98.1666949299844; Fri, 28 Oct 2022 02:28:19 -0700 (PDT) Received: from google.com (64.227.90.34.bc.googleusercontent.com. [34.90.227.64]) by smtp.gmail.com with ESMTPSA id 24-20020a170906311800b007933047f930sm1909751ejx.157.2022.10.28.02.28.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Oct 2022 02:28:19 -0700 (PDT) Date: Fri, 28 Oct 2022 09:28:15 +0000 From: Quentin Perret To: Oliver Upton Cc: Will Deacon , kvmarm@lists.linux.dev, Sean Christopherson , Vincent Donnefort , Alexandru Elisei , Catalin Marinas , Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= , James Morse , Chao Peng , Suzuki K Poulose , Mark Rutland , Fuad Tabba , Marc Zyngier , kernel-team@android.com, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH v5 02/25] KVM: arm64: Allow attaching of non-coalescable pages to a hyp pool Message-ID: References: <20221020133827.5541-1-will@kernel.org> <20221020133827.5541-3-will@kernel.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221028_022822_305483_7ADF9452 X-CRM114-Status: GOOD ( 26.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hey Oliver, On Friday 28 Oct 2022 at 00:17:40 (+0000), Oliver Upton wrote: > On Thu, Oct 20, 2022 at 02:38:04PM +0100, Will Deacon wrote: > > From: Quentin Perret > > > > All the contiguous pages used to initialize a 'struct hyp_pool' are > > considered coalescable, which means that the hyp page allocator will > > actively try to merge them with their buddies on the hyp_put_page() path. > > However, using hyp_put_page() on a page that is not part of the inital > > memory range given to a hyp_pool() is currently unsupported. > > > > In order to allow dynamically extending hyp pools at run-time, add a > > check to __hyp_attach_page() to allow inserting 'external' pages into > > the free-list of order 0. This will be necessary to allow lazy donation > > of pages from the host to the hypervisor when allocating guest stage-2 > > page-table pages at EL2. > > Is it ever going to be the case that we wind up mixing static and > dynamic memory within the same buddy allocator? Reading ahead a bit it > would seem pKVM uses separate allocators (i.e. pkvm_hyp_vm::pool for > donated memory) but just wanted to make sure. So, in the code we have that builds on top of this, yes, but to be frank it's a bit of a corner case. The per-guest pool is initially populated with a physically contiguous memory range that covers the need to allocate the guest's stage-2 PGD, which may be up to 16 contiguous pages IIRC. But aside from that, all subsequent allocations will be at order 0 granularity, and those pages are added to the pool dynamically. > I suppose what I'm getting at is the fact that the pool range makes > little sense in this case. Adding a field to hyp_pool describing the > type of pool that it is would make this more readable, such that we know > a pool contains only donated memory, and thus zero order pages should > never be coalesced. Right. In fact I think it would be fair to say that the buddy allocator is overkill for what we need so far. The only high-order allocations we do are for concatenated stage-2 PGDs (host and guest), but these could be easily special-cased, and we'd be left with only page-size allocs for which a simple free list would do just fine. I've considered removing the buddy allocator TBH, but it doesn't really hurt to have it, and it might turn out to be useful in the future (e.g. SMMU support might require high-order allocs IIUC, and the ability to coalesce would come handy then). _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel