From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1kwjrW-0002lZ-Uh for mharc-grub-devel@gnu.org; Tue, 05 Jan 2021 05:46:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:53942) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kwjrU-0002l7-LJ for grub-devel@gnu.org; Tue, 05 Jan 2021 05:46:48 -0500 Received: from out5-smtp.messagingengine.com ([66.111.4.29]:36081) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kwjrH-0000CM-W5 for grub-devel@gnu.org; Tue, 05 Jan 2021 05:46:48 -0500 Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id AD38D5C00BD; Tue, 5 Jan 2021 05:46:32 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Tue, 05 Jan 2021 05:46:32 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm2; bh=BCt2cNUmSJ1u/2uuTXDj0zSUUyJ CaMYGAMHbZVdUJmw=; b=ZIJ9IgzNnMRo/vplQeKNpb9wE/B/wDFrRycDdHuyuYG Ck39hDquVOM2BFssipFfxXCHp+Lq33/hFtVdEht5zktoJI3kVYLTw8QTjre9xD3q vEv51ODBRTw59rOsWzoMjDKSJJNThIvniM0etp3A8HO82v0ordr2/JUIS9mmrQ76 snIINJIIp9vZOAKk/EoH2MpixItlnBEsurk7A7oaUuV/60EdFQPA9OlbgfDnzF2g 8Ju343AAiYPopVEniVjeGm2FGV3F3zZIcm1aMuX2fCeEdelnIRm/kH2AIPt3wWQU f/B0+ZbFHpPPfJfPmyTWwMjoq9TZ5vIfcHhV8lZhz8g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=BCt2cN UmSJ1u/2uuTXDj0zSUUyJCaMYGAMHbZVdUJmw=; b=R6iHxIrrBJM39WspQGmOnm 0r+n797QWL6EEURRxn1Ur0kQlAryBkqIzgTocf6jAP5K/8CXKZ3Fuo6M6P6lspI/ V7OyPJjLqb56xlgKP2ypgGGxCSc432h+fdqODQCJY8tF/ckGPstboQETP5+T40LB TJC+/fdRn/WXBgnvlcz5mMkqN+djCvGMvTx14VuOkYd/lICSMnsIZdcePFAk/g6K PJgzW2zXwNqAPJb4GHGDHKe12BcTBSLlBS3doJaOKUDzT7Lm4rnEI1T0Dp8A61TM L8zTjZfzSpXq5YQEv7leT7N5vwgFtJc/o8okc1YhVjAXDA7NkQmM0VaaaO0RYm0A == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrvdefhedgudehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeefleekvddvveeuveejtdevleffjeegvdegfefhieetueefvedugefgleevvdehleen ucffohhmrghinhepghhnuhdrohhrghenucfkphepkeelrdduvddrfeeirddvtddvnecuve hluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhs rdhimh X-ME-Proxy: Received: from vm-mail.pks.im (dynamic-089-012-036-202.89.12.pool.telefonica.de [89.12.36.202]) by mail.messagingengine.com (Postfix) with ESMTPA id 28CC2108005B; Tue, 5 Jan 2021 05:46:31 -0500 (EST) Received: from localhost (tanuki [10.192.0.23]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id af5a471c (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 5 Jan 2021 10:46:24 +0000 (UTC) Date: Tue, 5 Jan 2021 11:47:36 +0100 From: Patrick Steinhardt To: The development of GNU GRUB Cc: Hanson Char Subject: Re: [PATCH] Make EFI minimum heap size configurable via --enable-efi-min-heap-mb Message-ID: References: <9B5320C5-BCDE-4EEB-876B-8A0965A488FF@amazon.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="e6BWyWc9bLJ9vkAt" Content-Disposition: inline In-Reply-To: Received-SPF: pass client-ip=66.111.4.29; envelope-from=ps@pks.im; helo=out5-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: grub-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: The development of GNU GRUB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2021 10:46:48 -0000 --e6BWyWc9bLJ9vkAt Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jan 05, 2021 at 10:02:12AM +0100, Paul Menzel wrote: > Dear Hanson, >=20 >=20 > Am 04.01.21 um 20:34 schrieb Char, Hanson via Grub-devel: > > When booted in UEFI mode, Grub would fail to load a ramdisk of size lar= ger than "(total_pages >> 2)" with > >=20 > > "error: out of memory" > >=20 > > (https://git.savannah.gnu.org/cgit/grub.git/tree/grub-core/kern/efi/mm.= c#n616) > >=20 > > This proposed patch adds a new option that can be used to specify the "= EFI min heap size" in MB. For example, > >=20 > > ./configure --with-platform=3Defi --target=3Dx86_64 --enable-efi-m= in-heap-mb=3D600 > >=20 > > The patch has been successfully tested to load large ramdisk with size = that would otherwise fail. >=20 > Thank you for the patch and description. Could you please add it as the= =20 > git commit message, and send a v2, maybe even using `git send-email`? >=20 > > From 9bd4220a6b0fe2d49c6aed347f5d2bbff5fb2d8d Mon Sep 17 00:00:00 2001 > > From: Hanson Char > > Date: Thu, 31 Dec 2020 12:49:33 -0800 > > Subject: [PATCH] Make EFI minimum heap size configurable via --enable-e= fi-min-heap-mb >=20 > Please excuse my ignorance, but I do not fully understand the problem=20 > and solution yet. What has the minimum heap size to do with the problem,= =20 > and why does increasing it fixes it? The memory allocator is quite limited in that it can only allocate a given maximum amount of bytes. I don't know about the author's problem context here, but I faced this problem when implementing Argon2i support for LUKS2 disk encryption. Due to its memory-hardness, it typically requires at least 1GB of RAM, which is currently impossible to allocate via the EFI allocator. I guess this patch here squarely aims at a similar problem. I'm not sure making this configurable at build time is the right thing to do, though. It's quite an easy fix, but it doesn't sound like it's going to be viable for distributions. They cannot know up front how much RAM is available in their users' machines, so picking a value that suits all users is next to impossible. I do have a patch series on this mailing list which instead alters the memory allocation strategy to allocate more RAM via the EFI service when we're hitting the limit [1]. It's not polished and the current strategy is probably debatable, but I think going into that direction is a much saner approach as it dynamically scales with whatever the system requires without having to make it a build-time switch. It's been some time since I last visited that patch, which can be blamed on the pending v2.06 release. But as soon as that's out of the door, I want to revisit it to finally get usable Argon2i support. Anyway, I don't want to discourage you to work on this topic, I just wanted to make you aware of my plans here to make the allocator itself more flexible. Patrick [1]: https://lists.gnu.org/archive/html/grub-devel/2020-06/msg00009.html > ``` > /* By default, request a quarter of the available memory. */ > total_pages =3D get_total_pages (filtered_memory_map, desc_size, > filtered_memory_map_end); > required_pages =3D (total_pages >> 2); > if (required_pages < BYTES_TO_PAGES (MIN_HEAP_SIZE)) > required_pages =3D BYTES_TO_PAGES (MIN_HEAP_SIZE); > else if (required_pages > BYTES_TO_PAGES (MAX_HEAP_SIZE)) > required_pages =3D BYTES_TO_PAGES (MAX_HEAP_SIZE); > ``` >=20 > Currently, MIN_HEAP_SIZE is 1 MB, and setting it to 600 MB fixes the=20 > issue for you. If `required_pages` is smaller than =E2=80=9C1 MB pages=E2= =80=9D it=E2=80=99s=20 > also smaller than =E2=80=9C600 MB pages=E2=80=9C, isn=E2=80=99t it? >=20 >=20 > Kind regards, >=20 > Paul >=20 > > --- > > config.h.in | 2 ++ > > configure.ac | 18 ++++++++++++++++++ > > grub-core/kern/efi/mm.c | 2 +- > > 3 files changed, 21 insertions(+), 1 deletion(-) > >=20 > > diff --git a/config.h.in b/config.h.in > > index 9e8f9911b..413287410 100644 > > --- a/config.h.in > > +++ b/config.h.in > > @@ -13,6 +13,8 @@ > > #define DISK_CACHE_STATS @DISK_CACHE_STATS@ > > #define BOOT_TIME_STATS @BOOT_TIME_STATS@ > >=20 > > +#define EFI_MIN_HEAP_MB @EFI_MIN_HEAP_MB@ > > + > > /* We don't need those. */ > > #define MINILZO_CFG_SKIP_LZO_PTR 1 > > #define MINILZO_CFG_SKIP_LZO_UTIL 1 > > diff --git a/configure.ac b/configure.ac > > index 7c10a4db7..8c4bb820e 100644 > > --- a/configure.ac > > +++ b/configure.ac > > @@ -1934,6 +1934,21 @@ AC_SUBST(HOST_CCASFLAGS) > >=20 > > AC_SUBST(BUILD_LIBM) > >=20 > > +# > > +# EFI minimum heap size in MB ranging from 1 to 1600. Default is 1. > > +# > > +AC_ARG_ENABLE([efi-min-heap-mb], > > + [AS_HELP_STRING([--enable-efi-min-heap-mb=3DSIZE_IN_MB], > > + [set the EFI minimum heap size in MB bet= ween 1 and 1600 inclusive])]) > > +AS_IF([test x"$enable_efi_min_heap_mb" =3D x -o x"$enable_efi_min_heap= _mb" =3D xno], > > + [EFI_MIN_HEAP_MB=3D1], > > + [AS_IF([test x"${enable_efi_min_heap_mb//@<:@0-9@:>@/}" !=3D x \ > > + || test "$enable_efi_min_heap_mb" -lt 1 -o "$enable_efi_min= _heap_mb" -gt 1600], > > + [AC_MSG_ERROR([EFI minimum heap size must be between 1 an= d 1600 inclusive])], > > + [EFI_MIN_HEAP_MB=3D$enable_efi_min_heap_mb])]) > > + > > +AC_SUBST([EFI_MIN_HEAP_MB]) > > + > > # > > # Automake conditionals > > # > > @@ -2132,5 +2147,8 @@ echo "Without liblzma (no support for XZ-compress= ed mips images) ($liblzma_excus > > else > > echo "With liblzma from $LIBLZMA (support for XZ-compressed mips images= )" > > fi > > +if [ x"$enable_efi_min_heap_mb" !=3D x ]; then > > +echo efi-min-heap-mb: $EFI_MIN_HEAP_MB > > +fi > > echo "*******************************************************" > > ] > > diff --git a/grub-core/kern/efi/mm.c b/grub-core/kern/efi/mm.c > > index 457772d57..ac5112b57 100644 > > --- a/grub-core/kern/efi/mm.c > > +++ b/grub-core/kern/efi/mm.c > > @@ -39,7 +39,7 @@ > > #define MEMORY_MAP_SIZE 0x3000 > >=20 > > /* The minimum and maximum heap size for GRUB itself. */ > > -#define MIN_HEAP_SIZE 0x100000 > > +#define MIN_HEAP_SIZE (EFI_MIN_HEAP_MB * 0x100000) > > #define MAX_HEAP_SIZE (1600 * 0x100000) > >=20 > > static void *finish_mmap_buf =3D 0; > > -- > > 2.29.2 --e6BWyWc9bLJ9vkAt Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEF9hrgiFbCdvenl/rVbJhu7ckPpQFAl/0Q8cACgkQVbJhu7ck PpSogA/9HK90k9VP1ZZLzivIcnnn2lMvJkp+reP0eHvMR9V56DhilLGQ3QCyjJoK nQ0yw02wRk1VBrtDNylv7qwIlYo0XS859qaPMVdpzAFZVG/lOagdOKNVNEtB1qqS AhLsbnrUbUV86AIHKsD9CMLg/Q77e7eAMGmwyL3Rsq/5MobuFj36J7xApr6Z09PT CGzV9+CXjcsoczejgPG17Or6lLfRlNwP+qWIBfbPSaZpViUVczZFoBLbWDr8k0QK XXFemsBf9Q2d6k1RMULo+O/5576ASFpnK1jfHuPdC70F8JvOMB5yf/OG24/VHpU2 kON7Pkazusg83wxZb2+TyJJqKrcyMAAZonM0iklfcXTlQnzqngEQumSAvSglfBsA JcuoTTXQdOX2uLo+DZ3XBbFRAzpmxDnYLHDJU1DPinFHDqCDko5Jgn10umJ/Uie+ 2gPUK+bcsyI9Y8cdB6aXzzXwly6BJsM37h1ZXjInpJOmHw0aXkw7CdaD5i6YntlO JdJsJj1bluWqy1Z1gY4g80aNU78J0ECw8xdhXGWZJXY34VQjQeyo8WCTOWQIH296 u2snr57hbPN2SUNeyGdfTXYtBgWDcuEbX6CTiqD0s/uIqNJS7DzzK73I0f8dCg5O 22/36Dq+ta0m/Ron5usercK5w2/te6eBeP8Gn5pc/Vfxgarlhm0= =DW00 -----END PGP SIGNATURE----- --e6BWyWc9bLJ9vkAt--