From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1nhQeW-0002A6-DX for mharc-grub-devel@gnu.org; Thu, 21 Apr 2022 02:50:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55542) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nhQeI-00026a-Ho for grub-devel@gnu.org; Thu, 21 Apr 2022 02:50:49 -0400 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]:33771) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nhQeG-000167-Ku for grub-devel@gnu.org; Thu, 21 Apr 2022 02:50:42 -0400 Received: by mail-pl1-x62d.google.com with SMTP id c23so4012943plo.0 for ; Wed, 20 Apr 2022 23:50:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=axtens.net; s=google; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version; bh=ep+KYAE+VMJkeLKYaSsyabkMPhPNS/vBqjsjXtiTVv0=; b=dOV+KJcC1EJgPNchq1U1GoURXHhTbb/68SNCxoAF8xEomUYMoSIXhYNG7mdzKd56Zr IFNbW49ILgjhewVl06xnNpZRsJ6YOllAQjmStog2/TFTeG1s2B2DqDdKDBtfTRiDz2rn MAFisp9uHXY+zrwyLZw0vua5umBbAU5L2X4tE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=ep+KYAE+VMJkeLKYaSsyabkMPhPNS/vBqjsjXtiTVv0=; b=OyvBYreLkp1n4EhbmXZU7sLixR0NbA2WtFDqWsw3HW/jV7/3yg6Mij94+7qhy15qT/ 4MwHDCKf85hdnNMoL+QU/UQsAB3aTE+0mklaZsCOfvQxBGI4M0UKfQQoe7JmKgCszrq1 PqcYVQRyXLmdSlqpg1DPqnpdqKEZbk6lY/LmLiqQElMnH4tRPpf2qYGDhvAXQNrpZ1PD SmV01MAUs7u1961Rl04dc4ntK8IMKGOuU4Bhs+VNhAT0gIDpNAUjXlKMGEwtjhggJosH RN3vv95TqoSAUX4XU88kiFTj1iT++wEQxaxsorwssGQruFIf6ZOQvOzghfexog4CQII7 sdgw== X-Gm-Message-State: AOAM531R/KD26G+McMMOXu19Y+1J/cAvoKmEYbb3N274j+RYVUmMSnS6 1igRZip1klUXzW4MDlTuklY4rrlH5ji3ug== X-Google-Smtp-Source: ABdhPJz+U1QA4skmCWn0PGHBVInNUNO9qSrWA21Qrkt0mQQmDBg5/FG3BxozZ1wn0eDviUKVF1reQQ== X-Received: by 2002:a17:903:20cc:b0:158:cacc:3655 with SMTP id i12-20020a17090320cc00b00158cacc3655mr24319574plb.148.1650523837617; Wed, 20 Apr 2022 23:50:37 -0700 (PDT) Received: from localhost ([2001:4479:e000:e400:4260:be14:52a8:8c94]) by smtp.gmail.com with ESMTPSA id c7-20020a17090a8d0700b001cd4989ff51sm1390349pjo.24.2022.04.20.23.50.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Apr 2022 23:50:37 -0700 (PDT) From: Daniel Axtens To: grub-devel@gnu.org Cc: leif@nuviainc.com, stefanb@linux.ibm.com, ps@pks.im, dkiper@net-space.pl, Daniel Kiper Subject: Re: [PATCH v3 06/15] mm: Allow dynamically requesting additional memory regions In-Reply-To: <20220421052427.1389987-7-dja@axtens.net> References: <20220421052427.1389987-1-dja@axtens.net> <20220421052427.1389987-7-dja@axtens.net> Date: Thu, 21 Apr 2022 16:50:34 +1000 Message-ID: <87ee1rdj5x.fsf@dja-thinkpad.axtens.net> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=2607:f8b0:4864:20::62d; envelope-from=dja@axtens.net; helo=mail-pl1-x62d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: grub-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: The development of GNU GRUB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Apr 2022 06:50:54 -0000 I genuinely do not know how I missed this, but we do need one more tweak so as not to break grub-emu builds: diff --git a/include/grub/mm.h b/include/grub/mm.h index 5d916809666c..f3bf87fa0f9a 100644 --- a/include/grub/mm.h +++ b/include/grub/mm.h @@ -42,7 +42,9 @@ typedef grub_err_t (*grub_mm_add_region_func_t) (grub_size_t, unsigned int); * Set this function pointer to enable adding memory-regions at runtime in case * a memory allocation cannot be satisfied with existing regions. */ +#ifndef GRUB_MACHINE_EMU extern grub_mm_add_region_func_t EXPORT_VAR(grub_mm_add_region_fn); +#endif void grub_mm_init_region (void *addr, grub_size_t size); void *EXPORT_FUNC(grub_calloc) (grub_size_t nmemb, grub_size_t size); I've updated by GH branches. Daniel K, are you right to fold this in when you merge this? Kind regards, Daniel Daniel Axtens writes: > From: Patrick Steinhardt > > Currently, all platforms will set up their heap on initialization of the > platform code. While this works mostly fine, it poses some limitations > on memory management on us. Most notably, allocating big chunks of > memory in the gigabyte range would require us to pre-request this many > bytes from the firmware and add it to the heap from the beginning on > some platforms like EFI. As this isn't needed for most configurations, > it is inefficient and may even negatively impact some usecases when, > e.g., chainloading. Nonetheless, allocating big chunks of memory is > required sometimes, where one example is the upcoming support for the > Argon2 key derival function in LUKS2. > > In order to avoid pre-allocating big chunks of memory, this commit > implements a runtime mechanism to add more pages to the system. When a > given allocation cannot be currently satisfied, we'll call a given > callback set up by the platform's own memory management subsystem, > asking it to add a memory area with at least `n` bytes. If this > succeeds, we retry searching for a valid memory region, which should now > succeed. > > If this fails, we try asking for `n` bytes, possibly spread across > multiple regions, in hopes that region merging means that we end up > with enough memory for things to work out. > > Signed-off-by: Patrick Steinhardt > [dja: add this to the documentation at the top of mm.c > v2: fallback to non-contiguous] > Signed-off-by: Daniel Axtens > Tested-by: Stefan Berger > Reviewed-by: Daniel Kiper > --- > grub-core/kern/mm.c | 30 ++++++++++++++++++++++++++++++ > include/grub/mm.h | 16 ++++++++++++++++ > 2 files changed, 46 insertions(+) > > diff --git a/grub-core/kern/mm.c b/grub-core/kern/mm.c > index 41e5ea07dc5a..ed782e8d1f56 100644 > --- a/grub-core/kern/mm.c > +++ b/grub-core/kern/mm.c > @@ -28,6 +28,9 @@ > - multiple regions may be used as free space. They may not be > contiguous. > > + - if existing regions are insufficient to satisfy an allocation, a new > + region can be requested from firmware. > + > Regions are managed by a singly linked list, and the meta information is > stored in the beginning of each region. Space after the meta information > is used to allocate memory. > @@ -81,6 +84,7 @@ > > > grub_mm_region_t grub_mm_base; > +grub_mm_add_region_func_t grub_mm_add_region_fn; > > /* Get a header from the pointer PTR, and set *P and *R to a pointer > to the header and a pointer to its region, respectively. PTR must > @@ -437,6 +441,32 @@ grub_memalign (grub_size_t align, grub_size_t size) > count++; > goto again; > > + case 1: > + /* Request additional pages, contiguous */ > + count++; > + > + if (grub_mm_add_region_fn != NULL && > + grub_mm_add_region_fn (size, GRUB_MM_ADD_REGION_CONSECUTIVE) == GRUB_ERR_NONE) > + goto again; > + > + /* fallthrough */ > + > + case 2: > + /* Request additional pages, anything at all */ > + count++; > + > + if (grub_mm_add_region_fn != NULL) > + { > + /* > + * Try again even if this fails, in case it was able to partially > + * satisfy the request > + */ > + grub_mm_add_region_fn (size, GRUB_MM_ADD_REGION_NONE); > + goto again; > + } > + > + /* fallthrough */ > + > default: > break; > } > diff --git a/include/grub/mm.h b/include/grub/mm.h > index 44fde7cb9033..5d916809666c 100644 > --- a/include/grub/mm.h > +++ b/include/grub/mm.h > @@ -20,6 +20,7 @@ > #ifndef GRUB_MM_H > #define GRUB_MM_H 1 > > +#include > #include > #include > #include > @@ -28,6 +29,21 @@ > # define NULL ((void *) 0) > #endif > > +#define GRUB_MM_ADD_REGION_NONE 0 > +#define GRUB_MM_ADD_REGION_CONSECUTIVE (1 << 0) > + > +/* > + * Function used to request memory regions of `grub_size_t` bytes. The second > + * parameter is a bitfield of `GRUB_MM_ADD_REGION` flags. > + */ > +typedef grub_err_t (*grub_mm_add_region_func_t) (grub_size_t, unsigned int); > + > +/* > + * Set this function pointer to enable adding memory-regions at runtime in case > + * a memory allocation cannot be satisfied with existing regions. > + */ > +extern grub_mm_add_region_func_t EXPORT_VAR(grub_mm_add_region_fn); > + > void grub_mm_init_region (void *addr, grub_size_t size); > void *EXPORT_FUNC(grub_calloc) (grub_size_t nmemb, grub_size_t size); > void *EXPORT_FUNC(grub_malloc) (grub_size_t size); > -- > 2.32.0