[PATCH v2 0/2] x86: add support for reserved memory defined by DT

[PATCH v2 0/2] x86: add support for reserved memory defined by DT

[Grzegorz Jaszczyk]

Currently x86 allows to boot with ACPI and DT at the same time and basic DT
support is already in place but processing DT reserved memory was missing.

The DT reserved-memory nodes can be present in DT as described in
Documentation/devicetree/bindings/reserved-memory/reserved-memory.yaml.
Similar to other architecture, which supports DT, there is a need to
scan and register reserved memory regions on x86 for such nodes. It is required
by drivers (e.g. open-dice driver) to process DT reserved-memory regions.

Patch #1 extends of/reserved_mem and adds the possibility to register an arch
specific hook, which will allow to validate if reserved-memory region passed by
DT is valid.

Patch #2, uses introduced in Patch #1 API and registers x86 specific hook, which
will validate if reserved-memory region passed by DT is covered by E820 reserved
region entry.

For more details please refer to the commit log description of individual
patches.

v1 -> v2: simplify and fix off-by-one in x86_is_region_reserved as suggested by
          Dmytro Maluka offline

Grzegorz Jaszczyk (2):
  of: fdt: allow to register arch specific hook validating reserved
    region
  x86/of: add support for reserved memory defined by DT

 arch/x86/kernel/devicetree.c |  9 +++++++++
 drivers/of/of_reserved_mem.c | 14 ++++++++++++++
 include/linux/of_fdt.h       |  4 ++++
 3 files changed, 27 insertions(+)

-- 
2.49.0.805.g082f7c87e0-goog

[PATCH v2 1/2] of: fdt: allow to register arch specific hook validating reserved region

[Grzegorz Jaszczyk]

From: Grzegorz Jaszczyk <jaszczyk@google.com>

Add a possibility to register arch specific hook which will validate if
reserved-memory region passed by DT is valid (e.g. in case of x86, which
allows to boot with both ACPI and DT at the same time, registered hook
will validate if reserved-memory region passed by DT is covered by E820
reserved region entry).

Signed-off-by: Grzegorz Jaszczyk <jaszczyk@google.com>
---
 drivers/of/of_reserved_mem.c | 14 ++++++++++++++
 include/linux/of_fdt.h       |  4 ++++
 2 files changed, 18 insertions(+)

diff --git a/drivers/of/of_reserved_mem.c b/drivers/of/of_reserved_mem.c
index ee2e31522d7e..bb66a019e3e2 100644
--- a/drivers/of/of_reserved_mem.c
+++ b/drivers/of/of_reserved_mem.c
@@ -27,6 +27,7 @@
 
 #include "of_private.h"
 
+static bool __initdata (*arch_is_region_reserved)(phys_addr_t base, phys_addr_t size);
 static struct reserved_mem reserved_mem_array[MAX_RESERVED_REGIONS] __initdata;
 static struct reserved_mem *reserved_mem __refdata = reserved_mem_array;
 static int total_reserved_mem_cnt = MAX_RESERVED_REGIONS;
@@ -131,6 +132,13 @@ static void __init fdt_reserved_mem_save_node(unsigned long node, const char *un
 static int __init early_init_dt_reserve_memory(phys_addr_t base,
 					       phys_addr_t size, bool nomap)
 {
+	if (arch_is_region_reserved && !arch_is_region_reserved(base, size)) {
+		phys_addr_t end = base + size - 1;
+
+		pr_err("mem %pa-%pa not arch reserved\n", &base, &end);
+		return -EINVAL;
+	}
+
 	if (nomap) {
 		/*
 		 * If the memory is already reserved (by another region), we
@@ -146,6 +154,12 @@ static int __init early_init_dt_reserve_memory(phys_addr_t base,
 	return memblock_reserve(base, size);
 }
 
+void __init early_init_set_rsv_region_verifier(bool (*is_mem_reserved)(phys_addr_t base,
+								       phys_addr_t size))
+{
+	arch_is_region_reserved = is_mem_reserved;
+}
+
 /*
  * __reserved_mem_reserve_reg() - reserve all memory described in 'reg' property
  */
diff --git a/include/linux/of_fdt.h b/include/linux/of_fdt.h
index b8d6c0c20876..c7769323e720 100644
--- a/include/linux/of_fdt.h
+++ b/include/linux/of_fdt.h
@@ -65,6 +65,8 @@ extern void early_init_dt_check_for_usable_mem_range(void);
 extern int early_init_dt_scan_chosen_stdout(void);
 extern void early_init_fdt_scan_reserved_mem(void);
 extern void early_init_fdt_reserve_self(void);
+extern void early_init_set_rsv_region_verifier(bool (*is_mem_reserved)(phys_addr_t base,
+								       phys_addr_t size));
 extern void early_init_dt_add_memory_arch(u64 base, u64 size);
 extern u64 dt_mem_next_cell(int s, const __be32 **cellp);
 
@@ -89,6 +91,8 @@ static inline void early_init_dt_check_for_usable_mem_range(void) {}
 static inline int early_init_dt_scan_chosen_stdout(void) { return -ENODEV; }
 static inline void early_init_fdt_scan_reserved_mem(void) {}
 static inline void early_init_fdt_reserve_self(void) {}
+static inline void early_init_set_rsv_region_verifier(bool (*is_mem_reserved)(phys_addr_t base,
+									      phys_addr_t size)) {};
 static inline const char *of_flat_dt_get_machine_name(void) { return NULL; }
 static inline void unflatten_device_tree(void) {}
 static inline void unflatten_and_copy_device_tree(void) {}
-- 
2.49.0.805.g082f7c87e0-goog

[PATCH v2 2/2] x86/of: add support for reserved memory defined by DT

[Grzegorz Jaszczyk]

From: Grzegorz Jaszczyk <jaszczyk@google.com>

The DT reserved-memory nodes can be present in DT as described in
Documentation/devicetree/bindings/reserved-memory/reserved-memory.yaml.
Similar to other architecture, which supports DT, there is a need to
create reserved memory regions for such nodes.

Additionally, the x86 architecture builds its memory map based on E820
description passed by bootloader and not on DT. Since x86 already has
some DT support and allows booting with both ACPI and DT at the same
time, let's register an arch specific hook which will validate if a
reserved-memory region passed by DT is valid (covered by E820 reserved
region entry).

Without this check, the reserved memory from DT could be successfully
registered, even though such a region could conflict with e820
description e.g. it could be described as E820_RAM and could be already
used at early x86 boot stage for memblock initialization (which happens
before DT parsing).

Co-developed-by: Bartłomiej Grzesik <bgrzesik@google.com>
Signed-off-by: Bartłomiej Grzesik <bgrzesik@google.com>
Signed-off-by: Grzegorz Jaszczyk <jaszczyk@google.com>
---
 arch/x86/kernel/devicetree.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/arch/x86/kernel/devicetree.c b/arch/x86/kernel/devicetree.c
index dd8748c45529..9a93eddfcedb 100644
--- a/arch/x86/kernel/devicetree.c
+++ b/arch/x86/kernel/devicetree.c
@@ -18,6 +18,7 @@
 #include <linux/of_pci.h>
 #include <linux/initrd.h>
 
+#include <asm/e820/api.h>
 #include <asm/irqdomain.h>
 #include <asm/hpet.h>
 #include <asm/apic.h>
@@ -289,6 +290,11 @@ static void __init x86_dtb_parse_smp_config(void)
 	dtb_apic_setup();
 }
 
+static bool __init x86_is_region_reserved(phys_addr_t base, phys_addr_t size)
+{
+	return e820__mapped_all(base, base + size, E820_TYPE_RESERVED);
+}
+
 void __init x86_flattree_get_config(void)
 {
 #ifdef CONFIG_OF_EARLY_FLATTREE
@@ -307,6 +313,9 @@ void __init x86_flattree_get_config(void)
 		}
 
 		early_init_dt_verify(dt, __pa(dt));
+
+		early_init_set_rsv_region_verifier(x86_is_region_reserved);
+		early_init_fdt_scan_reserved_mem();
 	}
 
 	unflatten_and_copy_device_tree();
-- 
2.49.0.805.g082f7c87e0-goog

Re: [PATCH v2 2/2] x86/of: add support for reserved memory defined by DT

[Rob Herring]

On Fri, Apr 18, 2025 at 12:47:18PM +0000, Grzegorz Jaszczyk wrote:
> From: Grzegorz Jaszczyk <jaszczyk@google.com>
> 
> The DT reserved-memory nodes can be present in DT as described in
> Documentation/devicetree/bindings/reserved-memory/reserved-memory.yaml.
> Similar to other architecture, which supports DT, there is a need to
> create reserved memory regions for such nodes.
> 
> Additionally, the x86 architecture builds its memory map based on E820
> description passed by bootloader and not on DT. Since x86 already has
> some DT support and allows booting with both ACPI and DT at the same
> time, let's register an arch specific hook which will validate if a
> reserved-memory region passed by DT is valid (covered by E820 reserved
> region entry).
> 
> Without this check, the reserved memory from DT could be successfully
> registered, even though such a region could conflict with e820
> description e.g. it could be described as E820_RAM and could be already
> used at early x86 boot stage for memblock initialization (which happens
> before DT parsing).

Sorry, I don't get how it conflicts. Wouldn't the E820_RAM be registered 
with memblock and memblock then handles the conflict (or should).

Rob

Re: [PATCH v2 2/2] x86/of: add support for reserved memory defined by DT

[Grzegorz Jaszczyk]

On Wed, Apr 23, 2025 at 4:09 PM Rob Herring <robh@kernel.org> wrote:
>
> On Fri, Apr 18, 2025 at 12:47:18PM +0000, Grzegorz Jaszczyk wrote:
> > From: Grzegorz Jaszczyk <jaszczyk@google.com>
> >
> > The DT reserved-memory nodes can be present in DT as described in
> > Documentation/devicetree/bindings/reserved-memory/reserved-memory.yaml.
> > Similar to other architecture, which supports DT, there is a need to
> > create reserved memory regions for such nodes.
> >
> > Additionally, the x86 architecture builds its memory map based on E820
> > description passed by bootloader and not on DT. Since x86 already has
> > some DT support and allows booting with both ACPI and DT at the same
> > time, let's register an arch specific hook which will validate if a
> > reserved-memory region passed by DT is valid (covered by E820 reserved
> > region entry).
> >
> > Without this check, the reserved memory from DT could be successfully
> > registered, even though such a region could conflict with e820
> > description e.g. it could be described as E820_RAM and could be already
> > used at early x86 boot stage for memblock initialization (which happens
> > before DT parsing).
>
> Sorry, I don't get how it conflicts. Wouldn't the E820_RAM be registered
> with memblock and memblock then handles the conflict (or should).
>

On x86, early memblock setup is performed by e820__memblock_setup()
and regions which are marked as E820_RAM are added to the memblock
"memory" type and such regions can be later on used for memblock
allocation on early x86 setup. If memblock allocation is performed
after e820__memblock_setup and before x86_flattree_get_config,  the
reserved region described in DT (but described as RAM in e820) could
be silently used before we scan DT for reserved memory regions.

Additionally there are more reasons why we want to make sure that e820
reserved regions are in sync with DT reserved memory: resource tree
building and setup pci gap based on e820.
On the x86 resource tree is built taking into account e820 entries
(e820__reserve_resources()) while on other arch like e.g. arm64, which
relies on DT, the resource tree is built taking into account
information from DT(request_standard_resources). Mixing both on x86
seems problematic and at first glance could be achieved by e.g.
patching e820_table via e820__range_update so other part of the early
x86 kernel setup such as e820__setup_pci_gap() will also not use
region which is described in DT as reserved-memory. But it is not
straight-forward (initially I've tried to go through this path) e.g.
it will require handling DT earlier (x86_flattree_get_config) but at
the same time x86_flattree_get_config relies on the memblock being set
up. Therefore it seems that making a requirement that the e820
reserved region should be in sync with DT reserved-memory on x86 is
reasonable.

Best regards,
Grzegorz

Re: [PATCH v2 2/2] x86/of: add support for reserved memory defined by DT

[Rob Herring]

On Thu, Apr 24, 2025 at 08:06:33PM +0200, Grzegorz Jaszczyk wrote:
> On Wed, Apr 23, 2025 at 4:09 PM Rob Herring <robh@kernel.org> wrote:
> >
> > On Fri, Apr 18, 2025 at 12:47:18PM +0000, Grzegorz Jaszczyk wrote:
> > > From: Grzegorz Jaszczyk <jaszczyk@google.com>
> > >
> > > The DT reserved-memory nodes can be present in DT as described in
> > > Documentation/devicetree/bindings/reserved-memory/reserved-memory.yaml.
> > > Similar to other architecture, which supports DT, there is a need to
> > > create reserved memory regions for such nodes.
> > >
> > > Additionally, the x86 architecture builds its memory map based on E820
> > > description passed by bootloader and not on DT. Since x86 already has
> > > some DT support and allows booting with both ACPI and DT at the same
> > > time, let's register an arch specific hook which will validate if a
> > > reserved-memory region passed by DT is valid (covered by E820 reserved
> > > region entry).
> > >
> > > Without this check, the reserved memory from DT could be successfully
> > > registered, even though such a region could conflict with e820
> > > description e.g. it could be described as E820_RAM and could be already
> > > used at early x86 boot stage for memblock initialization (which happens
> > > before DT parsing).
> >
> > Sorry, I don't get how it conflicts. Wouldn't the E820_RAM be registered
> > with memblock and memblock then handles the conflict (or should).
> >
> 
> On x86, early memblock setup is performed by e820__memblock_setup()
> and regions which are marked as E820_RAM are added to the memblock
> "memory" type and such regions can be later on used for memblock
> allocation on early x86 setup. If memblock allocation is performed
> after e820__memblock_setup and before x86_flattree_get_config,  the
> reserved region described in DT (but described as RAM in e820) could
> be silently used before we scan DT for reserved memory regions.
> 
> Additionally there are more reasons why we want to make sure that e820
> reserved regions are in sync with DT reserved memory: resource tree
> building and setup pci gap based on e820.
> On the x86 resource tree is built taking into account e820 entries
> (e820__reserve_resources()) while on other arch like e.g. arm64, which
> relies on DT, the resource tree is built taking into account
> information from DT(request_standard_resources). Mixing both on x86
> seems problematic and at first glance could be achieved by e.g.
> patching e820_table via e820__range_update so other part of the early
> x86 kernel setup such as e820__setup_pci_gap() will also not use
> region which is described in DT as reserved-memory. But it is not
> straight-forward (initially I've tried to go through this path) e.g.
> it will require handling DT earlier (x86_flattree_get_config) but at
> the same time x86_flattree_get_config relies on the memblock being set
> up. Therefore it seems that making a requirement that the e820
> reserved region should be in sync with DT reserved-memory on x86 is
> reasonable.

x86_flattree_get_config() is a bit odd in that the DT is mapped and 
unflattened in one shot. Usually the flat DT is mapped and scanned 
early, and then only unflattened once memblock is up. You will be better 
off moving the early mapping and scanning earlier. Then the next thing 
you want from the DT early will be there. For example, the console or 
handling for kexec (which is its own reserved regions).

Rob

Re: [PATCH v2 2/2] x86/of: add support for reserved memory defined by DT

[Grzegorz Jaszczyk]

On Fri, Apr 25, 2025 at 9:18 PM Rob Herring <robh@kernel.org> wrote:
>
> On Thu, Apr 24, 2025 at 08:06:33PM +0200, Grzegorz Jaszczyk wrote:
> > On Wed, Apr 23, 2025 at 4:09 PM Rob Herring <robh@kernel.org> wrote:
> > >
> > > On Fri, Apr 18, 2025 at 12:47:18PM +0000, Grzegorz Jaszczyk wrote:
> > > > From: Grzegorz Jaszczyk <jaszczyk@google.com>
> > > >
> > > > The DT reserved-memory nodes can be present in DT as described in
> > > > Documentation/devicetree/bindings/reserved-memory/reserved-memory.yaml.
> > > > Similar to other architecture, which supports DT, there is a need to
> > > > create reserved memory regions for such nodes.
> > > >
> > > > Additionally, the x86 architecture builds its memory map based on E820
> > > > description passed by bootloader and not on DT. Since x86 already has
> > > > some DT support and allows booting with both ACPI and DT at the same
> > > > time, let's register an arch specific hook which will validate if a
> > > > reserved-memory region passed by DT is valid (covered by E820 reserved
> > > > region entry).
> > > >
> > > > Without this check, the reserved memory from DT could be successfully
> > > > registered, even though such a region could conflict with e820
> > > > description e.g. it could be described as E820_RAM and could be already
> > > > used at early x86 boot stage for memblock initialization (which happens
> > > > before DT parsing).
> > >
> > > Sorry, I don't get how it conflicts. Wouldn't the E820_RAM be registered
> > > with memblock and memblock then handles the conflict (or should).
> > >
> >
> > On x86, early memblock setup is performed by e820__memblock_setup()
> > and regions which are marked as E820_RAM are added to the memblock
> > "memory" type and such regions can be later on used for memblock
> > allocation on early x86 setup. If memblock allocation is performed
> > after e820__memblock_setup and before x86_flattree_get_config,  the
> > reserved region described in DT (but described as RAM in e820) could
> > be silently used before we scan DT for reserved memory regions.
> >
> > Additionally there are more reasons why we want to make sure that e820
> > reserved regions are in sync with DT reserved memory: resource tree
> > building and setup pci gap based on e820.
> > On the x86 resource tree is built taking into account e820 entries
> > (e820__reserve_resources()) while on other arch like e.g. arm64, which
> > relies on DT, the resource tree is built taking into account
> > information from DT(request_standard_resources). Mixing both on x86
> > seems problematic and at first glance could be achieved by e.g.
> > patching e820_table via e820__range_update so other part of the early
> > x86 kernel setup such as e820__setup_pci_gap() will also not use
> > region which is described in DT as reserved-memory. But it is not
> > straight-forward (initially I've tried to go through this path) e.g.
> > it will require handling DT earlier (x86_flattree_get_config) but at
> > the same time x86_flattree_get_config relies on the memblock being set
> > up. Therefore it seems that making a requirement that the e820
> > reserved region should be in sync with DT reserved-memory on x86 is
> > reasonable.
>
> x86_flattree_get_config() is a bit odd in that the DT is mapped and
> unflattened in one shot. Usually the flat DT is mapped and scanned
> early, and then only unflattened once memblock is up. You will be better
> off moving the early mapping and scanning earlier. Then the next thing
> you want from the DT early will be there. For example, the console or
> handling for kexec (which is its own reserved regions).

But reserved memory scanning relies on memblcok being already setup
(see: early_init_fdt_scan_reserved_mem->__reserved_mem_reserve_reg->early_init_dt_reserve_memory()
which uses:
memblock_overlaps_region, memblock_is_region_reserved,
memblock_mark_nomap and memblock_reserve and therefore we can't move
scanning earlier than e820__memblock_setup(). We can move early
mapping and reserved memory scanning part(actually entire
x86_flattree_get_config) at the end of  e820__memblock_setup but there
will be still remaining issue with e820 not being with sync which will
affect mentioned earlier e820 based assumptions for e.g.
e820__setup_pci_gap and e820__reserve_resources.

I've prepared v3 which moves x86_flattree_get_config earlier in the
setup process and prepared patch, which updates e820 table
accordingly: https://lore.kernel.org/all/20250430084138.2287031-1-jaszczyk@chromium.org/.
Could you please take a look?

Best regards,
Grzegorz