[PATCH] efi: kernel param for legacy NVDIMM support

[PATCH] efi: kernel param for legacy NVDIMM support

[Yigal Korman]

The 'efi_legacy_pmem' parameter will convert EFI persistent memory range
(type 14) into E820 legacy NVDIMM (type 12) memory range.

Background:

In contrast with the NVDIMM E820 types where we can clearly distinguish
between old NVDIMMs (type-12) and ACPI 6.0 NVDIMMs (type-7), the EFI
memory types for NVDIMMs are the same before ACPI 6.0 and after
(type-14).
This means that old NVDIMMs under EFI aren't supported even though
they work fine if booted with BIOS (E820).

So allow the user to explicitly request the kernel to identify NVDIMMs
as legacy under EFI.

Signed-off-by: Yigal Korman <yigal-/8YdC2HfS5554TAoqtyWWQ@public.gmane.org>
---
 Documentation/kernel-parameters.txt |  3 +++
 arch/x86/platform/efi/efi.c         | 31 +++++++++++++++++++++++++++++++
 2 files changed, 34 insertions(+)

diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index 9a53c92..58f2a9b 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -394,6 +394,9 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
 	add_efi_memmap	[EFI; X86] Include EFI memory map in
 			kernel's map of available physical RAM.
 
+	efi_legacy_pmem	[EFI; X86] Convert EFI_PERSISTENT_MEMORY to E820_PRAM
+			and add it to E820 memmap for legacy NVDIMM support.
+
 	agp=		[AGP]
 			{ off | try_unsupported }
 			off: disable AGP support
diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
index ad28540..5e82532 100644
--- a/arch/x86/platform/efi/efi.c
+++ b/arch/x86/platform/efi/efi.c
@@ -78,6 +78,14 @@ static int __init setup_add_efi_memmap(char *arg)
 }
 early_param("add_efi_memmap", setup_add_efi_memmap);
 
+static int add_legacy_pmem __initdata;
+static int __init setup_add_legacy_pmem(char *arg)
+{
+	add_legacy_pmem = 1;
+	return 0;
+}
+early_param("efi_legacy_pmem", setup_add_legacy_pmem);
+
 static efi_status_t __init phys_efi_set_virtual_address_map(
 	unsigned long memory_map_size,
 	unsigned long descriptor_size,
@@ -191,6 +199,26 @@ static void __init do_add_efi_memmap(void)
 	sanitize_e820_map(e820.map, ARRAY_SIZE(e820.map), &e820.nr_map);
 }
 
+static void __init do_add_legacy_pmem(void)
+{
+	void *p;
+
+	for (p = memmap.map; p < memmap.map_end; p += memmap.desc_size) {
+		efi_memory_desc_t *md = p;
+
+		if (md->type == EFI_PERSISTENT_MEMORY) {
+			pr_info("Registering %lluMB as PRAM in E820\n",
+				(md->num_pages >> (20 - EFI_PAGE_SHIFT)));
+
+			e820_add_region(md->phys_addr,
+					md->num_pages << EFI_PAGE_SHIFT,
+					E820_PRAM);
+		}
+
+	}
+	sanitize_e820_map(e820.map, ARRAY_SIZE(e820.map), &e820.nr_map);
+}
+
 int __init efi_memblock_x86_reserve_range(void)
 {
 	struct efi_info *e = &boot_params.efi_info;
@@ -455,6 +483,9 @@ static int __init efi_memmap_init(void)
 	if (add_efi_memmap)
 		do_add_efi_memmap();
 
+	if (add_legacy_pmem)
+		do_add_legacy_pmem();
+
 	set_bit(EFI_MEMMAP, &efi.flags);
 
 	return 0;
-- 
1.9.3

Re: [PATCH] efi: kernel param for legacy NVDIMM support

[Dan Williams]

On Thu, Jun 9, 2016 at 10:42 AM, Yigal Korman <yigal-/8YdC2HfS5554TAoqtyWWQ@public.gmane.org> wrote:
> The 'efi_legacy_pmem' parameter will convert EFI persistent memory range
> (type 14) into E820 legacy NVDIMM (type 12) memory range.
>
> Background:
>
> In contrast with the NVDIMM E820 types where we can clearly distinguish
> between old NVDIMMs (type-12) and ACPI 6.0 NVDIMMs (type-7), the EFI
> memory types for NVDIMMs are the same before ACPI 6.0 and after
> (type-14).
> This means that old NVDIMMs under EFI aren't supported even though
> they work fine if booted with BIOS (E820).
>
> So allow the user to explicitly request the kernel to identify NVDIMMs
> as legacy under EFI.
>

I'm concerned with the potential for this command line parameter to
collide with NFIT defined ranges.  At a minimum it should confirm that
there is not already an NFIT describing the same address ranges.

However, we have the ability to override / inject ACPI tables and
methods from the kernel.  Why not use that facility to custom craft an
NFIT when the BIOS fails to provide one?  That way EFI type-14
maintains a constant interpretation as just a reserved memory range
with no other side effects.

Re: [PATCH] efi: kernel param for legacy NVDIMM support

[Yigal Korman]

On Thu, Jun 9, 2016 at 11:27 PM, Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:
>
> On Thu, Jun 9, 2016 at 10:42 AM, Yigal Korman <yigal-/8YdC2HfS5554TAoqtyWWQ@public.gmane.org> wrote:
> > The 'efi_legacy_pmem' parameter will convert EFI persistent memory range
> > (type 14) into E820 legacy NVDIMM (type 12) memory range.
> >
> > Background:
> >
> > In contrast with the NVDIMM E820 types where we can clearly distinguish
> > between old NVDIMMs (type-12) and ACPI 6.0 NVDIMMs (type-7), the EFI
> > memory types for NVDIMMs are the same before ACPI 6.0 and after
> > (type-14).
> > This means that old NVDIMMs under EFI aren't supported even though
> > they work fine if booted with BIOS (E820).
> >
> > So allow the user to explicitly request the kernel to identify NVDIMMs
> > as legacy under EFI.
> >
>
> I'm concerned with the potential for this command line parameter to
> collide with NFIT defined ranges.  At a minimum it should confirm that
> there is not already an NFIT describing the same address ranges.
>

That's a valid concern, but not related to this patch directly, the
same might happen today when an 'memmap=XX!YY' kernel parameter
collides with an NFIT on the same range.

Or, albeit a far fetched scenario, a platform vendor will decide to
provide an NFIT for a non-ACPI 6.0 and leave the old E820 type-12.

> However, we have the ability to override / inject ACPI tables and
> methods from the kernel.  Why not use that facility to custom craft an
> NFIT when the BIOS fails to provide one?  That way EFI type-14
> maintains a constant interpretation as just a reserved memory range
> with no other side effects.

That might be an interesting way to implement memmap=XX!YY in general
and can also replace the funny code in arch/x86/kernel/pmem.c.

But, it's more complex and probably has its own caveats, this patch is
simpler and straight forward, providing direct value.

Thanks,
Yigal

Re: [PATCH] efi: kernel param for legacy NVDIMM support

[Dan Williams]

On Fri, Jun 10, 2016 at 1:48 AM, Yigal Korman <yigal-/8YdC2HfS5554TAoqtyWWQ@public.gmane.org> wrote:
> On Thu, Jun 9, 2016 at 11:27 PM, Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:
>>
>> On Thu, Jun 9, 2016 at 10:42 AM, Yigal Korman <yigal-/8YdC2HfS5554TAoqtyWWQ@public.gmane.org> wrote:
>> > The 'efi_legacy_pmem' parameter will convert EFI persistent memory range
>> > (type 14) into E820 legacy NVDIMM (type 12) memory range.
>> >
>> > Background:
>> >
>> > In contrast with the NVDIMM E820 types where we can clearly distinguish
>> > between old NVDIMMs (type-12) and ACPI 6.0 NVDIMMs (type-7), the EFI
>> > memory types for NVDIMMs are the same before ACPI 6.0 and after
>> > (type-14).
>> > This means that old NVDIMMs under EFI aren't supported even though
>> > they work fine if booted with BIOS (E820).
>> >
>> > So allow the user to explicitly request the kernel to identify NVDIMMs
>> > as legacy under EFI.
>> >
>>
>> I'm concerned with the potential for this command line parameter to
>> collide with NFIT defined ranges.  At a minimum it should confirm that
>> there is not already an NFIT describing the same address ranges.
>>
>
> That's a valid concern, but not related to this patch directly, the
> same might happen today when an 'memmap=XX!YY' kernel parameter
> collides with an NFIT on the same range.

We should fix that too, i.e. always prefer NFIT over plain / imprecise
type-12 / efi-type-14 range.  Actually, in the type-12 vs NFIT case we
should be even more strict...

> Or, albeit a far fetched scenario, a platform vendor will decide to
> provide an NFIT for a non-ACPI 6.0 and leave the old E820 type-12.

If a platform has e820 type-12 (without memmap=) and an NFIT we should
throw a WARN_TAINT firmware warning, because that platform has taken
the time to update to ACPI 6.0 NFIT, but still use the out of spec
type-12.  That memory type is something we only supported as a
stop-gap for platforms that had no other option prior to ACPI 6.

>> However, we have the ability to override / inject ACPI tables and
>> methods from the kernel.  Why not use that facility to custom craft an
>> NFIT when the BIOS fails to provide one?  That way EFI type-14
>> maintains a constant interpretation as just a reserved memory range
>> with no other side effects.
>
> That might be an interesting way to implement memmap=XX!YY in general
> and can also replace the funny code in arch/x86/kernel/pmem.c.
>
> But, it's more complex and probably has its own caveats, this patch is
> simpler and straight forward, providing direct value.

No, it makes a bad situation worse.  In the type-12 case it is
obviously out of spec and the only thing that address range could be
is a legacy platform.  The efi-type-14 case is accidentally in-spec
and collides / aliases with the ACPI 6 definition.

We should always look for an NFIT in the type-14 case.  Injecting a
crafted NFIT for this legacy case removes confusion as there is only
one action the kernel takes and no possibility for collisions.  I also
do not want to make it easier for firmware developers to skip the
necessary due diligence to decide whether attributes like numa
topology, posted write queue flush mechanisms, and media health,
etc...  need description.

RE: [PATCH] efi: kernel param for legacy NVDIMM support

[Elliott, Robert (Persistent Memory)]

> -----Original Message-----
> From: Linux-nvdimm [mailto:linux-nvdimm-bounces-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org] On
> Behalf Of Dan Williams
> Sent: Friday, June 10, 2016 9:28 AM
> To: Yigal Korman <yigal-/8YdC2HfS5554TAoqtyWWQ@public.gmane.org>
> Cc: Matt Fleming <matt-mF/unelCI9GS6iBeEJttW/XRex20P6io@public.gmane.org>; linux-
> efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org
> Subject: Re: [PATCH] efi: kernel param for legacy NVDIMM support
> 
> On Fri, Jun 10, 2016 at 1:48 AM, Yigal Korman <yigal-/8YdC2HfS5554TAoqtyWWQ@public.gmane.org>
> wrote:
> > On Thu, Jun 9, 2016 at 11:27 PM, Dan Williams
...
> We should always look for an NFIT in the type-14 case.  Injecting a
> crafted NFIT for this legacy case removes confusion as there is only
> one action the kernel takes and no possibility for collisions.  I
> also
> do not want to make it easier for firmware developers to skip the
> necessary due diligence to decide whether attributes like numa
> topology, posted write queue flush mechanisms, and media health,
> etc...  need description.

I'll also reiterate that NVDIMM ranges may also be reported using
the UEFI and E820 reserved types.  The simple E820 and UEFI memory
map persistent memory type values do not contain enough information
to properly talk to an NVDIMM; the information in ACPI NFIT is
crucial.

---
Robert Elliott, HPE Persistent Memory