Confusions with reserve_early, reserve_bootmem, e820, efi, ... on x86_64

Confusions with reserve_early, reserve_bootmem, e820, efi, ... on x86_64

[Paul Jackson]

Bernhard, Johannes, Huang, and Yinghai:

I am running into some complications with reserve_bootmem vs
reserve_early, and a reserve_early of "EFI memmap" in the
efi_reserve_early() routine conflicting with reserve_early of
"BIOS reserved" in reserve_ebda_region().

This is on x86_64 arch, using Ingo's x86-latest, with possibly
still buggy EFI firmware in a lab system I'm helping to develop.

I have three concerns:

 1) The find_overlapped_early() check called from reserve_early is
    failing on my lab system , causing panic, when it tries to
    register "EFI memmap" at addresses <0xe0660, 0xe0778> that
    overlaps with the "BIOS reserved" early reserve at addresses
    <0x9f000, 0x100000>

 2) I'm a bit puzzled and concerned when I notice that what had been
    two separate reserve_ebda_region() routines, one for x86 32 and
    one for x86 64 have now become one routine -- the problem being
    that the earlier pair of these two reserve_ebda_region() routines
    were not the same.  They differed by their last line:

	--- /tmp/pj32   2008-06-12 02:09:03.414709042 -0700
	+++ /tmp/pj64   2008-06-12 02:08:42.458348590 -0700
	@@ -33,5 +33,5 @@ static void __init reserve_ebda_region(v
	                lowmem = 0x9f000;
	
	        /* reserve all memory between lowmem and the 1MB mark */
	-       reserve_bootmem(lowmem, 0x100000 - lowmem, BOOTMEM_DEFAULT);
	+       reserve_early(lowmem, 0x100000, "BIOS reserved");
	 }

    Perhaps this is fine ... I was just surprised to see what was
    essentially a change for the x86 32 side from reserve_bootmem(),
    to what is now called in both cases, reserve_early()

 3) Most likely my real problem is with my the EFI firmware on my
    lab system that is still under development.  It might be setting
    up addresses incorrectly.  However I am unable to make sense of
    the several patches in this area over the last week.

    I am unsure if what is there now should work, or whether my
    concern (1) above might indicate a still outstanding issue with
    the current code.

If anything I stated as fact above is incorrect, then I apologize.
I'm not entirely sure I have my details correct here.

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.940.382.4214

Re: Confusions with reserve_early, reserve_bootmem, e820, efi, ... on x86_64

[Yinghai Lu]

On Thu, Jun 12, 2008 at 3:06 AM, Paul Jackson <pj@sgi.com> wrote:
> Bernhard, Johannes, Huang, and Yinghai:
>
> I am running into some complications with reserve_bootmem vs
> reserve_early, and a reserve_early of "EFI memmap" in the
> efi_reserve_early() routine conflicting with reserve_early of
> "BIOS reserved" in reserve_ebda_region().
>
> This is on x86_64 arch, using Ingo's x86-latest, with possibly
> still buggy EFI firmware in a lab system I'm helping to develop.
>
> I have three concerns:
>
>  1) The find_overlapped_early() check called from reserve_early is
>    failing on my lab system , causing panic, when it tries to
>    register "EFI memmap" at addresses <0xe0660, 0xe0778> that
>    overlaps with the "BIOS reserved" early reserve at addresses
>    <0x9f000, 0x100000>

then reserve_ebda_region() need to be fixed.

YH

Re: Confusions with reserve_early, reserve_bootmem, e820, efi, ... on x86_64

[Paul Jackson]

Yinghai wrote:
> then reserve_ebda_region() need to be fixed.

What do you have in mind here?

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.940.382.4214

Re: Confusions with reserve_early, reserve_bootmem, e820, efi, ... on x86_64

[Yinghai Lu]

On Thu, Jun 12, 2008 at 9:19 AM, Paul Jackson <pj@sgi.com> wrote:
> Yinghai wrote:
>> then reserve_ebda_region() need to be fixed.
>
> What do you have in mind here?
>
that function seems to increase the ebda to workaround some broken
machine blindly.

YH

Re: Confusions with reserve_early, reserve_bootmem, e820, efi, ... on x86_64

[Paul Jackson]

Yinghai wrote:
> that function seems to increase the ebda to workaround some broken
> machine blindly.

Yes, true.

The reserve_ebda_region() has probably been like that for a while;
do you understand why it would be a problem conflicting with the
EFI memmap only now?

My other two concerns remain as well:

 2) Is what amounts to a change from calling reserve_bootmem()
    to calling reserve_early(), at the end of reserve_ebda_region()
    in the 32 bit case (now that it is merged with the 64 bit code)
    understood to be a desired and correct change?

 3) There has been quite a few patches restructuring this early
    memory reservation code over the last week.  What is the
    state of that work ... is it still "in progress" or is it
    thought to be complete?

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.940.382.4214

Re: Confusions with reserve_early, reserve_bootmem, e820, efi, ... on x86_64

[Yinghai Lu]

On Thu, Jun 12, 2008 at 1:10 PM, Paul Jackson <pj@sgi.com> wrote:
> Yinghai wrote:
>> that function seems to increase the ebda to workaround some broken
>> machine blindly.
>
> Yes, true.
>
> The reserve_ebda_region() has probably been like that for a while;
> do you understand why it would be a problem conflicting with the
> EFI memmap only now?
>
> My other two concerns remain as well:
>
>  2) Is what amounts to a change from calling reserve_bootmem()
>    to calling reserve_early(), at the end of reserve_ebda_region()
>    in the 32 bit case (now that it is merged with the 64 bit code)
>    understood to be a desired and correct change?
>
>  3) There has been quite a few patches restructuring this early
>    memory reservation code over the last week.  What is the
>    state of that work ... is it still "in progress" or is it
>    thought to be complete?

please check that commit that increase the range to 1M.
it seems at that time, andi and peter had different idea, because
increase 1M could waste K bytes memory.

also wonder if EFI is there, we still need to ebda?

YH

commit f6eb62b6924b99ec7da97fb6f554685a9ad6dce4
Author: Alexander van Heukelum <heukelum@mailshack.com>
Date:   Mon Feb 25 19:07:51 2008 +0100

    x86: reserve_early end-of-conventional-memory to 1MB, 64-bit

    Explicitly reserve_early the whole address range from the end of
    conventional memory as reported by the bios data area up to the
    1Mb mark. Regard the info retrieved from the BIOS data area with
    a bit of paranoia, though, because some biosses forget to register
    the EBDA correctly.

    Signed-off-by: Alexander van Heukelum <heukelum@fastmail.fm>
    Signed-off-by: Ingo Molnar <mingo@elte.hu>

diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 38f32e7..b684552 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -49,33 +49,42 @@ static void __init copy_bootdata(char *real_mode_data)
        }
 }

-#define EBDA_ADDR_POINTER 0x40E
+#define BIOS_EBDA_SEGMENT 0x40E
+#define BIOS_LOWMEM_KILOBYTES 0x413

+/*
+ * The BIOS places the EBDA/XBDA at the top of conventional
+ * memory, and usually decreases the reported amount of
+ * conventional memory (int 0x12) too.
+ */
 static __init void reserve_ebda(void)
 {
-       unsigned ebda_addr, ebda_size;
+       unsigned int lowmem, ebda_addr;

-       /*
-        * there is a real-mode segmented pointer pointing to the
-        * 4K EBDA area at 0x40E
-        */
-       ebda_addr = *(unsigned short *)__va(EBDA_ADDR_POINTER);
+       /* end of low (conventional) memory */
+       lowmem = *(unsigned short *)__va(BIOS_LOWMEM_KILOBYTES);
+       lowmem <<= 10;
+
+       /* start of EBDA area */
+       ebda_addr = *(unsigned short *)__va(BIOS_EBDA_SEGMENT);
        ebda_addr <<= 4;

-       if (!ebda_addr)
-               return;
+       /* Fixup: bios puts an EBDA in the top 64K segment */
+       /* of conventional memory, but does not adjust lowmem. */
+       if ((lowmem - ebda_addr) <= 0x10000)
+               lowmem = ebda_addr;

-       ebda_size = *(unsigned short *)__va(ebda_addr);
+       /* Fixup: bios does not report an EBDA at all. */
+       /* Some old Dells seem to need 4k anyhow (bugzilla 2990) */
+       if ((ebda_addr == 0) && (lowmem >= 0x9f000))
+               lowmem = 0x9f000;

-       /* Round EBDA up to pages */
-       if (ebda_size == 0)
-               ebda_size = 1;
-       ebda_size <<= 10;
-       ebda_size = round_up(ebda_size + (ebda_addr & ~PAGE_MASK), PAGE_SIZE);
-       if (ebda_size > 64*1024)
-               ebda_size = 64*1024;
+       /* Paranoia: should never happen, but... */
+       if (lowmem >= 0x100000)
+               lowmem = 0xa0000;

-       reserve_early(ebda_addr, ebda_addr + ebda_size, "EBDA");
+       /* reserve all memory between lowmem and the 1MB mark */
+       reserve_early(lowmem, 0x100000, "BIOS reserved");
 }

 void __init x86_64_start_kernel(char * real_mode_data)

Re: Confusions with reserve_early, reserve_bootmem, e820, efi, ... on x86_64

[H. Peter Anvin]

Yinghai Lu wrote:
> 
> please check that commit that increase the range to 1M.
> it seems at that time, andi and peter had different idea, because
> increase 1M could waste K bytes memory.
> 
> also wonder if EFI is there, we still need to ebda?
> 

You're not actually suggesting we should throw away this memory, are 
you?  Reserving it all early is fine, but the memory unused should 
eventually be reclaimed.

It's probably safer to reserve the EBDA even with EFI; it's only a page 
or so so it doesn't matter much.

	-hpa

Re: Confusions with reserve_early, reserve_bootmem, e820, efi, ... on x86_64

[Paul Jackson]

hpa writes:
> It's probably safer to reserve the EBDA even with EFI; it's only a page 
> or so so it doesn't matter much.

Reserving EBDA with EFI is fine by what little I know.

But the current code doesn't like it -- if the EFI memmap range is inside
the EBDA range, then the reserve_early() routine panics with:

    "Overlapping early reservations"

Something needs tweaking here somehow ... and I'm not the one to know what.

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.940.382.4214

Re: Confusions with reserve_early, reserve_bootmem, e820, efi, ... on x86_64

[Yinghai Lu]

On Thu, Jun 12, 2008 at 2:52 PM, Paul Jackson <pj@sgi.com> wrote:
> hpa writes:
>> It's probably safer to reserve the EBDA even with EFI; it's only a page
>> or so so it doesn't matter much.
>
> Reserving EBDA with EFI is fine by what little I know.
>
> But the current code doesn't like it -- if the EFI memmap range is inside
> the EBDA range, then the reserve_early() routine panics with:
>
>    "Overlapping early reservations"
>
> Something needs tweaking here somehow ... and I'm not the one to know what.

what is addr that the EFI memmap should be according to spec?

YH

Re: Confusions with reserve_early, reserve_bootmem, e820, efi, ... on x86_64

[Paul Jackson]

Yinghai wrote:
> what is addr that the EFI memmap should be according to spec?

The EFI memmap has no specific address by specification.  Rather the
BIOS/firmware can put it where it will, and then pass that address via
the boot_parms.efi_info.memmap field.  See include/asm-x86/bootparam.h
for these structures.

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.940.382.4214

Re: Confusions with reserve_early, reserve_bootmem, e820, efi, ... on x86_64

[H. Peter Anvin]

Paul Jackson wrote:
> 
> Reserving EBDA with EFI is fine by what little I know.
> 
> But the current code doesn't like it -- if the EFI memmap range is inside
> the EBDA range, then the reserve_early() routine panics with:
> 
>     "Overlapping early reservations"
> 
> Something needs tweaking here somehow ... and I'm not the one to know what.
> 

Right of course... in this particular case, one of the reservations is 
conservative "just in case" and the other is normative, but the code 
doesn't deal with that since it's potentially flagging a bug.

	-hpa

Re: Confusions with reserve_early, reserve_bootmem, e820, efi, ... on x86_64

[Huang, Ying]

On Thu, 2008-06-12 at 05:06 -0500, Paul Jackson wrote:
> Bernhard, Johannes, Huang, and Yinghai:
> 
> I am running into some complications with reserve_bootmem vs
> reserve_early, and a reserve_early of "EFI memmap" in the
> efi_reserve_early() routine conflicting with reserve_early of
> "BIOS reserved" in reserve_ebda_region().
> 
> This is on x86_64 arch, using Ingo's x86-latest, with possibly
> still buggy EFI firmware in a lab system I'm helping to develop.
> 
> I have three concerns:
> 
>  1) The find_overlapped_early() check called from reserve_early is
>     failing on my lab system , causing panic, when it tries to
>     register "EFI memmap" at addresses <0xe0660, 0xe0778> that
>     overlaps with the "BIOS reserved" early reserve at addresses
>     <0x9f000, 0x100000>

I think this can be resolved by avoiding memory area <0x9f000, 0x100000>
when allocating memory for EFI memory map in boot-loader.

Best Regards,
Huang Ying