xen-kernel and EDD

xen-kernel and EDD

[Lin-Bao Zhang]

Now , I am using RHEL5.7's xen-kernel, I must add "edd=off" ,
otherwise ,it will print this error :
1706-Smart Array Controller EBDA Memory Corrupted  ，INT 13h BIOS
Cannot Coninue - Sytem Halted

RHEL said that their RHEL5.7 xen-kernel must make edd off ; while
non-xen kernel doesn't need to make edd off.

why ?
for community XEN developing , is there limit regarding to
EDD(Enhanced Disk Drive)  ?

welcome your comments .thanks .
-Bob

Re: xen-kernel and EDD

[Keir Fraser]

The BIOS message might indicate that the hypervisor has corrupted BIOS
memory. Maybe this wouldn't happen with a newer version of Xen; the RHEL-5
version is old.

 -- Keir

On 18/08/2011 06:08, "Lin-Bao Zhang" <2004.zhang@gmail.com> wrote:

> Now , I am using RHEL5.7's xen-kernel, I must add "edd=off" ,
> otherwise ,it will print this error :
> 1706-Smart Array Controller EBDA Memory Corrupted  ，INT 13h BIOS
> Cannot Coninue - Sytem Halted
> 
> RHEL said that their RHEL5.7 xen-kernel must make edd off ; while
> non-xen kernel doesn't need to make edd off.
> 
> why ?
> for community XEN developing , is there limit regarding to
> EDD(Enhanced Disk Drive)  ?
> 
> welcome your comments .thanks .
> -Bob
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: xen-kernel and EDD

[Lin-Bao Zhang]

2011/8/18 Keir Fraser <keir.xen@gmail.com>:
> The BIOS message might indicate that the hypervisor has corrupted BIOS
> memory. Maybe this wouldn't happen with a newer version of Xen; the RHEL-
> 5 version is old.
>
>  -- Keir
in our system, there are two main devices , P410i(smart array card) and FC card.
your guess is reasonable.
another possibility is
My guess is the HBA option ROM is stepping on something the p410i
option ROM sets up:
P410i option ROM runs and sets up EBDA area
HBA option ROM runs and trashes EBDA area the p410i firmware setup
System tries to boot from the p410i, and the first thing the FW does
is check the EBDA area, and sends this message if it is messed up.
 --------
if we unplug FC(HBA) card , this issue can't happen again . so I feel
that the most possibility is FC firmware destroied P410i card's EBDA
area.


let's discuss your guess now.

I know for BIOS ,it exists 640K limitation ,especially for realmode.
1,EBDA is always immediately below 0xA0000 in memory：
http://wiki.osdev.org/Memory_Map_(x86)#BIOS_Data_Area_.28BDA.29
 [original text]
Note: the EBDA is a variable-sized memory area (on different BIOSes).
If it exists, it is always immediately below 0xA0000 in memory. It is
absolutely guaranteed to be less than 128 KiB in size. It is often 1
KiB. The biggest ones ever actually seen are 8 KiB. You can determine
the size of the EBDA by using BIOS function INT 12h, or (often) by
examining the word at 0x40E in the BDA (see below). Both of those
methods will tell you the location of the bottom of the EBDA.
-- 
so in my system , my EBDA area should be 95400 ~ a0000  ,about 43KBytes.
from xen.gz (hypervisor) code , trampoline.S , we know
----
#define BOOT_TRAMPOLINE 0x90000
#define bootsym_phys(sym)                                 \
    (((unsigned long)&(sym)-(unsigned long)&trampoline_start)+BOOT_TRAMPOLINE)
-----
0x95400 - 0x90000 = 0x5400 = 21KBytes.

21Kbytes is enough for gen.gz running(hypervisor) ?
if hypervisor will run more area , maybe it will overlap EBDA area .

so who can tell me xen.gz 's end address ? or max long ?



from RHEL5.7xen kernel, xen.gz loading address is 1M,but

Re: xen-kernel and EDD

[Lin-Bao Zhang]

in xen/arch/x86/boot/head.S:
there is a code ,it will copy bootstrap trampoline to low memory below 1MB.

I am not very familar with assembler code, who can explain its target :
I guess maybe 0x98000 has overlapped my EBDA area(95400 ~a0000) .
-----------------------
 /* Copy bootstrap trampoline to low memory, below 1MB. */
        mov     $sym_phys(trampoline_start),%esi
        mov     $bootsym_phys(trampoline_start),%edi
        mov     $trampoline_end - trampoline_start,%ecx
        rep     movsb

        mov     $0x98000,%esp
        call    cmdline_parse_early

        /* Jump into the relocated trampoline. */
        jmp     $BOOT_CS32,$bootsym_phys(trampoline_boot_cpu_entry)

#include "cmdline.S"

        .globl trampoline_start, trampoline_end
trampoline_start:
#include "trampoline.S"
trampoline_end:

        .text
__high_start:
#ifdef __x86_64__
#include "x86_64.S"
#else
#include "x86_32.S"
#endif

Re: xen-kernel and EDD

[Keir Fraser]

On 18/08/2011 11:52, "Lin-Bao Zhang" <2004.zhang@gmail.com> wrote:

> in xen/arch/x86/boot/head.S:
> there is a code ,it will copy bootstrap trampoline to low memory below 1MB.
> 
> I am not very familar with assembler code, who can explain its target :
> I guess maybe 0x98000 has overlapped my EBDA area(95400 ~a0000) .

Well, yes, obviously it does. :-)

> -----------------------
>  /* Copy bootstrap trampoline to low memory, below 1MB. */
>         mov     $sym_phys(trampoline_start),%esi
>         mov     $bootsym_phys(trampoline_start),%edi
>         mov     $trampoline_end - trampoline_start,%ecx
>         rep     movsb
> 
>         mov     $0x98000,%esp
>         call    cmdline_parse_early
> 
>         /* Jump into the relocated trampoline. */
>         jmp     $BOOT_CS32,$bootsym_phys(trampoline_boot_cpu_entry)
> 
> #include "cmdline.S"
> 
>         .globl trampoline_start, trampoline_end
> trampoline_start:
> #include "trampoline.S"
> trampoline_end:
> 
>         .text
> __high_start:
> #ifdef __x86_64__
> #include "x86_64.S"
> #else
> #include "x86_32.S"
> #endif

Re: xen-kernel and EDD

[Keir Fraser]

On 18/08/2011 11:43, "Lin-Bao Zhang" <2004.zhang@gmail.com> wrote:

> -- 
> so in my system , my EBDA area should be 95400 ~ a0000  ,about 43KBytes.
> from xen.gz (hypervisor) code , trampoline.S , we know
> ----
> #define BOOT_TRAMPOLINE 0x90000
> #define bootsym_phys(sym)                                 \
>     (((unsigned long)&(sym)-(unsigned long)&trampoline_start)+BOOT_TRAMPOLINE)
> -----
> 0x95400 - 0x90000 = 0x5400 = 21KBytes.
> 
> 21Kbytes is enough for gen.gz running(hypervisor) ?
> if hypervisor will run more area , maybe it will overlap EBDA area .
> 
> so who can tell me xen.gz 's end address ? or max long ?

It's almost certainly this. In latest Xen we run the trampoline code at
0x7c000. So you can try rebuilding your hypervisor with the following
changes:
1. Change BOOT_TRAMPOLINE to 0x7c000
2. In arch/x86/boot/head.S, change the line 'mov $0x98000,%esp' to 'mov
$0x80000,%esp'.

That will probably fix your problem.

 -- Keir

Re: xen-kernel and EDD

[Lin-Bao Zhang]

2011/8/18 Keir Fraser <keir@xen.org>:
> It's almost certainly this. In latest Xen we run the trampoline code at
> 0x7c000. So you can try rebuilding your hypervisor with the following
> changes:
> 1. Change BOOT_TRAMPOLINE to 0x7c000
> 2. In arch/x86/boot/head.S, change the line 'mov $0x98000,%esp' to 'mov
> $0x80000,%esp'.
>
> That will probably fix your problem.
>
>  -- Keir
>
I also think you are right , good idea , thanks very much! I will let
you all know the final result ASAP.

Re: xen-kernel and EDD

[Lin-Bao Zhang]

2011/8/18 Lin-Bao Zhang <2004.zhang@gmail.com>
>
> 2011/8/18 Keir Fraser <keir@xen.org>:
> > It's almost certainly this. In latest Xen we run the trampoline code at
> > 0x7c000. So you can try rebuilding your hypervisor with the following
> > changes:
> > 1. Change BOOT_TRAMPOLINE to 0x7c000
> > 2. In arch/x86/boot/head.S, change the line 'mov $0x98000,%esp' to 'mov
> > $0x80000,%esp'.
> >
> > That will probably fix your problem.
> >
> >  -- Keir

Hi Keir ,
maybe 0x80000 is not very good, I check xen-4.1.0 code:
/* Copy bootstrap trampoline to low memory, below 1MB. */
       mov     $sym_phys(trampoline_start),%esi
       mov     $bootsym_phys(trampoline_start),%edi
       mov     $trampoline_end - trampoline_start,%ecx
       rep     movsb

       mov     $bootsym_phys(early_stack),%esp  //eary_stack.
       call    cmdline_parse_early

       /* Jump into the relocated trampoline. */
       jmp     $BOOT_CS32,$bootsym_phys(trampoline_boot_cpu_entry)

it seems that direct number is not very compatible .

1, another point : it seems that this code should copy code (from
trampoline_start to trampoline_end) to BOOT_TRAMPOLINE(0x90000)) , I
will consider if trampoline code size bigger than
0x95400-0x90000=21KBytes .
but my binary xen.gz doesn't include symbols , so I can't see address
of trampoline_start and trampoline_end.  how to get them ? must
recompile it ?I think maybe redhat has deleted symbols . yet I don't
trampoline will be over 21KBytes,because it seems small code.


2, about definition of trampoline_start and trampoline_end ?
I just can see:
#define BOOT_TRAMPOLINE 0x90000
#define bootsym_phys(sym)                                 \
   (((unsigned long)&(sym)-(unsigned long)&trampoline_start)+BOOT_TRAMPOLINE)
#define bootsym(sym)                                      \
   (*RELOC_HIDE((typeof(&(sym)))__va(__pa(&(sym))),      \
                BOOT_TRAMPOLINE-__pa(trampoline_start)))
#ifndef __ASSEMBLY__
extern char trampoline_start[], trampoline_end[];
extern char trampoline_realmode_entry[];
extern unsigned int trampoline_xen_phys_start;
extern unsigned char trampoline_cpu_started;
#endif

。。。。。。。。。。
head.S :
       .globl trampoline_start, trampoline_end
trampoline_start:  //  it seems that I must find this symbol to
confirm its value , nm -a this file , displaying no symbols.
#include "trampoline.S"
trampoline_end:


3 , in sles11:

I find sles11 again , I did the same steps .
in sles11 x86_64 version ,there are also :
xen-syms-3.3.1_18546_12-3.1 and xen.gz :
nm -a xen(unzip xen.gz) ,can't trampoline_start/end
while nm -a xen-syms-3.3.1_18546_12-3.1 , I can finally find
"trampoline_start" and "trampoline_end" :
--bob comment: I begin another email subject "about __XEN_VIRT_START
and 0x100000" to ask the difference between these 2 files.

from this output ,useful information:
ffff828c80100000 A _start
...
ffff828c8020d5b6 t trampoline_boot_cpu_entry
ffff828c8020d549 T trampoline_cpu_started
ffff828c8020fe50 T trampoline_end                                 //here
ffff828c8020d511 t trampoline_gdt
ffff828c8020d54a t trampoline_protmode_entry
ffff828c8020d4e0 T trampoline_realmode_entry
ffff828c8020d4e0 T trampoline_start                                  //here
ffff828c8020d545 T trampoline_xen_phys_start
ffff828c80172ab0 t transition_pstate
...
from sles11 , trampoline code size   0xFFFF828C8_020FE50 -
ffff828c8_020d4e0 =  0x2970  (about 10KBytes)
if BOOT_TRAMPOLINE == 0x90000 , then trampoline end will be 0x92970
,while my EBDA is 0x954000 ~ 0xa0000 . so I think don't overlap.
certainly , this number is from sles11 xen-kernel ,but I think their
trampoline.S basily same ,length should be almost same.

Re: xen-kernel and EDD

[Keir Fraser]

On 19/08/2011 09:27, "Lin-Bao Zhang" <2004.zhang@gmail.com> wrote:

> 2011/8/18 Lin-Bao Zhang <2004.zhang@gmail.com>
>> 
>> 2011/8/18 Keir Fraser <keir@xen.org>:
>>> It's almost certainly this. In latest Xen we run the trampoline code at
>>> 0x7c000. So you can try rebuilding your hypervisor with the following
>>> changes:
>>> 1. Change BOOT_TRAMPOLINE to 0x7c000
>>> 2. In arch/x86/boot/head.S, change the line 'mov $0x98000,%esp' to 'mov
>>> $0x80000,%esp'.
>>> 
>>> That will probably fix your problem.
>>> 
>>>  -- Keir
> 
> Hi Keir ,
> maybe 0x80000 is not very good, I check xen-4.1.0 code:
> /* Copy bootstrap trampoline to low memory, below 1MB. */
>        mov     $sym_phys(trampoline_start),%esi
>        mov     $bootsym_phys(trampoline_start),%edi
>        mov     $trampoline_end - trampoline_start,%ecx
>        rep     movsb
> 
>        mov     $bootsym_phys(early_stack),%esp  //eary_stack.
>        call    cmdline_parse_early
> 
>        /* Jump into the relocated trampoline. */
>        jmp     $BOOT_CS32,$bootsym_phys(trampoline_boot_cpu_entry)
> 
> it seems that direct number is not very compatible .

I was just suggesting a minimal patch for you to try.

> 1, another point : it seems that this code should copy code (from
> trampoline_start to trampoline_end) to BOOT_TRAMPOLINE(0x90000)) , I
> will consider if trampoline code size bigger than
> 0x95400-0x90000=21KBytes .
> but my binary xen.gz doesn't include symbols , so I can't see address
> of trampoline_start and trampoline_end.  how to get them ? must
> recompile it ?I think maybe redhat has deleted symbols . yet I don't
> trampoline will be over 21KBytes,because it seems small code.

You could not move the trampoline code, and just move the stack, which is th
thing that is actually overlapping your EBDA currently. I.e., just change
the mov $0x98000,%esp instruction.

> 
> 2, about definition of trampoline_start and trampoline_end ?
> I just can see:
> #define BOOT_TRAMPOLINE 0x90000
> #define bootsym_phys(sym)                                 \
>    (((unsigned long)&(sym)-(unsigned long)&trampoline_start)+BOOT_TRAMPOLINE)
> #define bootsym(sym)                                      \
>    (*RELOC_HIDE((typeof(&(sym)))__va(__pa(&(sym))),      \
>                 BOOT_TRAMPOLINE-__pa(trampoline_start)))
> #ifndef __ASSEMBLY__
> extern char trampoline_start[], trampoline_end[];
> extern char trampoline_realmode_entry[];
> extern unsigned int trampoline_xen_phys_start;
> extern unsigned char trampoline_cpu_started;
> #endif
> 
> 。。。。。。。。。。
> head.S :
>        .globl trampoline_start, trampoline_end
> trampoline_start:  //  it seems that I must find this symbol to
> confirm its value , nm -a this file , displaying no symbols.
> #include "trampoline.S"
> trampoline_end:
> 
> 
> 3 , in sles11:
> 
> I find sles11 again , I did the same steps .
> in sles11 x86_64 version ,there are also :
> xen-syms-3.3.1_18546_12-3.1 and xen.gz :
> nm -a xen(unzip xen.gz) ,can't trampoline_start/end
> while nm -a xen-syms-3.3.1_18546_12-3.1 , I can finally find
> "trampoline_start" and "trampoline_end" :
> --bob comment: I begin another email subject "about __XEN_VIRT_START
> and 0x100000" to ask the difference between these 2 files.
> 
> from this output ,useful information:
> ffff828c80100000 A _start
> ...
> ffff828c8020d5b6 t trampoline_boot_cpu_entry
> ffff828c8020d549 T trampoline_cpu_started
> ffff828c8020fe50 T trampoline_end                                 //here
> ffff828c8020d511 t trampoline_gdt
> ffff828c8020d54a t trampoline_protmode_entry
> ffff828c8020d4e0 T trampoline_realmode_entry
> ffff828c8020d4e0 T trampoline_start                                  //here
> ffff828c8020d545 T trampoline_xen_phys_start
> ffff828c80172ab0 t transition_pstate
> ...
> from sles11 , trampoline code size   0xFFFF828C8_020FE50 -
> ffff828c8_020d4e0 =  0x2970  (about 10KBytes)
> if BOOT_TRAMPOLINE == 0x90000 , then trampoline end will be 0x92970
> ,while my EBDA is 0x954000 ~ 0xa0000 . so I think don't overlap.
> certainly , this number is from sles11 xen-kernel ,but I think their
> trampoline.S basily same ,length should be almost same.

Yes, you could probably leave the trampoline code where it is.

 -- Keir

Re: xen-kernel and EDD

[Lin-Bao Zhang]

[-- Attachment #1.1.1: Type: text/plain, Size: 115 bytes --]

I attach some other important information :
hopefully helpful for discussing further .



--- kernel booting info:

[-- Attachment #1.1.2: Type: text/html, Size: 370 bytes --]

[-- Attachment #1.2: image008.jpg --]
[-- Type: image/jpeg, Size: 85481 bytes --]

[-- Attachment #1.3: image006.jpg --]
[-- Type: image/jpeg, Size: 100792 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: xen-kernel and EDD

[Keir Fraser]

On 19/08/2011 10:15, "Lin-Bao Zhang" <2004.zhang@gmail.com> wrote:

> I attach some other important information :
> hopefully helpful for discussing further . 

The reserved area 0xf0000-0x100000 is for the BIOS ROM. Actually PCs do not
have any usable memory between 0xa0000 and 0x100000 -- no OS will try to
allocate memory from that range.

 -- Keir

> 
> 
> 
> --- kernel booting info:
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: xen-kernel and EDD

[Lin-Bao Zhang]

2011/8/19 Keir Fraser-5 [via Xen]
<ml-node+4714940-222445260-235647@n5.nabble.com>
>
> > it seems that direct number is not very compatible .
> I was just suggesting a minimal patch for you to try.
>
yes, I understand, thanks so much!
it seems that this stack indeed overlap my EBDA area wrongly.
any arbitrary or fixed value for this stack will cause un-expect error.

from higher xen-4.1.0 :
 mov     $bootsym_phys(early_stack),%esp
call    cmdline_parse_early
I guess , the author want to setup stack after trampoline code, from
symbols , we can see early_stack value is same as trampoline_end(from
sles11, I think rhel5.7 should be same).
the key points are stack beginning address and its length can't
overlap BIOS conventional area, i.e. EBDA.

how can we think out the length of this stack ?  xen developer must
make sure the length can't overlap EBDA? how to make sure it by method
?
I suppose xen-kernel should pre-know EBDA 's beginning address
,because xen can know space reserved by e820 output.

Re: Re: xen-kernel and EDD

[Lin-Bao Zhang]

I have replied to another subject "about __XEN_VIRT_START and
0x100000" ,because key discussing has moved there .
however, I sent a copy here , because ,here is the original discussion
about "EBDA corrupted".  sorry for my 2 emails .


> Stacks grow downwards/backwards in memory. The early_stack definition is
> immediately *before* the early_stack label.
>
>> b) what is function of ”   mov     $bootsym_phys(early_stack),%esp" ?
>> is this related with trampoline relocation ?
>
> The trampoline stack (early_stack) is now allocated within the trampoline
> code/data area, rather than being hard-coded at an address (e.g., 0x98000).
>
a) the Stack ,how to grow downwards ?
if device changed/added,the stack will grows much ?
what device or function code will use this stack ?
what time does this stack grow ?
Can this stack overwrite other things when grow downwards ? in
xen-4.1.0 , this stack top equals to trampoline_end ,will it overwrite
trampoline_code ?
I think maybe impossible ,because trampoline code has been moved to
BOOT_TRAMPOLINE .
why do I have such question ,please see my b)

b)
it seems that the core problem is the "early_stack" , in my case ,
Stack top is overlap P410i EBDA area when stack grows downwards,so
system can't boot again.
but But there is another question puzzled :
If 0x98000 value is wrong , we will think in any case , system will be
corrupted ,but if we disable or unplug FC card , system will boot
normally.
Very mystery.
Following is my guess:
I guess :    0x95400   ~   <------  0x98000 (stack top) ~  0xa0000
0x95400 … P410i_EBDA_start ….. P410i_EBDA_end …. 0x98000

We know stacks grow downwards/backwards in memory , so I guess P410i
EBDA area should be located in sub-area in 0x95400 ~0x98000.
I feel the stack size will change as device number.
Maybe ,if we plug FC card , the stack should be more larger ,
occasionally reach  P410i EBDA area. If unplug FC card , the stack
will be so small that it can’t reach P410i EBDA . who knows , maybe
only P410i firmware guys and FC card firmware guys , because only they
know their EBDA area range.
I will consult our firmware guys to verify my idea.

Re: xen-kernel and EDD

[Paolo Bonzini]

On 08/18/2011 01:04 PM, Keir Fraser wrote:
>> >  0x95400 - 0x90000 = 0x5400 = 21KBytes.
>> >
>> >  21Kbytes is enough for gen.gz running(hypervisor) ?

The trampoline is even smaller than 21K.  The problem is that the stack 
(at 0x98000) is overlapping the EBDA.

I suggest that you open a support case with Red Hat.  We do plan to make 
this change, but the process is much easier if we have an open ticket 
that requests it.

Paolo

Re: xen-kernel and EDD

[Lin-Bao Zhang]

yes, redhat has adopted this modification:

1. Change BOOT_TRAMPOLINE to 0x7c000
 2. In xen/arch/x86/boot/head.S, change the line 'mov $0x98000,%esp' to 'mov
 $0x80000,%esp'.
we are testing various cases.

I will update our testing result ASAP.


2011/8/23 Paolo Bonzini <pbonzini@redhat.com>:
> On 08/18/2011 01:04 PM, Keir Fraser wrote:
>>>
>>> >  0x95400 - 0x90000 = 0x5400 = 21KBytes.
>>> >
>>> >  21Kbytes is enough for gen.gz running(hypervisor) ?
>
> The trampoline is even smaller than 21K.  The problem is that the stack (at
> 0x98000) is overlapping the EBDA.
>
> I suggest that you open a support case with Red Hat.  We do plan to make
> this change, but the process is much easier if we have an open ticket that
> requests it.
>
> Paolo
>

Re: xen-kernel and EDD

[Lin-Bao Zhang]

what a pity !

it seems that our modification :
1. Change BOOT_TRAMPOLINE to 0x7c000
2. In xen/arch/x86/boot/head.S, change the line 'mov $0x98000,%esp' to
'mov $0x80000,%esp'.
has make things worse ,
after this picture
:https://picasaweb.google.com/lh/photo/_XQU3pCYJ9bSL4UgvBhFJem26SCOd2WkElE8x5uAaOk?feat=directlink
 system no response again ,and then reset .
I no longer can see
- 1706-Smart Array Controller EBDA Memory Corrupted
  INT 13h BIOS Cannot Coninue - Sytem Halted

Again !
 I doubt , maybe xen-kernel has corrupted before “get_edd” due to our
modification.
this is indeed a gloomy day indeed!

Now , it seems that things get more complicated :
I think there are two factors will affect :
 - Ensure Stack can't overlap HP's P410i EBDA area
 - Ensure Brocade card's option rom code can't overwrite P410i EBDA.

I also did another experiment :for speaking exactly, I use
:5.7s_xen-kernel(0x98000 stack and 0x90000 for BOOT_TRAMPOLINE) and
modified-xen-kernel(0x80000 STACK and 0x7c000 for BOOT_TRAMPOLINE) .

a)enable Brocade card(two ports) , but disable its option rom code) .
https://picasaweb.google.com/lh/photo/lVIqQjo2PESpX1JIzpEu0oIUdNTBh7One3tGCKQ3AHk?feat=directlink

 * 5.7s_xen-kernel can boot successfully(edd=on)
      from this point ,it seems that 0x98000 didn't overlap P410i EBDA. I just
disabled Brocade option rom.
 *  MODIFIED_xen-kernel can't boot successfully ,it also just print
https://picasaweb.google.com/lh/photo/_XQU3pCYJ9bSL4UgvBhFJem26SCOd2WkElE8x5uAaOk?feat=directlink
 no "INT13
BIOS Cannot Continue - System Halted".
 and no response, and then system reset.

-Bob

Re: xen-kernel and EDD

[Keir Fraser]

On 24/08/2011 08:50, "Lin-Bao Zhang" <2004.zhang@gmail.com> wrote:

> what a pity !
> 
> it seems that our modification :
> 1. Change BOOT_TRAMPOLINE to 0x7c000
> 2. In xen/arch/x86/boot/head.S, change the line 'mov $0x98000,%esp' to
> 'mov $0x80000,%esp'.
> has make things worse ,

Ah yes, the BOOT_TRAMPOLINE address is hardcoded into the trampoline_gdt
array in trampoline.S. You need to replace the definitions at
xen/arch/x86/boot/trampoline.S:trampoline_gdt with the following:

trampoline_gdt:
        /* 0x0000: unused */
        .quad   0x0000000000000000
        /* 0x0008: ring 0 code, 32-bit mode */
        .quad   0x00cf9a000000ffff
        /* 0x0010: ring 0 code, 64-bit mode */
        .quad   0x00af9a000000ffff
        /* 0x0018: ring 0 data */
        .quad   0x00cf92000000ffff
        /* 0x0020: real-mode code @ BOOT_TRAMPOLINE */
        .long   0x0000ffff | ((BOOT_TRAMPOLINE & 0x00ffff) << 16)
        .long   0x00009a00 | ((BOOT_TRAMPOLINE & 0xff0000) >> 16)
        /* 0x0028: real-mode data @ BOOT_TRAMPOLINE */
        .long   0x0000ffff | ((BOOT_TRAMPOLINE & 0x00ffff) << 16)
        .long   0x00009200 | ((BOOT_TRAMPOLINE & 0xff0000) >> 16)

That should fix things. This is from xen-unstable:16147 by the way.

 -- Keir

> after this picture
> :https://picasaweb.google.com/lh/photo/_XQU3pCYJ9bSL4UgvBhFJem26SCOd2WkElE8x5u
> AaOk?feat=directlink
>  system no response again ,and then reset .
> I no longer can see
> - 1706-Smart Array Controller EBDA Memory Corrupted
>   INT 13h BIOS Cannot Coninue - Sytem Halted
> 
> Again !
>  I doubt , maybe xen-kernel has corrupted before ³get_edd² due to our
> modification.
> this is indeed a gloomy day indeed!
> 
> Now , it seems that things get more complicated :
> I think there are two factors will affect :
>  - Ensure Stack can't overlap HP's P410i EBDA area
>  - Ensure Brocade card's option rom code can't overwrite P410i EBDA.
> 
> I also did another experiment :for speaking exactly, I use
> :5.7s_xen-kernel(0x98000 stack and 0x90000 for BOOT_TRAMPOLINE) and
> modified-xen-kernel(0x80000 STACK and 0x7c000 for BOOT_TRAMPOLINE) .
> 
> a)enable Brocade card(two ports) , but disable its option rom code) .
> https://picasaweb.google.com/lh/photo/lVIqQjo2PESpX1JIzpEu0oIUdNTBh7One3tGCKQ3
> AHk?feat=directlink
> 
>  * 5.7s_xen-kernel can boot successfully(edd=on)
>       from this point ,it seems that 0x98000 didn't overlap P410i EBDA. I just
> disabled Brocade option rom.
>  *  MODIFIED_xen-kernel can't boot successfully ,it also just print
> https://picasaweb.google.com/lh/photo/_XQU3pCYJ9bSL4UgvBhFJem26SCOd2WkElE8x5uA
> aOk?feat=directlink
>  no "INT13
> BIOS Cannot Continue - System Halted".
>  and no response, and then system reset.
> 
> -Bob

Re: xen-kernel and EDD

[BobZhang]

[-- Attachment #1.1: Type: text/plain, Size: 1232 bytes --]

Hi  Keir ,

rhel5.7s‘ xen version is :
kernel-2.6.18-268.el5.src.rpm
xen-3.0.3-132.el5.src.rpm

thanks very much for your sustained help on this issue.
Do you mean this link ?
http://xenbits.xensource.com/hg/xen-unstable.hg/rev/3b481059a54d
from your xen.org , I find http://xenbits.xen.org/hg/
there are many branches and one unstable hg。


I have a doubt , from this link , I can see this last update date is 2007 year.
and in config.h BOOT_TRAMPOLINE was still 0x94000 . It seems that 5
days ago : BOOT_TRAMPOLINE has just been set 0x7c000 (certainly, I we
all think 0x7c000 is very very reasonable,this should be the most
right value) .

I want to confirm :
Do you  think BOOT_TRAMPOLINE ’s value is independent on any other
later code ?  I prefer this idea to be right. it seems that your
comments page code that just make BOOT_TRAMPOLINE definition
independent.
if yes , I will try your comments plus BOOT_TRAMPOLINE 0X7C000

anway , I will try both 0x94000 and 0x7c000(this will be widely
suitable for all platform ).


--
View this message in context: http://xen.1045712.n5.nabble.com/xen-kernel-and-EDD-tp4710939p4729847.html
Sent from the Xen - Dev mailing list archive at Nabble.com.

[-- Attachment #1.2: Type: text/html, Size: 1732 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: xen-kernel and EDD

[Keir Fraser]

You can ignore the bit of the patch that touches BOOT_TRAMPOLINE in
include/asm-x86/config.h. You just need to apply the parts of the patch that
affect arch/x86/boot/trampoline.S.

 K.


On 24/08/2011 10:36, "BobZhang" <2004.zhang@gmail.com> wrote:

> Hi  Keir , rhel5.7s‘ xen version is : kernel-2.6.18-268.el5.src.rpm
> xen-3.0.3-132.el5.src.rpm thanks very much for your sustained help on this
> issue. Do you mean this link ?
> http://xenbits.xensource.com/hg/xen-unstable.hg/rev/3b481059a54dfrom your
> xen.org , I find http://xenbits.xen.org/hg/there are many branches and one
> unstable hg。 I have a doubt , from this link , I can see this last update date
> is 2007 year. and in config.h BOOT_TRAMPOLINE was still 0x94000 . It seems
> that 5 days ago : BOOT_TRAMPOLINE has just been set 0x7c000 (certainly, I we
> all think 0x7c000 is very very reasonable,this should be the most right value)
> . I want to confirm : Do you  think BOOT_TRAMPOLINE ’s value is independent on
> any other later code ?  I prefer this idea to be right. it seems that your
> comments page code that just make BOOT_TRAMPOLINE definition independent. if
> yes , I will try your comments plus BOOT_TRAMPOLINE 0X7C000 anway , I will try
> both 0x94000 and 0x7c000(this will be widely suitable for all platform ).
> 
> View this message in context: Re: xen-kernel and EDD
> <http://xen.1045712.n5.nabble.com/xen-kernel-and-EDD-tp4710939p4729847.html>
> Sent from the Xen - Dev mailing list archive
> <http://xen.1045712.n5.nabble.com/Xen-Dev-f2473738.html>  at Nabble.com.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: Re: xen-kernel and EDD

[Lin-Bao Zhang]

2011/8/24 Keir Fraser <keir.xen@gmail.com>:
> You can ignore the bit of the patch that touches BOOT_TRAMPOLINE in
> include/asm-x86/config.h. You just need to apply the parts of the patch that
> affect arch/x86/boot/trampoline.S.
>
>  K.
>

thanks very much！we have successfully booted xen-hypervisor .
Following is our final patch: I still define BOOT_TRAMPOLINE 0X7C000 .
We are using both HP P410i  and Brocade FC card. now it seems that
this bug doesn't relate directly with Brocade option rom.
but I guess Brocade option rom code may make Stack grow much more than
that Brocade card doesn't exist .
so when old 0x98000 stack top location , this stack will destroy P410i
EBDA when stack grows downwards.
meaning that , Brocade option code will make stack larger(this can be
verified by disable Brocade option rom code , 0x98000 still can work
in our system).


anway , following patch can indeed fix this bug, quite thank Keir's
continuous help!


diff --git a/arch/x86/boot/head.S b/arch/x86/boot/head.S
index 7e6c9e6..f761ef9 100644
--- a/arch/x86/boot/head.S
+++ b/arch/x86/boot/head.S
@@ -168,7 +168,7 @@ __start:
         mov     $trampoline_end - trampoline_start,%ecx
         rep     movsb

-        mov     $0x98000,%esp
+        mov     $(BOOT_TRAMPOLINE + 0x4000),%esp
         call    cmdline_parse_early

         /* Jump into the relocated trampoline. */
diff --git a/arch/x86/boot/trampoline.S b/arch/x86/boot/trampoline.S
index 4687be2..6ffaa81 100644
--- a/arch/x86/boot/trampoline.S
+++ b/arch/x86/boot/trampoline.S
@@ -23,12 +23,20 @@ idt_48: .word   0, 0, 0 # base = limit = 0
 gdt_48: .word   6*8-1
         .long   bootsym_phys(trampoline_gdt)
 trampoline_gdt:
-        .quad   0x0000000000000000     /* 0x0000: unused */
-        .quad   0x00cf9a000000ffff     /* 0x0008: ring 0 code, 32-bit mode */
-        .quad   0x00af9a000000ffff     /* 0x0010: ring 0 code, 64-bit mode */
-        .quad   0x00cf92000000ffff     /* 0x0018: ring 0 data */
-        .quad   0x00009a090000ffff     /* 0x0020: real-mode code @ 0x90000 */
-        .quad   0x000092090000ffff     /* 0x0028: real-mode data @ 0x90000 */
+        /* 0x0000: unused */
+        .quad   0x0000000000000000
+        /* 0x0008: ring 0 code, 32-bit mode */
+        .quad   0x00cf9a000000ffff
+        /* 0x0010: ring 0 code, 64-bit mode */
+        .quad   0x00af9a000000ffff
+        /* 0x0018: ring 0 data */
+        .quad   0x00cf92000000ffff
+        /* 0x0020: real-mode code @ BOOT_TRAMPOLINE */
+        .long   0x0000ffff | ((BOOT_TRAMPOLINE & 0x00ffff) << 16)
+        .long   0x00009a00 | ((BOOT_TRAMPOLINE & 0xff0000) >> 16)
+        /* 0x0028: real-mode data @ BOOT_TRAMPOLINE */
+        .long   0x0000ffff | ((BOOT_TRAMPOLINE & 0x00ffff) << 16)
+        .long   0x00009200 | ((BOOT_TRAMPOLINE & 0xff0000) >> 16)

 cpuid_ext_features:
         .long   0
@@ -142,7 +150,7 @@ trampoline_boot_cpu_entry:
         mov     %ax,%es
         mov     %ax,%ss

-        /* Stack grows down from 0x93000. Initialise IDT and enable irqs. */
+        /* Stack grows down from +0x3000. Initialise IDT and enable irqs. */
         mov     $0x3000,%sp
         lidt    bootsym(rm_idt)
         sti
diff --git a/include/asm-x86/config.h b/include/asm-x86/config.h
index 7fd368f..5efb374 100644
--- a/include/asm-x86/config.h
+++ b/include/asm-x86/config.h
@@ -89,7 +89,7 @@

 #define CONFIG_DMA_BITSIZE 32

-#define BOOT_TRAMPOLINE 0x90000
+#define BOOT_TRAMPOLINE 0x7c000
 #define bootsym_phys(sym)                                 \
     (((unsigned long)&(sym)-(unsigned long)&trampoline_start)+BOOT_TRAMPOLINE)
 #define bootsym(sym)

Re: Re: xen-kernel and EDD

[Lin-Bao Zhang]

Hi Keir,

I have another question about debugging Xen-hypervisor.
Do you have some advanced debugging method ?

I am reviewing xen source code, I just find in ../Config.mk there is a
debug = y
in xen/Rules.mk ,there is only "verbose" and "crash_debug" .

but just now , I recompile xen source to restore the INT13 error , but
I didn't see any useful and valuable messages.

do you have some comments ?thanks very much!
in a word, can you share your debugging experience ?  I think this
will be very useful for all xen source developers and debuggers.

-Bob

Re: Re: xen-kernel and EDD

[Keir Fraser]

On 25/08/2011 10:50, "Lin-Bao Zhang" <2004.zhang@gmail.com> wrote:

> Hi Keir,
> 
> I have another question about debugging Xen-hypervisor.
> Do you have some advanced debugging method ?
> 
> I am reviewing xen source code, I just find in ../Config.mk there is a
> debug = y
> in xen/Rules.mk ,there is only "verbose" and "crash_debug" .
> 
> but just now , I recompile xen source to restore the INT13 error , but
> I didn't see any useful and valuable messages.
> 
> do you have some comments ?thanks very much!
> in a word, can you share your debugging experience ?  I think this
> will be very useful for all xen source developers and debuggers.

I build with debug=y to turn on assertions and more verbose printing. I then
generally add extra printk tracing to close in on a specific bug. That's it
really.

 -- Keir

> -Bob

Re: Re: xen-kernel and EDD

[Lin-Bao Zhang]

2011/8/25 Keir Fraser <keir@xen.org>:
> I build with debug=y to turn on assertions and more verbose printing. I then
> generally add extra printk tracing to close in on a specific bug. That's it
> really.
>
>  -- Keir
>
Keir , thanks very much! I was ever ARM bootloader developer , I have
hardware ICE can set CPU stop when RESET . that's cool.   maybe I need
to persuade our boss to buy a x86_64 ICE debugger , that will be cool
and very expensive.

hopefully my dream can become true.
-Bob

Re: Re: xen-kernel and EDD

[Lin-Bao Zhang]

about this topic , it doesn't end.
in fact, we can't arbitrarily hard-code any number ,including
BOOT_TRAMPOLINE and stack(%esp).

I will special begin a new thread to discuss this question whose
subject is "define BOOT_TRAMPOLINE and stack based on result of
probing EBDA area by INT12"

Re: Re: xen-kernel and EDD

[Keir Fraser]

On 31/08/2011 08:50, "Lin-Bao Zhang" <2004.zhang@gmail.com> wrote:

> about this topic , it doesn't end.
> in fact, we can't arbitrarily hard-code any number ,including
> BOOT_TRAMPOLINE and stack(%esp).
> 
> I will special begin a new thread to discuss this question whose
> subject is "define BOOT_TRAMPOLINE and stack based on result of
> probing EBDA area by INT12"

Since the maximum EBDA size is 128kB, we should be fine sitting entirely
below 0x80000? Anyway, you otherwise have a chicken-and-egg situation: we
set up the trampoline to be able to execute real-mode code, and we need to
be in real mode to make legacy BIOS calls, hence we'd need to have the
trampoline set up to be able to make the BIOS call to decide where to locate
the trampoline. Which is a circular dependency.

 -- Keir