Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux

Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux

[Tristan Gingold]

Le Mercredi 27 Septembre 2006 11:36, Magnus Damm a écrit :
> On 9/15/06, Horms <horms@verge.net.au> wrote:
> > Hi,
> >
> > as some of you may be aware I am working on porting kexec
> > to xen/ia64. I have made a reasoble ammount of progress to this end.
> > I'll try and get a new patch set on xen-devel some time next week.
> > However I have a problem that I need some ideas on how to solve.
> >
> > At the moment when kexecing from xen to linux the boot halts on a call
> > to efi_gettimeofday(), or more specifically efi.get_time. I'm assuming
> > that this is more or less the first efi runtime call that is made, and
> > that it is halting because of a discrepancy in the virtual mapping set
> > up by efi.set_virtual_address_map().
> >
> > The problem as I see it is that linux uses a page_offset that covers the
> > most significant 3 bits, wherase xen uses the first 4. The unfortunate
> > thing is that efi.set_virtual_address_map() can only be called once,
> > and I don't think its possible to change the mappings at all once
> > its been called.
> >
> > One idea that I had was to make sure that the efi calls are always made
> > in real mode, and never call efi.set_virtual_address_map() at all - efi
> > calls have to be made using virtual addressing after
> > efi.set_virtual_address_map() is called. But can this work?
> >
> > Another idea from my colleague Magnus was to map the efi runtime calls
> > into some area of memory that is agreed upon by both Linux and Xen (and
> > any other kexec-able OS/hypervisor). This seems to be tedious at best.
>
> To clarify this a bit, my plan was to extend the bootloader to provide
> some kind resident efi filter code. This code should act as a filter
> for all efi run time calls including the dreaded now use-once
> set_virtual_address_map() function. The most important task for this
> code would be to support an unlimited number of
> set_virtual_address_map() calls. I suspect this can be done by always
> calling the efi functions from real mode. This technique does however
> require switching back and forth to real mode, and I'm not sure how
> well that will work out with NMI:s and data that crosses page
> boundaries.
>
> A first step would probably be to try to convert Linux into calling
> efi runtime functions from real mode. If that works well then the code
> can be broken out, made resident and placed into the bootloader.

Linux and xen call efi in real mode if set_virtual_address_map fails.
You may add an option in both xen and linux to force calling efi in real mode.
This should be really simple and you will be able to make progress.

The only possible drawback is performance.

Tristan.

Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux

[Horms]

On Wed, Sep 27, 2006 at 11:52:12AM +0200, Tristan Gingold wrote:
> 
> Linux and xen call efi in real mode if set_virtual_address_map fails.
> You may add an option in both xen and linux to force calling efi in real mode.
> This should be really simple and you will be able to make progress.

Great, I will test this out and see how it goes.

> The only possible drawback is performance.

What kind of performance issues would you expect?

-- 
Horms
  H: http://www.vergenet.net/~horms/
  W: http://www.valinux.co.jp/en/

Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux

[Tristan Gingold]

Le Jeudi 28 Septembre 2006 03:27, Horms a écrit :
> On Wed, Sep 27, 2006 at 11:52:12AM +0200, Tristan Gingold wrote:
> > Linux and xen call efi in real mode if set_virtual_address_map fails.
> > You may add an option in both xen and linux to force calling efi in real
> > mode. This should be really simple and you will be able to make progress.
>
> Great, I will test this out and see how it goes.
>
> > The only possible drawback is performance.
>
> What kind of performance issues would you expect?
Making EFI calls in physical mode is slower: Linux must switch from and to 
virtual mode.

However EFI calls are very unfrequent so the impact should be almost nul.

Tristan.

Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux

[Horms]

On Thu, Sep 28, 2006 at 08:55:22AM +0200, Tristan Gingold wrote:
> Le Jeudi 28 Septembre 2006 03:27, Horms a écrit :
> > On Wed, Sep 27, 2006 at 11:52:12AM +0200, Tristan Gingold wrote:
> > > Linux and xen call efi in real mode if set_virtual_address_map fails.
> > > You may add an option in both xen and linux to force calling efi in real
> > > mode. This should be really simple and you will be able to make progress.
> >
> > Great, I will test this out and see how it goes.
> >
> > > The only possible drawback is performance.
> >
> > What kind of performance issues would you expect?
> Making EFI calls in physical mode is slower: Linux must switch from and to 
> virtual mode.
> 
> However EFI calls are very unfrequent so the impact should be almost nul.

Thats what I was thinking. Lets see how it pans out.

-- 
Horms
  H: http://www.vergenet.net/~horms/
  W: http://www.valinux.co.jp/en/

Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux

[Magnus Damm]

On 9/27/06, Tristan Gingold <Tristan.Gingold@bull.net> wrote:
> Le Mercredi 27 Septembre 2006 11:36, Magnus Damm a écrit :
> > On 9/15/06, Horms <horms@verge.net.au> wrote:
> > > Hi,
> > >
> > > as some of you may be aware I am working on porting kexec
> > > to xen/ia64. I have made a reasoble ammount of progress to this end.
> > > I'll try and get a new patch set on xen-devel some time next week.
> > > However I have a problem that I need some ideas on how to solve.
> > >
> > > At the moment when kexecing from xen to linux the boot halts on a call
> > > to efi_gettimeofday(), or more specifically efi.get_time. I'm assuming
> > > that this is more or less the first efi runtime call that is made, and
> > > that it is halting because of a discrepancy in the virtual mapping set
> > > up by efi.set_virtual_address_map().
> > >
> > > The problem as I see it is that linux uses a page_offset that covers the
> > > most significant 3 bits, wherase xen uses the first 4. The unfortunate
> > > thing is that efi.set_virtual_address_map() can only be called once,
> > > and I don't think its possible to change the mappings at all once
> > > its been called.
> > >
> > > One idea that I had was to make sure that the efi calls are always made
> > > in real mode, and never call efi.set_virtual_address_map() at all - efi
> > > calls have to be made using virtual addressing after
> > > efi.set_virtual_address_map() is called. But can this work?
> > >
> > > Another idea from my colleague Magnus was to map the efi runtime calls
> > > into some area of memory that is agreed upon by both Linux and Xen (and
> > > any other kexec-able OS/hypervisor). This seems to be tedious at best.
> >
> > To clarify this a bit, my plan was to extend the bootloader to provide
> > some kind resident efi filter code. This code should act as a filter
> > for all efi run time calls including the dreaded now use-once
> > set_virtual_address_map() function. The most important task for this
> > code would be to support an unlimited number of
> > set_virtual_address_map() calls. I suspect this can be done by always
> > calling the efi functions from real mode. This technique does however
> > require switching back and forth to real mode, and I'm not sure how
> > well that will work out with NMI:s and data that crosses page
> > boundaries.
> >
> > A first step would probably be to try to convert Linux into calling
> > efi runtime functions from real mode. If that works well then the code
> > can be broken out, made resident and placed into the bootloader.
>
> Linux and xen call efi in real mode if set_virtual_address_map fails.
> You may add an option in both xen and linux to force calling efi in real mode.
> This should be really simple and you will be able to make progress.

Oh, is that so? I have not looked at the code yet, only discussed this
issue with Horms so far. But that's good news. Thanks for the
suggestion!

/ magnus

Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux

[Magnus Damm]

On 9/28/06, Tristan Gingold <Tristan.Gingold@bull.net> wrote:
> Le Jeudi 28 Septembre 2006 03:27, Horms a écrit :
> > On Wed, Sep 27, 2006 at 11:52:12AM +0200, Tristan Gingold wrote:
> > > Linux and xen call efi in real mode if set_virtual_address_map fails.
> > > You may add an option in both xen and linux to force calling efi in real
> > > mode. This should be really simple and you will be able to make progress.
> >
> > Great, I will test this out and see how it goes.
> >
> > > The only possible drawback is performance.
> >
> > What kind of performance issues would you expect?
> Making EFI calls in physical mode is slower: Linux must switch from and to
> virtual mode.
>
> However EFI calls are very unfrequent so the impact should be almost nul.

This makes me wonder - is it really worth having two code paths in that case?

/ magnus

Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux

[Tristan Gingold]

Le Jeudi 28 Septembre 2006 14:34, Magnus Damm a écrit :
> On 9/28/06, Tristan Gingold <Tristan.Gingold@bull.net> wrote:
> > Le Jeudi 28 Septembre 2006 03:27, Horms a écrit :
> > > On Wed, Sep 27, 2006 at 11:52:12AM +0200, Tristan Gingold wrote:
> > > > Linux and xen call efi in real mode if set_virtual_address_map fails.
> > > > You may add an option in both xen and linux to force calling efi in
> > > > real mode. This should be really simple and you will be able to make
> > > > progress.
> > >
> > > Great, I will test this out and see how it goes.
> > >
> > > > The only possible drawback is performance.
> > >
> > > What kind of performance issues would you expect?
> >
> > Making EFI calls in physical mode is slower: Linux must switch from and
> > to virtual mode.
> >
> > However EFI calls are very unfrequent so the impact should be almost nul.
>
> This makes me wonder - is it really worth having two code paths in that
> case?
According to linux/arch/ia64/kernel/efi.c, the code to switch efi to virtual 
mode was added later.  I don't know why this was added.  Maybe the answer is 
in the web?

Tristan.

RE: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux

[Zou, Nanhai]

> -----Original Message-----
> From: Magnus Damm [mailto:magnus.damm@gmail.com]
> Sent: 2006Äê9ÔÂ28ÈÕ 20:34
> To: Tristan Gingold
> Cc: Horms; Zou, Nanhai; Linux-IA64; xen-ia64-devel@lists.xensource.com
> Subject: Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux
> 
> On 9/28/06, Tristan Gingold <Tristan.Gingold@bull.net> wrote:
> > Le Jeudi 28 Septembre 2006 03:27, Horms a ¨¦crit :
> > > On Wed, Sep 27, 2006 at 11:52:12AM +0200, Tristan Gingold wrote:
> > > > Linux and xen call efi in real mode if set_virtual_address_map fails.
> > > > You may add an option in both xen and linux to force calling efi in real
> > > > mode. This should be really simple and you will be able to make progress.
> > >
> > > Great, I will test this out and see how it goes.
> > >
> > > > The only possible drawback is performance.
> > >
> > > What kind of performance issues would you expect?
> > Making EFI calls in physical mode is slower: Linux must switch from and to
> > virtual mode.
> >
> > However EFI calls are very unfrequent so the impact should be almost nul.
> 
> This makes me wonder - is it really worth having two code paths in that case?
> 
> / magnus

I am still not quiet clear about the particular issue on Xen.

For native IA64,
I put an empty efi.set_virtual_address_map() in purgatory code, 
So when the second kernel boots, it will still call to set_virtual_address_map() as if it successes. 
I guess you can modify the empty function in purgatory to return an error.
So the second kernel will call EFI in physical mode.

Thanks
Zou Nan hai 

Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux

[Horms]

On Fri, Sep 29, 2006 at 10:21:33AM +0800, Zou, Nanhai wrote:
> > -----Original Message-----
> > From: Magnus Damm [mailto:magnus.damm@gmail.com]
> > Sent: 2006年9月28日 20:34
> > To: Tristan Gingold
> > Cc: Horms; Zou, Nanhai; Linux-IA64; xen-ia64-devel@lists.xensource.com
> > Subject: Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux
> > 
> > On 9/28/06, Tristan Gingold <Tristan.Gingold@bull.net> wrote:
> > > Le Jeudi 28 Septembre 2006 03:27, Horms a écrit :
> > > > On Wed, Sep 27, 2006 at 11:52:12AM +0200, Tristan Gingold wrote:
> > > > > Linux and xen call efi in real mode if set_virtual_address_map fails.
> > > > > You may add an option in both xen and linux to force calling efi in real
> > > > > mode. This should be really simple and you will be able to make progress.
> > > >
> > > > Great, I will test this out and see how it goes.
> > > >
> > > > > The only possible drawback is performance.
> > > >
> > > > What kind of performance issues would you expect?
> > > Making EFI calls in physical mode is slower: Linux must switch from and to
> > > virtual mode.
> > >
> > > However EFI calls are very unfrequent so the impact should be almost nul.
> > 
> > This makes me wonder - is it really worth having two code paths in that case?
> > 
> > / magnus
> 
> I am still not quiet clear about the particular issue on Xen.
> 
> For native IA64,
> I put an empty efi.set_virtual_address_map() in purgatory code, So
> when the second kernel boots, it will still call to
> set_virtual_address_map() as if it successes.  I guess you can modify
> the empty function in purgatory to return an error.  So the second
> kernel will call EFI in physical mode.

I'm not 100% sure what the issue with xen->linux and linux->xen is
either. However my theory is that it relates to the fact that in linux
PAGE_OFFSET is 0xe000..., while in Xen it is 0xf000..., and thus any
addressing setup using efi.set_virtual_address_map() in one would seem
to be invalid in the other.

In any case, the problem that I am seeing is that while linux->linux and
xen->xen (UP only for now) can be made to work, both xen->linux and
linux->xen halt when making an EFI call fairly early on in boot.

More specifically, it halts on a call to efi_gettimeofday()
which is backed by efi.get_time. Other than the call to 
efi.set_virtual_address_map() I'm assuming that this is 
more or less the first efi runtime call that is made.

-- 
Horms
  H: http://www.vergenet.net/~horms/
  W: http://www.valinux.co.jp/en/

Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux

[Horms]

On Wed, Sep 27, 2006 at 11:52:12AM +0200, Tristan Gingold wrote:
> Linux and xen call efi in real mode if set_virtual_address_map fails.
> You may add an option in both xen and linux to force calling efi in real mode.
> This should be really simple and you will be able to make progress.

I took a stab at forcing the call to stay in real mode, 
by simply replacing the call to set_virtual_address_map with
a return. This should both prevent the mapping from occuring,
and force the kernel to use the real mode variants of the calls.

Unfortunately, the boot fails with the following log.
A bit of investigation has found that is it dying in

   SAL_CALL()
   called by: ia64_sal_cache_flush()
   called by: ia64_sal_cache_flush()
   called by: ia64_sal_init()

Well, when I say dying in, its probably more accurate to say,
never returning from.

The trace below is with 2.6.18. I had identical results with a recent
checkout of ia64-test. I can post my config if needed.

I'm a bit of a loss to know why this is occuring. Though
I do wonder if SAL (and in this case indrectly PAL) calls are
are running into trouble by runing in real mode without
set_virtual_address_map() having being called.

I tried poking around a bit, but to no avail.
Any ideas on where to poke around?

-- 
Horms
  H: http://www.vergenet.net/~horms/
  W: http://www.valinux.co.jp/en/

Linux version 2.6.18-kexec-dirty (horms@tabatha.lab.ultramonkey.org)
(gcc version 3.4.5) #2 SMP Fri Sep 29 12:38:35 JST 2006 EFI v1.10 by INTEL: SALsystab=0x7fe54980 ACPI=0x7ff99000 ACPI
2.0=0x7ff98000 MPS=0x7ff97000 SMBIOS=0xf0000
Forcing EFI calls to stay in physical mode
booting generic kernel on platform dig
Early serial console at I/O port 0x2f8 (options '115200')
Initial ramdisk at: 0xe00000007aecd000 (10463216 bytes)
SAL 3.20: Intel Corp                       SR870BH2                         version 3.0
SAL Platform features: BusLock
SAL: AP wakeup using external interrupt vector 0xf0
swapper[0]: General Exception: IA-64 Reserved Register/Field fault (data access) 2199023255600 [1]
Modules linked in:

Pid: 0, CPU 0, comm:              swapper
psr : 00001210084a2010 ifs : 8000000000000183 ip  : [<000000007fe21dc1>]
Not tainted
ip is at 0x7fe21dc1
unat: 0000000000000000 pfs : 0000000000000818 rsc : 0000000000000003
rnat: 0000000000000000 bsps: 0000000000000000 pr  : 80000000aff55d2b
ldrs: 0000000000000000 ccv : 0000000000000000 fpsr: 0009804c8a70033f
csd : 0930ffff00090000 ssd : 0930ffff00090000
b0  : 000000007fe21ca0 b6  : 000000007fe08010 b7  : a000000100205420
f6  : 0fffafffffffff0000000 f7  : 0ffdef000000000000000
f8  : 10002f000000000000000 f9  : 100038000000000000000
f10 : 0fffeeffffffff1000000 f11 : 1003e0000000000000000
r1  : e000000080095280 r2  : 0000000000000000 r3  : e00000007fe59340
r8  : fffffffffffffffe r9  : 000000000000000f r10 : 0000000000000000
r11 : 0000000000000000 r12 : a00000010057fc30 r13 : a000000100578000
r14 : 000000007fe20300 r15 : 00000000000004d5 r16 : 0000000000000000
r17 : 00001010084a2010 r18 : 000000000007e000 r19 : 80000000aff54cab
r20 : 00000000000006c2 r21 : 0000000000000000 r22 : 0000000000000000
r23 : 0000000000000609 r24 : 00000000003ffc00 r25 : 0000000000000008
r26 : 0000000000000000 r27 : 00000010084a2010 r28 : 000000000000003f
r29 : 0000000000000000 r30 : 0000000000000001 r31 : 0000000000000000

Call Trace:
 [<a0000001000124f0>] show_stack+0x50/0xa0
                                sp 0000010057f640 bsp 00000100579368
 <4>kernel unaligned access to 0xffffffffffff45c3, ip=0xa0000001000d9451
kernel unaligned access to 0xffffffffffff45c3, ip=0xa0000001000d9470
irq 239, desc: a0000001005b1c80, depth: 1, count: 0, unhandled: 0
->handle_irq():  a00000010050dcd0, __stop___param+0x3550/0x15880
->chip(): a000000100612f88, no_irq_chip+0x0/0x80
->action(): 0000000000000000
  IRQ_DISABLED set
Unexpected irq vector 0xef on CPU 0!

RE: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux

[Zou, Nanhai]

> -----Original Message-----
> From: linux-ia64-owner@vger.kernel.org
> [mailto:linux-ia64-owner@vger.kernel.org] On Behalf Of Horms
> Sent: 2006年9月29日 11:03
> To: Zou, Nanhai
> Cc: Magnus Damm; Tristan Gingold; Linux-IA64;
> xen-ia64-devel@lists.xensource.com
> Subject: Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux
> 
> On Fri, Sep 29, 2006 at 10:21:33AM +0800, Zou, Nanhai wrote:
> > > -----Original Message-----
> > > From: Magnus Damm [mailto:magnus.damm@gmail.com]
> > > Sent: 2006年9月28日 20:34
> > > To: Tristan Gingold
> > > Cc: Horms; Zou, Nanhai; Linux-IA64; xen-ia64-devel@lists.xensource.com
> > > Subject: Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux
> > >
> > > On 9/28/06, Tristan Gingold <Tristan.Gingold@bull.net> wrote:
> > > > Le Jeudi 28 Septembre 2006 03:27, Horms a écrit :
> > > > > On Wed, Sep 27, 2006 at 11:52:12AM +0200, Tristan Gingold wrote:
> > > > > > Linux and xen call efi in real mode if set_virtual_address_map fails.
> > > > > > You may add an option in both xen and linux to force calling efi in
> real
> > > > > > mode. This should be really simple and you will be able to make progress.
> > > > >
> > > > > Great, I will test this out and see how it goes.
> > > > >
> > > > > > The only possible drawback is performance.
> > > > >
> > > > > What kind of performance issues would you expect?
> > > > Making EFI calls in physical mode is slower: Linux must switch from and
> to
> > > > virtual mode.
> > > >
> > > > However EFI calls are very unfrequent so the impact should be almost nul.
> > >
> > > This makes me wonder - is it really worth having two code paths in that
> case?
> > >
> > > / magnus
> >
> > I am still not quiet clear about the particular issue on Xen.
> >
> > For native IA64,
> > I put an empty efi.set_virtual_address_map() in purgatory code, So
> > when the second kernel boots, it will still call to
> > set_virtual_address_map() as if it successes.  I guess you can modify
> > the empty function in purgatory to return an error.  So the second
> > kernel will call EFI in physical mode.
> 
> I'm not 100% sure what the issue with xen->linux and linux->xen is
> either. However my theory is that it relates to the fact that in linux
> PAGE_OFFSET is 0xe000..., while in Xen it is 0xf000..., and thus any
> addressing setup using efi.set_virtual_address_map() in one would seem
> to be invalid in the other.
> 
 That is why I suggest you to return error in the set_virtual_address_map() in purgatory code. That will force the second kernel to use EFI physical call.

Zou Nan hai
-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux

[Horms]

On Fri, Sep 29, 2006 at 01:13:17PM +0800, Zou, Nanhai wrote:
> > -----Original Message-----
> > From: linux-ia64-owner@vger.kernel.org
> > [mailto:linux-ia64-owner@vger.kernel.org] On Behalf Of Horms
> > Sent: 2006年9月29日 11:03
> > To: Zou, Nanhai
> > Cc: Magnus Damm; Tristan Gingold; Linux-IA64;
> > xen-ia64-devel@lists.xensource.com
> > Subject: Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux
> > 
> > On Fri, Sep 29, 2006 at 10:21:33AM +0800, Zou, Nanhai wrote:
> > > > -----Original Message-----
> > > > From: Magnus Damm [mailto:magnus.damm@gmail.com]
> > > > Sent: 2006年9月28日 20:34
> > > > To: Tristan Gingold
> > > > Cc: Horms; Zou, Nanhai; Linux-IA64; xen-ia64-devel@lists.xensource.com
> > > > Subject: Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux
> > > >
> > > > On 9/28/06, Tristan Gingold <Tristan.Gingold@bull.net> wrote:
> > > > > Le Jeudi 28 Septembre 2006 03:27, Horms a écrit :
> > > > > > On Wed, Sep 27, 2006 at 11:52:12AM +0200, Tristan Gingold wrote:
> > > > > > > Linux and xen call efi in real mode if set_virtual_address_map fails.
> > > > > > > You may add an option in both xen and linux to force calling efi in
> > real
> > > > > > > mode. This should be really simple and you will be able to make progress.
> > > > > >
> > > > > > Great, I will test this out and see how it goes.
> > > > > >
> > > > > > > The only possible drawback is performance.
> > > > > >
> > > > > > What kind of performance issues would you expect?
> > > > > Making EFI calls in physical mode is slower: Linux must switch from and
> > to
> > > > > virtual mode.
> > > > >
> > > > > However EFI calls are very unfrequent so the impact should be almost nul.
> > > >
> > > > This makes me wonder - is it really worth having two code paths in that
> > case?
> > > >
> > > > / magnus
> > >
> > > I am still not quiet clear about the particular issue on Xen.
> > >
> > > For native IA64,
> > > I put an empty efi.set_virtual_address_map() in purgatory code, So
> > > when the second kernel boots, it will still call to
> > > set_virtual_address_map() as if it successes.  I guess you can modify
> > > the empty function in purgatory to return an error.  So the second
> > > kernel will call EFI in physical mode.
> > 
> > I'm not 100% sure what the issue with xen->linux and linux->xen is
> > either. However my theory is that it relates to the fact that in linux
> > PAGE_OFFSET is 0xe000..., while in Xen it is 0xf000..., and thus any
> > addressing setup using efi.set_virtual_address_map() in one would seem
> > to be invalid in the other.
> > 
>  That is why I suggest you to return error in the set_virtual_address_map() in purgatory code. That will force the second kernel to use EFI physical call.

Sure. I think that is a pretty good idea. Except that we
need to make sure that set_virtual_address_map() isn't called
by the original code (well at least not a real working version),
as once it is called real mode EFI calls can no longer be made.

-- 
Horms
  H: http://www.vergenet.net/~horms/
  W: http://www.valinux.co.jp/en/

Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux

[Horms]

On Fri, Sep 29, 2006 at 12:48:19PM +0900, Horms wrote:
> On Wed, Sep 27, 2006 at 11:52:12AM +0200, Tristan Gingold wrote:
> > Linux and xen call efi in real mode if set_virtual_address_map fails.
> > You may add an option in both xen and linux to force calling efi in real mode.
> > This should be really simple and you will be able to make progress.
> 
> I took a stab at forcing the call to stay in real mode, 
> by simply replacing the call to set_virtual_address_map with
> a return. This should both prevent the mapping from occuring,
> and force the kernel to use the real mode variants of the calls.
> 
> Unfortunately, the boot fails with the following log.
> A bit of investigation has found that is it dying in
> 
>    SAL_CALL()
>    called by: ia64_sal_cache_flush()
>    called by: ia64_sal_cache_flush()
>    called by: ia64_sal_init()
> 
> Well, when I say dying in, its probably more accurate to say,
> never returning from.
> 
> The trace below is with 2.6.18. I had identical results with a recent
> checkout of ia64-test. I can post my config if needed.
> 
> I'm a bit of a loss to know why this is occuring. Though
> I do wonder if SAL (and in this case indrectly PAL) calls are
> are running into trouble by runing in real mode without
> set_virtual_address_map() having being called.

It turns out that this was indeed the case, and running SAL calls
in physical mode (regardless of if EFI is making physical or virtual
mode calls) seems to fix this boot failure. Though I am yet to determin
it it solves the xen->linux, linux->xen kexec problem.

I will post a patch shortly.

-- 
Horms
  H: http://www.vergenet.net/~horms/
  W: http://www.valinux.co.jp/en/

Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux

[Bjorn Helgaas]

On Wednesday 04 October 2006 20:53, Horms wrote:
> On Fri, Sep 29, 2006 at 12:48:19PM +0900, Horms wrote:
> > On Wed, Sep 27, 2006 at 11:52:12AM +0200, Tristan Gingold wrote:
> > > Linux and xen call efi in real mode if set_virtual_address_map fails.
> > > You may add an option in both xen and linux to force calling efi in real mode.
> > > This should be really simple and you will be able to make progress.
> > 
> > I took a stab at forcing the call to stay in real mode, 
> > by simply replacing the call to set_virtual_address_map with
> > a return. This should both prevent the mapping from occuring,
> > and force the kernel to use the real mode variants of the calls.
> > 
> > Unfortunately, the boot fails with the following log.
> > A bit of investigation has found that is it dying in
> > 
> >    SAL_CALL()
> >    called by: ia64_sal_cache_flush()
> >    called by: ia64_sal_cache_flush()
> >    called by: ia64_sal_init()
> > 
> > Well, when I say dying in, its probably more accurate to say,
> > never returning from.

I haven't been following this thread and don't know whether
this is related, but someone here is seeing an MCA in this
early SAL_CACHE_FLUSH call.  I haven't looked at it in detail
yet, but apparently it happens because we're making the call
in virtual mode before we have completely set up the TLB and VHPT.

> > I'm a bit of a loss to know why this is occuring. Though
> > I do wonder if SAL (and in this case indrectly PAL) calls are
> > are running into trouble by runing in real mode without
> > set_virtual_address_map() having being called.

SAL_CACHE_FLUSH is one of a few SAL calls that invoke PAL
procedures, and they can't be used in virtual mode until the
OS registers the virtual address of PAL (see SAL spec section
9.1.1).

> It turns out that this was indeed the case, and running SAL calls
> in physical mode (regardless of if EFI is making physical or virtual
> mode calls) seems to fix this boot failure. Though I am yet to determin
> it it solves the xen->linux, linux->xen kexec problem.

In general (I'm not speaking to xen or kexec), I think we want
to run SAL calls in virtual mode.  It looks like we have a problem
with the specific SAL_CACHE_FLUSH call above, but we should be able
to fix that specific problem without changing how we do other SAL
calls.

Bjorn

Re: [Xen-ia64-devel] Re: ia64 kexec: xen -> linux

[Horms]

On Thu, Oct 05, 2006 at 10:52:58AM -0600, Bjorn Helgaas wrote:
> On Wednesday 04 October 2006 20:53, Horms wrote:
> > On Fri, Sep 29, 2006 at 12:48:19PM +0900, Horms wrote:
> > > On Wed, Sep 27, 2006 at 11:52:12AM +0200, Tristan Gingold wrote:
> > > > Linux and xen call efi in real mode if set_virtual_address_map fails.
> > > > You may add an option in both xen and linux to force calling efi in real mode.
> > > > This should be really simple and you will be able to make progress.
> > > 
> > > I took a stab at forcing the call to stay in real mode, 
> > > by simply replacing the call to set_virtual_address_map with
> > > a return. This should both prevent the mapping from occuring,
> > > and force the kernel to use the real mode variants of the calls.
> > > 
> > > Unfortunately, the boot fails with the following log.
> > > A bit of investigation has found that is it dying in
> > > 
> > >    SAL_CALL()
> > >    called by: ia64_sal_cache_flush()
> > >    called by: ia64_sal_cache_flush()
> > >    called by: ia64_sal_init()
> > > 
> > > Well, when I say dying in, its probably more accurate to say,
> > > never returning from.
> 
> I haven't been following this thread and don't know whether
> this is related, but someone here is seeing an MCA in this
> early SAL_CACHE_FLUSH call.  I haven't looked at it in detail
> yet, but apparently it happens because we're making the call
> in virtual mode before we have completely set up the TLB and VHPT.
> 
> > > I'm a bit of a loss to know why this is occuring. Though
> > > I do wonder if SAL (and in this case indrectly PAL) calls are
> > > are running into trouble by runing in real mode without
> > > set_virtual_address_map() having being called.
> 
> SAL_CACHE_FLUSH is one of a few SAL calls that invoke PAL
> procedures, and they can't be used in virtual mode until the
> OS registers the virtual address of PAL (see SAL spec section
> 9.1.1).
> 
> > It turns out that this was indeed the case, and running SAL calls
> > in physical mode (regardless of if EFI is making physical or virtual
> > mode calls) seems to fix this boot failure. Though I am yet to determin
> > it it solves the xen->linux, linux->xen kexec problem.
> 
> In general (I'm not speaking to xen or kexec), I think we want
> to run SAL calls in virtual mode.  It looks like we have a problem
> with the specific SAL_CACHE_FLUSH call above, but we should be able
> to fix that specific problem without changing how we do other SAL
> calls.

Right now, all EFI and SAL calls are made in virtual mode, as
the code path to allow them to run in physical mode is broken -
it tries to run EFI calls in physlcal mode but SAL calls in virtual
mode. 

I have a patch to fix that problem which I will post ASAP.
It allows the physical path to work. And it allows you to force
the OS to make physical calls. But by default the existing,
try to use virtual behaviour is used.

I was trying to scratch my head as to why to use virtual at all,
but you may well have a good reason as you explained above.

-- 
Horms
  H: http://www.vergenet.net/~horms/
  W: http://www.valinux.co.jp/en/