MCA and processors in SAL_BOOT_RENDEZ status

MCA and processors in SAL_BOOT_RENDEZ status

[John Lee]

Hi Tony and MCA experts, 

I have several questions w/r/t the global MCA escalation and APs left
waken-not when maxcpu is used to limit the number of processors for
Linux.
Per SAL spec, they are in BOOT_RENDEZV and hence not initialized/set to
handle the MC_RENDEZV_VECTOR that Linux MCA is to set. I.e., they are
active processors from SAL's view but nothing from Linux's.

Q1: Do they participate in SAL's monarch selection anyway and can
possibly be the monarch to execute OS_MCA code ? Or they just cannot
join the RENDEZVOUS and receive INIT later ?

Q2: the same question w/ the processors in HALT/HALT_LIGHT status.

Q3: the same question w/ the processors that are logically deconfigured
from Linux kernel at runtime but the deconfiguration has no report back
to SAL. In this case, SAL still sees the processor and can select it as
a monarch to execute OS_MCA which will conflict with Linux kernel.

Thanks,
John

RE: MCA and processors in SAL_BOOT_RENDEZ status

[Luck, Tony]

Interesting questions ... 

>Q1: Do they participate in SAL's monarch selection anyway and can
>possibly be the monarch to execute OS_MCA code ? Or they just cannot
>join the RENDEZVOUS and receive INIT later ?

Cpus that were never started by the OS shouldn't be a part of an
MCA rendezvous.

>Q2: the same question w/ the processors in HALT/HALT_LIGHT status.

Once woken, by the OS cpus become part of the system and so will be
rendezvoused for MCA.  PAL_HALT_LIGHT won't change this, I don't think
that PAL_HALT would either.

>Q3: the same question w/ the processors that are logically deconfigured
>from Linux kernel at runtime but the deconfiguration has no report back
>to SAL. In this case, SAL still sees the processor and can select it as
>a monarch to execute OS_MCA which will conflict with Linux kernel.

This is where it gets interesting ... I don't think that there is a
clearly defined way for the OS to deconfigure a processor to prevent
it coming back from the dead after the OS takes it offline.

-Tony

RE: MCA and processors in SAL_BOOT_RENDEZ status

[John Lee]

> From: Luck, Tony [mailto:tony.luck@intel.com] 
> Sent: Wednesday, June 02, 2004 1:47 PM
> To: John Lee; linux-ia64@vger.kernel.org
> Subject: RE: MCA and processors in SAL_BOOT_RENDEZ status
> 
> Interesting questions ... 
> 
> >Q1: Do they participate in SAL's monarch selection anyway and can
> >possibly be the monarch to execute OS_MCA code ? Or they just cannot
> >join the RENDEZVOUS and receive INIT later ?
> 
> Cpus that were never started by the OS shouldn't be a part of an
> MCA rendezvous.

Is that because the MC_RENDEZV_VECTOR is not set or they are in
BOOT_RENDEZV yet ?
What if the processors are waken and executing something EFI runtime
code but not participating in Linux domain? Can they be a monarch
processor to execute OS_MCA ?

> 
> >Q2: the same question w/ the processors in HALT/HALT_LIGHT status.
> 
> Once woken, by the OS cpus become part of the system and so will be
> rendezvoused for MCA.  PAL_HALT_LIGHT won't change this, I don't think
> that PAL_HALT would either.
> 
> >Q3: the same question w/ the processors that are logically 
> deconfigured
> >from Linux kernel at runtime but the deconfiguration has no 
> report back
> >to SAL. In this case, SAL still sees the processor and can 
> select it as
> >a monarch to execute OS_MCA which will conflict with Linux kernel.
> 
> This is where it gets interesting ... I don't think that there is a
> clearly defined way for the OS to deconfigure a processor to prevent
> it coming back from the dead after the OS takes it offline.
> 
> -Tony
> 
> 

Thanks
John

Re: MCA and processors in SAL_BOOT_RENDEZ status

[Ashok Raj]

On Wed, Jun 02, 2004 at 01:46:49PM -0700, Luck, Tony wrote:
> Interesting questions ... 
> 
> >Q1: Do they participate in SAL's monarch selection anyway and can
> >possibly be the monarch to execute OS_MCA code ? Or they just cannot
> >join the RENDEZVOUS and receive INIT later ?
> 
> Cpus that were never started by the OS shouldn't be a part of an
> MCA rendezvous.

Correct. Or to be more specific SAL should'nt select them as monarch
since officially OS has not taken control of that CPU.
> 
> >Q2: the same question w/ the processors in HALT/HALT_LIGHT status.
> 
> Once woken, by the OS cpus become part of the system and so will be
> rendezvoused for MCA.  PAL_HALT_LIGHT won't change this, I don't think
> that PAL_HALT would either.
> 
> >Q3: the same question w/ the processors that are logically deconfigured
> >from Linux kernel at runtime but the deconfiguration has no report back
> >to SAL. In this case, SAL still sees the processor and can select it as
> >a monarch to execute OS_MCA which will conflict with Linux kernel.
> 
> This is where it gets interesting ... I don't think that there is a
> clearly defined way for the OS to deconfigure a processor to prevent
> it coming back from the dead after the OS takes it offline.

A first step solution is to put the processor back in BOOT_RENDEZ mode.
which should be the last step to hand off the cpu to SAL.

In the current cpu hotplug code we just spin inside idle thread. 
in arch/ia64/kernel/process.c/play_dead(). Ideally this step
(will) cause a jump to the SAL code, by performing a BIG jump to
br0 saved when the processor was woken into AP mode.

See SAL spec 3.2.5 BR0 holds the return into SAL. If we save enough context
and branch, that would techinically put the processor back in the same mode.

(the above change is under devl for cpu hotplug support)

Cheers,
Ashok Raj
- Linux OS Team

RE: MCA and processors in SAL_BOOT_RENDEZ status

[John Lee]

>> >Q1: Do they participate in SAL's monarch selection anyway and can
>> >possibly be the monarch to execute OS_MCA code ? Or they just cannot
>> >join the RENDEZVOUS and receive INIT later ?
>>
>> Cpus that were never started by the OS shouldn't be a part of an
>> MCA rendezvous.
>
>Correct. Or to be more specific SAL should'nt select them as monarch
>since officially OS has not taken control of that CPU.
 
Is this because they are still in BOOT_RENDEZV state ? 
These CPUs are guaranteed not to participate in MCA processing?
How does SAL tell them from CPUs under Linux in monarch selection?
 
>> >Q3: the same question w/ the processors that are logically
deconfigured
>> >from Linux kernel at runtime but the deconfiguration has no report
back
>> >to SAL. In this case, SAL still sees the processor and can select it
as
>> >a monarch to execute OS_MCA which will conflict with Linux kernel.
>>
>> This is where it gets interesting ... I don't think that there is a
>> clearly defined way for the OS to deconfigure a processor to prevent
>> it coming back from the dead after the OS takes it offline.
>
>A first step solution is to put the processor back in BOOT_RENDEZ mode.
>which should be the last step to hand off the cpu to SAL.
>
>In the current cpu hotplug code we just spin inside idle thread.
>in arch/ia64/kernel/process.c/play_dead(). Ideally this step
>(will) cause a jump to the SAL code, by performing a BIG jump to
>br0 saved when the processor was woken into AP mode.
>
>See SAL spec 3.2.5 BR0 holds the return into SAL. If we save enough
context
>and branch, that would techinically put the processor back in the same
mode.
>
>(the above change is under devl for cpu hotplug support)

What if the CPU logically deconfigured from Linux is not put back in
BOOT_RENDEZV but is running its own independent code ? If global MCA can
come thru the CPU, is there any way to make SAL not choose the CPU -
such as a SAL call for Linux to call to specify disqualified processors
to reflect Linux's logical view?

Thanks
John

Re: MCA and processors in SAL_BOOT_RENDEZ status

[Ashok Raj]

On Mon, Jun 14, 2004 at 06:01:08PM -0700, John Lee wrote:
> 
>    >> >Q1: Do they participate in SAL's monarch selection anyway and can
>    >>  >possibly  be  the  monarch  to execute OS_MCA code ? Or they just
>    cannot
>    >> >join the RENDEZVOUS and receive INIT later ?
>    >>
>    >> Cpus that were never started by the OS shouldn't be a part of an
>    >> MCA rendezvous.
>    >
>    >Correct. Or to be more specific SAL should'nt select them as monarch
>    >since officially OS has not taken control of that CPU.
> 
> 
>    Is this because they are still in BOOT_RENDEZV state ?
>    These CPUs are guaranteed not to participate in MCA processing?
>    How does SAL tell them from CPUs under Linux in monarch selection?

SAL knows which processors are in OS control, since it also processes the wakeup
vector when OS sends a wakeup to wakeup the CPU. The wakeup vector is 
obtained from the sal tables via efi vars.

> 
>    >>  >Q3:  the  same  question  w/  the  processors  that are logically
>    deconfigured
>    mode.
>    What  if  the  processor  logically deconfigured from Linux is not put
>    back in BOOT_RENDEZV but is running its own independent code ? I think
>    global  MCA  can come thru the processor. Is there any way to make SAL
>    not choose the processor for global MCAs - for example, a SAL call for
>    Linux  to  call  to specify disqualified processors to reflect Linux's
>    logical view?


Humm, just curious, is this because you are trying to use this one cpu for some other 
processing outside of kernel resources, say for dedicated processing something...

I dont know of the top of my head, i will let you know once i find out.


-
Cheers,
Ashok Raj
- Linux OS Technologies Team