PowerPC + SMP

PowerPC + SMP

[Stuart Yoder]

[-- Attachment #1: Type: text/plain, Size: 429 bytes --]

Hi.
 
I am trying to figure out where in the PowerPC kernel the HID1 register
is updated to enable bits dealing with cache coherency in an SMP system.
Grepping through the arch/ppc source does not reveal much.
 
I have two 7447A processors and somewhere the ABE and SYNCBE bits in
HID1 need to be turned on to enable cache coherency.   Is supposed to
happen in the bootloader prior to the kernel running??
 
Thanks,
Stuart Yoder

[-- Attachment #2: Type: text/html, Size: 1408 bytes --]

Re: PowerPC + SMP

[Kumar Gala]

On Apr 25, 2005, at 4:10 PM, Stuart Yoder wrote:

> Hi.
> =A0
> I am trying to figure out where in the PowerPC kernel the HID1=20
> register is updated to enable bits dealing with cache coherency in an=20=

> SMP system.=A0=A0 Grepping through the arch/ppc source does not reveal=20=

> much.
> =A0
> I have=A0two 7447A processors and somewhere the ABE and SYNCBE bits in=20=

> HID1=A0need to be turned on to enable cache coherency.=A0=A0 Is =
supposed to=20
> happen in the bootloader prior to the kernel running??

The expectation is that the bootloader normally handles such things.

- kumar

RE: PowerPC + SMP

[Stuart Yoder]

Thanks Kumar.

How can I know what assumptions the kernel makes about the state of the
CPU/system when it begins execution? Is this clearly documented
anywhere?

Do you know of open source bootloaders for Linux that would demonstrate
how to set up the CPUs as the Linux kernel expects.

Thanks,
Stuart

> -----Original Message-----
> From: Kumar Gala [mailto:kumar.gala@freescale.com]=20
> Sent: Monday, April 25, 2005 4:40 PM
> To: Stuart Yoder
> Cc: linuxppc-embedded@ozlabs.org
> Subject: Re: PowerPC + SMP
>=20
>=20
> On Apr 25, 2005, at 4:10 PM, Stuart Yoder wrote:
>=20
> > Hi.
> > =A0
> > I am trying to figure out where in the PowerPC kernel the HID1
> > register is updated to enable bits dealing with cache=20
> coherency in an=20
> > SMP system.=A0=A0 Grepping through the arch/ppc source does not =
reveal=20
> > much.
> > =A0
> > I have=A0two 7447A processors and somewhere the ABE and SYNCBE bits =
in
> > HID1=A0need to be turned on to enable cache coherency.=A0=A0 Is=20
> supposed to=20
> > happen in the bootloader prior to the kernel running??
>=20
> The expectation is that the bootloader normally handles such things.
>=20
> - kumar
>=20
>=20
>=20
>=20

Re: PowerPC + SMP

[Kumar Gala]

On Apr 26, 2005, at 2:16 PM, Stuart Yoder wrote:

> Thanks Kumar.
>
> How can I know what assumptions the kernel makes about the state of the
>  CPU/system when it begins execution? Is this clearly documented
>  anywhere?

Unfortunately this is something we dont have clearly documented today.  
As they say the code is the documentation.

> Do you know of open source bootloaders for Linux that would demonstrate
>  how to set up the CPUs as the Linux kernel expects.

No, maybe take a look at linuxbios, not sure if it does SMP or not.  
However, I'm not aware of any open source PPC SMP bootloaders.

- kumar

PowerPC + SMP

[Stuart Yoder]

[-- Attachment #1: Type: text/plain, Size: 421 bytes --]

Hi.
 
I am trying to figure out where in the PowerPC kernel the HID1 register
is updated to enable bits dealing with cache coherency in an SMP system.
Grepping through the arch/ppc source does not reveal much.
 
I have two 7447A processors and somewhere the ABE and SYNCBE bits need
to be turned on to enable cache coherency.   Is supposed to happen in
the bootloader prior to the kernel running??
 
Thanks,
Stuart Yoder

[-- Attachment #2: Type: text/html, Size: 1439 bytes --]

Re: PowerPC + SMP

[Benjamin Herrenschmidt]

On Mon, 2005-04-25 at 16:11 -0500, Stuart Yoder wrote:
> Hi.
>  
> I am trying to figure out where in the PowerPC kernel the HID1
> register is updated to enable bits dealing with cache coherency in an
> SMP system.   Grepping through the arch/ppc source does not reveal
> much.
>  
> I have two 7447A processors and somewhere the ABE and SYNCBE bits need
> to be turned on to enable cache coherency.   Is supposed to happen in
> the bootloader prior to the kernel running??

It's usually expected to happen in the firmware yes, though the kernel
does some of it's own 'fixups' (look at setup_cpu_6xx.S)

Ben.

RE: PowerPC + SMP

[Stuart Yoder]

Thanks Ben.

That raises a question though-- how can I know what assumptions the
kernel makes about the state of the CPU/system when it begins execution?
Is this clearly documented anywhere?

I am using U-boot on my SMP system and U-boot is not SMP aware.  Do you
know of other open source bootloaders for Linux that would set up the
CPUs as the kernel expects.

Thanks,
Stuart Yoder


> -----Original Message-----
> From: Benjamin Herrenschmidt [mailto:benh@kernel.crashing.org] 
> Sent: Monday, April 25, 2005 10:37 PM
> To: Stuart Yoder
> Cc: linuxppc-dev list
> Subject: Re: PowerPC + SMP
> 
> 
> On Mon, 2005-04-25 at 16:11 -0500, Stuart Yoder wrote:
> > Hi.
> >  
> > I am trying to figure out where in the PowerPC kernel the HID1 
> > register is updated to enable bits dealing with cache 
> coherency in an
> > SMP system.   Grepping through the arch/ppc source does not reveal
> > much.
> >  
> > I have two 7447A processors and somewhere the ABE and 
> SYNCBE bits need
> > to be turned on to enable cache coherency.   Is supposed to 
> happen in
> > the bootloader prior to the kernel running??
> 
> It's usually expected to happen in the firmware yes, though 
> the kernel does some of it's own 'fixups' (look at setup_cpu_6xx.S)
> 
> Ben.
> 
> 
> 
> 
> 

Re: PowerPC + SMP

[Wolfgang Denk]

In message <054301c54a6f$d1cf82b0$2f010a0a@foundation.com> you wrote:
> 
> I am using U-boot on my SMP system and U-boot is not SMP aware.  Do you

Strictly speaking this statement is wrong.

So far I am not aware of a port of U-Boot to a SMP system.  If  there
was  such  a  port, and if it was properly done, then U-Boot probably
_were_ SMP aware.

Best regards,

Wolfgang Denk

-- 
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
Every program has at least one bug and can be shortened by  at  least
one  instruction  --  from  which,  by induction, one can deduce that
every program can be reduced to one instruction which doesn't work.

RE: PowerPC + SMP

[Stuart Yoder]

We've run into a few issues with U-boot, but they are not unsolveable--

-it doesn't provide an alternate execution path for the 2nd CPU when it
is started by the kernel
-U-boot needs to detect which CPU is executing
-U-boot on the 2nd CPU should not execute any platform/board specific
setup code-- this is stuff you only want to do once
-U-boot on the 2nd CPU cannot relocate itself to SDRAM because Linux is
already running there and assumes it owns all of memory
-it needs some way of getting to the Linux kernel entry point for
secondary CPUs
-it also appears that it should set up HID1 (other stuff??) for SMP
systems

Stuart

> -----Original Message-----
> From: Wolfgang Denk [mailto:wd@denx.de] 
> Sent: Tuesday, April 26, 2005 12:35 PM
> To: Stuart Yoder
> Cc: 'linuxppc-dev list'
> Subject: Re: PowerPC + SMP 
> 
> 
> In message <054301c54a6f$d1cf82b0$2f010a0a@foundation.com> you wrote:
> > 
> > I am using U-boot on my SMP system and U-boot is not SMP aware.  Do 
> > you
> 
> Strictly speaking this statement is wrong.
> 
> So far I am not aware of a port of U-Boot to a SMP system.  
> If  there was  such  a  port, and if it was properly done, 
> then U-Boot probably _were_ SMP aware.
> 
> Best regards,
> 
> Wolfgang Denk
> 
> -- 
> Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
> Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: 
> wd@denx.de Every program has at least one bug and can be 
> shortened by  at  least one  instruction  --  from  which,  
> by induction, one can deduce that every program can be 
> reduced to one instruction which doesn't work.
> 
> 
> 

RE: PowerPC + SMP

[Benjamin Herrenschmidt]

On Tue, 2005-04-26 at 09:54 -0500, Stuart Yoder wrote:
> Thanks Ben.
> 
> That raises a question though-- how can I know what assumptions the
> kernel makes about the state of the CPU/system when it begins execution?
> Is this clearly documented anywhere?
> 
> I am using U-boot on my SMP system and U-boot is not SMP aware.  Do you
> know of other open source bootloaders for Linux that would set up the
> CPUs as the kernel expects.

No, it's up to you to configure the CPU properly. I'm surprised that
uBoot doesn't set it up properly though, but then, when you bringup a
board, you rarely use a bootloader like that "as-is" :)

In general, the kernel except the CPU to be in working condition, which
I consider meanings having necessary HID bits set for proper coherency
etc... on an SMP system :)

Ben.

RE: PowerPC + SMP

[Paul Mackerras]

Stuart Yoder writes:

> That raises a question though-- how can I know what assumptions the
> kernel makes about the state of the CPU/system when it begins execution?
> Is this clearly documented anywhere?

In general the kernel assumes that firmware has set up the HID
registers, caches, northbridge, memory controller, memory, etc., in a
suitable state for the machine to run at full speed.  There is code to
set and clear various HID bits, depending on the specific cpu model -
look for the __setup_cpu_* functions in arch/ppc/kernel/*.S.  I guess
we could add code to set ABE and SYNCBE on 7447A processors on SMP
systems; I can't imagine that any SMP system would require them to be
clear.

Paul.

RE: PowerPC + SMP

[Frank]

--- Stuart YoYoderststuartoyoderoconformativeom> wrote:
> 
> We've run into a few issues with U-boot, but they are not
> ununsolveable
> 
> -it doesn't provide an alternate execution path for the 2nd
> CPU when it
> is started by the kernel

You have to set a flag or something to sync. the booting process
of the other cpu's

> -U-boot needs to detect which CPU is executing
> -U-boot on the 2nd CPU should not execute any platform/board
> specific

We did this by having the cpcpuead a register that is updated as
each cpcpuomes up.

> setup code-- this is stuff you only want to do once
> -U-boot on the 2nd CPU cannot relocate itself to SDSDRAMecause
> Linux is
> already running there and assumes it owns all of memory

You have to carve the memory up by dividing the total memory by
the number of cpus. This can be done at complie time with a
#define. I did this in board_init_f I believe. We had a quad
7450 board with each one booting in sequence and orchestrated by
the master CPU (cpu 0). The master held each one in reset until
the previous cpu reported in. Unfortunately, I don't work for
the company anymore and don't have access to the source.
BTW, it was done right because it worked.:-)

> -it needs some way of getting to the Linux kernel entry point
> for
> secondary CPUs
> -it also appears that it should set up HID1 (other stuff??)
> for SMSMP> systems
> 
> Stuart
> 
> > -----Original Message-----
> > From: Wolfgang DeDenkmamailtodwdedenxede
> > Sent: Tuesday, April 26, 2005 12:35 PM
> > To: Stuart YoYoder> > Cc: 'lilinuxppcedevist'
> > Subject: Re: PoPowerPC SMSMP
> > 
> > 
> > In message <054301c54a6f$d1cf82b0$2f010a0a@foundation.com>
> you wrote:
> > > 
> > > I am using U-boot on my SMSMPystem and U-boot is not
SMSMP> aware.  Do 
> > > you
> > 
> > Strictly speaking this statement is wrong.
> > 
> > So far I am not aware of a port of U-Boot to a SMSMPystem.  
> > If  there was  such  a  port, and if it was properly done, 
> > then U-Boot probably _were_ SMSMPware.
> > 
> > Best regards,
> > 
> > Wolfgang DeDenk> > 
> > -- 
> > Software Engineering:  Embedded and ReRealtimeystems, 
> Embedded Linux
> > Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: 
> > wdwdedenxedevery program has at least one bug and can be 
> > shortened by  at  least one  instruction  --  from  which,  
> > by induction, one can deduce that every program can be 
> > reduced to one instruction which doesn't work.
> > 
> > 
> > 
> 
> 
> _______________________________________________
> LiLinuxppcedevailing list
> LiLinuxppcedevzozlabsrorg>
hthttps/ozozlabsrorgailman/lilistinfoilinuxppcedev> 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com