Multiple processor support?

Multiple processor support?

[Matthew Dharm]

Does the MIPS port of linux support multiple-processor architectures?

Matt Dharm

--
Matthew D. Dharm                            Senior Software Designer
Momentum Computer Inc.                      1815 Aston Ave.  Suite 107
(760) 431-8663 X-115                        Carlsbad, CA 92008-7310
Momentum Works For You                      www.momenco.com

Re: Multiple processor support?

[Justin Carlson]

On Fri, 23 Mar 2001, you wrote:
> Does the MIPS port of linux support multiple-processor architectures?
> 

MIPS or MIPS64?

-Justin

RE: Multiple processor support?

[Matthew Dharm]

Well, I'd like to know about both, frankly.  Tho I'm more interested
in whichever is designed to run on RM7000 series processors.

Matt

--
Matthew D. Dharm                            Senior Software Designer
Momentum Computer Inc.                      1815 Aston Ave.  Suite 107
(760) 431-8663 X-115                        Carlsbad, CA 92008-7310
Momentum Works For You                      www.momenco.com

> -----Original Message-----
> From: Justin Carlson [mailto:carlson@sibyte.com]
> Sent: Friday, March 23, 2001 2:58 PM
> To: Matthew Dharm
> Cc: linux-mips@oss.sgi.com
> Subject: Re: Multiple processor support?
>
>
> On Fri, 23 Mar 2001, you wrote:
> > Does the MIPS port of linux support multiple-processor
> architectures?
> >
>
> MIPS or MIPS64?
>
> -Justin
>
>

RE: Multiple processor support?

[Matthew Dharm]

Well, I'd like to know about both, frankly.  Tho I'm more interested
in whichever is designed to run on RM7000 series processors.

Matt

--
Matthew D. Dharm                            Senior Software Designer
Momentum Computer Inc.                      1815 Aston Ave.  Suite 107
(760) 431-8663 X-115                        Carlsbad, CA 92008-7310
Momentum Works For You                      www.momenco.com

> -----Original Message-----
> From: Justin Carlson [mailto:carlson@sibyte.com]
> Sent: Friday, March 23, 2001 2:58 PM
> To: Matthew Dharm
> Cc: linux-mips@oss.sgi.com
> Subject: Re: Multiple processor support?
>
>
> On Fri, 23 Mar 2001, you wrote:
> > Does the MIPS port of linux support multiple-processor
> architectures?
> >
>
> MIPS or MIPS64?
>
> -Justin
>
>

RE: Multiple processor support?

[Justin Carlson]

On Fri, 23 Mar 2001, Matthew Dharm wrote:
> Well, I'd like to know about both, frankly.  Tho I'm more interested
> in whichever is designed to run on RM7000 series processors.

To the best of my knowledge, the mips64 tree only works in SMP on the ip-27
which is r10K based.  There would be a bit of work to get an RM7K  based
multiprocessor system to run. A fair amount of the "generic" code in
that tree is also pretty ip-27 specific, and so would need to be cleaned up.

I'm working on mips32 SMP support at the moment; there are no existing ports of
this tree to an SMP platform.  The mips64 stuff is certainly much, much more 
mature.  I don't know of any reasons not to use the mips64 side for an RM7K.

-Justin

RE: Multiple processor support?

[Justin Carlson]

On Fri, 23 Mar 2001, Matthew Dharm wrote:
> Well, I'd like to know about both, frankly.  Tho I'm more interested
> in whichever is designed to run on RM7000 series processors.

To the best of my knowledge, the mips64 tree only works in SMP on the ip-27
which is r10K based.  There would be a bit of work to get an RM7K  based
multiprocessor system to run. A fair amount of the "generic" code in
that tree is also pretty ip-27 specific, and so would need to be cleaned up.

I'm working on mips32 SMP support at the moment; there are no existing ports of
this tree to an SMP platform.  The mips64 stuff is certainly much, much more 
mature.  I don't know of any reasons not to use the mips64 side for an RM7K.

-Justin

Re: Multiple processor support?

[Keith M Wesolowski]

On Fri, Mar 23, 2001 at 04:08:13PM -0800, Justin Carlson wrote:

> To the best of my knowledge, the mips64 tree only works in SMP on the ip-27
> which is r10K based.  There would be a bit of work to get an RM7K  based
> multiprocessor system to run. A fair amount of the "generic" code in
> that tree is also pretty ip-27 specific, and so would need to be cleaned up.

In the particular case of SMP, this is probably still true.  However,
as of last weekend this tree compiles properly and boots part way on
Indy; I have more to do but the separation of the obviously
ip27-dependent stuff is done; you should at least be able to compile
other machines in mips64...if there were any.

-- 
Keith M Wesolowski <wesolows@foobazco.org> http://foobazco.org/~wesolows
------(( Project Foobazco Coordinator and Network Administrator ))------
"I should have crushed his marketing-addled skull with a fucking bat."

Re: Multiple processor support?

[Kevin D. Kissell]

> > Well, I'd like to know about both, frankly.  Tho I'm more interested
> > in whichever is designed to run on RM7000 series processors.

None are "designed" as such for the RM7000.  I don't have a
full RM7000 manual in the shop, but the short-form advance
manual that I do have, while it goes into a fair amount of
detail about the cache operations and external interface
protocols, makes no mention of any support for hardware
coherence of the sort provided by the R10000/R12000
(and the R4000/R4400 for that matter).  If there is no support
for cached/coherent memory attributes and cache interventions
from the system side, an SMP kernel design for an MP SGI box
might not be useful for an MP RM7K configuration.  It is possible,
but tricky, and at times unavoidably inefficient to build a
software-coherent SMP system.  I have not heard of anyone
doing so with MIPS/Linux.

> To the best of my knowledge, the mips64 tree only works in SMP on the
ip-27
> which is r10K based.  There would be a bit of work to get an RM7K  based
> multiprocessor system to run. A fair amount of the "generic" code in
> that tree is also pretty ip-27 specific, and so would need to be cleaned
up.
>
> I'm working on mips32 SMP support at the moment; there are no existing
ports of
> this tree to an SMP platform.  The mips64 stuff is certainly much, much
more
> mature.  I don't know of any reasons not to use the mips64 side for an
RM7K.

Well, one reason might be memory footprint...

            Regards,

            Kevin K.

Re: Multiple processor support?

[Kevin D. Kissell]

> > Well, I'd like to know about both, frankly.  Tho I'm more interested
> > in whichever is designed to run on RM7000 series processors.

None are "designed" as such for the RM7000.  I don't have a
full RM7000 manual in the shop, but the short-form advance
manual that I do have, while it goes into a fair amount of
detail about the cache operations and external interface
protocols, makes no mention of any support for hardware
coherence of the sort provided by the R10000/R12000
(and the R4000/R4400 for that matter).  If there is no support
for cached/coherent memory attributes and cache interventions
from the system side, an SMP kernel design for an MP SGI box
might not be useful for an MP RM7K configuration.  It is possible,
but tricky, and at times unavoidably inefficient to build a
software-coherent SMP system.  I have not heard of anyone
doing so with MIPS/Linux.

> To the best of my knowledge, the mips64 tree only works in SMP on the
ip-27
> which is r10K based.  There would be a bit of work to get an RM7K  based
> multiprocessor system to run. A fair amount of the "generic" code in
> that tree is also pretty ip-27 specific, and so would need to be cleaned
up.
>
> I'm working on mips32 SMP support at the moment; there are no existing
ports of
> this tree to an SMP platform.  The mips64 stuff is certainly much, much
more
> mature.  I don't know of any reasons not to use the mips64 side for an
RM7K.

Well, one reason might be memory footprint...

            Regards,

            Kevin K.

Re: Multiple processor support?

[Ralf Baechle]

On Sat, Mar 24, 2001 at 01:40:47AM +0100, Kevin D. Kissell wrote:

> Well, one reason might be memory footprint...

As of now the memory footprint of the kernel is large but userspace which
is all 32-bit software has unchanged footprint.  I've got plans for 2.5
to reduce the memory footprint of the kernel by introducing a 2-level
pagetable.

  Ralf

Re: Multiple processor support?

[Ralf Baechle]

On Sat, Mar 24, 2001 at 01:40:47AM +0100, Kevin D. Kissell wrote:

> Well, one reason might be memory footprint...

As of now the memory footprint of the kernel is large but userspace which
is all 32-bit software has unchanged footprint.  I've got plans for 2.5
to reduce the memory footprint of the kernel by introducing a 2-level
pagetable.

  Ralf

Re: Multiple processor support?

[Jun Sun]

"Kevin D. Kissell" wrote:
> 
> (Software cache coherency) It is possible,
> but tricky, and at times unavoidably inefficient to build a
> software-coherent SMP system.  I have not heard of anyone
> doing so with MIPS/Linux.
>

How would it be possible?  Any reference to the previous implementations?

I imagine you would need at least some kind of atomic operation (like ll/sc)
working reliably (which itself may require cache coherency).  Also, any such
scheme should not require massive change in the programming.

I am very curious....

Jun

Re: Multiple processor support?

[Kevin D. Kissell]

> > (Software cache coherency) It is possible,
> > but tricky, and at times unavoidably inefficient to build a
> > software-coherent SMP system.  I have not heard of anyone
> > doing so with MIPS/Linux.
> >
>
> How would it be possible?  Any reference to the previous implementations?

Lots of work on software coherent schemes was done in the
mid-late 1980s.  Check out the ASPLOS, and ISCA proceedings
from the period for references.  In essence, such schemes involve
the identification of critical regions at risk, the use of barriers around
such regions, and an explicit cache flush/purge protocol.  You can think
of the more common MP "TLB shootdown" protocols as being a variant
of a software cache coherence scheme.

> I imagine you would need at least some kind of atomic operation (like
ll/sc)
> working reliably (which itself may require cache coherency).

MIPS ll/sc, as defined and implemented, does require hardware
coherency support for correct multiprocessor operation.  But one
can, in principle, construct a software-coherent SMP system even
in the absence of such a primitive - many of the implementations
of software coherent SMPs used software coherence precisely
because they were based on simple switch/crossbar interconnects
where snooping was not possible.

> Also, any such
> scheme should not require massive change in the programming.

Whether progams need to change depends on the coherency
and consistency models assumed by the program.  Certainly
a naive multithreaded program that assumes an SGI-like model
could not be dropped onto a software-coherent MP system without
recompilation with specialized compilers at a minimum, and
more likely not without recoding.  On the other hand, if one's objective
is to run multiple, independent programs on different CPUs in
an SMP system, it should only be the OS that should need to
change to deal with the coherence issues for shared user pages
and shared kernel data structures, and to ensure that any
multithreaded application that is not explicitly set up to handle
software cache coherency has its threads bound to the same
CPU and caches (defeats some of the point of having a
multithreaded program, I know, but...).

            Regards,

            Kevin K.

Re: Multiple processor support?

[Kevin D. Kissell]

> > (Software cache coherency) It is possible,
> > but tricky, and at times unavoidably inefficient to build a
> > software-coherent SMP system.  I have not heard of anyone
> > doing so with MIPS/Linux.
> >
>
> How would it be possible?  Any reference to the previous implementations?

Lots of work on software coherent schemes was done in the
mid-late 1980s.  Check out the ASPLOS, and ISCA proceedings
from the period for references.  In essence, such schemes involve
the identification of critical regions at risk, the use of barriers around
such regions, and an explicit cache flush/purge protocol.  You can think
of the more common MP "TLB shootdown" protocols as being a variant
of a software cache coherence scheme.

> I imagine you would need at least some kind of atomic operation (like
ll/sc)
> working reliably (which itself may require cache coherency).

MIPS ll/sc, as defined and implemented, does require hardware
coherency support for correct multiprocessor operation.  But one
can, in principle, construct a software-coherent SMP system even
in the absence of such a primitive - many of the implementations
of software coherent SMPs used software coherence precisely
because they were based on simple switch/crossbar interconnects
where snooping was not possible.

> Also, any such
> scheme should not require massive change in the programming.

Whether progams need to change depends on the coherency
and consistency models assumed by the program.  Certainly
a naive multithreaded program that assumes an SGI-like model
could not be dropped onto a software-coherent MP system without
recompilation with specialized compilers at a minimum, and
more likely not without recoding.  On the other hand, if one's objective
is to run multiple, independent programs on different CPUs in
an SMP system, it should only be the OS that should need to
change to deal with the coherence issues for shared user pages
and shared kernel data structures, and to ensure that any
multithreaded application that is not explicitly set up to handle
software cache coherency has its threads bound to the same
CPU and caches (defeats some of the point of having a
multithreaded program, I know, but...).

            Regards,

            Kevin K.

Re: Multiple processor support?

[Joe George]

Unfortunately the RM7000 does not have hardware cache coherency
support.

Joe George

Matthew Dharm wrote:

> Well, I'd like to know about both, frankly.  Tho I'm more interested
> in whichever is designed to run on RM7000 series processors.
> 
> Matt
> 
> --
> Matthew D. Dharm                            Senior Software Designer
> Momentum Computer Inc.                      1815 Aston Ave.  Suite 107
> (760) 431-8663 X-115                        Carlsbad, CA 92008-7310
> Momentum Works For You                      www.momenco.com
> 
> 
>> -----Original Message-----
>> From: Justin Carlson [mailto:carlson@sibyte.com]
>> Sent: Friday, March 23, 2001 2:58 PM
>> To: Matthew Dharm
>> Cc: linux-mips@oss.sgi.com
>> Subject: Re: Multiple processor support?
>> 
>> 
>> On Fri, 23 Mar 2001, you wrote:
>> 
>>> Does the MIPS port of linux support multiple-processor
>> 
>> architectures?
>> 
>> MIPS or MIPS64?
>> 
>> -Justin
>> 
>>