MIPS64 status?

MIPS64 status?

[Matthew Dharm]

So, lots of people are asking me about 64-bit support for MIPS Linux.  And
this is something I think I know, but I'm not sure.  And before I go
spreading any more bad data, I thought I'd ask the experts. :)

As I understand it, 64-bit support is really two different things:  64-bit
data path (i.e. unsigned long long) and 64-bit addressing (for more than 4G
of RAM).

My understanding is that "MIPS64" generally refers to a kernel which
supports a 64-bit data path, but we're still limited to 32-bit addressing.
Is that correct?

I suspect that this is very much a toolchain issue, as I don't think gcc
will generate 64-bit addressing code.

Comments?  Corrections?  Smack-downs? :)

Matt

-- 
Matthew Dharm                              Work: mdharm@momenco.com
Senior Software Designer, Momentum Computer

Re: MIPS64 status?

[Jason Gunthorpe]

On Sun, 13 Jan 2002, Matthew Dharm wrote:

> As I understand it, 64-bit support is really two different things:  64-bit
> data path (i.e. unsigned long long) and 64-bit addressing (for more than 4G
> of RAM).

I suppose you could say that. I think I saw someone post to this list
that they were working on a patch to enable 64 bit registers with a 32 bit
kernel.

> My understanding is that "MIPS64" generally refers to a kernel which
> supports a 64-bit data path, but we're still limited to 32-bit addressing.
> Is that correct?

The mips64 tree in CVS is one that is a 64 bit addressing capable kernel.
AFAIK the linx convention is that <foo>64 refers to addressing (which
probably impiles register width too :>)

I'm not sure what the current state of the ELF64 MIPS toolchain is.

Jason

Re: MIPS64 status?

[Dominic Sweetman]

Matthew,

> As I understand it, 64-bit support is really two different things:
> 64-bit data path (i.e. unsigned long long) and 64-bit addressing
> (for more than 4G of RAM).

Yes: the MIPS architecture is designed so there are lots of different
things which can be "64-bit", and you don't have to go for them all at
once.  This kind of choice can be as much curse as blessing, of course.

Ralf etc have worked (with some sponsorship from SGI) on a full-blown
system where you get:

o A very large physical address space
o Very large virtual address spaces, using 64-bit pointer types.
o C "long" (and perhaps even "int") becomes 64-bit.

In such a 64-bit Linux system, though, you might still want to be able
to run 32-bit applications with 32-bit pointers, int and long - either
for compatibility or economy (32-bit data types make for a smaller
program).  SGI do this in Irix: I don't know whether the 64-bit
Linux/MIPS systems got around to it.

There are other potentially useful combinations:

o A Linux where all machine-supported integer data types are 32-bit,
  but capable of addressing physical memory outside of a 4Gbyte map.
  (In practice, you need to use this kind of system to get outside of
  a 512Mbyte map - so it's urgent).

  Ralf says he has done this: it could be done without using any
  64-bit operations, but it might be easier with them.

o A system using 32-bit pointers and 'long' throughout, but with
  support for 'long long' 64-bit integer data types in registers.
  
o A system using 64-bit addressing within the kernel, but not for
  applications. 

However, it's unlikely to make sense to do all of them!

> I suspect that this is very much a toolchain issue, as I don't think
> gcc will generate 64-bit addressing code.

I suspect that the generic GNU toolchains are pretty buggy when you
switch on 64-bit MIPS operation; but it's bug-fixes which are needed,
not wholesale new features.

Politics: MIPS Technologies' advocacy for their "MIPS32" instruction
set dialect in embedded systems means there are now some quite capable
MIPS CPUs (eg Alchemy's 500Mhz integrated CPUs) which don't have
64-bit datapaths or arithmetic.  So casual dependence on 64-bit
operations should probably be avoided.

-- 
Dominic Sweetman
Algorithmics Ltd
The Fruit Farm, Ely Road, Chittering, CAMBS CB5 9PH, ENGLAND
phone: +44 1223 706200 / fax: +44 1223 706250 / home: +44 20 7226 0032
http://www.algor.co.uk

Re: MIPS64 status?

[Kevin D. Kissell]

> > As I understand it, 64-bit support is really two different things:
> > 64-bit data path (i.e. unsigned long long) and 64-bit addressing
> > (for more than 4G of RAM).
> 
> Yes: the MIPS architecture is designed so there are lots of different
> things which can be "64-bit", and you don't have to go for them all at
> once.  This kind of choice can be as much curse as blessing, of course.

Careful, Dom.  As far as user-mode programs are concerned,
older 64-bit MIPS designs (R4xxxx/R5xxxx/R7xxxx), one cannot
enable 64-bit arithmetic without enabling 64-bit addressing,
both of these functions being enabled by the Status.UX bit.
SGI's IRIX OS allowed an execution model that provided
64-bit registers and math, while *simulating* a 32-bit address
space, based on sign-extending 32-bit addresses to 64-bits.
The user was spared doubling the footprint of all his pointers,
but the OS still had to manage the larger page tables.

The official MIPS64[tm] architecture spec from MIPS 
Technologies also provides a bit (Status.PX) which enables
the 64-bit data path without affecting address generation
and translation, which removes this quirk.  Only the very
most recent 64-bit cores and CPUs implement it, however.


            Regards,

            Kevin K.

Re: MIPS64 status?

[Kevin D. Kissell]

> > As I understand it, 64-bit support is really two different things:
> > 64-bit data path (i.e. unsigned long long) and 64-bit addressing
> > (for more than 4G of RAM).
> 
> Yes: the MIPS architecture is designed so there are lots of different
> things which can be "64-bit", and you don't have to go for them all at
> once.  This kind of choice can be as much curse as blessing, of course.

Careful, Dom.  As far as user-mode programs are concerned,
older 64-bit MIPS designs (R4xxxx/R5xxxx/R7xxxx), one cannot
enable 64-bit arithmetic without enabling 64-bit addressing,
both of these functions being enabled by the Status.UX bit.
SGI's IRIX OS allowed an execution model that provided
64-bit registers and math, while *simulating* a 32-bit address
space, based on sign-extending 32-bit addresses to 64-bits.
The user was spared doubling the footprint of all his pointers,
but the OS still had to manage the larger page tables.

The official MIPS64[tm] architecture spec from MIPS 
Technologies also provides a bit (Status.PX) which enables
the 64-bit data path without affecting address generation
and translation, which removes this quirk.  Only the very
most recent 64-bit cores and CPUs implement it, however.


            Regards,

            Kevin K.

Re: MIPS64 status?

[Maciej W. Rozycki]

On Mon, 14 Jan 2002, Kevin D. Kissell wrote:

> Careful, Dom.  As far as user-mode programs are concerned,
> older 64-bit MIPS designs (R4xxxx/R5xxxx/R7xxxx), one cannot
> enable 64-bit arithmetic without enabling 64-bit addressing,
> both of these functions being enabled by the Status.UX bit.
> SGI's IRIX OS allowed an execution model that provided
> 64-bit registers and math, while *simulating* a 32-bit address
> space, based on sign-extending 32-bit addresses to 64-bits.
> The user was spared doubling the footprint of all his pointers,
> but the OS still had to manage the larger page tables.

 Note that there is usually no point in using a 64D/32A mode unless you
have weird toolchain problems.  You may get a 64D/32A user program by
mapping its segments appropriately in the 31-bit address space.  Since the
MIPS user segment always starts at zero such a program won't see a
difference between a 64D/32A and a 64D/64A mode.

 Why would an OS need larger page tables?

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

RE: MIPS64 status?

[Matthew Dharm]

See, it's answers like this that make me skeptical and confused...

So MIPS64 denotes 64-bit data _and_ address?  Great.  But, what is the
current state of the toolchain?  I mean, what point is having the code
if the tools won't compile it properly?

I guess what I'm looking for is what I should tell someone who wants
to run MIPS Linux 64-bit with 8 gigs of RAM.

Matt

--
Matthew D. Dharm                            Senior Software Designer
Momentum Computer Inc.                      1815 Aston Ave.  Suite 107
(760) 431-8663 X-115                        Carlsbad, CA 92008-7310
Momentum Works For You                      www.momenco.com

> -----Original Message-----
> From: Jason Gunthorpe [mailto:jgg@debian.org]
> Sent: Monday, January 14, 2002 12:13 AM
> To: Matthew Dharm
> Cc: linux-mips@oss.sgi.com
> Subject: Re: MIPS64 status?
>
>
>
> On Sun, 13 Jan 2002, Matthew Dharm wrote:
>
> > As I understand it, 64-bit support is really two
> different things:  64-bit
> > data path (i.e. unsigned long long) and 64-bit addressing
> (for more than 4G
> > of RAM).
>
> I suppose you could say that. I think I saw someone post to
> this list
> that they were working on a patch to enable 64 bit
> registers with a 32 bit
> kernel.
>
> > My understanding is that "MIPS64" generally refers to a
> kernel which
> > supports a 64-bit data path, but we're still limited to
> 32-bit addressing.
> > Is that correct?
>
> The mips64 tree in CVS is one that is a 64 bit addressing
> capable kernel.
> AFAIK the linx convention is that <foo>64 refers to
> addressing (which
> probably impiles register width too :>)
>
> I'm not sure what the current state of the ELF64 MIPS toolchain is.
>
> Jason
>

RE: MIPS64 status?

[Matthew Dharm]

See, it's answers like this that make me skeptical and confused...

So MIPS64 denotes 64-bit data _and_ address?  Great.  But, what is the
current state of the toolchain?  I mean, what point is having the code
if the tools won't compile it properly?

I guess what I'm looking for is what I should tell someone who wants
to run MIPS Linux 64-bit with 8 gigs of RAM.

Matt

--
Matthew D. Dharm                            Senior Software Designer
Momentum Computer Inc.                      1815 Aston Ave.  Suite 107
(760) 431-8663 X-115                        Carlsbad, CA 92008-7310
Momentum Works For You                      www.momenco.com

> -----Original Message-----
> From: Jason Gunthorpe [mailto:jgg@debian.org]
> Sent: Monday, January 14, 2002 12:13 AM
> To: Matthew Dharm
> Cc: linux-mips@oss.sgi.com
> Subject: Re: MIPS64 status?
>
>
>
> On Sun, 13 Jan 2002, Matthew Dharm wrote:
>
> > As I understand it, 64-bit support is really two
> different things:  64-bit
> > data path (i.e. unsigned long long) and 64-bit addressing
> (for more than 4G
> > of RAM).
>
> I suppose you could say that. I think I saw someone post to
> this list
> that they were working on a patch to enable 64 bit
> registers with a 32 bit
> kernel.
>
> > My understanding is that "MIPS64" generally refers to a
> kernel which
> > supports a 64-bit data path, but we're still limited to
> 32-bit addressing.
> > Is that correct?
>
> The mips64 tree in CVS is one that is a 64 bit addressing
> capable kernel.
> AFAIK the linx convention is that <foo>64 refers to
> addressing (which
> probably impiles register width too :>)
>
> I'm not sure what the current state of the ELF64 MIPS toolchain is.
>
> Jason
>

RE: MIPS64 status?

[Jason Gunthorpe]

On Mon, 14 Jan 2002, Matthew Dharm wrote:

> So MIPS64 denotes 64-bit data _and_ address?  Great.  But, what is the
> current state of the toolchain?  I mean, what point is having the code
> if the tools won't compile it properly?

Well, working on the kernel is a good way to get the tools working. That
is how several of the Linux archs have worked in the past.

> I guess what I'm looking for is what I should tell someone who wants
> to run MIPS Linux 64-bit with 8 gigs of RAM.

I have such hardware myself. IMHO the best option is the mips64 kernel and
I hope to try it someday. But in practice I'd guess that Ralf's highmem
patch is more likely to be usuable first (?). 

Jason

RE: MIPS64 status?

[Dominic Sweetman]

Matthew Dharm (mdharm@momenco.com) writes:

> I guess what I'm looking for is what I should tell someone who wants
> to run MIPS Linux 64-bit with 8 gigs of RAM.

"Ask Ralf".  (at least, that's the best advice for the kernel, which
he worked on at SGI).

For a compiler, you should ask Algorithmics.  We haven't tried the
64-bit configuration yet, but we've done lots of GCC variants and it's
only a program...

-- 
Dominic Sweetman
Algorithmics Ltd
The Fruit Farm, Ely Road, Chittering, CAMBS CB5 9PH, ENGLAND
phone: +44 1223 706200 / fax: +44 1223 706250 / home: +44 20 7226 0032
http://www.algor.co.uk

RE: MIPS64 status?

[Dominic Sweetman]

Matthew Dharm (mdharm@momenco.com) writes:

> I guess what I'm looking for is what I should tell someone who wants
> to run MIPS Linux 64-bit with 8 gigs of RAM.

"Ask Ralf".  (at least, that's the best advice for the kernel, which
he worked on at SGI).

For a compiler, you should ask Algorithmics.  We haven't tried the
64-bit configuration yet, but we've done lots of GCC variants and it's
only a program...

-- 
Dominic Sweetman
Algorithmics Ltd
The Fruit Farm, Ely Road, Chittering, CAMBS CB5 9PH, ENGLAND
phone: +44 1223 706200 / fax: +44 1223 706250 / home: +44 20 7226 0032
http://www.algor.co.uk

Re: MIPS64 status?

[Ralf Baechle]

On Sun, Jan 13, 2002 at 09:13:23PM -0800, Matthew Dharm wrote:

> As I understand it, 64-bit support is really two different things:  64-bit
> data path (i.e. unsigned long long) and 64-bit addressing (for more than 4G
> of RAM).

Right but due to the CPU architecture of pre-MIPS64 CPUs they always come
together unless the software does funny attempts at truncating OS support
to just 32-bit.  So the 32-bit kernel gives you none of the two, the mips64
kernel both.

> My understanding is that "MIPS64" generally refers to a kernel which
> supports a 64-bit data path, but we're still limited to 32-bit addressing.
> Is that correct?

MIPS64 is MIPS's MIPS64 processor architecture, mips64 is the 64-bit kernel.
That may sound like nitpicking but it's important to understand that both
are not the same.

> I suspect that this is very much a toolchain issue, as I don't think gcc
> will generate 64-bit addressing code.

Gcc is fine; the problem are binutils, that is as and ld.  As a result of
the gcc problems we don't have a 64-bit userspace either so all software
running on 64-bit kernels is currently old 32-bit software running in
compatibility mode.

  Ralf

Re: MIPS64 status?

[Ralf Baechle]

On Mon, Jan 14, 2002 at 01:42:22PM -0700, Jason Gunthorpe wrote:

> I have such hardware myself. IMHO the best option is the mips64 kernel and
> I hope to try it someday. But in practice I'd guess that Ralf's highmem
> patch is more likely to be usuable first (?). 

Depends if you can live with the problems of the current mips64 kernel.
If you can then it's highmem-free memory managment is certainly the way
to go.  It's also not limited to peanuts numbers of gigabytes but can
support as much memory as your can tack on a MIPS.

  Ralf

Re: MIPS64 status?

[Ralf Baechle]

On Mon, Jan 14, 2002 at 09:17:12PM +0000, Dominic Sweetman wrote:

> > I guess what I'm looking for is what I should tell someone who wants
> > to run MIPS Linux 64-bit with 8 gigs of RAM.
> 
> "Ask Ralf".  (at least, that's the best advice for the kernel, which
> he worked on at SGI).

Tested with upto 128gb :-)

> For a compiler, you should ask Algorithmics.  We haven't tried the
> 64-bit configuration yet, but we've done lots of GCC variants and it's
> only a program...

May I have full support for N32 and N64 tomorrow after lunch, please ;-)

  Ralf

Re: MIPS64 status?

[Ralf Baechle]

On Mon, Jan 14, 2002 at 11:17:08AM +0000, Dominic Sweetman wrote:

> o Very large virtual address spaces, using 64-bit pointer types.

Actually I only implemented support for something like 0.5TB.  As for
supercomputing that's peanuts (Like five years ago a customer requested
SGI to increase the per process size of the address space from 1TB, the
limit of the R4000 to 16TB, the limit of R10000 class processors.)

> o C "long" (and perhaps even "int") becomes 64-bit.

We follow the MIPS ABI which uses 32-bit ints and 64-bit longs.

> In such a 64-bit Linux system, though, you might still want to be able
> to run 32-bit applications with 32-bit pointers, int and long - either
> for compatibility or economy (32-bit data types make for a smaller
> program).  SGI do this in Irix: I don't know whether the 64-bit
> Linux/MIPS systems got around to it.

Yes.  The environment provided however is slightly different.  32-bit
software on the mips64 kernel is running with UX=1 thus 64-bit instructions
are allowed.

> There are other potentially useful combinations:
> 
> o A Linux where all machine-supported integer data types are 32-bit,

I don't want to support 32-bit char and short, sorry :-)

>   but capable of addressing physical memory outside of a 4Gbyte map.
>   (In practice, you need to use this kind of system to get outside of
>   a 512Mbyte map - so it's urgent).

I'd be working on this right now if you'd not be bothering me with email ;-)

>   Ralf says he has done this: it could be done without using any
>   64-bit operations, but it might be easier with them.

There are still MIPS32 systems which don't support 64-bit operations just
may have an address space of upto 36 bits.

> o A system using 32-bit pointers and 'long' throughout, but with
>   support for 'long long' 64-bit integer data types in registers.
>   
> o A system using 64-bit addressing within the kernel, but not for
>   applications. 
> 
> However, it's unlikely to make sense to do all of them!

Correct.  We may add support for the one or other code of these models
over time.

> > I suspect that this is very much a toolchain issue, as I don't think
> > gcc will generate 64-bit addressing code.
> 
> I suspect that the generic GNU toolchains are pretty buggy when you
> switch on 64-bit MIPS operation; but it's bug-fixes which are needed,
> not wholesale new features.

Actually in the past somebody was doing paid work to get the combo
g++ + SGI as + GNU ld to work for N32.  Due to the similarity of N32 and
N64 that already brought us quite a bit closer to N64 support.  That
still leaves alot of work including plenty of gas work.

> Politics: MIPS Technologies' advocacy for their "MIPS32" instruction
> set dialect in embedded systems means there are now some quite capable
> MIPS CPUs (eg Alchemy's 500Mhz integrated CPUs) which don't have
> 64-bit datapaths or arithmetic.  So casual dependence on 64-bit
> operations should probably be avoided.

I'm doing the best to avoid that.

  Ralf

Re: MIPS64 status?

[Ralf Baechle]

On Mon, Jan 14, 2002 at 01:54:42PM +0100, Kevin D. Kissell wrote:

> The official MIPS64[tm] architecture spec from MIPS 
> Technologies also provides a bit (Status.PX) which enables
> the 64-bit data path without affecting address generation
> and translation, which removes this quirk.  Only the very
> most recent 64-bit cores and CPUs implement it, however.

And Linux doesn't use PX at all.

  Ralf

RE: MIPS64 status?

[Matthew Dharm]

Ralf --

Thanks for the info.  Too bad "MIPS64" and "mips64" sound exactly the
same on the telephone.

But, I need to be pedantic, just to be clear on a couple of
questions...

So, the "mips64" kernel can use 64-bits of address, for RAM >4G?
But, the apps running are always 32-bit?
Does this mean that any individual application can only use 4G of
memory, tho you could have several applications in physical memory
doing this? (i.e. multiple applications using 1G of RAM each, but not
swapping to disk?)
Does this mean we could map PCI memory/IO addresses above 4G and have
it work?

Matt

--
Matthew D. Dharm                            Senior Software Designer
Momentum Computer Inc.                      1815 Aston Ave.  Suite 107
(760) 431-8663 X-115                        Carlsbad, CA 92008-7310
Momentum Works For You                      www.momenco.com

> -----Original Message-----
> From: owner-linux-mips@oss.sgi.com
> [mailto:owner-linux-mips@oss.sgi.com]On Behalf Of Ralf Baechle
> Sent: Monday, January 14, 2002 3:06 PM
> To: Matthew Dharm
> Cc: linux-mips@oss.sgi.com
> Subject: Re: MIPS64 status?
>
>
> On Sun, Jan 13, 2002 at 09:13:23PM -0800, Matthew Dharm wrote:
>
> > As I understand it, 64-bit support is really two
> different things:  64-bit
> > data path (i.e. unsigned long long) and 64-bit addressing
> (for more than 4G
> > of RAM).
>
> Right but due to the CPU architecture of pre-MIPS64 CPUs
> they always come
> together unless the software does funny attempts at
> truncating OS support
> to just 32-bit.  So the 32-bit kernel gives you none of the
> two, the mips64
> kernel both.
>
> > My understanding is that "MIPS64" generally refers to a
> kernel which
> > supports a 64-bit data path, but we're still limited to
> 32-bit addressing.
> > Is that correct?
>
> MIPS64 is MIPS's MIPS64 processor architecture, mips64 is
> the 64-bit kernel.
> That may sound like nitpicking but it's important to
> understand that both
> are not the same.
>
> > I suspect that this is very much a toolchain issue, as I
> don't think gcc
> > will generate 64-bit addressing code.
>
> Gcc is fine; the problem are binutils, that is as and ld.
> As a result of
> the gcc problems we don't have a 64-bit userspace either so
> all software
> running on 64-bit kernels is currently old 32-bit software
> running in
> compatibility mode.
>
>   Ralf
>

RE: MIPS64 status?

[Matthew Dharm]

Ralf --

Thanks for the info.  Too bad "MIPS64" and "mips64" sound exactly the
same on the telephone.

But, I need to be pedantic, just to be clear on a couple of
questions...

So, the "mips64" kernel can use 64-bits of address, for RAM >4G?
But, the apps running are always 32-bit?
Does this mean that any individual application can only use 4G of
memory, tho you could have several applications in physical memory
doing this? (i.e. multiple applications using 1G of RAM each, but not
swapping to disk?)
Does this mean we could map PCI memory/IO addresses above 4G and have
it work?

Matt

--
Matthew D. Dharm                            Senior Software Designer
Momentum Computer Inc.                      1815 Aston Ave.  Suite 107
(760) 431-8663 X-115                        Carlsbad, CA 92008-7310
Momentum Works For You                      www.momenco.com

> -----Original Message-----
> From: owner-linux-mips@oss.sgi.com
> [mailto:owner-linux-mips@oss.sgi.com]On Behalf Of Ralf Baechle
> Sent: Monday, January 14, 2002 3:06 PM
> To: Matthew Dharm
> Cc: linux-mips@oss.sgi.com
> Subject: Re: MIPS64 status?
>
>
> On Sun, Jan 13, 2002 at 09:13:23PM -0800, Matthew Dharm wrote:
>
> > As I understand it, 64-bit support is really two
> different things:  64-bit
> > data path (i.e. unsigned long long) and 64-bit addressing
> (for more than 4G
> > of RAM).
>
> Right but due to the CPU architecture of pre-MIPS64 CPUs
> they always come
> together unless the software does funny attempts at
> truncating OS support
> to just 32-bit.  So the 32-bit kernel gives you none of the
> two, the mips64
> kernel both.
>
> > My understanding is that "MIPS64" generally refers to a
> kernel which
> > supports a 64-bit data path, but we're still limited to
> 32-bit addressing.
> > Is that correct?
>
> MIPS64 is MIPS's MIPS64 processor architecture, mips64 is
> the 64-bit kernel.
> That may sound like nitpicking but it's important to
> understand that both
> are not the same.
>
> > I suspect that this is very much a toolchain issue, as I
> don't think gcc
> > will generate 64-bit addressing code.
>
> Gcc is fine; the problem are binutils, that is as and ld.
> As a result of
> the gcc problems we don't have a 64-bit userspace either so
> all software
> running on 64-bit kernels is currently old 32-bit software
> running in
> compatibility mode.
>
>   Ralf
>

Re: MIPS64 status?

[Ralf Baechle]

On Mon, Jan 14, 2002 at 03:25:44PM -0800, Matthew Dharm wrote:

> Thanks for the info.  Too bad "MIPS64" and "mips64" sound exactly the
> same on the telephone.
> 
> But, I need to be pedantic, just to be clear on a couple of
> questions...
> 
> So, the "mips64" kernel can use 64-bits of address, for RAM >4G?
> But, the apps running are always 32-bit?

In theory the kernel has the capability to run 64-bit applications.  In
practice that doesn't work due to the lack of 64-bit apps and stuff.

> Does this mean that any individual application can only use 4G of
> memory, tho you could have several applications in physical memory
> doing this? (i.e. multiple applications using 1G of RAM each, but not
> swapping to disk?)

In theory we don't limit the address space of 32-bit applications in 64-bit
mode so they could go and use all memory and syscalls on the 64-bit
address space also.  In practice that's just too ugly to be usable so
consider 32-bit apps on the 64-bit kernel as limited to 2gb as they are
currently.  You can however run an arbitrary number of these processes.

> Does this mean we could map PCI memory/IO addresses above 4G and have
> it work?

Sure.

  Ralf

RE: MIPS64 status?

[Jason Gunthorpe]

On Mon, 14 Jan 2002, Matthew Dharm wrote:

> Does this mean we could map PCI memory/IO addresses above 4G and have
> it work?

Ooh, don't go there :> We looked at that and actually did it then backed
it out.

The PCI spec (particuarly PCI-X) tries to make it possible, but in a
general system with PCI sockets/etc it is just is not feasible. PCI
bridges need to be located below 4G, as do the majority of devices made. 
There is also a performance hit for having device registers > 4G.

You'd definately need the mips64 kernel to do that, or use ugly wired TLB
entries with normal mips.

Jason

Re: MIPS64 status?

[Jason Gunthorpe]

On Mon, 14 Jan 2002, Ralf Baechle wrote:

> On Mon, Jan 14, 2002 at 01:42:22PM -0700, Jason Gunthorpe wrote:
> 
> > I have such hardware myself. IMHO the best option is the mips64 kernel and
> > I hope to try it someday. But in practice I'd guess that Ralf's highmem
> > patch is more likely to be usuable first (?). 
> 
> Depends if you can live with the problems of the current mips64 kernel.
> If you can then it's highmem-free memory managment is certainly the way
> to go.  It's also not limited to peanuts numbers of gigabytes but can
> support as much memory as your can tack on a MIPS.

Yep, totally.

I thought about starting with mips64 but it was simpler to go with the
pre-existing processor support in the mips32 kernel. I'd like to patch
mips64 up to support the CPU's I have here, but not this month :>

Could someone quickly comment on how it is running? Can it boot normal 32
bit user space OK? I know with sparc64 there were (are?) a number of
problems with missing 32 bit syscall translations for some obscure
things.. 

Jason

RE: MIPS64 status?

[Matthew Dharm]

Hrm...

Were you actually using 64-bit addresses on the PCI bus?

My guess is that with some creative address mappings, this could be
done.  The PCI bus itself would use only 32-bit address, but the CPU
would use a base address offset into the >4G range.

Yeah, I could see how that could get ugly...

Matt

--
Matthew D. Dharm                            Senior Software Designer
Momentum Computer Inc.                      1815 Aston Ave.  Suite 107
(760) 431-8663 X-115                        Carlsbad, CA 92008-7310
Momentum Works For You                      www.momenco.com

> -----Original Message-----
> From: owner-linux-mips@oss.sgi.com
> [mailto:owner-linux-mips@oss.sgi.com]On Behalf Of Jason Gunthorpe
> Sent: Monday, January 14, 2002 4:00 PM
> To: Matthew Dharm
> Cc: linux-mips@oss.sgi.com
> Subject: RE: MIPS64 status?
>
>
>
> On Mon, 14 Jan 2002, Matthew Dharm wrote:
>
> > Does this mean we could map PCI memory/IO addresses above
> 4G and have
> > it work?
>
> Ooh, don't go there :> We looked at that and actually did
> it then backed
> it out.
>
> The PCI spec (particuarly PCI-X) tries to make it possible, but in a
> general system with PCI sockets/etc it is just is not feasible. PCI
> bridges need to be located below 4G, as do the majority of
> devices made.
> There is also a performance hit for having device registers > 4G.
>
> You'd definately need the mips64 kernel to do that, or use
> ugly wired TLB
> entries with normal mips.
>
> Jason
>

RE: MIPS64 status?

[Matthew Dharm]

Hrm...

Were you actually using 64-bit addresses on the PCI bus?

My guess is that with some creative address mappings, this could be
done.  The PCI bus itself would use only 32-bit address, but the CPU
would use a base address offset into the >4G range.

Yeah, I could see how that could get ugly...

Matt

--
Matthew D. Dharm                            Senior Software Designer
Momentum Computer Inc.                      1815 Aston Ave.  Suite 107
(760) 431-8663 X-115                        Carlsbad, CA 92008-7310
Momentum Works For You                      www.momenco.com

> -----Original Message-----
> From: owner-linux-mips@oss.sgi.com
> [mailto:owner-linux-mips@oss.sgi.com]On Behalf Of Jason Gunthorpe
> Sent: Monday, January 14, 2002 4:00 PM
> To: Matthew Dharm
> Cc: linux-mips@oss.sgi.com
> Subject: RE: MIPS64 status?
>
>
>
> On Mon, 14 Jan 2002, Matthew Dharm wrote:
>
> > Does this mean we could map PCI memory/IO addresses above
> 4G and have
> > it work?
>
> Ooh, don't go there :> We looked at that and actually did
> it then backed
> it out.
>
> The PCI spec (particuarly PCI-X) tries to make it possible, but in a
> general system with PCI sockets/etc it is just is not feasible. PCI
> bridges need to be located below 4G, as do the majority of
> devices made.
> There is also a performance hit for having device registers > 4G.
>
> You'd definately need the mips64 kernel to do that, or use
> ugly wired TLB
> entries with normal mips.
>
> Jason
>

Re: MIPS64 status?

[Maciej W. Rozycki]

On Mon, 14 Jan 2002, Ralf Baechle wrote:

> > The official MIPS64[tm] architecture spec from MIPS 
> > Technologies also provides a bit (Status.PX) which enables
> > the 64-bit data path without affecting address generation
> > and translation, which removes this quirk.  Only the very
> > most recent 64-bit cores and CPUs implement it, however.
> 
> And Linux doesn't use PX at all.

 Especially as there is no real need for the bit for the standard MIPS64
TLB organization.  The functionality of the PX bit can be replaced by the
UX bit together with an appropriate mapping of segments to the 31-bit
address space.  Of course the kernel needs to handle the XTLB exception
for UX as opposed to the TLB one for PX, but for programs it doesn't
matter.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: MIPS64 status?

[Thiemo Seufer]

Dominic Sweetman wrote:
[snip]
> > I suspect that this is very much a toolchain issue, as I don't think
> > gcc will generate 64-bit addressing code.

gcc does 64bit adressing, but binutils (ld/gas) do only partially yet.

> I suspect that the generic GNU toolchains are pretty buggy when you
> switch on 64-bit MIPS operation; but it's bug-fixes which are needed,
> not wholesale new features.

It needs definitely more than bug fixing. I commited some support
for n64 non-PIC to binutils CVS some time ago, but support for
n64 PIC and n32 still needs to be done.


Thiemo

Re: MIPS64 status?

[John Heil]

On Mon, 14 Jan 2002, Ralf Baechle wrote:

> Date: Mon, 14 Jan 2002 15:07:51 -0800
> From: Ralf Baechle <ralf@oss.sgi.com>
> To: Jason Gunthorpe <jgg@debian.org>
> Cc: Matthew Dharm <mdharm@momenco.com>, linux-mips@oss.sgi.com
> Subject: Re: MIPS64 status?
> 
> On Mon, Jan 14, 2002 at 01:42:22PM -0700, Jason Gunthorpe wrote:
> 
> > I have such hardware myself. IMHO the best option is the mips64 kernel and
> > I hope to try it someday. But in practice I'd guess that Ralf's highmem
> > patch is more likely to be usuable first (?). 
> 
> Depends if you can live with the problems of the current mips64 kernel.
> If you can then it's highmem-free memory managment is certainly the way
> to go.  It's also not limited to peanuts numbers of gigabytes but can
> support as much memory as your can tack on a MIPS.
> 
>   Ralf
> 
Who, if anyone, has a MIPS64 reference design board available ?
...With >4G memory capacity ?


Thanx
Johnh

Re: MIPS64 status?

[Ralf Baechle]

On Tue, Jan 15, 2002 at 12:00:12PM -0800, John Heil wrote:

> Who, if anyone, has a MIPS64 reference design board available ?
> ...With >4G memory capacity ?

I think such boards are currently quite rare.  2gb using latest memory
modules seems to be as much as you can get right now.

  Ralf

Re: MIPS64 status?

[Ralf Baechle]

On Mon, Jan 14, 2002 at 05:08:24PM -0700, Jason Gunthorpe wrote:

> Could someone quickly comment on how it is running? Can it boot normal 32
> bit user space OK? I know with sparc64 there were (are?) a number of
> problems with missing 32 bit syscall translations for some obscure
> things.. 

Same situation for mips64.

  Ralf

Re: MIPS64 status?

[Dominic Sweetman]

> Who, if anyone, has a MIPS64 reference design board available ?

Algorithmics P-6064 (http://www.algor.co.uk/algor/info/p6064.html) 

> ...With >4G memory capacity ?

A mere 2Gbytes today - we've got two DIMM slots and 1Gbyte SDRAM DIMMs
are readily available.  The 1Gbyte boards have 32 x 256Mbit chips, but
I think there may even now be boards with more chips "stacked" in
pairs.

We've got no significant addressing restrictions on the memory
controller and the existing DIMM modules still have address lines
remaining...  We can support even larger memories when and if the
DIMMs show up.