anybody tried NPTL?

anybody tried NPTL?

[Jun Sun]

I am looking into porting NPTL to MIPS.  Just curious if
anybody has tried this before.

I notice there was a discussion about the ABI extension
for TLS (thread local storage) support.  Before that support
becomes a reality it seems one can still use NPTL with 
the help of additional system calls.

A rough search of latest glibc source shows there is
zero MIPS code for nptl.  A couple of other arches
are missing as well (such as ARM)

Jun

Re: anybody tried NPTL?

[Kumba]

Jun Sun wrote:

> I am looking into porting NPTL to MIPS.  Just curious if
> anybody has tried this before.
> 
> I notice there was a discussion about the ABI extension
> for TLS (thread local storage) support.  Before that support
> becomes a reality it seems one can still use NPTL with 
> the help of additional system calls.
> 
> A rough search of latest glibc source shows there is
> zero MIPS code for nptl.  A couple of other arches
> are missing as well (such as ARM)
> 
> Jun

All I've heard about this is that some kernel changes are (still?) 
needed, then just the glibc support along w/ TLS (Maybe compiler support?).

I believe I heard reports that the glibc people were looking to 
deprecate linuxthreads within a another release or two (but don't know 
specifics or anything), so it sounds like NPTL should be something to 
get working.


--Kumba

-- 
"Such is oft the course of deeds that move the wheels of the world: 
small hands do them because they must, while the eyes of the great are 
elsewhere."  --Elrond

Re: anybody tried NPTL?

[Jun Sun]

On Wed, Aug 04, 2004 at 09:08:56PM -0400, Kumba wrote:
> Jun Sun wrote:
> 
> > I am looking into porting NPTL to MIPS.  Just curious if
> > anybody has tried this before.
> > 
> > I notice there was a discussion about the ABI extension
> > for TLS (thread local storage) support.  Before that support
> > becomes a reality it seems one can still use NPTL with 
> > the help of additional system calls.
> > 
> > A rough search of latest glibc source shows there is
> > zero MIPS code for nptl.  A couple of other arches
> > are missing as well (such as ARM)
> > 
> > Jun
> 
> All I've heard about this is that some kernel changes are (still?) 
> needed, then just the glibc support along w/ TLS (Maybe compiler support?).
> 

TLS support requires ABI change, which involves work in gcc and binutils.
At current stage I think only a few arches have added TLS support.
MIPS is definitely not one of them.  Does anybody know about the current
status, for MIPS and other arches?

I think the ABI change and TLS support might take a long time to 
be ready.  It appears meanwhile NPTL can run without TLS, but would
need a couple of additional system calls that get and set thread
local area.

> I believe I heard reports that the glibc people were looking to 
> deprecate linuxthreads within a another release or two (but don't know 
> specifics or anything), so it sounds like NPTL should be something to 
> get working.
> 

That surely puts some urgency on this matter. :)

Jun

Re: anybody tried NPTL?

[Ralf Baechle]

On Wed, Aug 04, 2004 at 09:08:56PM -0400, Kumba wrote:

> >I am looking into porting NPTL to MIPS.  Just curious if
> >anybody has tried this before.
> >
> >I notice there was a discussion about the ABI extension
> >for TLS (thread local storage) support.  Before that support
> >becomes a reality it seems one can still use NPTL with 
> >the help of additional system calls.
> >
> >A rough search of latest glibc source shows there is
> >zero MIPS code for nptl.  A couple of other arches
> >are missing as well (such as ARM)
> >
> >Jun
> 
> All I've heard about this is that some kernel changes are (still?) 
> needed, then just the glibc support along w/ TLS (Maybe compiler support?).
> 
> I believe I heard reports that the glibc people were looking to 
> deprecate linuxthreads within a another release or two (but don't know 
> specifics or anything), so it sounds like NPTL should be something to 
> get working.

Threading support means breaking the ABI compatibility.  There are other
issues that can best be solved by breakin the ABI which is why this is
stretching out.

  Ralf

Re: anybody tried NPTL?

[Dominic Sweetman]

Jun Sun,

> I am looking into porting NPTL to MIPS.  Just curious if
> anybody has tried this before.
> 
> I notice there was a discussion about the ABI extension
> for TLS (thread local storage) support.  Before that support
> becomes a reality it seems one can still use NPTL with 
> the help of additional system calls.

Well, this is an area of substantial interest to MIPS Technologies.
We are working on our multi-threading extension to the MIPS
architecture, and one of our longer-term aims is to achieve really
good NPTL performance.

As you said, this motivates a revision of the ABI.  Any incompatible
change to the ABI is painful, since everything has to be recompiled;
so our reasoning at the moment is that we might as well be radical...

MIPS Technologies would write up the definition; make and circulate
changes to the compiler, binutils and debugger; and make and circulate
supporting changes to the kernel and glibc.  We're aware this is only
a part of the problem, so we do need to figure out what will attract
community approval.

A few years ago I was volubly arguing in favour of keeping o32 until
we could move to n32/n64.  But times change: SGI compatibility
no longer seems important, and Linux on 32-bit MIPS CPUs needs
long-term support.

In many ways it's easier to get away from the ingenious but
troublesome stack-structure-based calling convention.  The "stack
structure" trick was invented so that o32 could work without function
prototypes - but function prototypes are now required, and something
with fixed-size arguments is simpler.  Something like the old
Cygnus/WRS EABI proposal, in fact...

So we're proposing:

o The register name<->number mapping is that of n64.

o Calling convention: register-, not slot-based. Each argument is
  represented by a register value. Arguments 0-7 travel in registers
  a0-7 (or fa0-7 as required for floating point types). If there are
  more than eight arguments, further ones are formed as if put in a
  register and then saved on the stack into a 64-bit slot (more than 8
  arguments is rare enough that we can afford to standardise on the
  big slots).
  
o Use floating point registers for double and float arguments, and
  integer registers for all integer/pointer values which will
  fit. Larger or structured data items are implicitly passed by
  reference: to maintain pass-by-value semantics, the compiler uses a
  copy-on-write trick if software writes a by-reference argument (or
  takes its address).  I'm told gcc is happy enough to do that.

o The return value comes back in two registers, with the second
  return-register used only when the return value consists of two
  scalars (ie a complex or double-precision number). [Folklore insists
  this is essential for Fortran support of complex numbers, and I
  don't want to fight folklore].

  All other non-scalar return values are returned via a pointer
  specified by the caller as an implicit first argument.

o Reserved registers: all the traditional ones. But now:

  - gp will be the GOT pointer in Linux, and should be defined as
    saved (ie a function must preserve values in this registers, which
    means it will need to save-and-restore the register if it is
    written locally).
    
  - we'll define some other register as a per-thread data pointer.

Some details are still to be worked out.  But do you think this is on
the right lines?  And who would like to take an active part in
specifying or reviewing?

--
Dominic Sweetman
MIPS Technologies

Re: anybody tried NPTL?

[Alec Voropay]

> Well, this is an area of substantial interest to MIPS Technologies.
> We are working on our multi-threading extension to the MIPS
> architecture, and one of our longer-term aims is to achieve really
> good NPTL performance.

 Sorry for a bit offtopic..., as far as I remember, the Windows NT
MIPS edition has a working multithread implementation. Is this
implementation very copyrighted and is it possible to use something
ftom there for the NPTL implementation ?




--
-=AV=-

Re: anybody tried NPTL?

[Alec Voropay]

> Well, this is an area of substantial interest to MIPS Technologies.
> We are working on our multi-threading extension to the MIPS
> architecture, and one of our longer-term aims is to achieve really
> good NPTL performance.

 Sorry for a bit offtopic..., as far as I remember, the Windows NT
MIPS edition has a working multithread implementation. Is this
implementation very copyrighted and is it possible to use something
ftom there for the NPTL implementation ?




--
-=AV=-

Re: anybody tried NPTL?

[David Daney]

Dominic Sweetman wrote:
> So we're proposing:
> 
> o The register name<->number mapping is that of n64.
> 
> o Calling convention: register-, not slot-based. Each argument is
>   represented by a register value. Arguments 0-7 travel in registers
>   a0-7 (or fa0-7 as required for floating point types). If there are
>   more than eight arguments, further ones are formed as if put in a
>   register and then saved on the stack into a 64-bit slot (more than 8
>   arguments is rare enough that we can afford to standardise on the
>   big slots).
>   
> o Use floating point registers for double and float arguments, and
>   integer registers for all integer/pointer values which will
>   fit. Larger or structured data items are implicitly passed by
>   reference: to maintain pass-by-value semantics, the compiler uses a
>   copy-on-write trick if software writes a by-reference argument (or
>   takes its address).  I'm told gcc is happy enough to do that.
> 
> o The return value comes back in two registers, with the second
>   return-register used only when the return value consists of two
>   scalars (ie a complex or double-precision number). [Folklore insists
>   this is essential for Fortran support of complex numbers, and I
>   don't want to fight folklore].
> 
>   All other non-scalar return values are returned via a pointer
>   specified by the caller as an implicit first argument.
> 
> o Reserved registers: all the traditional ones. But now:
> 
>   - gp will be the GOT pointer in Linux, and should be defined as
>     saved (ie a function must preserve values in this registers, which
>     means it will need to save-and-restore the register if it is
>     written locally).
>     
>   - we'll define some other register as a per-thread data pointer.
> 
> Some details are still to be worked out.  But do you think this is on
> the right lines?  And who would like to take an active part in
> specifying or reviewing?
> 
All of this sounds good to me.  However my current concerns are how to
make my code run on a mips32[r2] core with no floating point.  We are
using several different systems with variations of this cpu type.

So for me, making sure that a soft-float variant of the ABI is well
specified is also important.  I suppose it would be to treat
float/double values as appropriate encoding of 32/64 bit integer values.

David Daney.

Re: anybody tried NPTL?

[Jun Sun]

On Thu, Aug 19, 2004 at 03:17:10PM +0100, Dominic Sweetman wrote:
> 
> Jun Sun,
> 
> > I am looking into porting NPTL to MIPS.  Just curious if
> > anybody has tried this before.
> > 
> > I notice there was a discussion about the ABI extension
> > for TLS (thread local storage) support.  Before that support
> > becomes a reality it seems one can still use NPTL with 
> > the help of additional system calls.
> 
> Well, this is an area of substantial interest to MIPS Technologies.
> We are working on our multi-threading extension to the MIPS
> architecture, and one of our longer-term aims is to achieve really
> good NPTL performance.
> 
> As you said, this motivates a revision of the ABI.  Any incompatible
> change to the ABI is painful, since everything has to be recompiled;
> so our reasoning at the moment is that we might as well be radical...
> 
> MIPS Technologies would write up the definition; make and circulate
> changes to the compiler, binutils and debugger; and make and circulate
> supporting changes to the kernel and glibc.  We're aware this is only
> a part of the problem, so we do need to figure out what will attract
> community approval.
> 
> A few years ago I was volubly arguing in favour of keeping o32 until
> we could move to n32/n64.  But times change: SGI compatibility
> no longer seems important, and Linux on 32-bit MIPS CPUs needs
> long-term support.
> 
> In many ways it's easier to get away from the ingenious but
> troublesome stack-structure-based calling convention.  The "stack
> structure" trick was invented so that o32 could work without function
> prototypes - but function prototypes are now required, and something
> with fixed-size arguments is simpler.  Something like the old
> Cygnus/WRS EABI proposal, in fact...
> 
> So we're proposing:
> 
> o The register name<->number mapping is that of n64.
> 
> o Calling convention: register-, not slot-based. Each argument is
>   represented by a register value. Arguments 0-7 travel in registers
>   a0-7 (or fa0-7 as required for floating point types). If there are
>   more than eight arguments, further ones are formed as if put in a
>   register and then saved on the stack into a 64-bit slot (more than 8
>   arguments is rare enough that we can afford to standardise on the
>   big slots).
>   
> o Use floating point registers for double and float arguments, and
>   integer registers for all integer/pointer values which will
>   fit. Larger or structured data items are implicitly passed by
>   reference: to maintain pass-by-value semantics, the compiler uses a
>   copy-on-write trick if software writes a by-reference argument (or
>   takes its address).  I'm told gcc is happy enough to do that.
> 
> o The return value comes back in two registers, with the second
>   return-register used only when the return value consists of two
>   scalars (ie a complex or double-precision number). [Folklore insists
>   this is essential for Fortran support of complex numbers, and I
>   don't want to fight folklore].
> 
>   All other non-scalar return values are returned via a pointer
>   specified by the caller as an implicit first argument.
> 
> o Reserved registers: all the traditional ones. But now:
> 
>   - gp will be the GOT pointer in Linux, and should be defined as
>     saved (ie a function must preserve values in this registers, which
>     means it will need to save-and-restore the register if it is
>     written locally).
>     
>   - we'll define some other register as a per-thread data pointer.
> 
> Some details are still to be worked out.  But do you think this is on
> the right lines?  And who would like to take an active part in
> specifying or reviewing?
>

I am not against any of those.  However, from NPTL implementation
point of view I really just care about the per-thread register.
What is the motivation of other changes?  Taking this chance to make
things all right in one shot?

Jun

Re: anybody tried NPTL?

[Dominic Sweetman]

Alec,

> > We are working on our multi-threading extension to the MIPS
> > architecture, and one of our longer-term aims is to achieve really
> > good NPTL performance.
> 
> Sorry for a bit offtopic..., as far as I remember, the Windows NT
> MIPS edition has a working multithread implementation. Is this
> implementation very copyrighted and is it possible to use something
> ftom there for the NPTL implementation ?

It's Microsoft, so yes it will be copyrighted and it would be better
if our staff didn't even look at it.

--
Dominic Sweetman
MIPS Technologies

Re: anybody tried NPTL?

[Dominic Sweetman]

Alec,

> > We are working on our multi-threading extension to the MIPS
> > architecture, and one of our longer-term aims is to achieve really
> > good NPTL performance.
> 
> Sorry for a bit offtopic..., as far as I remember, the Windows NT
> MIPS edition has a working multithread implementation. Is this
> implementation very copyrighted and is it possible to use something
> ftom there for the NPTL implementation ?

It's Microsoft, so yes it will be copyrighted and it would be better
if our staff didn't even look at it.

--
Dominic Sweetman
MIPS Technologies

Re: anybody tried NPTL?

[Dominic Sweetman]

David Daney (ddaney@avtrex.com) writes:

> All of this sounds good to me.  However my current concerns are how to
> make my code run on a mips32[r2] core with no floating point.  We are
> using several different systems with variations of this cpu type.
> 
> So for me, making sure that a soft-float variant of the ABI is well
> specified is also important.  I suppose it would be to treat
> float/double values as appropriate encoding of 32/64 bit integer values.

Exactly - that's certainly what we do with o32 -msoft-float.  I don't
think there are even any corner cases to worry about.

We've recently made some major improvements to our soft maths library
to make it many times faster, but still IEEE-compliant.  It was
originally written by Algorithmics - a GPL'd version is used in the
Linux/MIPS kernel to handle FPU exceptions.  I should probably check
before saying for certain, but I think we will distribute this under
straight GPL conditions.

If we had the time to do it, I guess we should put it on one of the OS
project sites... but I'll cc: Nigel Stephens (who wrote it).  If it
might be of use to you (which seems quite likely) perhaps we could do
a deal to get it on a site somewhere?  Let me know if you'd like sight
of it.

--
Dominic

Re: anybody tried NPTL?

[Thiemo Seufer]

Dominic Sweetman wrote:
[snip]
> So we're proposing:
> 
> o The register name<->number mapping is that of n64.
> 
> o Calling convention: register-, not slot-based. Each argument is
>   represented by a register value. Arguments 0-7 travel in registers
>   a0-7 (or fa0-7 as required for floating point types).

This suggests to have no fp temporaries left: fv0-1, fa0-7, fs0-5.
Is this intentional?


Thiemo

Re: anybody tried NPTL?

[Dominic Sweetman]

Jun Sun (jsun@mvista.com) writes:

> > [large expanse of new ABI stuff omitted]
>
> What is the motivation of other changes?  Taking this chance to make
> things all right in one shot?

Yes, and I did kind of cover this in the large expanse of stuff:

> > As you said, this motivates a revision of the ABI.  Any
> > incompatible change to the ABI is painful, since everything has to
> > be recompiled; so our reasoning at the moment is that we might as
> > well be radical...

o32 puts only four arguments in registers, which is less than optimal,
and continually reloads the GOT pointer (which is not defined as a
saved register in o32).

n32 (despite its name) runs exclusively on 64-bit CPUs.

So yes, we have strong reasons for making a larger change to the ABI,
and knowing how difficult it is...

> However, from NPTL implementation point of view I really just care
> about the per-thread register.

I guess our main message was that we felt it would be a mistake just
to add a thread register to o32 (which produces a substantially
incompatible new ABI anyway).

Until that all works, what we had in mind is that we'd do NPTL over
o32 by defining a system call to return a per-thread ID which is or
can be converted into a per-thread data pointer.  We suspected that
NPTL's per-thread-data model allows the use of cunning macros or
library functions to make that look OK.

Ought we to go further and see exactly how that can be done?

--
Dominic

Re: anybody tried NPTL?

[Dominic Sweetman]

Thiemo Seufer (ica2_ts@csv.ica.uni-stuttgart.de) writes:

> > So we're proposing:
> > 
> > o The register name<->number mapping is that of n64.
> > 
> > o Calling convention: register-, not slot-based. Each argument is
> >   represented by a register value. Arguments 0-7 travel in registers
> >   a0-7 (or fa0-7 as required for floating point types).
> 
> This suggests to have no fp temporaries left: fv0-1, fa0-7, fs0-5.
> Is this intentional?

No, there are lots really...

Long, long ago early MIPS CPUs had to use FP registers in pairs to get
FP doubles, so had only 16 effective registers.

But there are 32 real double-precision FP registers in all known
modern MIPS CPUs (well, all known pretty old ones, really) once you
turn off the backward-compatibility bit in the status register.  I
forgot to mention that... (or I could claim, weasel words, that this
is implied by the "name<->number mapping of n64").

--
Dominic

Re: anybody tried NPTL?

[Ralf Baechle]

On Thu, Aug 19, 2004 at 06:31:45PM +0400, Alec Voropay wrote:

> > Well, this is an area of substantial interest to MIPS Technologies.
> > We are working on our multi-threading extension to the MIPS
> > architecture, and one of our longer-term aims is to achieve really
> > good NPTL performance.
> 
>  Sorry for a bit offtopic..., as far as I remember, the Windows NT
> MIPS edition has a working multithread implementation. Is this
> implementation very copyrighted and is it possible to use something
> ftom there for the NPTL implementation ?

In addition to what Dom has already answered - there are very significant
differences between the multithreading as implemented in the Windows OS
family and the varioius threading implementations for Linux like classic
libpthreads, Linuxthreads, NPTL, Mozilla and more.  If we legally could
look at MS's code I'd not expect to find much useful for us there ...

  Ralf

Re: anybody tried NPTL?

[Daniel Jacobowitz]

On Fri, Aug 20, 2004 at 02:46:11PM +0100, Dominic Sweetman wrote:
> I guess our main message was that we felt it would be a mistake just
> to add a thread register to o32 (which produces a substantially
> incompatible new ABI anyway).

Completely agree...

> Until that all works, what we had in mind is that we'd do NPTL over
> o32 by defining a system call to return a per-thread ID which is or
> can be converted into a per-thread data pointer.  We suspected that
> NPTL's per-thread-data model allows the use of cunning macros or
> library functions to make that look OK.
> 
> Ought we to go further and see exactly how that can be done?

It shouldn't be at all hard.  The way NPTL's __thread support works,
the only things that should have to know where the TLS base is are
(A) GCC, so it can load it and (B) GDB, via some new ptrace op.  I
don't know if you'd want to open-code the syscall or take the overhead
of a function call.  Ralf had some ideas?

-- 
Daniel Jacobowitz

RE: anybody tried NPTL?

[Alec Voropay]

Ralf Baechle [mailto:ralf@linux-mips.org] wrote:

> In addition to what Dom has already answered - there are very 
> significant differences between the multithreading as
> implemented in the  Windows OS family and the varioius
> threading implementations for Linux  like classic
> libpthreads, Linuxthreads, NPTL, Mozilla and more.
> If we legally could look at MS's code I'd not expect to find
> much useful for us there ...

 OK, OK. You are right. However, as it is known, there is at
least one project "to bridge" Win32/multithread and *NIX :
 WINE.
http://winehq.com/site/docs/wine-devel/x3398

 Yes, the Win32/MIPS API (and ABI) is dead, but MIPS/multithreading
lives in the WindowsCE/MIPS HPCs.


 Unfortunately, I can't find any details about Win32/MIPS
implementation.


--
-=VA=-

RE: anybody tried NPTL?

[Alec Voropay]

Ralf Baechle [mailto:ralf@linux-mips.org] wrote:

> In addition to what Dom has already answered - there are very 
> significant differences between the multithreading as
> implemented in the  Windows OS family and the varioius
> threading implementations for Linux  like classic
> libpthreads, Linuxthreads, NPTL, Mozilla and more.
> If we legally could look at MS's code I'd not expect to find
> much useful for us there ...

 OK, OK. You are right. However, as it is known, there is at
least one project "to bridge" Win32/multithread and *NIX :
 WINE.
http://winehq.com/site/docs/wine-devel/x3398

 Yes, the Win32/MIPS API (and ABI) is dead, but MIPS/multithreading
lives in the WindowsCE/MIPS HPCs.


 Unfortunately, I can't find any details about Win32/MIPS
implementation.


--
-=VA=-

Re: anybody tried NPTL?

[Ralf Baechle]

On Mon, Aug 23, 2004 at 09:28:53AM -0400, Daniel Jacobowitz wrote:

> On Fri, Aug 20, 2004 at 02:46:11PM +0100, Dominic Sweetman wrote:
> > I guess our main message was that we felt it would be a mistake just
> > to add a thread register to o32 (which produces a substantially
> > incompatible new ABI anyway).
> 
> Completely agree...
> 
> > Until that all works, what we had in mind is that we'd do NPTL over
> > o32 by defining a system call to return a per-thread ID which is or
> > can be converted into a per-thread data pointer.  We suspected that
> > NPTL's per-thread-data model allows the use of cunning macros or
> > library functions to make that look OK.
> > 
> > Ought we to go further and see exactly how that can be done?
> 
> It shouldn't be at all hard.  The way NPTL's __thread support works,
> the only things that should have to know where the TLS base is are
> (A) GCC, so it can load it and (B) GDB, via some new ptrace op.  I
> don't know if you'd want to open-code the syscall or take the overhead
> of a function call.  Ralf had some ideas?

Thiemo and have been compiling various pieces of code with different
gcc versions trying to find the best possible register for that purpose.
We used code bloat as (weak ...) indicator for register pressure.  It
turned out that $t9 was the best choice for all tested compiler versions;
thanks to the much improved register allocation of newer gcc the choice
of a particular register made far less difference on recent compilers
than on older compilers.

I've also implemented a fast system call for reading the thread registers.
Benchmarks did show that to have about half the latency of a regular
syscall; the hope was if gcc was doing clever optimization that overhead
would effectivly become zero.

I was favoring this low-overhead syscall approach because it would avoid
the loss of a register thus leaving performance of non-threaded code
unchanged but other developers generally favor the permanent allocation
of $t9 as a thread register.

Other crazy ideas did include a per-thread mapping containing the thread
pointer - and possibly more information in the future.

Finally one of the ideas was using one of $k0 / $k1 as thread pointer.
This would require changes to the exception handlers; any extra
instruction in the TLB refill handler would be particularly painful.

On the positive side if we had multiple register sets on a MIPSxx V2
processor we could exploit that to get rid of this overheade and do
other nice optimizations for TLB reload also.  Unfortunately these
register sets are optional feature of the architecture only.

  Ralf

Re: anybody tried NPTL?

[Ralf Baechle]

On Mon, Aug 23, 2004 at 07:09:31PM +0400, Alec Voropay wrote:

>  OK, OK. You are right. However, as it is known, there is at
> least one project "to bridge" Win32/multithread and *NIX :
>  WINE.
> http://winehq.com/site/docs/wine-devel/x3398
> 
>  Yes, the Win32/MIPS API (and ABI) is dead, but MIPS/multithreading
> lives in the WindowsCE/MIPS HPCs.
> 
> 
>  Unfortunately, I can't find any details about Win32/MIPS
> implementation.

That's because for Windows on x86 there are tons of important binary-only
applications available and Linux would have to be bought if they're
available at all.  On MIPS we're not in this dependency situation, so
we hardly care about Win32.

  Ralf

Re: anybody tried NPTL?

[Jun Sun]

On Mon, Aug 23, 2004 at 09:28:53AM -0400, Daniel Jacobowitz wrote:
> On Fri, Aug 20, 2004 at 02:46:11PM +0100, Dominic Sweetman wrote:
> > I guess our main message was that we felt it would be a mistake just
> > to add a thread register to o32 (which produces a substantially
> > incompatible new ABI anyway).
> 
> Completely agree...
> 
> > Until that all works, what we had in mind is that we'd do NPTL over
> > o32 by defining a system call to return a per-thread ID which is or
> > can be converted into a per-thread data pointer.  We suspected that
> > NPTL's per-thread-data model allows the use of cunning macros or
> > library functions to make that look OK.
> > 
> > Ought we to go further and see exactly how that can be done?
> 
> It shouldn't be at all hard.  The way NPTL's __thread support works,
> the only things that should have to know where the TLS base is are
> (A) GCC, so it can load it and (B) GDB, via some new ptrace op. 

Are you implying one can implement TLS support without changing O32
ABI?  Interesting...

I know Boris Hu has tried to implemented NPTL with another approach which
does not rely on TLS support (use "--without-tls").  According to him
this approach is getting harder these days.

Jun

Re: anybody tried NPTL?

[Daniel Jacobowitz]

On Mon, Aug 23, 2004 at 07:12:57PM +0200, Ralf Baechle wrote:
> Thiemo and have been compiling various pieces of code with different
> gcc versions trying to find the best possible register for that purpose.
> We used code bloat as (weak ...) indicator for register pressure.  It
> turned out that $t9 was the best choice for all tested compiler versions;
> thanks to the much improved register allocation of newer gcc the choice
> of a particular register made far less difference on recent compilers
> than on older compilers.
> 
> I've also implemented a fast system call for reading the thread registers.
> Benchmarks did show that to have about half the latency of a regular
> syscall; the hope was if gcc was doing clever optimization that overhead
> would effectivly become zero.
> 
> I was favoring this low-overhead syscall approach because it would avoid
> the loss of a register thus leaving performance of non-threaded code
> unchanged but other developers generally favor the permanent allocation
> of $t9 as a thread register.

Personally, I favor doing the low-overhead syscall for o32 and then
moving to the new ABI that MIPS is talking about with a thread
register.  I'm not sure what to do about n32/n64.

> Other crazy ideas did include a per-thread mapping containing the thread
> pointer - and possibly more information in the future.

Does MIPS have an efficient way to do this for SMP?

> On the positive side if we had multiple register sets on a MIPSxx V2
> processor we could exploit that to get rid of this overheade and do
> other nice optimizations for TLB reload also.  Unfortunately these
> register sets are optional feature of the architecture only.

That's more or less what was talked about for ARM v6.

-- 
Daniel Jacobowitz

Re: anybody tried NPTL?

[Ralf Baechle]

On Mon, Aug 23, 2004 at 01:44:47PM -0400, Daniel Jacobowitz wrote:

> Personally, I favor doing the low-overhead syscall for o32 and then
> moving to the new ABI that MIPS is talking about with a thread
> register.

I was always wondering how far gcc could optimize code to minimize this
special system call.  After all on Alpha something similar PAL-code based
was the method of choice.

> I'm not sure what to do about n32/n64.

I believe N32 / N64 are not widespead enough yet to be a big roadblock
for moving to new ABIs.  Whoever disagress should better speak up soon :-)

If we deciede to move to something entirely different than the existing
ABIs in the future will be able to support compatibility the same way
we're already doing.

> > Other crazy ideas did include a per-thread mapping containing the thread
> > pointer - and possibly more information in the future.
> 
> Does MIPS have an efficient way to do this for SMP?

It can be done making the TLB fault for that page about as expensive as
a null syscall.

> > On the positive side if we had multiple register sets on a MIPSxx V2
> > processor we could exploit that to get rid of this overheade and do
> > other nice optimizations for TLB reload also.  Unfortunately these
> > register sets are optional feature of the architecture only.
> 
> That's more or less what was talked about for ARM v6.

I'm unARMed here ;-)

  Ralf

Re: anybody tried NPTL?

[Ralf Baechle]

On Mon, Aug 23, 2004 at 10:37:31AM -0700, Jun Sun wrote:

> Are you implying one can implement TLS support without changing O32
> ABI?  Interesting...
> 
> I know Boris Hu has tried to implemented NPTL with another approach which
> does not rely on TLS support (use "--without-tls").  According to him
> this approach is getting harder these days.

The whole TLS pointer thing is about making TLS more efficient.  If
you wanted to use TLS without any kernel changes you could do that
based on the THREAD result value of pthread_create or something like
that.  It'd work but it'd also not be terribly efficient ...

  Ralf

Re: anybody tried NPTL?

[Richard Sandiford]

Dominic Sweetman <dom@mips.com> writes:
> In many ways it's easier to get away from the ingenious but
> troublesome stack-structure-based calling convention.  The "stack
> structure" trick was invented so that o32 could work without function
> prototypes - but function prototypes are now required, and something
> with fixed-size arguments is simpler.  Something like the old
> Cygnus/WRS EABI proposal, in fact...

Indeed, all of this:

> So we're proposing:
> [...]
> o Calling convention: register-, not slot-based. Each argument is
>   represented by a register value. Arguments 0-7 travel in registers
>   a0-7 (or fa0-7 as required for floating point types). If there are
>   more than eight arguments, further ones are formed as if put in a
>   register and then saved on the stack into a 64-bit slot (more than 8
>   arguments is rare enough that we can afford to standardise on the
>   big slots).
>   
> o Use floating point registers for double and float arguments, and
>   integer registers for all integer/pointer values which will
>   fit. Larger or structured data items are implicitly passed by
>   reference: to maintain pass-by-value semantics, the compiler uses a
>   copy-on-write trick if software writes a by-reference argument (or
>   takes its address).  I'm told gcc is happy enough to do that.
>
> o The return value comes back in two registers, with the second
>   return-register used only when the return value consists of two
>   scalars (ie a complex or double-precision number). [Folklore insists
>   this is essential for Fortran support of complex numbers, and I
>   don't want to fight folklore].
>
>   All other non-scalar return values are returned via a pointer
>   specified by the caller as an implicit first argument.
>
> o Reserved registers: all the traditional ones. But now:
>
>   - gp will be the GOT pointer in Linux, and should be defined as
>     saved (ie a function must preserve values in this registers, which
>     means it will need to save-and-restore the register if it is
>     written locally).

...are pretty much what we already do for EABI.  There's some fuzziness
in what EABI considers to be "double and float arguments": things like
"struct { float f; }" are passed in the same way as scalar floats, for
example.  Complex integers are also returned by invisible reference
rather than in two GPRs.  But these are very much corner cases.

Rather than define new, slightly different, conventions, have you thought
about defining this new ABI as an extension of EABI?  I.e. taking EABI,
adding n32/n64-style PIC, and the all-important:

>   - we'll define some other register as a per-thread data pointer.

Richard