ll/sc emulation patch

ll/sc emulation patch

[Jan-Benedict Glaw]

Hi!

At Linuxtag 2001 in Stuttgart/Germany, I was talking to Karsten about
completing userspace. He mentioned a patch for ll/sc emulation in
kernel which was mentioned to be sent some days ago to l-k. I searched
the archives, but couldn't find it. Any pointers?

MfG, JBG

Re: ll/sc emulation patch

[Ralf Baechle]

On Thu, Jul 12, 2001 at 08:37:27PM +0200, Jan-Benedict Glaw wrote:

> At Linuxtag 2001 in Stuttgart/Germany, I was talking to Karsten about
> completing userspace. He mentioned a patch for ll/sc emulation in
> kernel which was mentioned to be sent some days ago to l-k. I searched
> the archives, but couldn't find it. Any pointers?

There is a bunch of patches for ll/sc and MIPS_ATOMIC_SET floating around.
I came to the conclusion that I don't like any of them so I'm just working
on fixing the thing once and for all.

  Ralf

Re: ll/sc emulation patch

[Maciej W. Rozycki]

On Thu, 12 Jul 2001, Ralf Baechle wrote:

> There is a bunch of patches for ll/sc and MIPS_ATOMIC_SET floating around.
> I came to the conclusion that I don't like any of them so I'm just working
> on fixing the thing once and for all.

 What about the _test_and_set syscall?  Having it implemented we could get
rid of the MIPS_ATOMIC_SET hack for post-2.4.  Glibc may be made ready for
the transition any time now.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: ll/sc emulation patch

[Ralf Baechle]

On Fri, Jul 13, 2001 at 12:52:55PM +0200, Maciej W. Rozycki wrote:

> > There is a bunch of patches for ll/sc and MIPS_ATOMIC_SET floating around.
> > I came to the conclusion that I don't like any of them so I'm just working
> > on fixing the thing once and for all.
> 
>  What about the _test_and_set syscall?  Having it implemented we could get
> rid of the MIPS_ATOMIC_SET hack for post-2.4.  Glibc may be made ready for
> the transition any time now.

I'm just making an attempt to re-implement the ll/sc emulation as light
as possible.  I hope to get the overhead down to the point were we don't
need _test_and_set anymore - in any case below the overhead of a syscall.

Have you ever profiled the number of calls to MIPS_ATOMIC_SET or
_test_and_set?  They'll be the other factor in a decission.

  Ralf

Re: ll/sc emulation patch

[Harald Koerfgen]

On Saturday 14 July 2001 12:53, Ralf Baechle wrote:
> I'm just making an attempt to re-implement the ll/sc emulation as light
> as possible.  I hope to get the overhead down to the point were we don't
> need _test_and_set anymore - in any case below the overhead of a syscall.

I'd love to see something like that. What 's your idea?


Greetings,
Harald
-- 
There is no distinctly native American criminal class except Congress.
		-- Mark Twain

Re: ll/sc emulation patch

[Jun Sun]

On Sat, Jul 14, 2001 at 12:53:12PM +0200, Ralf Baechle wrote:
> On Fri, Jul 13, 2001 at 12:52:55PM +0200, Maciej W. Rozycki wrote:
> 
> > > There is a bunch of patches for ll/sc and MIPS_ATOMIC_SET floating around.
> > > I came to the conclusion that I don't like any of them so I'm just working
> > > on fixing the thing once and for all.
> > 
> >  What about the _test_and_set syscall?  Having it implemented we could get
> > rid of the MIPS_ATOMIC_SET hack for post-2.4.  Glibc may be made ready for
> > the transition any time now.
> 
> I'm just making an attempt to re-implement the ll/sc emulation as light
> as possible.  I hope to get the overhead down to the point were we don't
> need _test_and_set anymore - in any case below the overhead of a syscall.
>

That is really cool.  

For compatibility reasons, I assume we still keep sysmips() around
for a while, right?  And if that is yes, we better take either flo's or
my fix (derived from Maceij's new syscall) to get rid of the illegal 
instruction bug, which really becomes an untolerable FAQ now.

Jun

Re: ll/sc emulation patch

[Ralf Baechle]

On Sun, Jul 15, 2001 at 09:32:24PM -0700, Jun Sun wrote:

> For compatibility reasons, I assume we still keep sysmips() around
> for a while, right?  And if that is yes, we better take either flo's or
> my fix (derived from Maceij's new syscall) to get rid of the illegal 
> instruction bug, which really becomes an untolerable FAQ now.

I've also debugged sysmips.

  Ralf

Re: ll/sc emulation patch

[Maciej W. Rozycki]

On Sun, 15 Jul 2001, Jun Sun wrote:

> For compatibility reasons, I assume we still keep sysmips() around
> for a while, right?  And if that is yes, we better take either flo's or
> my fix (derived from Maceij's new syscall) to get rid of the illegal 
> instruction bug, which really becomes an untolerable FAQ now.

 I don't think it would be wise to remove MIPS_ATOMIC_SET before 2.5.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: ll/sc emulation patch

[Florian Lohoff]

On Sat, Jul 14, 2001 at 12:53:12PM +0200, Ralf Baechle wrote:
> Have you ever profiled the number of calls to MIPS_ATOMIC_SET or
> _test_and_set?  They'll be the other factor in a decission.

Against glibc 2.2 compiled bins do it 20-30 times on startup ... Just
try an strace and you'll astonished ... Even sleep (What on earth
is atomic in sleep ?)

Flo
-- 
Florian Lohoff                  flo@rfc822.org             +49-5201-669912
     Why is it called "common sense" when nobody seems to have any?

Re: ll/sc emulation patch

[Maciej W. Rozycki]

On Sat, 14 Jul 2001, Ralf Baechle wrote:

> I'm just making an attempt to re-implement the ll/sc emulation as light
> as possible.  I hope to get the overhead down to the point were we don't
> need _test_and_set anymore - in any case below the overhead of a syscall.
> 
> Have you ever profiled the number of calls to MIPS_ATOMIC_SET or
> _test_and_set?  They'll be the other factor in a decission.

 I didn't profile it very extensively, yet when stracing `ls /usr/lib'
(fileutils 4.1 linked against glibc 2.2.3) on my system once I yielded
~4500 syscalls of which ~4000 were _test_and_set() (or MIPS_ATOMIC_SET,
depending on my kernel/glibc configuration) invocations.  Yes, libpthread
appears to assume atomic operations are cheap, which is justifiable as
they are indeed, for almost every other CPU type. 

 Also I feel having ll and sc opcodes in a pure MIPS I binary is somewhat
ugly (e.g. `objdump' won't disassemble them unless a MIPS II+ CPU is
specified), but I could probably live with it if performance was not
worse. 

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: ll/sc emulation patch

[Mike McDonald]

>Date: Mon, 16 Jul 2001 14:03:30 +0200 (MET DST)
>From: "Maciej W. Rozycki" <macro@ds2.pg.gda.pl>
>To: Ralf Baechle <ralf@oss.sgi.com>
>Subject: Re: ll/sc emulation patch
>
>On Sat, 14 Jul 2001, Ralf Baechle wrote:
>
>> I'm just making an attempt to re-implement the ll/sc emulation as light
>> as possible.  I hope to get the overhead down to the point were we don't
>> need _test_and_set anymore - in any case below the overhead of a syscall.
>> 
>> Have you ever profiled the number of calls to MIPS_ATOMIC_SET or
>> _test_and_set?  They'll be the other factor in a decission.
>
> I didn't profile it very extensively, yet when stracing `ls /usr/lib'
>(fileutils 4.1 linked against glibc 2.2.3) on my system once I yielded
>~4500 syscalls of which ~4000 were _test_and_set() (or MIPS_ATOMIC_SET,
>depending on my kernel/glibc configuration) invocations.  Yes, libpthread
>appears to assume atomic operations are cheap, which is justifiable as
>they are indeed, for almost every other CPU type. 

  Not knowing anything about the glibc architecture, I have a dumb
question: why is 'ls' doing anything at all with pthreads?

  Mike McDonald
  mikemac@mikemac.com

Re: ll/sc emulation patch

[H . J . Lu]

On Mon, Jul 16, 2001 at 11:47:04AM -0700, Mike McDonald wrote:
> 
>   Not knowing anything about the glibc architecture, I have a dumb
> question: why is 'ls' doing anything at all with pthreads?

You must be using a broken Linux. /bin/ls in my RedHat 7.1/mips doesn't
use pthreads.


H.J.

Re: ll/sc emulation patch

[Maciej W. Rozycki]

On Mon, 16 Jul 2001, H . J . Lu wrote:

> You must be using a broken Linux. /bin/ls in my RedHat 7.1/mips doesn't
> use pthreads.

 Which version of ls?  Versions up to 4.0 did not, indeed.

 Of course, you may force version 4.1 not to make use libpthreads, either. 
Just convince the configure script in the usual way not to use
clock_gettime().

 I just checked ls 4.1 compiled for i386-linux in a glibc 2.2.3
environment:

$ LD_TRACE_LOADED_OBJECTS=1 ls
	librt.so.1 => /lib/librt.so.1 (0x40020000)
	libc.so.6 => /lib/libc.so.6 (0x40032000)
	libpthread.so.0 => /lib/libpthread.so.0 (0x4015a000)
	/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)

so it's not MIPS-specific at all...  Why would it, anyway? 

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: ll/sc emulation patch

[H . J . Lu]

On Mon, Jul 16, 2001 at 09:51:12PM +0200, Maciej W. Rozycki wrote:
> 
>  Of course, you may force version 4.1 not to make use libpthreads, either. 
> Just convince the configure script in the usual way not to use
> clock_gettime().
> 

ls shouldn't bother with clock_gettime, which is in librt and librt
needs libpthreads. RedHat 7.1 has a similar patch to make 3.79.1 to
get around it. Otherwise, make won't work right due to the 2MB stack
limit imposed by libpthreads.


H.J.

Re: ll/sc emulation patch

[Maciej W. Rozycki]

On Mon, 16 Jul 2001, H . J . Lu wrote:

> ls shouldn't bother with clock_gettime, which is in librt and librt
> needs libpthreads. RedHat 7.1 has a similar patch to make 3.79.1 to
> get around it. Otherwise, make won't work right due to the 2MB stack
> limit imposed by libpthreads.

 ls.c contains an explicit reference to clock_gettime() in its
get_current_time() function.  The reference was added quite recently but
ChangeLog does not contain a relevant entry.  In any case clock_gettime()
provides a better resolution than gettimeofday() does.  Did you or someone
else contacted the maintainer to clarify the issue?  RedHat isn't the
whole world. 

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: ll/sc emulation patch

[H . J . Lu]

On Mon, Jul 16, 2001 at 10:33:42PM +0200, Maciej W. Rozycki wrote:
> On Mon, 16 Jul 2001, H . J . Lu wrote:
> 
> > ls shouldn't bother with clock_gettime, which is in librt and librt
> > needs libpthreads. RedHat 7.1 has a similar patch to make 3.79.1 to
> > get around it. Otherwise, make won't work right due to the 2MB stack
> > limit imposed by libpthreads.
> 
>  ls.c contains an explicit reference to clock_gettime() in its
> get_current_time() function.  The reference was added quite recently but
> ChangeLog does not contain a relevant entry.  In any case clock_gettime()
> provides a better resolution than gettimeofday() does.  Did you or someone
> else contacted the maintainer to clarify the issue?  RedHat isn't the
> whole world. 

I am happy to let Red Hat deal with it. FWIW, RedHat 7.1 has
fileutils 4.0.36.


H.J.

Re: ll/sc emulation patch

[Mike McDonald]

>To: "Maciej W. Rozycki" <macro@ds2.pg.gda.pl>
>Subject: Re: ll/sc emulation patch 
>Date: Mon, 16 Jul 2001 11:47:04 -0700
>From: Mike McDonald <mikemac@mikemac.com>
>
>
>>Date: Mon, 16 Jul 2001 14:03:30 +0200 (MET DST)
>>From: "Maciej W. Rozycki" <macro@ds2.pg.gda.pl>
>>To: Ralf Baechle <ralf@oss.sgi.com>
>>Subject: Re: ll/sc emulation patch

>> I didn't profile it very extensively, yet when stracing `ls /usr/lib'
>>(fileutils 4.1 linked against glibc 2.2.3) on my system once I yielded
>>~4500 syscalls of which ~4000 were _test_and_set() (or MIPS_ATOMIC_SET,
>>depending on my kernel/glibc configuration) invocations.  Yes, libpthread
>>appears to assume atomic operations are cheap, which is justifiable as
>>they are indeed, for almost every other CPU type. 
>
>  Not knowing anything about the glibc architecture, I have a dumb
>question: why is 'ls' doing anything at all with pthreads?

  OK, let me rephrase this: why are ~90% of ls's syscalls calls to
_test_and_set() when 'ls' is(??) a single threaded program? Does glibc
always assume it's running in a multithreaded environment?

  Mike McDonald
  mikemac@mikemac.com

Re: ll/sc emulation patch

[Maciej W. Rozycki]

On Mon, 16 Jul 2001, Mike McDonald wrote:

>   Not knowing anything about the glibc architecture, I have a dumb
> question: why is 'ls' doing anything at all with pthreads?

 A short answer: clock_gettime().

 A long one: clock_gettime(), which is defined by librt, which in turn
depends on libpthread.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: ll/sc emulation patch

[Ralf Baechle]

On Mon, Jul 16, 2001 at 02:03:30PM +0200, Maciej W. Rozycki wrote:

>  I didn't profile it very extensively, yet when stracing `ls /usr/lib'
> (fileutils 4.1 linked against glibc 2.2.3) on my system once I yielded
> ~4500 syscalls of which ~4000 were _test_and_set() (or MIPS_ATOMIC_SET,
> depending on my kernel/glibc configuration) invocations.  Yes, libpthread
> appears to assume atomic operations are cheap, which is justifiable as
> they are indeed, for almost every other CPU type. 

On a fast Indy those 4000 syscalls would cost about 3.2ms of CPU which is
a noticable fraction of the total execution time.

>  Also I feel having ll and sc opcodes in a pure MIPS I binary is somewhat
> ugly (e.g. `objdump' won't disassemble them unless a MIPS II+ CPU is
> specified), but I could probably live with it if performance was not
> worse. 

This behaviour of objdump sucks rocks anyway.  There are MIPS I CPUs which
have ll but no branch likely and many other MIPS ISA perversions.  Objdump
also will only hexdump anything that hasn't been marked as code with
.type.  Seems objdump's behaviour was choosen to be the most annoying
possible.

  Ralf

Re: ll/sc emulation patch

[Maciej W. Rozycki]

On Thu, 19 Jul 2001, Ralf Baechle wrote:

> This behaviour of objdump sucks rocks anyway.  There are MIPS I CPUs which
> have ll but no branch likely and many other MIPS ISA perversions.  Objdump
> also will only hexdump anything that hasn't been marked as code with
> .type.  Seems objdump's behaviour was choosen to be the most annoying
> possible.

 Well, actually I disagree.  I like this very much, since looking at the
output it's immediately obvious if some code contains opcodes
unrecognizable by a selected CPU (due to a bad compiler option or an
inline asm error).  And you can always select a desired CPU type with the
"-m mips:<model>" option (e.g. "-m mips:3000").  If there are MIPS I CPUs
which have have ll but no branch likely, etc. then their descriptions
should be appropriately set in objdump.  Otherwise, it's a bug in
binutils. 

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +