Re: horrible disk thorughput on itanium

Re: horrible disk thorughput on itanium

[Andi Kleen]

torvalds@transmeta.com (Linus Torvalds) writes:
> 
> "putc()" is a standard function.  If it sucks, let's get it fixed.  And
> instead of changing bonnie, how about pinging the _real_ people who
> write sucky code?

It is easy to fix. Just do #define putc putc_unlocked
There is just a slight problem: it'll fail if your application is threaded
and wants to use the same FILE from multiple threads.

It is a common problem on all OS that eventually got threadsafe stdio. 
I bet putc sucks on Solaris too.

-Andi

Re: horrible disk thorughput on itanium

[Padraig Brady]

Andi Kleen wrote:

> torvalds@transmeta.com (Linus Torvalds) writes:
> 
>>"putc()" is a standard function.  If it sucks, let's get it fixed.  And
>>instead of changing bonnie, how about pinging the _real_ people who
>>write sucky code?
>>
> 
> It is easy to fix. Just do #define putc putc_unlocked
> There is just a slight problem: it'll fail if your application is threaded
> and wants to use the same FILE from multiple threads.
> 
> It is a common problem on all OS that eventually got threadsafe stdio. 
> I bet putc sucks on Solaris too.
> 
> -Andi


Interesting thread on this:
http://sources.redhat.com/ml/bug-glibc/2001-11/msg00079.html

for glibc 2.2.4 with the following program with input file
of 354371 lines of text(/usr/share/doc/*), where the average line
length was 37 chars.

getc/putc
real 
2.181s
user 
2.150s
sys 
0.030s
getc_unlocked/putc_unlocked
real 
0.326s
user 
0.280s
sys 
0.040s

I.E. 669% faster!

Padraig.

-------------------
#include <stdio.h>

#ifndef  _REENTRANT
#    undef getc
#    define getc(x)   getc_unlocked(x)
#    undef putc
#    define putc(x,y) putc_unlocked(x,y)
#endif //_REENTRANT

void main(void)
{
     int c;
     while((c=getc(stdin))!=EOF) putc(c,stdout);
}

Re: horrible disk thorughput on itanium

[Richard Gooch]

Andi Kleen writes:
> torvalds@transmeta.com (Linus Torvalds) writes:
> > 
> > "putc()" is a standard function.  If it sucks, let's get it fixed.  And
> > instead of changing bonnie, how about pinging the _real_ people who
> > write sucky code?
> 
> It is easy to fix. Just do #define putc putc_unlocked
> There is just a slight problem: it'll fail if your application is threaded
> and wants to use the same FILE from multiple threads.
> 
> It is a common problem on all OS that eventually got threadsafe stdio. 
> I bet putc sucks on Solaris too.

This kind of thing should be covered by _REENTRANT. If you don't
compile with _REENTRANT (the default), then putc() should be the
unlocked version.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: horrible disk thorughput on itanium

[Robert Love]

On Fri, 2001-12-07 at 11:14, Richard Gooch wrote:

> This kind of thing should be covered by _REENTRANT. If you don't
> compile with _REENTRANT (the default), then putc() should be the
> unlocked version.

The link to the mailing list post from bug-glibc says otherwise, that is
the problem.  Using the unlocked version isn't implied by not setting
__REENTRANT.

	Robert Love

Re: horrible disk thorughput on itanium

[Richard Gooch]

Robert Love writes:
> On Fri, 2001-12-07 at 11:14, Richard Gooch wrote:
> 
> > This kind of thing should be covered by _REENTRANT. If you don't
> > compile with _REENTRANT (the default), then putc() should be the
> > unlocked version.
> 
> The link to the mailing list post from bug-glibc says otherwise,
> that is the problem.  Using the unlocked version isn't implied by
> not setting __REENTRANT.

The bug is in glibc. An application shouldn't need to be changed to
work around that bug. putc() is a well-known interface, and people
shouldn't have to code around a change in that interface.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: horrible disk thorughput on itanium

[Robert Love]

On Fri, 2001-12-07 at 12:40, Richard Gooch wrote:

> > The link to the mailing list post from bug-glibc says otherwise,
> > that is the problem.  Using the unlocked version isn't implied by
> > not setting __REENTRANT.
> 
> The bug is in glibc. An application shouldn't need to be changed to
> work around that bug. putc() is a well-known interface, and people
> shouldn't have to code around a change in that interface.

Right.  That's why I referenced a post on bug-glibc and called the issue
a problem.  I'm not defending the heaping mass known as glibc ... it
should be fixed.

	Robert Love

Re: horrible disk thorughput on itanium

[Linus Torvalds]

On 7 Dec 2001, Andi Kleen wrote:
> torvalds@transmeta.com (Linus Torvalds) writes:
> >
> > "putc()" is a standard function.  If it sucks, let's get it fixed.  And
> > instead of changing bonnie, how about pinging the _real_ people who
> > write sucky code?
>
> It is easy to fix. Just do #define putc putc_unlocked

Sure. And why don't you also do

	#define sin(x) (1)
	#define sqrt(x) (1)
	#define strlen(x) (1)
	...

to make other benchmarks happier?

bonnie is a _benchmark_. It's meant for finding bad performance. Changing
it to make it work better when performance is bad is _pointless_. You've
now made the whole point of bonnie go away.

> There is just a slight problem: it'll fail if your application is threaded
> and wants to use the same FILE from multiple threads.
>
> It is a common problem on all OS that eventually got threadsafe stdio.

It's a common problem with bad programming.

You can be thread-safe without sucking dead baby donkeys through a straw.
I already mentioned two possible ways to fix it so that you have locking
when you need to, and no locking when you don't.

		Linus

Re: horrible disk thorughput on itanium

[Andi Kleen]

> You can be thread-safe without sucking dead baby donkeys through a straw.
> I already mentioned two possible ways to fix it so that you have locking
> when you need to, and no locking when you don't.

Your proposals sound rather dangerous. They would silently break recompiled
threaded programs that need the locking and don't use -D__REENTRANT (most
people do not seem to use it). I doubt the possible pain from that is 
worth it for speeding up an basically obsolete interface like putc. 

i.e. if someone wants speed they definitely shouldn't use putc()

-Andi

Re: horrible disk thorughput on itanium

[Michael Poole]

Andi Kleen <ak@suse.de> writes:

> > You can be thread-safe without sucking dead baby donkeys through a straw.
> > I already mentioned two possible ways to fix it so that you have locking
> > when you need to, and no locking when you don't.
> 
> Your proposals sound rather dangerous. They would silently break recompiled
> threaded programs that need the locking and don't use -D__REENTRANT (most
> people do not seem to use it). I doubt the possible pain from that is 
> worth it for speeding up an basically obsolete interface like putc. 
> 
> i.e. if someone wants speed they definitely shouldn't use putc()

Threaded programs that need locking and don't define _THREAD_SAFE or
_REENTRANT or whatever is appropriate are already broken -- they just
don't know it yet.

FreeBSD #defines putc and getc to their unlocked versions unless
_THREAD_SAFE is defined, and people don't seem to think its libc is
broken.  Many lightly threaded programs, in fact, wouldn't need or
even want the locked variants to be the default.  One app I've worked
with only reads and writes any given FILE* from one thread, and I saw
an 4x speedup by switching to the unlocked variants.

It's generally a bad idea to make people pay for a feature they don't
ask for.  FreeBSD's libc understands this; glibc apaprently doesn't.

-- Michael Poole

Re: horrible disk thorughput on itanium

[Padraig Brady]

Michael Poole wrote:

> Andi Kleen <ak@suse.de> writes:
> 
> 
>>>You can be thread-safe without sucking dead baby donkeys through a straw.
>>>I already mentioned two possible ways to fix it so that you have locking
>>>when you need to, and no locking when you don't.
>>>
>>Your proposals sound rather dangerous. They would silently break recompiled
>>threaded programs that need the locking and don't use -D__REENTRANT (most
>>people do not seem to use it). I doubt the possible pain from that is 
>>worth it for speeding up an basically obsolete interface like putc. 
>>
>>i.e. if someone wants speed they definitely shouldn't use putc()
>>
> 
> Threaded programs that need locking and don't define _THREAD_SAFE or
> _REENTRANT or whatever is appropriate are already broken -- they just
> don't know it yet.
> 
> FreeBSD #defines putc and getc to their unlocked versions unless
> _THREAD_SAFE is defined, and people don't seem to think its libc is
> broken.  Many lightly threaded programs, in fact, wouldn't need or
> even want the locked variants to be the default.  One app I've worked
> with only reads and writes any given FILE* from one thread, and I saw
> an 4x speedup by switching to the unlocked variants.

This breaks for the case discussed @
http://sources.redhat.com/ml/bug-glibc/2001-11/msg00079.html
I.E. if you have a multithreaded lib being linked by
single threaded apps (Note multithreaded lib, not just a
threadsafe lib (I.E. the lib calls pthread_create())).

Padraig.

Re: horrible disk thorughput on itanium

[Michael Poole]

Padraig Brady <padraig@antefacto.com> writes:

> This breaks for the case discussed @
> http://sources.redhat.com/ml/bug-glibc/2001-11/msg00079.html
> I.E. if you have a multithreaded lib being linked by
> single threaded apps (Note multithreaded lib, not just a
> threadsafe lib (I.E. the lib calls pthread_create())).

That's an interesting, but very contrived, example.  Can you find a
single multi-threaded lib that uses FILE*'s shared with the
application using it?

Linus's suggestion to add hooks to pthread_create() gets around that
problem, anyway.  Alternatively, the multi-threaded library could
require any application linking to it to define _REENTRANT.

After all, it's silly to talk about a 'multi-threaded' library linked
to a 'single-threaded' application -- the application plus any
libraries, as a whole, are either multithreaded or not.  They have to
be on the same page to deal with *any* locking issues.

-- Michael

Re: horrible disk thorughput on itanium

[Padraig Brady]

Michael Poole wrote:

> Padraig Brady <padraig@antefacto.com> writes:
> 
> 
>>This breaks for the case discussed @
>>http://sources.redhat.com/ml/bug-glibc/2001-11/msg00079.html
>>I.E. if you have a multithreaded lib being linked by
>>single threaded apps (Note multithreaded lib, not just a
>>threadsafe lib (I.E. the lib calls pthread_create())).
>>
> 
> That's an interesting, but very contrived, example.  Can you find a
> single multi-threaded lib that uses FILE*'s shared with the
> application using it?


Well you have to deal with the general case. A single threaded

app linking against a multithreaded lib. It mightn't be just
shared FILE*'s that could cause problems.


> Linus's suggestion to add hooks to pthread_create() gets around that
> problem, anyway.


I don't think this will work as I said before current apps that
use _unlocked() functions directly manipulate the stdio structures,
hence a "new smarter locking stdio" would never get used by existing
compiled apps.

> Alternatively, the multi-threaded library could
> require any application linking to it to define _REENTRANT.


It could, but what if an existing interface (lib) is changed

from signle to multithreaded. You can't preclude this.


> After all, it's silly to talk about a 'multi-threaded' library linked
> to a 'single-threaded' application -- the application plus any
> libraries, as a whole, are either multithreaded or not.  They have to
> be on the same page to deal with *any* locking issues.
> 
> -- Michael

Padraig.

Re: horrible disk thorughput on itanium

[Michael Poole]

Padraig Brady <padraig@antefacto.com> writes:

> Well you have to deal with the general case. A single threaded
> app linking against a multithreaded lib. It mightn't be just
> shared FILE*'s that could cause problems.

The question was about putc(), and presumably other stdio functions.
They deal with FILE*'s.  They are also commonly used and small
operations, so forcing locking on an app that doesn't ask for it can
(and has in the past) cause a major performance degredation.  I bet
you could find very similar results for memory allocation in other
applications.

> > Linus's suggestion to add hooks to pthread_create() gets around that
> > problem, anyway.
> 
> I don't think this will work as I said before current apps that
> use _unlocked() functions directly manipulate the stdio structures,
> hence a "new smarter locking stdio" would never get used by existing
> compiled apps.

As Linus pointed out, you just need another test in those macros, so
they can be forced to call functions rather than using inline code.
When you call a function, it can use whatever new smarter locking
library you want.

> > Alternatively, the multi-threaded library could
> > require any application linking to it to define _REENTRANT.
> 
> It could, but what if an existing interface (lib) is changed
> from signle to multithreaded. You can't preclude this.

I certainly can preclude that.  I'd consider adding threads and their
locking considerations to the library a change in the API and ABI --
and quite a big one at that.  If you change the ABI (rather than just
extending it), you must increase the major version number, which
prevents linking with those applications that expect the semantics of
earlier versions.

(BSD variants have in the past had both libc and libc_r, where only
libc_r makes you pay locking overhead.  This is yet another way to
enforce the ABI separation between single- and multi-threaded
applications, with different tradeoffs than the others mentioned.)

-- Michael

Re: horrible disk thorughput on itanium

[Linus Torvalds]

On Fri, 7 Dec 2001, Andi Kleen wrote:
>
> Your proposals sound rather dangerous. They would silently break recompiled
> threaded programs that need the locking and don't use -D__REENTRANT

No it wouldn't.

Once you do a pthread_create(), the locking is there.

Before you do a pthread_create(), it doesn't lock.

What's the problem? Before you do a pthread_create(), you don't _NEED_
locking, because there is only one thread that accesses the stdio data
structures.

And there are no races - if there is only one thread, then another thread
couldn't be suddenly doing a pthread_create() during a stdio operations.

Safe, and efficient. Yes, it adds a flag test or a indirect branch, but
considering that you avoid a serialized locking instruction, the
optimization sounds obvious.

		Linus

Re: horrible disk thorughput on itanium

[Padraig Brady]

Linus Torvalds wrote:

> On Fri, 7 Dec 2001, Andi Kleen wrote:
> 
>>Your proposals sound rather dangerous. They would silently break recompiled
>>threaded programs that need the locking and don't use -D__REENTRANT
>>
> 
> No it wouldn't.
> 
> Once you do a pthread_create(), the locking is there.
> 
> Before you do a pthread_create(), it doesn't lock.
> 
> What's the problem? Before you do a pthread_create(), you don't _NEED_
> locking, because there is only one thread that accesses the stdio data
> structures.
> 
> And there are no races - if there is only one thread, then another thread
> couldn't be suddenly doing a pthread_create() during a stdio operations.
> 
> Safe, and efficient. Yes, it adds a flag test or a indirect branch, but
> considering that you avoid a serialized locking instruction, the
> optimization sounds obvious.
> 
> 		Linus

Re: horrible disk thorughput on itanium

[Padraig Brady]

Andi Kleen wrote:

>>You can be thread-safe without sucking dead baby donkeys through a straw.
>>I already mentioned two possible ways to fix it so that you have locking
>>when you need to, and no locking when you don't.
>>
> 
> Your proposals sound rather dangerous. They would silently break recompiled
> threaded programs that need the locking and don't use -D__REENTRANT (most
> people do not seem to use it).


I would worry about threaded progs that don't -D_REENTRANT as
they are broken.

> I doubt the possible pain from that is 
> worth it for speeding up an basically obsolete interface like putc. 
> 
> i.e. if someone wants speed they definitely shouldn't use putc()

It's not just putc, it's all of stdio.

Padraig.

Re: horrible disk thorughput on itanium

[Greg Hennessy]

In article <Pine.LNX.4.33.0112070941330.8465-100000@penguin.transmeta.com>,
Linus Torvalds <torvalds@transmeta.com> wrote:
> bonnie is a _benchmark_. It's meant for finding bad performance. Changing
> it to make it work better when performance is bad is _pointless_. You've
> now made the whole point of bonnie go away.

It isn't just bonnie showing bad performance. My application shows it,
bonnie shows it, and tiobench shows it. I think the focus on putc
may be too intense. It isn't suprizing to me that either the kernel
or glibc may not be optimised on ia64, I'm just trying to figure out
how to get better io rates out of my itanium machine.

Re: horrible disk thorughput on itanium

[Marco Colombo]

On Fri, 7 Dec 2001, Greg Hennessy wrote:

> In article <Pine.LNX.4.33.0112070941330.8465-100000@penguin.transmeta.com>,
> Linus Torvalds <torvalds@transmeta.com> wrote:
> > bonnie is a _benchmark_. It's meant for finding bad performance. Changing
> > it to make it work better when performance is bad is _pointless_. You've
> > now made the whole point of bonnie go away.
> 
> It isn't just bonnie showing bad performance. My application shows it,
> bonnie shows it, and tiobench shows it. I think the focus on putc
> may be too intense. It isn't suprizing to me that either the kernel
> or glibc may not be optimised on ia64, I'm just trying to figure out
> how to get better io rates out of my itanium machine.

Does a simple 'dd' show the problem? I mean,

time dd if=/dev/zero of=/somelargefile count=somelargenumber bs=8k

is it much slower on the itanium, too? dd doesn't use putc(), I hope.

Just for comparison, I've run the following command here:

# time dd if=/dev/zero of=/u2/test count=250000 bs=8k
250000+0 records in
250000+0 records out
0.14user 12.95system 1:23.15elapsed 15%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (117major+18minor)pagefaults 0swaps

(Almost idle box, 256MB RAM, UDMA disk, 2.2.x kernel)

You may try with a bigger file, expecially if you have more RAM.

.TM.
-- 
      ____/  ____/   /
     /      /       /			Marco Colombo
    ___/  ___  /   /		      Technical Manager
   /          /   /			 ESI s.r.l.
 _____/ _____/  _/		       Colombo@ESI.it

Re: horrible disk thorughput on itanium

[David S. Miller]

   From: Andi Kleen <ak@suse.de>
   Date: 07 Dec 2001 14:54:49 +0100
   
   It is a common problem on all OS that eventually got threadsafe stdio. 
   I bet putc sucks on Solaris too.

Nope, they even inline the full implementation it in their header
files so they beat even our putc_unlocked() to the point it is
embarassing.

[OT] fputc vs putc Re: horrible disk thorughput on itanium

[Tom Vier]

On Fri, Dec 07, 2001 at 01:03:16PM -0800, David S. Miller wrote:
> They do it also for putc() if you haven't defined the appropriate
> defines.

what exactly is the difference between putc() and fputc()? the man page is
very vague.

-- 
Tom Vier <tmv5@home.com>
DSA Key id 0x27371A2C

Re: [OT] fputc vs putc Re: horrible disk thorughput on itanium

[Eric W. Biederman]

Tom Vier <tmv5@home.com> writes:

> On Fri, Dec 07, 2001 at 01:03:16PM -0800, David S. Miller wrote:
> > They do it also for putc() if you haven't defined the appropriate
> > defines.
> 
> what exactly is the difference between putc() and fputc()? the man page is
> very vague.

fputc is a function putc is a macro because the overhead of a function call
in writing a character has always been a problem...

Eric

Re: [OT] fputc vs putc Re: horrible disk thorughput on itanium

[H. Peter Anvin]

Followup to:  <m1d71pw51p.fsf@frodo.biederman.org>
By author:    ebiederm@xmission.com (Eric W. Biederman)
In newsgroup: linux.dev.kernel
> 
> fputc is a function putc is a macro because the overhead of a function call
> in writing a character has always been a problem...
> 

putc() is frequently defined as

#define putc(__C)   fputc((__C), stdout)

... or some equivalent; I think the best way to say it's that it's a
shorthand.

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt	<amsp@zytor.com>

Re: [OT] fputc vs putc Re: horrible disk thorughput on itanium

[Tom Vier]

On Sun, Dec 09, 2001 at 01:27:51AM -0800, H. Peter Anvin wrote:
> putc() is frequently defined as
> 
> #define putc(__C)   fputc((__C), stdout)
> 
> ... or some equivalent; I think the best way to say it's that it's a
> shorthand.

according to the putc man page in debian stable, it takes the same args as
fputc. maybe it varies by glibc version (mine is 2.1.3-19). i guess anyone
using putc better use autoconf. also, "unix system programming for SVr4"
says the only difference is that putc is an inlined macro version of fputc.

-- 
Tom Vier <tmv5@home.com>
DSA Key id 0x27371A2C

Re: [OT] fputc vs putc Re: horrible disk thorughput on itanium

[H. Peter Anvin]

Followup to:  <20011209141408.A17671@zero>
By author:    Tom Vier <tmv5@home.com>
In newsgroup: linux.dev.kernel
> 
> according to the putc man page in debian stable, it takes the same args as
> fputc. maybe it varies by glibc version (mine is 2.1.3-19). i guess anyone
> using putc better use autoconf. also, "unix system programming for SVr4"
> says the only difference is that putc is an inlined macro version of fputc.
> 

Nevermind.  I was thinking about putc() and putchar(), additionally
confused by the annoying fact that fputs() is to putc() as puts() is
to putchar()... makes no fscking sense.

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt	<amsp@zytor.com>