Re: Linux syscall speed -- was X15 rootin-tootin webserver

Re: Linux syscall speed -- was X15 rootin-tootin webserver

[Michael Rothwell]

There seems to be a contingent of people on the LKML who think that it
is appropriate to flame people off-list, in order to bask in their own
superiority, or prove that they are smarter by pointing out that someone
is an idiot, etc. I would figure that most intelligent people would
simply ignore posts they don't like, rather than investing time and
bandwidth compounding the perceived offense. But I'm apparently too
optimistic on that point; any group of people the size of the LKML will
always contain some juviniles. A great many of us have suffered their
attention.

-M

On 04 May 2001 11:21:48 +1000, Rusty Russell wrote:
> In message <988856961.6355.1.camel@gromit> you write:
> > According to tests performed at IBM:
> > 
> > http://www-106.ibm.com/developerworks/linux/library/l-rt1/
> > 
> > Linux's sycalls are a little more than twice as fast as those of Windows
> 
> This post was pretty much a waste of space, wasn't it?
> 
> > 2000. 0.75usec vs 2.0msec.
> 
> That would be 2,666 times.
> 
> Rusty.
> --
> Premature optmztion is rt of all evl. --DK

Re: Linux syscall speed -- was X15 rootin-tootin webserver

[Dan Mann]

I followed the link and read the article.  I am glad you sent the link.  I
am glad to see the Linux kernel doing so well.  Does anyone else have any
Linux Kernel benchmark related links that are interesting?

Thank You,

Dan

----- Original Message -----
From: "Michael Rothwell" <rothwell@holly-springs.nc.us>
To: <linux-kernel@vger.kernel.org>
Sent: Friday, May 04, 2001 9:38 AM
Subject: Re: Linux syscall speed -- was X15 rootin-tootin webserver


> There seems to be a contingent of people on the LKML who think that it
> is appropriate to flame people off-list, in order to bask in their own
> superiority, or prove that they are smarter by pointing out that someone
> is an idiot, etc. I would figure that most intelligent people would
> simply ignore posts they don't like, rather than investing time and
> bandwidth compounding the perceived offense. But I'm apparently too
> optimistic on that point; any group of people the size of the LKML will
> always contain some juviniles. A great many of us have suffered their
> attention.
>
> -M
>
> On 04 May 2001 11:21:48 +1000, Rusty Russell wrote:
> > In message <988856961.6355.1.camel@gromit> you write:
> > > According to tests performed at IBM:
> > >
> > > http://www-106.ibm.com/developerworks/linux/library/l-rt1/
> > >
> > > Linux's sycalls are a little more than twice as fast as those of
Windows
> >
> > This post was pretty much a waste of space, wasn't it?
> >
> > > 2000. 0.75usec vs 2.0msec.
> >
> > That would be 2,666 times.
> >
> > Rusty.
> > --
> > Premature optmztion is rt of all evl. --DK
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: X15 alpha release: as fast as TUX but in user space

[Fabio Riccardi]

Linux 2.4 is surely one of the most advanced OSs ever happened, especially
from the optimization point of view and for the admirable economy of concepts
on which it lies. I definitively hope that X15 helps reinforcing the success
to this amazing system.

TUX has definitively been my performance yardstick for the development of
X15, but I had many sources of inspiration for the X15 architecture. Maybe
the most relevant are the Flash Web Server (Pai, Druschel, Zwaenepoel),
several Linus observations on this list about (web) server architecture and
kernnel services, and the reading of the Hennessy & Patterson architecture
books. Last but not least, aside from some heated discussions, research in
microkernel architecture has taught us many lessons on how to achieve an
efficient model of interaction across separate addressing spaces.

If i have to make some sort of educated guess and point at where the current
bottleneck lies for web server performance, I would say that it is somewhere
between the memory subsystem and the PCI bus.

With zero-copy sendfile data movement is not an issue anymore, asynchronous
network IO allows for really inexpensive thread scheduling, and system call
invocation adds a very negligible overhead in Linux. What we are left with
now is purely wait cycles, the CPUs and the NICs are contending for memory
and bus bandwidth. It would be really interesting to see where the network
shifts now that faster machines are becoming available.

On my whish list for future kernel developments I would definitively put disk
asynchronous IO and a more decent file descriptor passing implementation.
I'll detail this in subsequent messages.

I'll surely check out the impact of Ingo's patches on TUX performance
sometime this week.

I'd also like to reiterate my request for help for testing X15 on higher end
server architectures.

X15 is still very young alpha code and I can surely improve its performance
in many ways.

 - Fabio

Ingo Molnar wrote:

> On Fri, 27 Apr 2001, Fabio Riccardi wrote:
>
> > I'd like to announce the first release of X15 Alpha 1, a _user space_
> > web server that is as fast as TUX.
>
> great, the first TUX clone! ;-)
>
> This should put the accusations to rest that Linux got the outstandingly
> high SPECweb99 scores only because the webserver was in kernel-space. It's
> the 2.4 kernel's high performance that enabled those results, having the
> web-server in kernel-space didnt have much effect. TUX was and remains a
> testbed to test high-performance webserving (and FTP serving), without the
> API-exporting overhead of userspace.
>
> [i suspect the small performance advantage of X15 is due to subtle
> differences in the SPECweb99 user-space module: eg. while the TUX code was
> written, tested and ready to use mmap()-enabled
> TUXAPI_alloc_read_objectbuf(), it wasnt enabled actually. I sent Fabio a
> mail how to enable it, perhaps he can do some tests to confirm this
> suspicion?]
>
> doing a TUX 2.0 SPECweb99 benchmark on the latest -ac kernels, 86% of time
> is spent in generic parts of the kernel, 12% of time is spent in the
> user-space SPECweb99 module, and only 2% of time is spent in TUX-specific
> kernel code.
>
> doing the same test with the original TUX 1.0 code shows that more than
> 50% of CPU time was spent in TUX-specific code.
>
> what does this mean? In the roughly 6 months since TUX 1.0 was released,
> we moved much of the TUX 1.0 -only improvements into the generic kernel
> (most of which was made available to user-space as well), and TUX itself
> became smaller and smaller (and used more and more generic parts of the
> kernel). So in effect X15 is executing 50% TUX code :-)
>
> (there are still a number of performance improvement patches pending that
> are not integrated yet: the pagecache extreme-scalability patch and the
> smptimers patch. These patches speed both X15 and TUX up.)
>
> (there is one thing though that can never be 'exported to user-space': to
> isolate possibly untrusted binary application code from the server itself,
> without performance degradation. So we always have to be mentally open to
> the validity of kernel-space services.)
>
>         Ingo

Re: X15 alpha release: as fast as TUX but in user space

[Lincoln Dale]

Hi,

At 10:50 AM 2/05/2001 +0200, Ingo Molnar wrote:
>i think Zach's phhttpd is an important milestone as well, it's the first
>userspace webserver that shows how to use event-based, sigio-based async
>networking IO and sendfile() under Linux. (I believe it had some
>performance problems related to sigio queue overflow, these issues might
>be solved in the latest kernels.) The zerocopy enhancements should help
>phhttpd as well.

my experience with sigio-based event-handlers is that the net-gain of 
event-driven i/o is mitigated by the fact that SIGIO is based on signals.

the problem with signals for this purpose are:
  (a) you go thru a syncronization point in the kernel.  signals are protected
      by a spinlock.
      it doesn't scale with SMP.
  (b) SI_PAD_SIZE

explicitly, (b) means that you have an awful lot of memory-accesses going 
on for every signal.
my experience with the overhead is that it mitigates the advantages when 
you become bottlenecked on memory-bus-accesses.


cheers,

lincoln.

Linux syscall speed -- was X15 rootin-tootin webserver

[Michael Rothwell]

According to tests performed at IBM:

http://www-106.ibm.com/developerworks/linux/library/l-rt1/

Linux's sycalls are a little more than twice as fast as those of Windows
2000. 0.75usec vs 2.0msec. Not too shabby. And this "magic page" idea
means it may get faster.

-M