need better I/O scheduler for bulk file serving

[Felix von Leitner <felix-kernel@fefe.de>]

I'm doing some network scalability testing on Linux.  I'm using sendfile
and epoll on non-blocking file descriptors, and I am uploading several
large files (CD images in my case) to several downloaders.

When only one peer is downloading, I get wire speed bandwidth
utilization.

When three people are downloading different images (my server "only" has
512 MB RAM, so it can't hold even one of the images in memory) at the
same time, the bandwidth utilization goes down to 6 MB/sec on my fast
ethernet NIC.  The hard disk appears to be busy seeking.

What knobs exist to tweak this?  This appears to be exactly the kind of
problem the I/O schedulers are trying to solve, right?  And the kernel
has perfect knowledge of what is going to happen because sendfile is
called with file size as length.  Larger read-ahead maybe?

Felix