need better I/O scheduler for bulk file serving

need better I/O scheduler for bulk file serving

[Felix von Leitner]

I'm doing some network scalability testing on Linux.  I'm using sendfile
and epoll on non-blocking file descriptors, and I am uploading several
large files (CD images in my case) to several downloaders.

When only one peer is downloading, I get wire speed bandwidth
utilization.

When three people are downloading different images (my server "only" has
512 MB RAM, so it can't hold even one of the images in memory) at the
same time, the bandwidth utilization goes down to 6 MB/sec on my fast
ethernet NIC.  The hard disk appears to be busy seeking.

What knobs exist to tweak this?  This appears to be exactly the kind of
problem the I/O schedulers are trying to solve, right?  And the kernel
has perfect knowledge of what is going to happen because sendfile is
called with file size as length.  Larger read-ahead maybe?

Felix

Re: need better I/O scheduler for bulk file serving

[Andrew Morton]

Felix von Leitner <felix-kernel@fefe.de> wrote:
>
> Larger read-ahead maybe?

Or an anticipatory scheduler.  You don't say what kernel you're
using.

We can set the readahead per-fd now, but the fcntl/ioctl hasn't
been implemented.

Re: need better I/O scheduler for bulk file serving

[Andrea Arcangeli]

On Sun, May 18, 2003 at 02:06:42PM -0700, Andrew Morton wrote:
> Felix von Leitner <felix-kernel@fefe.de> wrote:
> >
> > Larger read-ahead maybe?
> 
> Or an anticipatory scheduler.  You don't say what kernel you're
> using.

yes, IMHO he could give a spin to my tree with better readahead and see
if it helps.

Andrea

Re: need better I/O scheduler for bulk file serving

[Chuck Ebbert]

Felix von Leitner wrote:

> When three people are downloading different images (my server "only" has
> 512 MB RAM, so it can't hold even one of the images in memory) at the
> same time, the bandwidth utilization goes down to 6 MB/sec on my fast
> ethernet NIC.  The hard disk appears to be busy seeking.

  Try RAID1.

Re: need better I/O scheduler for bulk file serving

[Joel Jaeggli]

it sounds like it's just slow disk.  our redhat mirror held steady at 
about 89Mb/s during the rh9 release. using 2.4.18 384MB of ram and a 
4x120GB sw raid5 stripe with 64k block size.

joelja

On Sun, 18 May 2003, Chuck Ebbert wrote:

> Felix von Leitner wrote:
> 
> > When three people are downloading different images (my server "only" has
> > 512 MB RAM, so it can't hold even one of the images in memory) at the
> > same time, the bandwidth utilization goes down to 6 MB/sec on my fast
> > ethernet NIC.  The hard disk appears to be busy seeking.
> 
>   Try RAID1.
> 
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

-- 
-------------------------------------------------------------------------- 
Joel Jaeggli	      Academic User Services   joelja@darkwing.uoregon.edu    
--    PGP Key Fingerprint: 1DE9 8FCA 51FB 4195 B42A 9C32 A30D 121E      --
  In Dr. Johnson's famous dictionary patriotism is defined as the last
  resort of the scoundrel.  With all due respect to an enlightened but
  inferior lexicographer I beg to submit that it is the first.
	   	            -- Ambrose Bierce, "The Devil's Dictionary"