RE: drive/block device write scheduling, buffer flushing?

RE: drive/block device write scheduling, buffer flushing?

[Nathan Black]

That is what I wanted to do...Write directly to the disk. But the
kernel(2.4.1) is caching the io...



-----Original Message-----
From: bert hubert [mailto:ahu@ds9a.nl]
Sent: Wednesday, January 31, 2001 12:51 PM
To: Nathan Black
Cc: linux-kernel@vger.kernel.org
Subject: Re: drive/block device write scheduling, buffer flushing?


On Wed, Jan 31, 2001 at 11:52:25AM -0500, Nathan Black wrote:
> I was wondering if there is a way to make the kernel write to disk faster.

> I need to maintain a 10 MB /sec write rate to a 10K scsi disk in a
computer,
> but it caches and doesn't start writing to disk until I hit about 700 MB.
At
> that point, it pauses(presumably while the kernel is flushing some of the
> buffers) and I will have missed data that I am trying to capture.

try opening with O_SYNC, or call fsync() every once in a while. Otherwise,
this sounds like an application for a raw device, whereby you can write
directly to the disk, with no caching in between.

Regards,

bert

-- 
PowerDNS                     Versatile DNS Services  
Trilab                       The Technology People   
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: drive/block device write scheduling, buffer flushing?

[Jens Axboe]

On Wed, Jan 31 2001, Nathan Black wrote:
> That is what I wanted to do...Write directly to the disk. But the
> kernel(2.4.1) is caching the io...

Bind the block device to a /dev/raw* and do I/O on that

-- 
Jens Axboe

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

how to use and get raw devices

[bert hubert]

On Wed, Jan 31, 2001 at 01:29:05PM -0500, Nathan Black wrote:

> That is what I wanted to do...Write directly to the disk. But the
> kernel(2.4.1) is caching the io...

Is it caching when you do O_SYNC? 

> try opening with O_SYNC, or call fsync() every once in a while. Otherwise,
> this sounds like an application for a raw device, whereby you can write
> directly to the disk, with no caching in between.

A raw device is something different than just /dev/sda1 - a raw device is
created with the 'raw' utility and is assigned /dev/raw or /dev/raw1. This
utility, and some documentation can be found on

     ftp://ftp.linux.org.uk/pub/linux/sct/fs/raw-io/raw-19990728.tar.gz

AFAIK you don't need any patches anymore if you run 2.4 or a recent 2.3.

Regards,

bert hubert

-- 
PowerDNS                     Versatile DNS Services  
Trilab                       The Technology People   
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

drive/block device write scheduling, buffer flushing?

[Nathan Black]

I was wondering if there is a way to make the kernel write to disk faster. 
I need to maintain a 10 MB /sec write rate to a 10K scsi disk in a computer,
but it caches and doesn't start writing to disk until I hit about 700 MB. At
that point, it pauses(presumably while the kernel is flushing some of the
buffers) and I will have missed data that I am trying to capture.

Any ideas?

Nathan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: drive/block device write scheduling, buffer flushing?

[bert hubert]

On Wed, Jan 31, 2001 at 11:52:25AM -0500, Nathan Black wrote:
> I was wondering if there is a way to make the kernel write to disk faster. 
> I need to maintain a 10 MB /sec write rate to a 10K scsi disk in a computer,
> but it caches and doesn't start writing to disk until I hit about 700 MB. At
> that point, it pauses(presumably while the kernel is flushing some of the
> buffers) and I will have missed data that I am trying to capture.

try opening with O_SYNC, or call fsync() every once in a while. Otherwise,
this sounds like an application for a raw device, whereby you can write
directly to the disk, with no caching in between.

Regards,

bert

-- 
PowerDNS                     Versatile DNS Services  
Trilab                       The Technology People   
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: drive/block device write scheduling, buffer flushing?

[Rogier Wolff]

bert hubert wrote:
> On Wed, Jan 31, 2001 at 11:52:25AM -0500, Nathan Black wrote:
> > I was wondering if there is a way to make the kernel write to disk faster. 
> > I need to maintain a 10 MB /sec write rate to a 10K scsi disk in a computer,
> > but it caches and doesn't start writing to disk until I hit about 700 MB. At
> > that point, it pauses(presumably while the kernel is flushing some of the
> > buffers) and I will have missed data that I am trying to capture.
> 
> try opening with O_SYNC, or call fsync() every once in a while. Otherwise,
> this sounds like an application for a raw device, whereby you can write
> directly to the disk, with no caching in between.

But you'll probably need to use "buffer" too. 

   capture | buffer -m 128m > outputfile

Otherwise the "fsync" can block you for say 1/10th of a second,
causing loss of a few frames. (10MB per seconds sure sounds like
video).


Roger. 


-- 
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2137555 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* There are old pilots, and there are bold pilots. 
* There are also old, bald pilots. 
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/