* drive/block device write scheduling, buffer flushing?
@ 2001-01-31 16:52 Nathan Black
2001-01-31 17:51 ` bert hubert
0 siblings, 1 reply; 6+ messages in thread
From: Nathan Black @ 2001-01-31 16:52 UTC (permalink / raw)
To: linux-kernel
I was wondering if there is a way to make the kernel write to disk faster.
I need to maintain a 10 MB /sec write rate to a 10K scsi disk in a computer,
but it caches and doesn't start writing to disk until I hit about 700 MB. At
that point, it pauses(presumably while the kernel is flushing some of the
buffers) and I will have missed data that I am trying to capture.
Any ideas?
Nathan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: drive/block device write scheduling, buffer flushing?
2001-01-31 16:52 drive/block device write scheduling, buffer flushing? Nathan Black
@ 2001-01-31 17:51 ` bert hubert
2001-02-01 8:56 ` Rogier Wolff
0 siblings, 1 reply; 6+ messages in thread
From: bert hubert @ 2001-01-31 17:51 UTC (permalink / raw)
To: Nathan Black; +Cc: linux-kernel
On Wed, Jan 31, 2001 at 11:52:25AM -0500, Nathan Black wrote:
> I was wondering if there is a way to make the kernel write to disk faster.
> I need to maintain a 10 MB /sec write rate to a 10K scsi disk in a computer,
> but it caches and doesn't start writing to disk until I hit about 700 MB. At
> that point, it pauses(presumably while the kernel is flushing some of the
> buffers) and I will have missed data that I am trying to capture.
try opening with O_SYNC, or call fsync() every once in a while. Otherwise,
this sounds like an application for a raw device, whereby you can write
directly to the disk, with no caching in between.
Regards,
bert
--
PowerDNS Versatile DNS Services
Trilab The Technology People
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 6+ messages in thread
* RE: drive/block device write scheduling, buffer flushing?
@ 2001-01-31 18:29 Nathan Black
2001-01-31 18:41 ` Jens Axboe
2001-01-31 18:48 ` how to use and get raw devices bert hubert
0 siblings, 2 replies; 6+ messages in thread
From: Nathan Black @ 2001-01-31 18:29 UTC (permalink / raw)
To: 'bert hubert'; +Cc: linux-kernel
That is what I wanted to do...Write directly to the disk. But the
kernel(2.4.1) is caching the io...
-----Original Message-----
From: bert hubert [mailto:ahu@ds9a.nl]
Sent: Wednesday, January 31, 2001 12:51 PM
To: Nathan Black
Cc: linux-kernel@vger.kernel.org
Subject: Re: drive/block device write scheduling, buffer flushing?
On Wed, Jan 31, 2001 at 11:52:25AM -0500, Nathan Black wrote:
> I was wondering if there is a way to make the kernel write to disk faster.
> I need to maintain a 10 MB /sec write rate to a 10K scsi disk in a
computer,
> but it caches and doesn't start writing to disk until I hit about 700 MB.
At
> that point, it pauses(presumably while the kernel is flushing some of the
> buffers) and I will have missed data that I am trying to capture.
try opening with O_SYNC, or call fsync() every once in a while. Otherwise,
this sounds like an application for a raw device, whereby you can write
directly to the disk, with no caching in between.
Regards,
bert
--
PowerDNS Versatile DNS Services
Trilab The Technology People
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: drive/block device write scheduling, buffer flushing?
2001-01-31 18:29 drive/block device write scheduling, buffer flushing? Nathan Black
@ 2001-01-31 18:41 ` Jens Axboe
2001-01-31 18:48 ` how to use and get raw devices bert hubert
1 sibling, 0 replies; 6+ messages in thread
From: Jens Axboe @ 2001-01-31 18:41 UTC (permalink / raw)
To: Nathan Black; +Cc: 'bert hubert', linux-kernel
On Wed, Jan 31 2001, Nathan Black wrote:
> That is what I wanted to do...Write directly to the disk. But the
> kernel(2.4.1) is caching the io...
Bind the block device to a /dev/raw* and do I/O on that
--
Jens Axboe
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 6+ messages in thread
* how to use and get raw devices
2001-01-31 18:29 drive/block device write scheduling, buffer flushing? Nathan Black
2001-01-31 18:41 ` Jens Axboe
@ 2001-01-31 18:48 ` bert hubert
1 sibling, 0 replies; 6+ messages in thread
From: bert hubert @ 2001-01-31 18:48 UTC (permalink / raw)
To: Nathan Black; +Cc: linux-kernel
On Wed, Jan 31, 2001 at 01:29:05PM -0500, Nathan Black wrote:
> That is what I wanted to do...Write directly to the disk. But the
> kernel(2.4.1) is caching the io...
Is it caching when you do O_SYNC?
> try opening with O_SYNC, or call fsync() every once in a while. Otherwise,
> this sounds like an application for a raw device, whereby you can write
> directly to the disk, with no caching in between.
A raw device is something different than just /dev/sda1 - a raw device is
created with the 'raw' utility and is assigned /dev/raw or /dev/raw1. This
utility, and some documentation can be found on
ftp://ftp.linux.org.uk/pub/linux/sct/fs/raw-io/raw-19990728.tar.gz
AFAIK you don't need any patches anymore if you run 2.4 or a recent 2.3.
Regards,
bert hubert
--
PowerDNS Versatile DNS Services
Trilab The Technology People
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: drive/block device write scheduling, buffer flushing?
2001-01-31 17:51 ` bert hubert
@ 2001-02-01 8:56 ` Rogier Wolff
0 siblings, 0 replies; 6+ messages in thread
From: Rogier Wolff @ 2001-02-01 8:56 UTC (permalink / raw)
To: bert hubert; +Cc: Nathan Black, linux-kernel
bert hubert wrote:
> On Wed, Jan 31, 2001 at 11:52:25AM -0500, Nathan Black wrote:
> > I was wondering if there is a way to make the kernel write to disk faster.
> > I need to maintain a 10 MB /sec write rate to a 10K scsi disk in a computer,
> > but it caches and doesn't start writing to disk until I hit about 700 MB. At
> > that point, it pauses(presumably while the kernel is flushing some of the
> > buffers) and I will have missed data that I am trying to capture.
>
> try opening with O_SYNC, or call fsync() every once in a while. Otherwise,
> this sounds like an application for a raw device, whereby you can write
> directly to the disk, with no caching in between.
But you'll probably need to use "buffer" too.
capture | buffer -m 128m > outputfile
Otherwise the "fsync" can block you for say 1/10th of a second,
causing loss of a few frames. (10MB per seconds sure sounds like
video).
Roger.
--
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2137555 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* There are old pilots, and there are bold pilots.
* There are also old, bald pilots.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2001-02-01 8:56 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2001-01-31 18:29 drive/block device write scheduling, buffer flushing? Nathan Black
2001-01-31 18:41 ` Jens Axboe
2001-01-31 18:48 ` how to use and get raw devices bert hubert
-- strict thread matches above, loose matches on Subject: below --
2001-01-31 16:52 drive/block device write scheduling, buffer flushing? Nathan Black
2001-01-31 17:51 ` bert hubert
2001-02-01 8:56 ` Rogier Wolff
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox