Re: Writeback cache, SCSI, and ReiserFS

[Chris Mason <mason@suse.com>]

On Wed, 2002-10-16 at 01:33, JP Howard wrote:
> Here's a little program to test fsync() performance--if you run it with
> the parameter '1' it does 10000 write/fsync combinations, with the
> parameter '0' it does 10000 writes, followed by a single fsync:
> ----
> int main(int argc, char **argv) {
>   int i, fd, mode;
>   mode = atoi(argv[1]);
>   fd = open("/some/path", O_CREAT | O_RDWR);
>   for (i=0; i<10000; i++) {
>     if (i%10 == 0)
>       lseek(fd, 0, SEEK_SET);
>     write(fd, buf, 8192);
>     if (mode == 1) {
>       fsync(fd);
>     }
>   }
>   fsync(fd);
> }
> ----
> 
> Now, according to our RAID controller (MegaRAID Series 475), we have
> writeback cache turned on. Therefore, I would have thought that the above
> program would run at the same speed regardless of using parameter '1' vs
> '0', since fsync should return immediately when using a writeback cache.

This is one of the things the data logging patches were optimized for. 
If you fsync after every write, the average transaction size is going to
be about 4 or 5 blocks.  If you fsync at the every end, the average
transaction size will be a few hundred blocks at least.

The data logging patches make small transactions much faster (even
without data=journal), and with data=journal the difference is huge.

Not 30x faster though.  You see that difference because of the lseek
back to zero, which means you are overwriting the same blocks over and
over again.  With fsync on, you wait for each write to hit the
controller's cache.  With fsync at the end, you wait for each write to
hit the page cache, and then wait for 8192 * 10 bytes to hit the
controller's cache at the end.

Even if the controller is writeback caching, fsync=1 sends many more
commands down the pipe, and each one has to be processed.

So, if you really want to test your writeback cache on the controller,
run your test with fsync = 1 with the writeback on, the run with fsync =
1 with the writeback off.

If you get the same numbers both ways, you don't have writeback on.

-chris