beta reiserfs data logging patches

beta reiserfs data logging patches

[Chris Mason]

Hello everyone,

The very daring among you can try:

ftp.suse.com/pub/people/mason/patches/data-logging

To help me benchmark and test the new data logging code.  This is all
experimental, and should not be tried on production machines, or
anything other than a pure test box right now.

data logging helps for fsync and O_SYNC heavy applications, like mail
servers, and databases.  Without the patches, using O_SYNC writes with
iozone to make a 10MB file results in ~600K per second.  With the
patches, it goes up to around 900K/s and in data=journal mode it goes to
2300K/s.

In addition to the data logging code, there are a number of speedups for
tiny transactions.  Wayne, you are cc'd because I believe this code is
faster than the first data-logging patch, you might want to run through
the benchmarks again.

The rest of the message details the changes:

01_commit_super-8.diff -- my patch to make write_super not be used by
sync().  This has been floating around for a while and is unchanged.

02_beta-data-logging-3.diff

Adds the data=journal option, logging data blocks whenever it is one. 
This is an extension of the nesting patch used by the reiserfs quota
code.

Gets rid of the static array of struct reiserfs_journal_list.  The
structs are now allocated on demand and stored in a time ordered
per-super list.  The old code was O(N) on the size of the array in a
bunch of places, and made it very difficult to quickly find lists in
need of flushing.

The new lists allow me to quickly find transactions that haven't been
touched by kreiserfsd yet, which makes it easy to send more than one
transaction worth of metadata to disk at once.  For tiny transactions,
this makes a huge performance difference.

Since the lists are time ordered, it is also easier to figure out which
transactions the next transaction might overlap, and which ones are very
old, which cuts down on CPU time spent in do_journal_end.

-chris

Re: beta reiserfs data logging patches

[Dieter Nützel]

Sorry Chris,

but there are "only" the complete files for the SuSE kernel.

02-beta-data-logging-3.diff is missing.

Cheers,
	Dieter
-- 
Dieter Nützel
Graduate Student, Computer Science

University of Hamburg
Department of Computer Science
@home: Dieter.Nuetzel@hamburg.de

Re: beta reiserfs data logging patches

[Chris Mason]

On Tue, 2002-05-28 at 17:37, Dieter Nützel wrote:
> Sorry Chris,
> 
> but there are "only" the complete files for the SuSE kernel.
> 
> 02-beta-data-logging-3.diff is missing.

Thanks Dieter,

I've fixed the ftp site, but you can use 03-beta-data-logging-3.diff
from the suse directory if you don't want to wait for the mirror to
update.

-chris

RE: beta reiserfs data logging patches

[berthiaume_wayne]

	I'll definitely run them through the benchmark. 

	BTW: I have finished the tests and am assembling the data. The
patches caused rather interesting results. ReiserFS now performs on the same
level as Ext3FS in 2.4.19-pre7 when write cache is not used. When using
barrier writes, performance outshined Ext3FS. I'll forward the data.

Wayne.

-----Original Message-----
From: Chris Mason [mailto:mason@suse.com]
Sent: Tuesday, May 28, 2002 4:44 PM
To: reiserfs-list@namesys.com
Cc: berthiaume_wayne@emc.com
Subject: beta reiserfs data logging patches


Hello everyone,

The very daring among you can try:

ftp.suse.com/pub/people/mason/patches/data-logging

To help me benchmark and test the new data logging code.  This is all
experimental, and should not be tried on production machines, or
anything other than a pure test box right now.

data logging helps for fsync and O_SYNC heavy applications, like mail
servers, and databases.  Without the patches, using O_SYNC writes with
iozone to make a 10MB file results in ~600K per second.  With the
patches, it goes up to around 900K/s and in data=journal mode it goes to
2300K/s.

In addition to the data logging code, there are a number of speedups for
tiny transactions.  Wayne, you are cc'd because I believe this code is
faster than the first data-logging patch, you might want to run through
the benchmarks again.

The rest of the message details the changes:

01_commit_super-8.diff -- my patch to make write_super not be used by
sync().  This has been floating around for a while and is unchanged.

02_beta-data-logging-3.diff

Adds the data=journal option, logging data blocks whenever it is one. 
This is an extension of the nesting patch used by the reiserfs quota
code.

Gets rid of the static array of struct reiserfs_journal_list.  The
structs are now allocated on demand and stored in a time ordered
per-super list.  The old code was O(N) on the size of the array in a
bunch of places, and made it very difficult to quickly find lists in
need of flushing.

The new lists allow me to quickly find transactions that haven't been
touched by kreiserfsd yet, which makes it easy to send more than one
transaction worth of metadata to disk at once.  For tiny transactions,
this makes a huge performance difference.

Since the lists are time ordered, it is also easier to figure out which
transactions the next transaction might overlap, and which ones are very
old, which cuts down on CPU time spent in do_journal_end.

-chris

RE: beta reiserfs data logging patches

[Chris Mason]

On Wed, 2002-05-29 at 09:01, berthiaume_wayne@emc.com wrote:
> 	I'll definitely run them through the benchmark. 
> 
> 	BTW: I have finished the tests and am assembling the data. The
> patches caused rather interesting results. ReiserFS now performs on the same
> level as Ext3FS in 2.4.19-pre7 when write cache is not used. When using
> barrier writes, performance outshined Ext3FS. I'll forward the data.

I suspect this means the barrier patch is being slightly unfair to ext3,
it should affect both filesystems equally.  I'll look closer when the
results come in.

-chris

RE: beta reiserfs data logging patches

[berthiaume_wayne]

	What I noticed is mount() would not allow me to disable write
barrier on ext3 - option not supported.

-----Original Message-----
From: Chris Mason [mailto:mason@suse.com]
Sent: Wednesday, May 29, 2002 9:36 AM
To: berthiaume_wayne@emc.com
Cc: reiserfs-list@namesys.com
Subject: RE: beta reiserfs data logging patches


On Wed, 2002-05-29 at 09:01, berthiaume_wayne@emc.com wrote:
> 	I'll definitely run them through the benchmark. 
> 
> 	BTW: I have finished the tests and am assembling the data. The
> patches caused rather interesting results. ReiserFS now performs on the
same
> level as Ext3FS in 2.4.19-pre7 when write cache is not used. When using
> barrier writes, performance outshined Ext3FS. I'll forward the data.

I suspect this means the barrier patch is being slightly unfair to ext3,
it should affect both filesystems equally.  I'll look closer when the
results come in.

-chris