Re: [PATCH CFT] tons of logging patches

Re: [PATCH CFT] tons of logging patches

[Dieter Nützel]

On Friday, Chris wrote:

> ftp.suse.com/pub/people/mason/patches/data-logging has a new
> beta-data-logging-10.diff.gz
>
> This one:
>
> compiles as a module (thanks Jeff Mahoney)
>
> has a highmem bug fixed with data=journal (help from Andrea)

Hello Chris,

should I run this with -aa kernel?

> Fixes a problem with relocated/different sized journal devices.  This
> was a merge error on my part.
>
> One tester reports a hang that I can't reproduce here, I'm still trying.

You do not mean me (page coloring)?

Regards,
	Dieter

BTW I have my dual Athlon MP 1900+ up and "flying"...
-- 
Dieter Nützel
Graduate Student, Computer Science

University of Hamburg
Department of Computer Science
@home: Dieter.Nuetzel@hamburg.de

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Sat, 2002-06-22 at 16:02, Dieter Nützel wrote:
> On Friday, Chris wrote:
> 
> > ftp.suse.com/pub/people/mason/patches/data-logging has a new
> > beta-data-logging-10.diff.gz
> >
> > This one:
> >
> > compiles as a module (thanks Jeff Mahoney)
> >
> > has a highmem bug fixed with data=journal (help from Andrea)
> 
> Hello Chris,
> 
> should I run this with -aa kernel?

You can, the differences are minor, just interdiff from the last data
logging patch and apply the incremental onto the aa data logging patch. 
I ran out of time before leaving for ottawa, so I didn't get the chance
to make an aa patch.

> 
> > Fixes a problem with relocated/different sized journal devices.  This
> > was a merge error on my part.
> >
> > One tester reports a hang that I can't reproduce here, I'm still trying.
> 
> You do not mean me (page coloring)?

No, this was a large 8 way smp machine.  

-chris

Re: [PATCH CFT] tons of logging patches

[Dieter Nützel]

On Monday, 8 July 2002 13:51, Chris Mason wrote:
> On Wed, 2002-07-03 at 20:28, Manuel Krause wrote:
>
> > So far the 13 prerelease works fine under rootflags boot and 
> > mount/remount option "data=ordered" on here.
>
> I think I've got the bug, it was newly introduced in -13.  New stuff
> will be uploaded today.

Where is your stuff?
It didn't appeared in your SuSE "home"
I'm during the preparation of 2.4.19-rc1-jam2-pc (2.4.19pre1aa2 + jam2 + page 
coloring).

Thanks,
	Dieter

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Tue, 2002-07-09 at 09:52, Dieter Nützel wrote:
> On Monday, 8 July 2002 13:51, Chris Mason wrote:
> > On Wed, 2002-07-03 at 20:28, Manuel Krause wrote:
> >
> > > So far the 13 prerelease works fine under rootflags boot and 
> > > mount/remount option "data=ordered" on here.
> >
> > I think I've got the bug, it was newly introduced in -13.  New stuff
> > will be uploaded today.
> 
> Where is your stuff?
> It didn't appeared in your SuSE "home"
> I'm during the preparation of 2.4.19-rc1-jam2-pc (2.4.19pre1aa2 + jam2 + page 
> coloring).

Sigh, further testing showed I'd only fixed half of the bug.  The other
half is somewhere deep writepage/truncate interaction.  I'm still trying
to find it.

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Tue, 2002-07-09 at 10:00, Chris Mason wrote:
> On Tue, 2002-07-09 at 09:52, Dieter Nützel wrote:
> > On Monday, 8 July 2002 13:51, Chris Mason wrote:
> > > On Wed, 2002-07-03 at 20:28, Manuel Krause wrote:
> > >
> > > > So far the 13 prerelease works fine under rootflags boot and 
> > > > mount/remount option "data=ordered" on here.
> > >
> > > I think I've got the bug, it was newly introduced in -13.  New stuff
> > > will be uploaded today.
> > >
> Sigh, further testing showed I'd only fixed half of the bug.  The other
> half is somewhere deep writepage/truncate interaction.  I'm still trying
> to find it.

Well, after a week of fixing minor things and adding debugging code,
this bug still goes unsolved.  The only good news is that I can also
trigger it on pure 2.4.19-rc1 and ext2.  The bug seems to either be in
the VM or my ram, so I'm having a little memtest86 party overnight.

I've uploaded 03-beta-data-logging-16, it should be more stable and
faster than -11 and -13 were, but includes more debugging checks to try
and make sure I'm not screwing things up.  You can grab the patches at:

ftp://ftp.suse.com/pub/people/mason/patches/data-logging

If anyone wants to try and reproduce my memory corruption bug (can and
probably will lead to FS corruption), just let 10 fsx processes run in a
loop for ~3 hours or so on an SMP machine.

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 07/12/2002 09:43 PM, Chris Mason wrote:
> On Tue, 2002-07-09 at 10:00, Chris Mason wrote:
> 
>>On Tue, 2002-07-09 at 09:52, Dieter Nützel wrote:
>>
>>>On Monday, 8 July 2002 13:51, Chris Mason wrote:
>>>
>>>>On Wed, 2002-07-03 at 20:28, Manuel Krause wrote:
>>>>
>>>>
>>>>>So far the 13 prerelease works fine under rootflags boot and 
>>>>>mount/remount option "data=ordered" on here.
>>>>
>>>>I think I've got the bug, it was newly introduced in -13.  New stuff
>>>>will be uploaded today.
>>>>
>>>
>>Sigh, further testing showed I'd only fixed half of the bug.  The other
>>half is somewhere deep writepage/truncate interaction.  I'm still trying
>>to find it.
> 
> 
> Well, after a week of fixing minor things and adding debugging code,
> this bug still goes unsolved.  The only good news is that I can also
> trigger it on pure 2.4.19-rc1 and ext2.  The bug seems to either be in
> the VM or my ram, so I'm having a little memtest86 party overnight.
> 
> I've uploaded 03-beta-data-logging-16, it should be more stable and
> faster than -11 and -13 were, but includes more debugging checks to try
> and make sure I'm not screwing things up.  You can grab the patches at:
> 
> ftp://ftp.suse.com/pub/people/mason/patches/data-logging
> 

Thank you, Chris!

It's working fine under rootflags set and mount/remount options 
"noatime,notail,data=ordered".
Nothing got wrong on here due to data-logging-13.

Netscape7 dropped its load time again by 3secs of (previously 14+1), 
OpenOffice1 increased by 1sec (of average 17secs previously). To explain 
the previous state: I've only changed -pre9 to -rc1 and data-logging 
from -10(-pre9)..13(-rc1)..11(-rc1)..16(-rc1, now!) from the old patchset.

> If anyone wants to try and reproduce my memory corruption bug (can and
> probably will lead to FS corruption), just let 10 fsx processes run in a
> loop for ~3 hours or so on an SMP machine.
> 
> -chris
> 

Is this possible to be reproduced on uniprocessor machines, too ?


Best regards,

Manuel

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

Hello everyone,

I've uploaded 03-data-logging-24.diff, which fixes a tail packing oops
in data=journal mode, and fixes the data=ordered slow down (5-10%)
introduced in -23.  

-23 flushed all the ordered buffer heads with the journal lock held,
which meant no new transactions could start while the ordered buffers
were being flushed.  This was safe but slower.

-24 flushes them right before the commit blocks (like -19 did), and only
flushes the tail conversion targets with the journal lock held.  Doing
the tail conversion targets later risks losing the tail data after a
crash.

Thanks to grev@namesys.com who reported the ordered write bug in -19 and
the data=journal oops in -23.

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Fri, 2002-08-02 at 08:28, Chris Mason wrote:
> Hello everyone,
> 
> I've uploaded 03-data-logging-24.diff, which fixes a tail packing oops
> in data=journal mode, and fixes the data=ordered slow down (5-10%)
> introduced in -23.  

Whoops, I neglected to mention that data=ordered is now the default in
the patch.  You can mount with -o data=writeback if you don't want
ordered data writes.

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Hans Reiser]

Chris Mason wrote:

>On Fri, 2002-08-02 at 08:28, Chris Mason wrote:
>  
>
>>Hello everyone,
>>
>>I've uploaded 03-data-logging-24.diff, which fixes a tail packing oops
>>in data=journal mode, and fixes the data=ordered slow down (5-10%)
>>introduced in -23.  
>>    
>>
>
>Whoops, I neglected to mention that data=ordered is now the default in
>the patch.  You can mount with -o data=writeback if you don't want
>ordered data writes.
>
>-chris
>
>
>
>
>  
>
we just saw a benchmark in which data=ordered caused a major slowdown. 
 Does this patch fix this?

Edward and Elena, I want thorough benchmarking of this.

-- 
Hans

Re: Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Fri, 2002-08-02 at 08:59, Hans Reiser wrote:
> >
> we just saw a benchmark in which data=ordered caused a major slowdown. 
>  Does this patch fix this?
> 
> Edward and Elena, I want thorough benchmarking of this.
> 

The last fract_tree benchmarks were on older patches, Manual Krause's
benchmark was more showing the difference between a used and a fresh
filesystem, and he was using -23 which had data=ordered performance
issues fixed in -24.

More benchmarking does need to be done, the patch isn't set in stone yet
;-)

-chris

RE: [PATCH CFT] tons of logging patches

[berthiaume_wayne]

	Chris, isn't there a danger, remotely, that with data=ordered you
could lose your data? For Ext3FS, if the machine should crash after the
filesystem data is updated but before the metadata is updated, fsck(8) will
reclaim the newly allocated blocks; therefore, the updated file is no longer
correct.
Wayne.

-----Original Message-----
From: Chris Mason [mailto:mason@suse.com]
Sent: Tuesday, June 04, 2002 9:12 AM
To: Manuel Krause
Cc: reiserfs-list
Subject: Re: [reiserfs-list] [PATCH CFT] tons of logging patches


On Mon, 2002-06-03 at 23:28, Manuel Krause wrote:

> 
> So, VMware is stable with it, too, on my well known "heavy-private-test" 
> of it (running Norton SpeedDisk at least twice within a most recent 
> VMware Win98). It doesn't show greatly different timings than to my 
> setup before though having a different disk i/o pattern (due to the 
> missing aa patches)... and me having a reduced RAM from 512to256MB at 
> the moment. And I should be honest to say I can't give exact timings as 
> the important disk contents changed during last weeks. But the 
> disk-access-times/related-to-the-content are definitively _not_ higher 
> than before!

same speed on 1/2 the ram isn't bad ;-)

> 
> > 
> > Great to hear, thanks for trying things out.
> > 
> > data=logging will be the slowest mode for everything except mail servers
> > and write heavy databases (or other apps that hammer on O_SYNC/fsync). 
> > This is because all the data gets written twice, once to log and once to
> > the main disk.  It helps synchronous writes by writing to the log in
> > quick sequential bursts, and then writing back to the main disk in
> > larger chunks.
> 
> Yes, I assumed/learnt that from your very previous explanations. But I 
> don't see the point "slowest", so far, on here.

try dbench, or any streaming write.  It should be roughtly 1/2 as fast
as the default mode.  If it isn't data=journal probably isn't really
activated (it sends a printk on mount about using journaled data mode).

> 
> This is a home/single session ?non?-production system on here. 1 to 2 
> IDE disks, one processor... notebook, you'd remember this setup from my 
>   mails.
> 
> > 
> > data=ordered will usually be fastest for streaming appends to a file. 
> > The rest of the time it will be the same or slightly slower than the
> > default (data=writeback).
> 
> What would be my recommended mount option later then (mentionned in the 
> manpages or on the webpage?!) Mmmh: I see I test that at the moment...

data=ordered will work best for most people.  The performance hit is
very small, and you won't have any more garbage in files after a crash.

> 
> Why is no combo possible? Meant as: Could/would that bring advantages in 
> future?!!

Since there are no userspace transactions, data=journal is only useful
in a very limited set of cases.  The fact that everything is logged
allows me to cheat, and skip some of the steps to keep block allocations
and deallocations safe in the writeback data mode.

intermezzo is the only app that really needs data=journal with the rest
of the FS mounted with some other write mode (for speed).  The patch has
a way for them to enable data logging mode on a single file.

I think I'll forward the patch announcement to the intermezzo list...

-chris

RE: [PATCH CFT] tons of logging patches

[Chris Mason]

On Tue, 2002-06-04 at 11:48, berthiaume_wayne@emc.com wrote:
> 	Chris, isn't there a danger, remotely, that with data=ordered you
> could lose your data? For Ext3FS, if the machine should crash after the
> filesystem data is updated but before the metadata is updated, fsck(8) will
> reclaim the newly allocated blocks; therefore, the updated file is no longer
> correct.

Yes, reiserfs does the same.  data=ordered only promises that before the
updated metadata is on disk, the data blocks they reference will be on
disk too.  This way you know that if you add new bytes to a file, after
a crash you will either have valid data in the file, or no new bytes
added at all.

The only way you can be sure the actual metadata is there is by waiting
for an fsync(file) to finish.

-chris

[PATCH CFT] tons of logging patches

[Chris Mason]

Hello everyone,

ftp.suse.com/pub/people/mason/data-logging

has beta code that gives you the following:

external logging devices
data logging mount option (makes fsync and O_SYNC faster)
ordered data mount option (no more garbage in your files after a crash)

optimizations to keep kupdate from triggering commits every 5 seconds
optimizations for tiny transactions to reduce CPU usage and IO.
optimizations for better locking and fewer calls to wakeup()

In short, it adds a bunch of features and makes everything faster.  The
changes are very large, but they also build on older patches that have
been circulating around for months.  It has been through a weekend of
load testing here, but I still won't be surprised if people find
corruption causing bugs.

It needs broad testing and benchmarking, in all the data modes, with a
standard and external journal.

-chris

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Sun, 2002-06-02 at 23:51, Chris Mason wrote:
> Hello everyone,
> 
> ftp.suse.com/pub/people/mason/data-logging
> 
> has beta code that gives you the following:
> 
> external logging devices

Whoops, I should have noted the external logging patch is from namesys,
my changes were just merging things around slightly.

-chris

Re: [PATCH CFT] tons of logging patches

[Matthias Andree]

Chris Mason <mason@suse.com> writes:

> Hello everyone,
>
> ftp.suse.com/pub/people/mason/data-logging

You seem to mean:

ftp://ftp.suse.com/pub/people/mason/patches/data-logging/

> optimizations to keep kupdate from triggering commits every 5 seconds

Does this mean the current released ReiserFS versions are causing the
persistent write activity at 5 s interval that I observe?

Or is it ext3fs also?

-- 
Matthias Andree

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Mon, 2002-06-03 at 04:46, Matthias Andree wrote:
> Chris Mason <mason@suse.com> writes:
> 
> > Hello everyone,
> >
> > ftp.suse.com/pub/people/mason/data-logging
> 
> You seem to mean:
> 
> ftp://ftp.suse.com/pub/people/mason/patches/data-logging/

Correct, thanks.

> 
> > optimizations to keep kupdate from triggering commits every 5 seconds
> 
> Does this mean the current released ReiserFS versions are causing the
> persistent write activity at 5 s interval that I observe?
> 
> Or is it ext3fs also?

No, there is a dirty flag on the super struct that should get cleaned
after all commits are done.  The commit_super-8 patch might fix a bug
where we are not properly cleaning the flag.

-chris

Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

Hi!

On 06/03/2002 05:51 AM, Chris Mason wrote:
> Hello everyone,
> 
> ftp.suse.com/pub/people/mason/data-logging
> 
> has beta code that gives you the following:
> 
> external logging devices
> data logging mount option (makes fsync and O_SYNC faster)
> ordered data mount option (no more garbage in your files after a crash)

Just a short dumb question inbetween: Is it allowed (...wished and 
healthy?!) to use both options together? Should look like "mount -o 
data=journal,data=ordered ...", yes? (mount-options referring to your 
latest ftp...README)

> 
> optimizations to keep kupdate from triggering commits every 5 seconds
> optimizations for tiny transactions to reduce CPU usage and IO.
> optimizations for better locking and fewer calls to wakeup()
> 
[...]

Thank you,

Manuel

Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 06/03/2002 05:51 AM, Chris Mason wrote:
> Hello everyone,
> 
> ftp.suse.com/pub/people/mason/data-logging
> 
> has beta code that gives you the following:
> 
> external logging devices
> data logging mount option (makes fsync and O_SYNC faster)
> ordered data mount option (no more garbage in your files after a crash)
> 
> optimizations to keep kupdate from triggering commits every 5 seconds
> optimizations for tiny transactions to reduce CPU usage and IO.
> optimizations for better locking and fewer calls to wakeup()
> 
> In short, it adds a bunch of features and makes everything faster.  The
> changes are very large, but they also build on older patches that have
> been circulating around for months.  It has been through a weekend of
> load testing here, but I still won't be surprised if people find
> corruption causing bugs.
> 
> It needs broad testing and benchmarking, in all the data modes, with a
> standard and external journal.
> 
> -chris
> 

I really got rid of my "usual setup": the speedup-compound-patch(with 
iicache-15?), akpm and aa patches for this testing now (mostly due to 
all these many rejects I got that Dieter had promised already... and due 
to the fact I have no time to understand->adjust them all manually at 
the moment).
I can't believe it, really, but I even left the other settings alone... ;-)

Now it is "only":
Kernel.2.4.19-pre9
ReiserFS.pending.01..03,05 (rest doesn't cooperate cleanly)
ReiserFS.mason.01..03 (maybe slightly modified for this)
rml.preempt-kernel (          -"-                 )

/ mounted -o noatime,notail,data=logging

Related to my previous state I once assumed to be lightning-fast I get 
-3secs (of former 17+1secs) load time of NS7 with all my plugins and 
-13+10secs (of former 27+10secs) load time of OOo.1.0 on a freshly 
booted machine. That is really nice!

Now I'll see VMwares results/reliability...
;-)

Very-very impressing, so far,

Manuel

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Mon, 2002-06-03 at 21:49, Manuel Krause wrote:
> 
> I really got rid of my "usual setup": the speedup-compound-patch(with 
> iicache-15?), akpm and aa patches for this testing now (mostly due to 
> all these many rejects I got that Dieter had promised already... and due 
> to the fact I have no time to understand->adjust them all manually at 
> the moment).
> I can't believe it, really, but I even left the other settings alone... ;-)

;-)

> 
> Now it is "only":
> Kernel.2.4.19-pre9
> ReiserFS.pending.01..03,05 (rest doesn't cooperate cleanly)
> ReiserFS.mason.01..03 (maybe slightly modified for this)
> rml.preempt-kernel (          -"-                 )
> 
> / mounted -o noatime,notail,data=logging
> 
> Related to my previous state I once assumed to be lightning-fast I get 
> -3secs (of former 17+1secs) load time of NS7 with all my plugins and 
> -13+10secs (of former 27+10secs) load time of OOo.1.0 on a freshly 
> booted machine. That is really nice!
> 
> Now I'll see VMwares results/reliability...
> ;-)
> 
> Very-very impressing, so far,

Great to hear, thanks for trying things out.

data=logging will be the slowest mode for everything except mail servers
and write heavy databases (or other apps that hammer on O_SYNC/fsync). 
This is because all the data gets written twice, once to log and once to
the main disk.  It helps synchronous writes by writing to the log in
quick sequential bursts, and then writing back to the main disk in
larger chunks.

data=ordered will usually be fastest for streaming appends to a file. 
The rest of the time it will be the same or slightly slower than the
default (data=writeback).

To answer your other question, you can only select one data mode at a
time.

-chris

Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 06/04/2002 04:20 AM, Chris Mason wrote:
> On Mon, 2002-06-03 at 21:49, Manuel Krause wrote:
> 
>>I really got rid of my "usual setup": the speedup-compound-patch(with 
>>iicache-15?), akpm and aa patches for this testing now (mostly due to 
>>all these many rejects I got that Dieter had promised already... and due 
>>to the fact I have no time to understand->adjust them all manually at 
>>the moment).
>>I can't believe it, really, but I even left the other settings alone... ;-)
> 
> 
> ;-)

   :-))

Makes some things and testings easier, I see, and some things faster??? 
;-) Mmh...

> 
> 
>>Now it is "only":
>>Kernel.2.4.19-pre9
>>ReiserFS.pending.01..03,05 (rest doesn't cooperate cleanly)
>>ReiserFS.mason.01..03 (maybe slightly modified for this)
>>rml.preempt-kernel (          -"-                 )
>>
>>/ mounted -o noatime,notail,data=logging
>>
>>Related to my previous state I once assumed to be lightning-fast I get 
>>-3secs (of former 17+1secs) load time of NS7 with all my plugins and 
>>-13+10secs (of former 27+10secs) load time of OOo.1.0 on a freshly 
>>booted machine. That is really nice!
>>
>>Now I'll see VMwares results/reliability...
>>;-)
>>
>>Very-very impressing, so far,
> 

So, VMware is stable with it, too, on my well known "heavy-private-test" 
of it (running Norton SpeedDisk at least twice within a most recent 
VMware Win98). It doesn't show greatly different timings than to my 
setup before though having a different disk i/o pattern (due to the 
missing aa patches)... and me having a reduced RAM from 512to256MB at 
the moment. And I should be honest to say I can't give exact timings as 
the important disk contents changed during last weeks. But the 
disk-access-times/related-to-the-content are definitively _not_ higher 
than before!

> 
> Great to hear, thanks for trying things out.
> 
> data=logging will be the slowest mode for everything except mail servers
> and write heavy databases (or other apps that hammer on O_SYNC/fsync). 
> This is because all the data gets written twice, once to log and once to
> the main disk.  It helps synchronous writes by writing to the log in
> quick sequential bursts, and then writing back to the main disk in
> larger chunks.

Yes, I assumed/learnt that from your very previous explanations. But I 
don't see the point "slowest", so far, on here.

This is a home/single session ?non?-production system on here. 1 to 2 
IDE disks, one processor... notebook, you'd remember this setup from my 
  mails.

> 
> data=ordered will usually be fastest for streaming appends to a file. 
> The rest of the time it will be the same or slightly slower than the
> default (data=writeback).

What would be my recommended mount option later then (mentionned in the 
manpages or on the webpage?!) Mmmh: I see I test that at the moment...

> 
> To answer your other question, you can only select one data mode at a
> time.

Why is no combo possible? Meant as: Could/would that bring advantages in 
future?!!

> 
> -chris
> 

Thanks,

Manuel

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Mon, 2002-06-03 at 23:28, Manuel Krause wrote:

> 
> So, VMware is stable with it, too, on my well known "heavy-private-test" 
> of it (running Norton SpeedDisk at least twice within a most recent 
> VMware Win98). It doesn't show greatly different timings than to my 
> setup before though having a different disk i/o pattern (due to the 
> missing aa patches)... and me having a reduced RAM from 512to256MB at 
> the moment. And I should be honest to say I can't give exact timings as 
> the important disk contents changed during last weeks. But the 
> disk-access-times/related-to-the-content are definitively _not_ higher 
> than before!

same speed on 1/2 the ram isn't bad ;-)

> 
> > 
> > Great to hear, thanks for trying things out.
> > 
> > data=logging will be the slowest mode for everything except mail servers
> > and write heavy databases (or other apps that hammer on O_SYNC/fsync). 
> > This is because all the data gets written twice, once to log and once to
> > the main disk.  It helps synchronous writes by writing to the log in
> > quick sequential bursts, and then writing back to the main disk in
> > larger chunks.
> 
> Yes, I assumed/learnt that from your very previous explanations. But I 
> don't see the point "slowest", so far, on here.

try dbench, or any streaming write.  It should be roughtly 1/2 as fast
as the default mode.  If it isn't data=journal probably isn't really
activated (it sends a printk on mount about using journaled data mode).

> 
> This is a home/single session ?non?-production system on here. 1 to 2 
> IDE disks, one processor... notebook, you'd remember this setup from my 
>   mails.
> 
> > 
> > data=ordered will usually be fastest for streaming appends to a file. 
> > The rest of the time it will be the same or slightly slower than the
> > default (data=writeback).
> 
> What would be my recommended mount option later then (mentionned in the 
> manpages or on the webpage?!) Mmmh: I see I test that at the moment...

data=ordered will work best for most people.  The performance hit is
very small, and you won't have any more garbage in files after a crash.

> 
> Why is no combo possible? Meant as: Could/would that bring advantages in 
> future?!!

Since there are no userspace transactions, data=journal is only useful
in a very limited set of cases.  The fact that everything is logged
allows me to cheat, and skip some of the steps to keep block allocations
and deallocations safe in the writeback data mode.

intermezzo is the only app that really needs data=journal with the rest
of the FS mounted with some other write mode (for speed).  The patch has
a way for them to enable data logging mode on a single file.

I think I'll forward the patch announcement to the intermezzo list...

-chris

Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 06/04/2002 03:12 PM, Chris Mason wrote:
 > On Mon, 2002-06-03 at 23:28, Manuel Krause wrote:
 >
 >
 >>So, VMware is stable with it, too, on my well known "heavy-private-test"
 >>of it (running Norton SpeedDisk at least twice within a most recent
 >>VMware Win98). It doesn't show greatly different timings than to my
 >>setup before though having a different disk i/o pattern (due to the
 >>missing aa patches)... and me having a reduced RAM from 512to256MB at
 >>the moment. And I should be honest to say I can't give exact timings as
 >>the important disk contents changed during last weeks. But the
 >>disk-access-times/related-to-the-content are definitively _not_ higher
 >>than before!
 >
 >
 > same speed on 1/2 the ram isn't bad ;-)
 >
[...]
Don't know where to reply best...

Hi, again!

I want to make some more comments on my latest words.

As I said I first used the data=journal mode and got nice timings. O.k. 
I really think after that "revision" my previous kernel setup wasn't 
that well configured as I thought and felt. Long time degression?!

I really had the reiserfs messages in my logs that it explicitely used
this mode. The only problem I obviously had, so far, was to distinguish 
the mount options at darkest night: data=logging is no mount-option but 
the description "data-logging", the mount option for it is data=journal
-- Passing rootflags=data=journal in lilo.conf and data=logging in fstab 
results in an uncontrollable kernel ;-) Huh!
Sorry, for my thoughtless testing. But my posted timings are quite 
relieble on here.

Concerning VMware the "same speed on 1/2 RAM" results are even more
impressing as VMware seems to buffer it's memory contents to /tmp/... fs 
again since I reduced the RAM. With 512MB it didn't seem to need this 
method usually.

The data=ordered mode saves 1..2secs from of my previously posted load
times for NS7 and OOo-1.0 and seems to be stable itself in "everydays 
usage" and for my VMware sessions, too. I didn't test the 
"crash->no-garbage-in-files" case and the more recent 
03-beta-data-logging-6.diff, yet.

I was extraordinary glad to see the explicit wording of the mounted 
partition in the logs we missed for so long time!

Thanks for your help,

Manuel

Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 06/05/2002 11:13 PM, Manuel Krause wrote:
> On 06/04/2002 03:12 PM, Chris Mason wrote:
>  > On Mon, 2002-06-03 at 23:28, Manuel Krause wrote:
>  >
>  >
>  >>So, VMware is stable with it, too, on my well known "heavy-private-test"
>  >>of it (running Norton SpeedDisk at least twice within a most recent
>  >>VMware Win98). It doesn't show greatly different timings than to my
>  >>setup before though having a different disk i/o pattern (due to the
>  >>missing aa patches)... and me having a reduced RAM from 512to256MB at
>  >>the moment. And I should be honest to say I can't give exact timings as
>  >>the important disk contents changed during last weeks. But the
>  >>disk-access-times/related-to-the-content are definitively _not_ higher
>  >>than before!
>  >
>  >
>  > same speed on 1/2 the ram isn't bad ;-)
>  >
> [...]
> Don't know where to reply best...
[...]

BTW, what is this "only" diff good for (is it worth to recompile, I mean):
# diff '03-beta-data-logging-6.diff' '03-beta-data-logging-5.diff'
2777c2777
< +  if (SB_JOURNAL(p_s_sb)->j_num_lists > 512) {
---
 > +  if (SB_JOURNAL(p_s_sb)->j_num_lists > 256) {

Thank you,

Manuel

> 
> I was extraordinary glad to see the explicit wording of the mounted 
> partition in the logs we missed for so long time!
> 
> Thanks for your help,
> 
> Manuel
> 

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Wed, 2002-06-05 at 17:27, Manuel Krause wrote:

> 
> BTW, what is this "only" diff good for (is it worth to recompile, I mean):
> # diff '03-beta-data-logging-6.diff' '03-beta-data-logging-5.diff'
> 2777c2777
> < +  if (SB_JOURNAL(p_s_sb)->j_num_lists > 512) {
> ---
>  > +  if (SB_JOURNAL(p_s_sb)->j_num_lists > 256) {

It is a performance tweak, my next patch removes that bit entirely. 
Thanks to some hints from Andrew Morton I've been able to do many more
optimizations for high load fsync heavy applications.  The new patch
will be out later tonight.

-chris

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

Ok, ftp.suse.com/pub/people/mason/patches/data-logging has my latest
updates, which have more optimizations for many threads all doing
synchronous transactions.

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 06/06/2002 02:09 AM, Chris Mason wrote:
> Ok, ftp.suse.com/pub/people/mason/patches/data-logging has my latest
> updates, which have more optimizations for many threads all doing
> synchronous transactions.
> 
> -chris
> 

Mmh. I recompiled with the new one when it appeared on the server and 
kept the kernel append line with the obvious problem not to be able to 
easily reboot to my ext2 maintenance partition.

I had some disappearing files/dirs from /lib/modules/* and from 
somewhere within /var/* the second time now. First occurrence is about 6 
days ago. That takes action within a good uptime (I mean: not at startup 
or upon special programs' interaction).

Jun 12 20:52:48 firehead kernel: is_leaf: item location seems wrong 
(second one): *3.6* [5273 5274 0x1 IND], item_len 4, item_location 2120, 
free_space(entry_count) 0
Jun 12 20:52:48 firehead kernel: vs-5150: search_by_key: invalid format 
found in block 8422. Fsck?
Jun 12 20:52:48 firehead kernel: vs-13070: reiserfs_read_inode2: i/o 
failure occurred trying to find stat data of [5199 261934 0x0 SD]
Jun 12 20:52:48 firehead kernel: is_leaf: item location seems wrong 
(second one): *3.6* [5273 5274 0x1 IND], item_len 4, item_location 2120, 
free_space(entry_count) 0
Jun 12 20:52:48 firehead kernel: vs-5150: search_by_key: invalid format 
found in block 8422. Fsck?
[...] <- many many more
Jun 12 20:54:46 firehead kernel: vs-13070: reiserfs_read_inode2: i/o 
failure occurred trying to find stat data of [5199 261934 0x0 SD]
Jun 12 20:54:46 firehead modprobe: modprobe: Can't open dependencies 
file /lib/modules/2.4.19-pre9/modules.dep (Permission denied)


I then needed to do a reiserfsck --rebuild-tree, as suggested by most 
recent reiserfsck (3.x.1c-pre4), the 3.x.1b-one said, it was fixable by 
--fix-fixable, but I then decided to use the latest. And then I had to 
adjust the lost&found manually. -- Huh, that took a long time, but 
nothing's missing, if I experience my system correctly now.

My system/partitions still look like the recently posted setup:
  Kernel.2.4.19-pre9
  ReiserFS.pending.01..03,05 (rest doesn't cooperate cleanly)
  ReiserFS.mason.01..03 (most recent)
  rml.preempt-kernel (maybe slightly modified for this setup)

/ mounted -o noatime,notail,data=ordered
and from applied lilo.conf: append = "... rootflags=data=ordered"


Thanks, but the term "speedup" seems to be relative for me for now,

Manuel

Re: Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Wed, 2002-06-12 at 18:32, Manuel Krause wrote:
> On 06/06/2002 02:09 AM, Chris Mason wrote:
> > Ok, ftp.suse.com/pub/people/mason/patches/data-logging has my latest
> > updates, which have more optimizations for many threads all doing
> > synchronous transactions.
> > 
> > -chris
> > 
> 
> Mmh. I recompiled with the new one when it appeared on the server and 
> kept the kernel append line with the obvious problem not to be able to 
> easily reboot to my ext2 maintenance partition.

Hmmm, I might have found it, but this isn't a new bug to the 07 patch. 
Have you been able to reproduce, or have you fallen back to the older
patches?

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 06/17/2002 02:47 AM, Chris Mason wrote:
> On Wed, 2002-06-12 at 18:32, Manuel Krause wrote:
> 
>>On 06/06/2002 02:09 AM, Chris Mason wrote:
>>
>>>Ok, ftp.suse.com/pub/people/mason/patches/data-logging has my latest
>>>updates, which have more optimizations for many threads all doing
>>>synchronous transactions.
>>>
>>>-chris
>>>
>>
>>Mmh. I recompiled with the new one when it appeared on the server and 
>>kept the kernel append line with the obvious problem not to be able to 
>>easily reboot to my ext2 maintenance partition.
> 
> 
> Hmmm, I might have found it, but this isn't a new bug to the 07 patch. 
> Have you been able to reproduce, or have you fallen back to the older
> patches?
> 

You mean 03-beta-data-logging-7.diff.gz when saying "07 patch"? Then I 
didn't go back according to my disk content.

When I boot up I read on screen and in boot.msg:
  ...
  EXT2-fs: Unrecognized mount option data
  reiserfs: using ordered data mode
  reiserfs: found format "3.6" with standard journal
  reiserfs: checking transaction log (ide0(3,7)) for (ide0(3,7))
  (...)
  Using r5 hash to sort names
  ...

Addon is: I use the ReiserFS pending patches that apply as I mentionned 
previously(?).

No, it looks o.k. here.


> -chris
> 

Manuel

Re: Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Sun, 2002-06-16 at 20:31, Manuel Krause wrote:

> > 
> > Hmmm, I might have found it, but this isn't a new bug to the 07 patch. 
> > Have you been able to reproduce, or have you fallen back to the older
> > patches?
> > 
> 
> You mean 03-beta-data-logging-7.diff.gz when saying "07 patch"? Then I 
> didn't go back according to my disk content.
> 

I've uploaded 03-beta-data-logging-8.diff.gz.  It will fix the oops Ed
hit, and hopefully your corruption problem as well.

-chris

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

ftp.suse.com/pub/people/mason/patches/data-logging has a new
beta-data-logging-10.diff.gz

This one:

compiles as a module (thanks Jeff Mahoney)

has a highmem bug fixed with data=journal (help from Andrea)

Fixes a problem with relocated/different sized journal devices.  This
was a merge error on my part.

One tester reports a hang that I can't reproduce here, I'm still trying.

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Chris Mason]

Hi everyone,

I've updated data logging patch, to fix a problem where
reiserfs_writepage didn't properly unlock pages in data logging mode.

The new code will be on
ftp.suse.com/pub/people/mason/patches/data-logging once the mirror
updates.

If you've already got 03-beta-data-logging-10, you only need the
incremental fix below.

-chris

--- 1.33/fs/reiserfs/inode.c	Fri Jun 28 09:34:33 2002
+++ edited/fs/reiserfs/inode.c	Fri Jun 28 09:37:42 2002
@@ -2034,6 +2034,7 @@
     }
     if (reiserfs_active_handle(&th)) {
         journal_end(&th, sb, nr) ;
+        UnlockPage(page);
     }
 }
 

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

Hello again,

Yet another data logging patch is available (once the mirror updates)
at:

ftp.suse.com/pub/people/mason/patches/data-logging

We're getting closer to stability here, so I'm very interested in
workloads where the patch performs poorly.

New in 03-beta-data-logging-13.diff:

Fix BUG() due to missing BKL before calling journal_begin

Make reiserfs unpack ioctl force a log flush in data journaling mode. 
This is required to make things safe if you crash after running lilo. 
It is also slow, but there is no other fix possible at the moment.

When the running transaction has many writers, the count of log blocks
reserved is much higher than the number actually used.  If a new writer
finds the transaction full, he now tries to let an existing writer
finish instead of forcing a commit.  This increases the average
transaction size when you have many procs writing to the log.

Smarter code to collect writers before forcing a commit during fsync.

In benchmarks with akpm's synctest program (it tries to simulate postfix
load), these last two changes brought the number of transactions during
a 50 thread run down from 6000 to 3300.  Not as good as I wanted, but it
is 20% faster than the last patch, so I'll take it.

Overall, for most mail server workloads, using the new patches +
data=journal should be around 3-4 times as fast as the stock 2.4.19-rc1
kernel.  I'm trying to get verification on this from actual mail server
benchmarks, instead of just simulations ;-)

-chris

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Tue, 2002-07-02 at 14:16, Chris Mason wrote:
> 
> Hello again,
> 
> Yet another data logging patch is available (once the mirror updates)
> at:
> 
> ftp.suse.com/pub/people/mason/patches/data-logging
> 
> We're getting closer to stability here, so I'm very interested in
> workloads where the patch performs poorly.
> 
> New in 03-beta-data-logging-13.diff:

And a buffer list corruption bug of some kind.  I've pulled
beta-data-logging-13.diff until I track the bug down.  I'm heading off
to the mountains for the 4th of July holiday, so it be fixed until
monday.

-chris

Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 07/03/2002 10:56 PM, Chris Mason wrote:
> On Tue, 2002-07-02 at 14:16, Chris Mason wrote:
> 
>>Hello again,
>>
>>Yet another data logging patch is available (once the mirror updates)
>>at:
>>
>>ftp.suse.com/pub/people/mason/patches/data-logging
>>
>>We're getting closer to stability here, so I'm very interested in
>>workloads where the patch performs poorly.
>>
>>New in 03-beta-data-logging-13.diff:
> 
> 
> And a buffer list corruption bug of some kind.  I've pulled
> beta-data-logging-13.diff until I track the bug down.  I'm heading off
> to the mountains for the 4th of July holiday, so it be fixed until
> monday.
> 
> -chris
> 
> 
> 

So far the 13 prerelease works fine under rootflags boot and 
mount/remount option "data=ordered" on here.

Can you, please, answer my old questions as of:

On 06/25/2002 02:55 AM, Manuel Krause mainly wrote (but too much more):

 > Is the rootflag needed any more? Does your patchset work with all
 > pending ReiserFS patches together, now?


Thanks,

I don't get a performance decrease with "13", and: Is it really 
dangerous to use "13" in this mode of operation?


Manuel

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Wed, 2002-07-03 at 20:28, Manuel Krause wrote:
 
> So far the 13 prerelease works fine under rootflags boot and 
> mount/remount option "data=ordered" on here.

I think I've got the bug, it was newly introduced in -13.  New stuff
will be uploaded today.

> 
> Can you, please, answer my old questions as of:
> 
> On 06/25/2002 02:55 AM, Manuel Krause mainly wrote (but too much more):
> 
>  > Is the rootflag needed any more? Does your patchset work with all
>  > pending ReiserFS patches together, now?

Yes, you still need the rootflag.  I haven't tried to merge it with all
the namesys pending patches, except for the journal relocation stuff and
the commit_super speedups.
 
> 
> Thanks,
> 
> I don't get a performance decrease with "13", and: Is it really 
> dangerous to use "13" in this mode of operation?

It is dangerous to use -13 at all, although the bug is pretty difficult
to trigger.  Please run reiserfsck --check before upgrading.

-chris

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Mon, 2002-06-03 at 21:49, Manuel Krause wrote:

> 
> Now it is "only":
> Kernel.2.4.19-pre9
> ReiserFS.pending.01..03,05 (rest doesn't cooperate cleanly)
> ReiserFS.mason.01..03 (maybe slightly modified for this)
> rml.preempt-kernel (          -"-                 )
> 
> / mounted -o noatime,notail,data=logging

Ah, still on the todo list for the current patch is safe handling of
remount.  Currently data=logging and data=ordered are ignored on
remount, which is probably why your results were so fast in data logging
mode.

You'll want to pass rootflags=data=ordered or rootflags=data=journal at
the lilo prompt.

-chris

Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 06/04/2002 04:57 AM, Chris Mason wrote:
> On Mon, 2002-06-03 at 21:49, Manuel Krause wrote:
> 
> 
>>Now it is "only":
>>Kernel.2.4.19-pre9
>>ReiserFS.pending.01..03,05 (rest doesn't cooperate cleanly)
>>ReiserFS.mason.01..03 (maybe slightly modified for this)
>>rml.preempt-kernel (          -"-                 )
>>
>>/ mounted -o noatime,notail,data=logging
> 
> 
> Ah, still on the todo list for the current patch is safe handling of
> remount.  

O.k.

 >
> Currently data=logging and data=ordered are ignored on
> remount, which is probably why your results were so fast in data logging
> mode.

Yes, as-is on SuSE typical bootup. But, o.k., I know to handle,  I've 
changed and reset my lilo-setup accordingly to your mail...

So, I just retested, with your recommended append to lilo.conf 
"rootflags=data=journal" -- but timings stay the same. And mount says 
still the same, too:
/dev/hda7 on / type reiserfs (rw,noatime,notail,data=logging)

So should you be glad or not!?
The NS7 and OOo.1.0 load timings are generally measured in secs +-1 and 
stay to be faster, like I posted before.

> 
> You'll want to pass rootflags=data=ordered or rootflags=data=journal at
> the lilo prompt.
> 
> -chris
> 

No, no mistake on this way, on my side, really, you could finally (if 
you don't find a solution within your new coding) blame it to my bad 
previous patches' setup... ;-)))
(yes, finally -- at the real end, of course!!! ;-)  )

Thank you, works very well,

Manuel

P.S.: Please, keep me informed, as Namesys seems to need a pushup for 
2.4. kernels... ;-) Sorry, Hans! ;-)

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Tue, 2002-06-04 at 00:16, Manuel Krause wrote:

> 
> P.S.: Please, keep me informed, as Namesys seems to need a pushup for 
> 2.4. kernels... ;-) Sorry, Hans! ;-)

BTW, for general usage, the most important speedup in these patches is
the commit_super patch.  The namesys guys already have that in the queue
of stuff to send for 2.4.20-pre.

The tweaks included in the data logging code improve a few of the corner
cases that were hurting us.

-chris

Re: [PATCH CFT] tons of logging patches

[Hans Reiser]

Chris Mason wrote:

>On Tue, 2002-06-04 at 00:16, Manuel Krause wrote:
>
>  
>
>>P.S.: Please, keep me informed, as Namesys seems to need a pushup for 
>>2.4. kernels... ;-) Sorry, Hans! ;-)
>>    
>>
>
>BTW, for general usage, the most important speedup in these patches is
>the commit_super patch.  The namesys guys already have that in the queue
>of stuff to send for 2.4.20-pre.
>
You will see substantial performance increases in 2.4.20

Oleg is doing a complete rewrite of our write function, and is seeing 
large file performance increases of 25%.

We are currently doing a complete analysis of our layout algorithms, and 
I expect some results from that.

Finally, there is V4 which is almost ready to work now.....

Hans

Re: [PATCH CFT] tons of logging patches

[Robert Brockway]

On Wed, 5 Jun 2002, Hans Reiser wrote:
> You will see substantial performance increases in 2.4.20
> 
> Oleg is doing a complete rewrite of our write function, and is seeing 
> large file performance increases of 25%.

This is very impressive.  Particularly considering I did some performance 
tests on reiserfs vs ext3 vs xfs last night & Reiserfs beat the pants off 
the other two for large file creation.  This was on a 2.4.18 kernel & the 
xfs used was the new version 1.1.

In order to eliminate other variables, I was using the same partition for 
all tests (just remade it over & over), same kernel, same system load, 
etc.

I'm going to develop a script which will go through & do some formal
testing on the different filesystems.  Last night was just a play to see
how it all went.  The proper testing will be done in single user mode.

I also intend to patch for jfs if it will play nicely with the others.  

I'm happy to post my results to the list when I finish if you like.  
Cheers,
	-Rob

-- Robert Brockway B.Sc. email: robert@timetraveller.org  ICQ: 104781119
   Linux counter project ID #16440 (http://counter.li.org)
   "The earth is but one country and mankind its citizens" -Baha'u'llah

Re: [PATCH CFT] tons of logging patches

[Oleg Drokin]

Hello!

On Wed, Jun 05, 2002 at 01:20:50AM +0400, Hans Reiser wrote:
> >BTW, for general usage, the most important speedup in these patches is
> >the commit_super patch.  The namesys guys already have that in the queue
> >of stuff to send for 2.4.20-pre.
> You will see substantial performance increases in 2.4.20
> Oleg is doing a complete rewrite of our write function, and is seeing 
> large file performance increases of 25%.

Note that I see only 25+% performance increase CPU-wise. Since the task
is io-bound real time does not change much (unless one runs tests
on the ramdisk or similar (by speed) kind of storage).

Bye,
    Oleg