Re: [PATCH CFT] tons of logging patches

Re: [PATCH CFT] tons of logging patches

[Dieter Nützel]

On Friday, Chris wrote:

> ftp.suse.com/pub/people/mason/patches/data-logging has a new
> beta-data-logging-10.diff.gz
>
> This one:
>
> compiles as a module (thanks Jeff Mahoney)
>
> has a highmem bug fixed with data=journal (help from Andrea)

Hello Chris,

should I run this with -aa kernel?

> Fixes a problem with relocated/different sized journal devices.  This
> was a merge error on my part.
>
> One tester reports a hang that I can't reproduce here, I'm still trying.

You do not mean me (page coloring)?

Regards,
	Dieter

BTW I have my dual Athlon MP 1900+ up and "flying"...
-- 
Dieter Nützel
Graduate Student, Computer Science

University of Hamburg
Department of Computer Science
@home: Dieter.Nuetzel@hamburg.de

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Sat, 2002-06-22 at 16:02, Dieter Nützel wrote:
> On Friday, Chris wrote:
> 
> > ftp.suse.com/pub/people/mason/patches/data-logging has a new
> > beta-data-logging-10.diff.gz
> >
> > This one:
> >
> > compiles as a module (thanks Jeff Mahoney)
> >
> > has a highmem bug fixed with data=journal (help from Andrea)
> 
> Hello Chris,
> 
> should I run this with -aa kernel?

You can, the differences are minor, just interdiff from the last data
logging patch and apply the incremental onto the aa data logging patch. 
I ran out of time before leaving for ottawa, so I didn't get the chance
to make an aa patch.

> 
> > Fixes a problem with relocated/different sized journal devices.  This
> > was a merge error on my part.
> >
> > One tester reports a hang that I can't reproduce here, I'm still trying.
> 
> You do not mean me (page coloring)?

No, this was a large 8 way smp machine.  

-chris

Re: [PATCH CFT] tons of logging patches

[Dieter Nützel]

On Monday, 8 July 2002 13:51, Chris Mason wrote:
> On Wed, 2002-07-03 at 20:28, Manuel Krause wrote:
>
> > So far the 13 prerelease works fine under rootflags boot and 
> > mount/remount option "data=ordered" on here.
>
> I think I've got the bug, it was newly introduced in -13.  New stuff
> will be uploaded today.

Where is your stuff?
It didn't appeared in your SuSE "home"
I'm during the preparation of 2.4.19-rc1-jam2-pc (2.4.19pre1aa2 + jam2 + page 
coloring).

Thanks,
	Dieter

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Tue, 2002-07-09 at 09:52, Dieter Nützel wrote:
> On Monday, 8 July 2002 13:51, Chris Mason wrote:
> > On Wed, 2002-07-03 at 20:28, Manuel Krause wrote:
> >
> > > So far the 13 prerelease works fine under rootflags boot and 
> > > mount/remount option "data=ordered" on here.
> >
> > I think I've got the bug, it was newly introduced in -13.  New stuff
> > will be uploaded today.
> 
> Where is your stuff?
> It didn't appeared in your SuSE "home"
> I'm during the preparation of 2.4.19-rc1-jam2-pc (2.4.19pre1aa2 + jam2 + page 
> coloring).

Sigh, further testing showed I'd only fixed half of the bug.  The other
half is somewhere deep writepage/truncate interaction.  I'm still trying
to find it.

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Tue, 2002-07-09 at 10:00, Chris Mason wrote:
> On Tue, 2002-07-09 at 09:52, Dieter Nützel wrote:
> > On Monday, 8 July 2002 13:51, Chris Mason wrote:
> > > On Wed, 2002-07-03 at 20:28, Manuel Krause wrote:
> > >
> > > > So far the 13 prerelease works fine under rootflags boot and 
> > > > mount/remount option "data=ordered" on here.
> > >
> > > I think I've got the bug, it was newly introduced in -13.  New stuff
> > > will be uploaded today.
> > >
> Sigh, further testing showed I'd only fixed half of the bug.  The other
> half is somewhere deep writepage/truncate interaction.  I'm still trying
> to find it.

Well, after a week of fixing minor things and adding debugging code,
this bug still goes unsolved.  The only good news is that I can also
trigger it on pure 2.4.19-rc1 and ext2.  The bug seems to either be in
the VM or my ram, so I'm having a little memtest86 party overnight.

I've uploaded 03-beta-data-logging-16, it should be more stable and
faster than -11 and -13 were, but includes more debugging checks to try
and make sure I'm not screwing things up.  You can grab the patches at:

ftp://ftp.suse.com/pub/people/mason/patches/data-logging

If anyone wants to try and reproduce my memory corruption bug (can and
probably will lead to FS corruption), just let 10 fsx processes run in a
loop for ~3 hours or so on an SMP machine.

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 07/12/2002 09:43 PM, Chris Mason wrote:
> On Tue, 2002-07-09 at 10:00, Chris Mason wrote:
> 
>>On Tue, 2002-07-09 at 09:52, Dieter Nützel wrote:
>>
>>>On Monday, 8 July 2002 13:51, Chris Mason wrote:
>>>
>>>>On Wed, 2002-07-03 at 20:28, Manuel Krause wrote:
>>>>
>>>>
>>>>>So far the 13 prerelease works fine under rootflags boot and 
>>>>>mount/remount option "data=ordered" on here.
>>>>
>>>>I think I've got the bug, it was newly introduced in -13.  New stuff
>>>>will be uploaded today.
>>>>
>>>
>>Sigh, further testing showed I'd only fixed half of the bug.  The other
>>half is somewhere deep writepage/truncate interaction.  I'm still trying
>>to find it.
> 
> 
> Well, after a week of fixing minor things and adding debugging code,
> this bug still goes unsolved.  The only good news is that I can also
> trigger it on pure 2.4.19-rc1 and ext2.  The bug seems to either be in
> the VM or my ram, so I'm having a little memtest86 party overnight.
> 
> I've uploaded 03-beta-data-logging-16, it should be more stable and
> faster than -11 and -13 were, but includes more debugging checks to try
> and make sure I'm not screwing things up.  You can grab the patches at:
> 
> ftp://ftp.suse.com/pub/people/mason/patches/data-logging
> 

Thank you, Chris!

It's working fine under rootflags set and mount/remount options 
"noatime,notail,data=ordered".
Nothing got wrong on here due to data-logging-13.

Netscape7 dropped its load time again by 3secs of (previously 14+1), 
OpenOffice1 increased by 1sec (of average 17secs previously). To explain 
the previous state: I've only changed -pre9 to -rc1 and data-logging 
from -10(-pre9)..13(-rc1)..11(-rc1)..16(-rc1, now!) from the old patchset.

> If anyone wants to try and reproduce my memory corruption bug (can and
> probably will lead to FS corruption), just let 10 fsx processes run in a
> loop for ~3 hours or so on an SMP machine.
> 
> -chris
> 

Is this possible to be reproduced on uniprocessor machines, too ?


Best regards,

Manuel

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

Hello everyone,

I've uploaded 03-data-logging-24.diff, which fixes a tail packing oops
in data=journal mode, and fixes the data=ordered slow down (5-10%)
introduced in -23.  

-23 flushed all the ordered buffer heads with the journal lock held,
which meant no new transactions could start while the ordered buffers
were being flushed.  This was safe but slower.

-24 flushes them right before the commit blocks (like -19 did), and only
flushes the tail conversion targets with the journal lock held.  Doing
the tail conversion targets later risks losing the tail data after a
crash.

Thanks to grev@namesys.com who reported the ordered write bug in -19 and
the data=journal oops in -23.

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Fri, 2002-08-02 at 08:28, Chris Mason wrote:
> Hello everyone,
> 
> I've uploaded 03-data-logging-24.diff, which fixes a tail packing oops
> in data=journal mode, and fixes the data=ordered slow down (5-10%)
> introduced in -23.  

Whoops, I neglected to mention that data=ordered is now the default in
the patch.  You can mount with -o data=writeback if you don't want
ordered data writes.

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Hans Reiser]

Chris Mason wrote:

>On Fri, 2002-08-02 at 08:28, Chris Mason wrote:
>  
>
>>Hello everyone,
>>
>>I've uploaded 03-data-logging-24.diff, which fixes a tail packing oops
>>in data=journal mode, and fixes the data=ordered slow down (5-10%)
>>introduced in -23.  
>>    
>>
>
>Whoops, I neglected to mention that data=ordered is now the default in
>the patch.  You can mount with -o data=writeback if you don't want
>ordered data writes.
>
>-chris
>
>
>
>
>  
>
we just saw a benchmark in which data=ordered caused a major slowdown. 
 Does this patch fix this?

Edward and Elena, I want thorough benchmarking of this.

-- 
Hans

Re: Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Fri, 2002-08-02 at 08:59, Hans Reiser wrote:
> >
> we just saw a benchmark in which data=ordered caused a major slowdown. 
>  Does this patch fix this?
> 
> Edward and Elena, I want thorough benchmarking of this.
> 

The last fract_tree benchmarks were on older patches, Manual Krause's
benchmark was more showing the difference between a used and a fresh
filesystem, and he was using -23 which had data=ordered performance
issues fixed in -24.

More benchmarking does need to be done, the patch isn't set in stone yet
;-)

-chris

[PATCH CFT] tons of logging patches

[Chris Mason]

Hello everyone,

ftp.suse.com/pub/people/mason/data-logging

has beta code that gives you the following:

external logging devices
data logging mount option (makes fsync and O_SYNC faster)
ordered data mount option (no more garbage in your files after a crash)

optimizations to keep kupdate from triggering commits every 5 seconds
optimizations for tiny transactions to reduce CPU usage and IO.
optimizations for better locking and fewer calls to wakeup()

In short, it adds a bunch of features and makes everything faster.  The
changes are very large, but they also build on older patches that have
been circulating around for months.  It has been through a weekend of
load testing here, but I still won't be surprised if people find
corruption causing bugs.

It needs broad testing and benchmarking, in all the data modes, with a
standard and external journal.

-chris

Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 06/03/2002 05:51 AM, Chris Mason wrote:
> Hello everyone,
> 
> ftp.suse.com/pub/people/mason/data-logging
> 
> has beta code that gives you the following:
> 
> external logging devices
> data logging mount option (makes fsync and O_SYNC faster)
> ordered data mount option (no more garbage in your files after a crash)
> 
> optimizations to keep kupdate from triggering commits every 5 seconds
> optimizations for tiny transactions to reduce CPU usage and IO.
> optimizations for better locking and fewer calls to wakeup()
> 
> In short, it adds a bunch of features and makes everything faster.  The
> changes are very large, but they also build on older patches that have
> been circulating around for months.  It has been through a weekend of
> load testing here, but I still won't be surprised if people find
> corruption causing bugs.
> 
> It needs broad testing and benchmarking, in all the data modes, with a
> standard and external journal.
> 
> -chris
> 

I really got rid of my "usual setup": the speedup-compound-patch(with 
iicache-15?), akpm and aa patches for this testing now (mostly due to 
all these many rejects I got that Dieter had promised already... and due 
to the fact I have no time to understand->adjust them all manually at 
the moment).
I can't believe it, really, but I even left the other settings alone... ;-)

Now it is "only":
Kernel.2.4.19-pre9
ReiserFS.pending.01..03,05 (rest doesn't cooperate cleanly)
ReiserFS.mason.01..03 (maybe slightly modified for this)
rml.preempt-kernel (          -"-                 )

/ mounted -o noatime,notail,data=logging

Related to my previous state I once assumed to be lightning-fast I get 
-3secs (of former 17+1secs) load time of NS7 with all my plugins and 
-13+10secs (of former 27+10secs) load time of OOo.1.0 on a freshly 
booted machine. That is really nice!

Now I'll see VMwares results/reliability...
;-)

Very-very impressing, so far,

Manuel

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Mon, 2002-06-03 at 21:49, Manuel Krause wrote:
> 
> I really got rid of my "usual setup": the speedup-compound-patch(with 
> iicache-15?), akpm and aa patches for this testing now (mostly due to 
> all these many rejects I got that Dieter had promised already... and due 
> to the fact I have no time to understand->adjust them all manually at 
> the moment).
> I can't believe it, really, but I even left the other settings alone... ;-)

;-)

> 
> Now it is "only":
> Kernel.2.4.19-pre9
> ReiserFS.pending.01..03,05 (rest doesn't cooperate cleanly)
> ReiserFS.mason.01..03 (maybe slightly modified for this)
> rml.preempt-kernel (          -"-                 )
> 
> / mounted -o noatime,notail,data=logging
> 
> Related to my previous state I once assumed to be lightning-fast I get 
> -3secs (of former 17+1secs) load time of NS7 with all my plugins and 
> -13+10secs (of former 27+10secs) load time of OOo.1.0 on a freshly 
> booted machine. That is really nice!
> 
> Now I'll see VMwares results/reliability...
> ;-)
> 
> Very-very impressing, so far,

Great to hear, thanks for trying things out.

data=logging will be the slowest mode for everything except mail servers
and write heavy databases (or other apps that hammer on O_SYNC/fsync). 
This is because all the data gets written twice, once to log and once to
the main disk.  It helps synchronous writes by writing to the log in
quick sequential bursts, and then writing back to the main disk in
larger chunks.

data=ordered will usually be fastest for streaming appends to a file. 
The rest of the time it will be the same or slightly slower than the
default (data=writeback).

To answer your other question, you can only select one data mode at a
time.

-chris

Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 06/04/2002 04:20 AM, Chris Mason wrote:
> On Mon, 2002-06-03 at 21:49, Manuel Krause wrote:
> 
>>I really got rid of my "usual setup": the speedup-compound-patch(with 
>>iicache-15?), akpm and aa patches for this testing now (mostly due to 
>>all these many rejects I got that Dieter had promised already... and due 
>>to the fact I have no time to understand->adjust them all manually at 
>>the moment).
>>I can't believe it, really, but I even left the other settings alone... ;-)
> 
> 
> ;-)

   :-))

Makes some things and testings easier, I see, and some things faster??? 
;-) Mmh...

> 
> 
>>Now it is "only":
>>Kernel.2.4.19-pre9
>>ReiserFS.pending.01..03,05 (rest doesn't cooperate cleanly)
>>ReiserFS.mason.01..03 (maybe slightly modified for this)
>>rml.preempt-kernel (          -"-                 )
>>
>>/ mounted -o noatime,notail,data=logging
>>
>>Related to my previous state I once assumed to be lightning-fast I get 
>>-3secs (of former 17+1secs) load time of NS7 with all my plugins and 
>>-13+10secs (of former 27+10secs) load time of OOo.1.0 on a freshly 
>>booted machine. That is really nice!
>>
>>Now I'll see VMwares results/reliability...
>>;-)
>>
>>Very-very impressing, so far,
> 

So, VMware is stable with it, too, on my well known "heavy-private-test" 
of it (running Norton SpeedDisk at least twice within a most recent 
VMware Win98). It doesn't show greatly different timings than to my 
setup before though having a different disk i/o pattern (due to the 
missing aa patches)... and me having a reduced RAM from 512to256MB at 
the moment. And I should be honest to say I can't give exact timings as 
the important disk contents changed during last weeks. But the 
disk-access-times/related-to-the-content are definitively _not_ higher 
than before!

> 
> Great to hear, thanks for trying things out.
> 
> data=logging will be the slowest mode for everything except mail servers
> and write heavy databases (or other apps that hammer on O_SYNC/fsync). 
> This is because all the data gets written twice, once to log and once to
> the main disk.  It helps synchronous writes by writing to the log in
> quick sequential bursts, and then writing back to the main disk in
> larger chunks.

Yes, I assumed/learnt that from your very previous explanations. But I 
don't see the point "slowest", so far, on here.

This is a home/single session ?non?-production system on here. 1 to 2 
IDE disks, one processor... notebook, you'd remember this setup from my 
  mails.

> 
> data=ordered will usually be fastest for streaming appends to a file. 
> The rest of the time it will be the same or slightly slower than the
> default (data=writeback).

What would be my recommended mount option later then (mentionned in the 
manpages or on the webpage?!) Mmmh: I see I test that at the moment...

> 
> To answer your other question, you can only select one data mode at a
> time.

Why is no combo possible? Meant as: Could/would that bring advantages in 
future?!!

> 
> -chris
> 

Thanks,

Manuel

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Mon, 2002-06-03 at 23:28, Manuel Krause wrote:

> 
> So, VMware is stable with it, too, on my well known "heavy-private-test" 
> of it (running Norton SpeedDisk at least twice within a most recent 
> VMware Win98). It doesn't show greatly different timings than to my 
> setup before though having a different disk i/o pattern (due to the 
> missing aa patches)... and me having a reduced RAM from 512to256MB at 
> the moment. And I should be honest to say I can't give exact timings as 
> the important disk contents changed during last weeks. But the 
> disk-access-times/related-to-the-content are definitively _not_ higher 
> than before!

same speed on 1/2 the ram isn't bad ;-)

> 
> > 
> > Great to hear, thanks for trying things out.
> > 
> > data=logging will be the slowest mode for everything except mail servers
> > and write heavy databases (or other apps that hammer on O_SYNC/fsync). 
> > This is because all the data gets written twice, once to log and once to
> > the main disk.  It helps synchronous writes by writing to the log in
> > quick sequential bursts, and then writing back to the main disk in
> > larger chunks.
> 
> Yes, I assumed/learnt that from your very previous explanations. But I 
> don't see the point "slowest", so far, on here.

try dbench, or any streaming write.  It should be roughtly 1/2 as fast
as the default mode.  If it isn't data=journal probably isn't really
activated (it sends a printk on mount about using journaled data mode).

> 
> This is a home/single session ?non?-production system on here. 1 to 2 
> IDE disks, one processor... notebook, you'd remember this setup from my 
>   mails.
> 
> > 
> > data=ordered will usually be fastest for streaming appends to a file. 
> > The rest of the time it will be the same or slightly slower than the
> > default (data=writeback).
> 
> What would be my recommended mount option later then (mentionned in the 
> manpages or on the webpage?!) Mmmh: I see I test that at the moment...

data=ordered will work best for most people.  The performance hit is
very small, and you won't have any more garbage in files after a crash.

> 
> Why is no combo possible? Meant as: Could/would that bring advantages in 
> future?!!

Since there are no userspace transactions, data=journal is only useful
in a very limited set of cases.  The fact that everything is logged
allows me to cheat, and skip some of the steps to keep block allocations
and deallocations safe in the writeback data mode.

intermezzo is the only app that really needs data=journal with the rest
of the FS mounted with some other write mode (for speed).  The patch has
a way for them to enable data logging mode on a single file.

I think I'll forward the patch announcement to the intermezzo list...

-chris

Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 06/04/2002 03:12 PM, Chris Mason wrote:
 > On Mon, 2002-06-03 at 23:28, Manuel Krause wrote:
 >
 >
 >>So, VMware is stable with it, too, on my well known "heavy-private-test"
 >>of it (running Norton SpeedDisk at least twice within a most recent
 >>VMware Win98). It doesn't show greatly different timings than to my
 >>setup before though having a different disk i/o pattern (due to the
 >>missing aa patches)... and me having a reduced RAM from 512to256MB at
 >>the moment. And I should be honest to say I can't give exact timings as
 >>the important disk contents changed during last weeks. But the
 >>disk-access-times/related-to-the-content are definitively _not_ higher
 >>than before!
 >
 >
 > same speed on 1/2 the ram isn't bad ;-)
 >
[...]
Don't know where to reply best...

Hi, again!

I want to make some more comments on my latest words.

As I said I first used the data=journal mode and got nice timings. O.k. 
I really think after that "revision" my previous kernel setup wasn't 
that well configured as I thought and felt. Long time degression?!

I really had the reiserfs messages in my logs that it explicitely used
this mode. The only problem I obviously had, so far, was to distinguish 
the mount options at darkest night: data=logging is no mount-option but 
the description "data-logging", the mount option for it is data=journal
-- Passing rootflags=data=journal in lilo.conf and data=logging in fstab 
results in an uncontrollable kernel ;-) Huh!
Sorry, for my thoughtless testing. But my posted timings are quite 
relieble on here.

Concerning VMware the "same speed on 1/2 RAM" results are even more
impressing as VMware seems to buffer it's memory contents to /tmp/... fs 
again since I reduced the RAM. With 512MB it didn't seem to need this 
method usually.

The data=ordered mode saves 1..2secs from of my previously posted load
times for NS7 and OOo-1.0 and seems to be stable itself in "everydays 
usage" and for my VMware sessions, too. I didn't test the 
"crash->no-garbage-in-files" case and the more recent 
03-beta-data-logging-6.diff, yet.

I was extraordinary glad to see the explicit wording of the mounted 
partition in the logs we missed for so long time!

Thanks for your help,

Manuel

Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 06/05/2002 11:13 PM, Manuel Krause wrote:
> On 06/04/2002 03:12 PM, Chris Mason wrote:
>  > On Mon, 2002-06-03 at 23:28, Manuel Krause wrote:
>  >
>  >
>  >>So, VMware is stable with it, too, on my well known "heavy-private-test"
>  >>of it (running Norton SpeedDisk at least twice within a most recent
>  >>VMware Win98). It doesn't show greatly different timings than to my
>  >>setup before though having a different disk i/o pattern (due to the
>  >>missing aa patches)... and me having a reduced RAM from 512to256MB at
>  >>the moment. And I should be honest to say I can't give exact timings as
>  >>the important disk contents changed during last weeks. But the
>  >>disk-access-times/related-to-the-content are definitively _not_ higher
>  >>than before!
>  >
>  >
>  > same speed on 1/2 the ram isn't bad ;-)
>  >
> [...]
> Don't know where to reply best...
[...]

BTW, what is this "only" diff good for (is it worth to recompile, I mean):
# diff '03-beta-data-logging-6.diff' '03-beta-data-logging-5.diff'
2777c2777
< +  if (SB_JOURNAL(p_s_sb)->j_num_lists > 512) {
---
 > +  if (SB_JOURNAL(p_s_sb)->j_num_lists > 256) {

Thank you,

Manuel

> 
> I was extraordinary glad to see the explicit wording of the mounted 
> partition in the logs we missed for so long time!
> 
> Thanks for your help,
> 
> Manuel
> 

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

Ok, ftp.suse.com/pub/people/mason/patches/data-logging has my latest
updates, which have more optimizations for many threads all doing
synchronous transactions.

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 06/06/2002 02:09 AM, Chris Mason wrote:
> Ok, ftp.suse.com/pub/people/mason/patches/data-logging has my latest
> updates, which have more optimizations for many threads all doing
> synchronous transactions.
> 
> -chris
> 

Mhh. I was just starting to try the ftp, but  no-go-for-now.  O.k now 
I've got an old Santana CD in... "Black magic ..." Thanks to Chris,

bye, very nice to listen to,

Manuel

Re: Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 06/06/2002 02:09 AM, Chris Mason wrote:
> Ok, ftp.suse.com/pub/people/mason/patches/data-logging has my latest
> updates, which have more optimizations for many threads all doing
> synchronous transactions.
> 
> -chris
> 

Mmh. I recompiled with the new one when it appeared on the server and 
kept the kernel append line with the obvious problem not to be able to 
easily reboot to my ext2 maintenance partition.

I had some disappearing files/dirs from /lib/modules/* and from 
somewhere within /var/* the second time now. First occurrence is about 6 
days ago. That takes action within a good uptime (I mean: not at startup 
or upon special programs' interaction).

Jun 12 20:52:48 firehead kernel: is_leaf: item location seems wrong 
(second one): *3.6* [5273 5274 0x1 IND], item_len 4, item_location 2120, 
free_space(entry_count) 0
Jun 12 20:52:48 firehead kernel: vs-5150: search_by_key: invalid format 
found in block 8422. Fsck?
Jun 12 20:52:48 firehead kernel: vs-13070: reiserfs_read_inode2: i/o 
failure occurred trying to find stat data of [5199 261934 0x0 SD]
Jun 12 20:52:48 firehead kernel: is_leaf: item location seems wrong 
(second one): *3.6* [5273 5274 0x1 IND], item_len 4, item_location 2120, 
free_space(entry_count) 0
Jun 12 20:52:48 firehead kernel: vs-5150: search_by_key: invalid format 
found in block 8422. Fsck?
[...] <- many many more
Jun 12 20:54:46 firehead kernel: vs-13070: reiserfs_read_inode2: i/o 
failure occurred trying to find stat data of [5199 261934 0x0 SD]
Jun 12 20:54:46 firehead modprobe: modprobe: Can't open dependencies 
file /lib/modules/2.4.19-pre9/modules.dep (Permission denied)


I then needed to do a reiserfsck --rebuild-tree, as suggested by most 
recent reiserfsck (3.x.1c-pre4), the 3.x.1b-one said, it was fixable by 
--fix-fixable, but I then decided to use the latest. And then I had to 
adjust the lost&found manually. -- Huh, that took a long time, but 
nothing's missing, if I experience my system correctly now.

My system/partitions still look like the recently posted setup:
  Kernel.2.4.19-pre9
  ReiserFS.pending.01..03,05 (rest doesn't cooperate cleanly)
  ReiserFS.mason.01..03 (most recent)
  rml.preempt-kernel (maybe slightly modified for this setup)

/ mounted -o noatime,notail,data=ordered
and from applied lilo.conf: append = "... rootflags=data=ordered"


Thanks, but the term "speedup" seems to be relative for me for now,

Manuel

Re: Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Wed, 2002-06-12 at 18:32, Manuel Krause wrote:
> On 06/06/2002 02:09 AM, Chris Mason wrote:
> > Ok, ftp.suse.com/pub/people/mason/patches/data-logging has my latest
> > updates, which have more optimizations for many threads all doing
> > synchronous transactions.
> > 
> > -chris
> > 
> 
> Mmh. I recompiled with the new one when it appeared on the server and 
> kept the kernel append line with the obvious problem not to be able to 
> easily reboot to my ext2 maintenance partition.

Hmmm, I might have found it, but this isn't a new bug to the 07 patch. 
Have you been able to reproduce, or have you fallen back to the older
patches?

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 06/17/2002 02:47 AM, Chris Mason wrote:
> On Wed, 2002-06-12 at 18:32, Manuel Krause wrote:
> 
>>On 06/06/2002 02:09 AM, Chris Mason wrote:
>>
>>>Ok, ftp.suse.com/pub/people/mason/patches/data-logging has my latest
>>>updates, which have more optimizations for many threads all doing
>>>synchronous transactions.
>>>
>>>-chris
>>>
>>
>>Mmh. I recompiled with the new one when it appeared on the server and 
>>kept the kernel append line with the obvious problem not to be able to 
>>easily reboot to my ext2 maintenance partition.
> 
> 
> Hmmm, I might have found it, but this isn't a new bug to the 07 patch. 
> Have you been able to reproduce, or have you fallen back to the older
> patches?
> 

You mean 03-beta-data-logging-7.diff.gz when saying "07 patch"? Then I 
didn't go back according to my disk content.

When I boot up I read on screen and in boot.msg:
  ...
  EXT2-fs: Unrecognized mount option data
  reiserfs: using ordered data mode
  reiserfs: found format "3.6" with standard journal
  reiserfs: checking transaction log (ide0(3,7)) for (ide0(3,7))
  (...)
  Using r5 hash to sort names
  ...

Addon is: I use the ReiserFS pending patches that apply as I mentionned 
previously(?).

No, it looks o.k. here.


> -chris
> 

Manuel

Re: Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Sun, 2002-06-16 at 20:31, Manuel Krause wrote:

> > 
> > Hmmm, I might have found it, but this isn't a new bug to the 07 patch. 
> > Have you been able to reproduce, or have you fallen back to the older
> > patches?
> > 
> 
> You mean 03-beta-data-logging-7.diff.gz when saying "07 patch"? Then I 
> didn't go back according to my disk content.
> 

I've uploaded 03-beta-data-logging-8.diff.gz.  It will fix the oops Ed
hit, and hopefully your corruption problem as well.

-chris

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

ftp.suse.com/pub/people/mason/patches/data-logging has a new
beta-data-logging-10.diff.gz

This one:

compiles as a module (thanks Jeff Mahoney)

has a highmem bug fixed with data=journal (help from Andrea)

Fixes a problem with relocated/different sized journal devices.  This
was a merge error on my part.

One tester reports a hang that I can't reproduce here, I'm still trying.

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 06/21/2002 02:25 AM, Chris Mason wrote:
 > ftp.suse.com/pub/people/mason/patches/data-logging has a new
 > beta-data-logging-10.diff.gz
 >
 > This one:
 >
 > compiles as a module (thanks Jeff Mahoney)
 >
 > has a highmem bug fixed with data=journal (help from Andrea)
 >
 > Fixes a problem with relocated/different sized journal devices.  This
 > was a merge error on my part.
 >
 > One tester reports a hang that I can't reproduce here, I'm still trying.
 >
 > -chris
 >
 >
 >

Many thanks to you, Chris!

I applied it on Sunday as I was on a short vacation to Hamburg.de, for
one week before, and your recent patch works very well under
noatime,notail,data=ordered on / still booted with the appropriate
rootflag. (That is my old setup.)
I remarked one difference, I didn't realize this before on ReiserFS.
Having a backup partition previously mounted with fstab settings:
"/dev/hda11  /mnt/beta   reiserfs
noauto,user,noatime,notail,data=ordered   0  0",
I was not able to execute any executable. So it should be normally as I
understand "mount" and its manpage. I had to add a "exec," to the mount
options. Now it works and [this] is right?! ;-)

Is the rootflag needed any more? Does your patchset work with all
pending ReiserFS patches together, now? (Yes, I remember your and your
teams words on 2.4.20-pre...) I didn't try this complete combo again,
just replaced the old beta-data-logging patch in my last (posted) setup.

Last week I got my old RAM renewed (to 512MB again) and some of my old
problems disappeared for now (boot-up overheating symptoms, Oleg told me
about, that are quite assumable (warranty time exceeded)). The load
timings I posted stay the same, so, my aa-akpm-rml patch setup was
really somekind of poor.

Mmh. Now I don't have the old crash pattern any more ;-)
Maybe I push my notebooks power button "accidentally", soon, to test...


Best wishes and greetings,

Manuel

Re: Re: [PATCH CFT] tons of logging patches

[Chris Mason]

Hi everyone,

I've updated data logging patch, to fix a problem where
reiserfs_writepage didn't properly unlock pages in data logging mode.

The new code will be on
ftp.suse.com/pub/people/mason/patches/data-logging once the mirror
updates.

If you've already got 03-beta-data-logging-10, you only need the
incremental fix below.

-chris

--- 1.33/fs/reiserfs/inode.c	Fri Jun 28 09:34:33 2002
+++ edited/fs/reiserfs/inode.c	Fri Jun 28 09:37:42 2002
@@ -2034,6 +2034,7 @@
     }
     if (reiserfs_active_handle(&th)) {
         journal_end(&th, sb, nr) ;
+        UnlockPage(page);
     }
 }