Re: Re: [PATCH CFT] tons of logging patches

Re: Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 06/06/2002 02:09 AM, Chris Mason wrote:
> Ok, ftp.suse.com/pub/people/mason/patches/data-logging has my latest
> updates, which have more optimizations for many threads all doing
> synchronous transactions.
> 
> -chris
> 

Mhh. I was just starting to try the ftp, but  no-go-for-now.  O.k now 
I've got an old Santana CD in... "Black magic ..." Thanks to Chris,

bye, very nice to listen to,

Manuel

Re: Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 06/06/2002 02:09 AM, Chris Mason wrote:
> Ok, ftp.suse.com/pub/people/mason/patches/data-logging has my latest
> updates, which have more optimizations for many threads all doing
> synchronous transactions.
> 
> -chris
> 

Mmh. I recompiled with the new one when it appeared on the server and 
kept the kernel append line with the obvious problem not to be able to 
easily reboot to my ext2 maintenance partition.

I had some disappearing files/dirs from /lib/modules/* and from 
somewhere within /var/* the second time now. First occurrence is about 6 
days ago. That takes action within a good uptime (I mean: not at startup 
or upon special programs' interaction).

Jun 12 20:52:48 firehead kernel: is_leaf: item location seems wrong 
(second one): *3.6* [5273 5274 0x1 IND], item_len 4, item_location 2120, 
free_space(entry_count) 0
Jun 12 20:52:48 firehead kernel: vs-5150: search_by_key: invalid format 
found in block 8422. Fsck?
Jun 12 20:52:48 firehead kernel: vs-13070: reiserfs_read_inode2: i/o 
failure occurred trying to find stat data of [5199 261934 0x0 SD]
Jun 12 20:52:48 firehead kernel: is_leaf: item location seems wrong 
(second one): *3.6* [5273 5274 0x1 IND], item_len 4, item_location 2120, 
free_space(entry_count) 0
Jun 12 20:52:48 firehead kernel: vs-5150: search_by_key: invalid format 
found in block 8422. Fsck?
[...] <- many many more
Jun 12 20:54:46 firehead kernel: vs-13070: reiserfs_read_inode2: i/o 
failure occurred trying to find stat data of [5199 261934 0x0 SD]
Jun 12 20:54:46 firehead modprobe: modprobe: Can't open dependencies 
file /lib/modules/2.4.19-pre9/modules.dep (Permission denied)


I then needed to do a reiserfsck --rebuild-tree, as suggested by most 
recent reiserfsck (3.x.1c-pre4), the 3.x.1b-one said, it was fixable by 
--fix-fixable, but I then decided to use the latest. And then I had to 
adjust the lost&found manually. -- Huh, that took a long time, but 
nothing's missing, if I experience my system correctly now.

My system/partitions still look like the recently posted setup:
  Kernel.2.4.19-pre9
  ReiserFS.pending.01..03,05 (rest doesn't cooperate cleanly)
  ReiserFS.mason.01..03 (most recent)
  rml.preempt-kernel (maybe slightly modified for this setup)

/ mounted -o noatime,notail,data=ordered
and from applied lilo.conf: append = "... rootflags=data=ordered"


Thanks, but the term "speedup" seems to be relative for me for now,

Manuel

Re: Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 06/17/2002 02:47 AM, Chris Mason wrote:
> On Wed, 2002-06-12 at 18:32, Manuel Krause wrote:
> 
>>On 06/06/2002 02:09 AM, Chris Mason wrote:
>>
>>>Ok, ftp.suse.com/pub/people/mason/patches/data-logging has my latest
>>>updates, which have more optimizations for many threads all doing
>>>synchronous transactions.
>>>
>>>-chris
>>>
>>
>>Mmh. I recompiled with the new one when it appeared on the server and 
>>kept the kernel append line with the obvious problem not to be able to 
>>easily reboot to my ext2 maintenance partition.
> 
> 
> Hmmm, I might have found it, but this isn't a new bug to the 07 patch. 
> Have you been able to reproduce, or have you fallen back to the older
> patches?
> 

You mean 03-beta-data-logging-7.diff.gz when saying "07 patch"? Then I 
didn't go back according to my disk content.

When I boot up I read on screen and in boot.msg:
  ...
  EXT2-fs: Unrecognized mount option data
  reiserfs: using ordered data mode
  reiserfs: found format "3.6" with standard journal
  reiserfs: checking transaction log (ide0(3,7)) for (ide0(3,7))
  (...)
  Using r5 hash to sort names
  ...

Addon is: I use the ReiserFS pending patches that apply as I mentionned 
previously(?).

No, it looks o.k. here.


> -chris
> 

Manuel

Re: Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Wed, 2002-06-12 at 18:32, Manuel Krause wrote:
> On 06/06/2002 02:09 AM, Chris Mason wrote:
> > Ok, ftp.suse.com/pub/people/mason/patches/data-logging has my latest
> > updates, which have more optimizations for many threads all doing
> > synchronous transactions.
> > 
> > -chris
> > 
> 
> Mmh. I recompiled with the new one when it appeared on the server and 
> kept the kernel append line with the obvious problem not to be able to 
> easily reboot to my ext2 maintenance partition.

Hmmm, I might have found it, but this isn't a new bug to the 07 patch. 
Have you been able to reproduce, or have you fallen back to the older
patches?

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Sun, 2002-06-16 at 20:31, Manuel Krause wrote:

> > 
> > Hmmm, I might have found it, but this isn't a new bug to the 07 patch. 
> > Have you been able to reproduce, or have you fallen back to the older
> > patches?
> > 
> 
> You mean 03-beta-data-logging-7.diff.gz when saying "07 patch"? Then I 
> didn't go back according to my disk content.
> 

I've uploaded 03-beta-data-logging-8.diff.gz.  It will fix the oops Ed
hit, and hopefully your corruption problem as well.

-chris

Re: [PATCH CFT] tons of logging patches

[Dieter Nützel]

On Friday, Chris wrote:

> ftp.suse.com/pub/people/mason/patches/data-logging has a new
> beta-data-logging-10.diff.gz
>
> This one:
>
> compiles as a module (thanks Jeff Mahoney)
>
> has a highmem bug fixed with data=journal (help from Andrea)

Hello Chris,

should I run this with -aa kernel?

> Fixes a problem with relocated/different sized journal devices.  This
> was a merge error on my part.
>
> One tester reports a hang that I can't reproduce here, I'm still trying.

You do not mean me (page coloring)?

Regards,
	Dieter

BTW I have my dual Athlon MP 1900+ up and "flying"...
-- 
Dieter Nützel
Graduate Student, Computer Science

University of Hamburg
Department of Computer Science
@home: Dieter.Nuetzel@hamburg.de

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Sat, 2002-06-22 at 16:02, Dieter Nützel wrote:
> On Friday, Chris wrote:
> 
> > ftp.suse.com/pub/people/mason/patches/data-logging has a new
> > beta-data-logging-10.diff.gz
> >
> > This one:
> >
> > compiles as a module (thanks Jeff Mahoney)
> >
> > has a highmem bug fixed with data=journal (help from Andrea)
> 
> Hello Chris,
> 
> should I run this with -aa kernel?

You can, the differences are minor, just interdiff from the last data
logging patch and apply the incremental onto the aa data logging patch. 
I ran out of time before leaving for ottawa, so I didn't get the chance
to make an aa patch.

> 
> > Fixes a problem with relocated/different sized journal devices.  This
> > was a merge error on my part.
> >
> > One tester reports a hang that I can't reproduce here, I'm still trying.
> 
> You do not mean me (page coloring)?

No, this was a large 8 way smp machine.  

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 06/21/2002 02:25 AM, Chris Mason wrote:
 > ftp.suse.com/pub/people/mason/patches/data-logging has a new
 > beta-data-logging-10.diff.gz
 >
 > This one:
 >
 > compiles as a module (thanks Jeff Mahoney)
 >
 > has a highmem bug fixed with data=journal (help from Andrea)
 >
 > Fixes a problem with relocated/different sized journal devices.  This
 > was a merge error on my part.
 >
 > One tester reports a hang that I can't reproduce here, I'm still trying.
 >
 > -chris
 >
 >
 >

Many thanks to you, Chris!

I applied it on Sunday as I was on a short vacation to Hamburg.de, for
one week before, and your recent patch works very well under
noatime,notail,data=ordered on / still booted with the appropriate
rootflag. (That is my old setup.)
I remarked one difference, I didn't realize this before on ReiserFS.
Having a backup partition previously mounted with fstab settings:
"/dev/hda11  /mnt/beta   reiserfs
noauto,user,noatime,notail,data=ordered   0  0",
I was not able to execute any executable. So it should be normally as I
understand "mount" and its manpage. I had to add a "exec," to the mount
options. Now it works and [this] is right?! ;-)

Is the rootflag needed any more? Does your patchset work with all
pending ReiserFS patches together, now? (Yes, I remember your and your
teams words on 2.4.20-pre...) I didn't try this complete combo again,
just replaced the old beta-data-logging patch in my last (posted) setup.

Last week I got my old RAM renewed (to 512MB again) and some of my old
problems disappeared for now (boot-up overheating symptoms, Oleg told me
about, that are quite assumable (warranty time exceeded)). The load
timings I posted stay the same, so, my aa-akpm-rml patch setup was
really somekind of poor.

Mmh. Now I don't have the old crash pattern any more ;-)
Maybe I push my notebooks power button "accidentally", soon, to test...


Best wishes and greetings,

Manuel

Re: Re: [PATCH CFT] tons of logging patches

[Chris Mason]

Hi everyone,

I've updated data logging patch, to fix a problem where
reiserfs_writepage didn't properly unlock pages in data logging mode.

The new code will be on
ftp.suse.com/pub/people/mason/patches/data-logging once the mirror
updates.

If you've already got 03-beta-data-logging-10, you only need the
incremental fix below.

-chris

--- 1.33/fs/reiserfs/inode.c	Fri Jun 28 09:34:33 2002
+++ edited/fs/reiserfs/inode.c	Fri Jun 28 09:37:42 2002
@@ -2034,6 +2034,7 @@
     }
     if (reiserfs_active_handle(&th)) {
         journal_end(&th, sb, nr) ;
+        UnlockPage(page);
     }
 }
 

Re: [PATCH CFT] tons of logging patches

[Dieter Nützel]

On Monday, 8 July 2002 13:51, Chris Mason wrote:
> On Wed, 2002-07-03 at 20:28, Manuel Krause wrote:
>
> > So far the 13 prerelease works fine under rootflags boot and 
> > mount/remount option "data=ordered" on here.
>
> I think I've got the bug, it was newly introduced in -13.  New stuff
> will be uploaded today.

Where is your stuff?
It didn't appeared in your SuSE "home"
I'm during the preparation of 2.4.19-rc1-jam2-pc (2.4.19pre1aa2 + jam2 + page 
coloring).

Thanks,
	Dieter

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Tue, 2002-07-09 at 09:52, Dieter Nützel wrote:
> On Monday, 8 July 2002 13:51, Chris Mason wrote:
> > On Wed, 2002-07-03 at 20:28, Manuel Krause wrote:
> >
> > > So far the 13 prerelease works fine under rootflags boot and 
> > > mount/remount option "data=ordered" on here.
> >
> > I think I've got the bug, it was newly introduced in -13.  New stuff
> > will be uploaded today.
> 
> Where is your stuff?
> It didn't appeared in your SuSE "home"
> I'm during the preparation of 2.4.19-rc1-jam2-pc (2.4.19pre1aa2 + jam2 + page 
> coloring).

Sigh, further testing showed I'd only fixed half of the bug.  The other
half is somewhere deep writepage/truncate interaction.  I'm still trying
to find it.

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Tue, 2002-07-09 at 10:00, Chris Mason wrote:
> On Tue, 2002-07-09 at 09:52, Dieter Nützel wrote:
> > On Monday, 8 July 2002 13:51, Chris Mason wrote:
> > > On Wed, 2002-07-03 at 20:28, Manuel Krause wrote:
> > >
> > > > So far the 13 prerelease works fine under rootflags boot and 
> > > > mount/remount option "data=ordered" on here.
> > >
> > > I think I've got the bug, it was newly introduced in -13.  New stuff
> > > will be uploaded today.
> > >
> Sigh, further testing showed I'd only fixed half of the bug.  The other
> half is somewhere deep writepage/truncate interaction.  I'm still trying
> to find it.

Well, after a week of fixing minor things and adding debugging code,
this bug still goes unsolved.  The only good news is that I can also
trigger it on pure 2.4.19-rc1 and ext2.  The bug seems to either be in
the VM or my ram, so I'm having a little memtest86 party overnight.

I've uploaded 03-beta-data-logging-16, it should be more stable and
faster than -11 and -13 were, but includes more debugging checks to try
and make sure I'm not screwing things up.  You can grab the patches at:

ftp://ftp.suse.com/pub/people/mason/patches/data-logging

If anyone wants to try and reproduce my memory corruption bug (can and
probably will lead to FS corruption), just let 10 fsx processes run in a
loop for ~3 hours or so on an SMP machine.

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Manuel Krause]

On 07/12/2002 09:43 PM, Chris Mason wrote:
> On Tue, 2002-07-09 at 10:00, Chris Mason wrote:
> 
>>On Tue, 2002-07-09 at 09:52, Dieter Nützel wrote:
>>
>>>On Monday, 8 July 2002 13:51, Chris Mason wrote:
>>>
>>>>On Wed, 2002-07-03 at 20:28, Manuel Krause wrote:
>>>>
>>>>
>>>>>So far the 13 prerelease works fine under rootflags boot and 
>>>>>mount/remount option "data=ordered" on here.
>>>>
>>>>I think I've got the bug, it was newly introduced in -13.  New stuff
>>>>will be uploaded today.
>>>>
>>>
>>Sigh, further testing showed I'd only fixed half of the bug.  The other
>>half is somewhere deep writepage/truncate interaction.  I'm still trying
>>to find it.
> 
> 
> Well, after a week of fixing minor things and adding debugging code,
> this bug still goes unsolved.  The only good news is that I can also
> trigger it on pure 2.4.19-rc1 and ext2.  The bug seems to either be in
> the VM or my ram, so I'm having a little memtest86 party overnight.
> 
> I've uploaded 03-beta-data-logging-16, it should be more stable and
> faster than -11 and -13 were, but includes more debugging checks to try
> and make sure I'm not screwing things up.  You can grab the patches at:
> 
> ftp://ftp.suse.com/pub/people/mason/patches/data-logging
> 

Thank you, Chris!

It's working fine under rootflags set and mount/remount options 
"noatime,notail,data=ordered".
Nothing got wrong on here due to data-logging-13.

Netscape7 dropped its load time again by 3secs of (previously 14+1), 
OpenOffice1 increased by 1sec (of average 17secs previously). To explain 
the previous state: I've only changed -pre9 to -rc1 and data-logging 
from -10(-pre9)..13(-rc1)..11(-rc1)..16(-rc1, now!) from the old patchset.

> If anyone wants to try and reproduce my memory corruption bug (can and
> probably will lead to FS corruption), just let 10 fsx processes run in a
> loop for ~3 hours or so on an SMP machine.
> 
> -chris
> 

Is this possible to be reproduced on uniprocessor machines, too ?


Best regards,

Manuel

Re: [PATCH CFT] tons of logging patches

[Chris Mason]

Hello everyone,

I've uploaded 03-data-logging-24.diff, which fixes a tail packing oops
in data=journal mode, and fixes the data=ordered slow down (5-10%)
introduced in -23.  

-23 flushed all the ordered buffer heads with the journal lock held,
which meant no new transactions could start while the ordered buffers
were being flushed.  This was safe but slower.

-24 flushes them right before the commit blocks (like -19 did), and only
flushes the tail conversion targets with the journal lock held.  Doing
the tail conversion targets later risks losing the tail data after a
crash.

Thanks to grev@namesys.com who reported the ordered write bug in -19 and
the data=journal oops in -23.

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Fri, 2002-08-02 at 08:28, Chris Mason wrote:
> Hello everyone,
> 
> I've uploaded 03-data-logging-24.diff, which fixes a tail packing oops
> in data=journal mode, and fixes the data=ordered slow down (5-10%)
> introduced in -23.  

Whoops, I neglected to mention that data=ordered is now the default in
the patch.  You can mount with -o data=writeback if you don't want
ordered data writes.

-chris

Re: Re: [PATCH CFT] tons of logging patches

[Hans Reiser]

Chris Mason wrote:

>On Fri, 2002-08-02 at 08:28, Chris Mason wrote:
>  
>
>>Hello everyone,
>>
>>I've uploaded 03-data-logging-24.diff, which fixes a tail packing oops
>>in data=journal mode, and fixes the data=ordered slow down (5-10%)
>>introduced in -23.  
>>    
>>
>
>Whoops, I neglected to mention that data=ordered is now the default in
>the patch.  You can mount with -o data=writeback if you don't want
>ordered data writes.
>
>-chris
>
>
>
>
>  
>
we just saw a benchmark in which data=ordered caused a major slowdown. 
 Does this patch fix this?

Edward and Elena, I want thorough benchmarking of this.

-- 
Hans

Re: Re: [PATCH CFT] tons of logging patches

[Chris Mason]

On Fri, 2002-08-02 at 08:59, Hans Reiser wrote:
> >
> we just saw a benchmark in which data=ordered caused a major slowdown. 
>  Does this patch fix this?
> 
> Edward and Elena, I want thorough benchmarking of this.
> 

The last fract_tree benchmarks were on older patches, Manual Krause's
benchmark was more showing the difference between a used and a fresh
filesystem, and he was using -23 which had data=ordered performance
issues fixed in -24.

More benchmarking does need to be done, the patch isn't set in stone yet
;-)

-chris