2.2 kernel - Ext3 & Raid patches

2.2 kernel - Ext3 & Raid patches

[Jon Hedlund]

Last September Stephan told someone on the linux-kernel list that 
Ext3 and Raid 1 didn't work together on the 2.2 kernel. 
Has this been fixed or have I just been lucky?  I've been using ext3 
on a Raid 1 array of two IBM 75GB ide drives with kernel 2.2.19.  
Three times in the last 9 months one of the drives reported errors 
and dropped offline, each time I have fdisked the bad drive, 
formatted it, fsck'ed it and found no problems, fdisked it again, and 
raidhotadd'ed it back in and it restored the array without problems.
Two questions:
1. Besides the faulty drive, is my data in danger from this software 
configuration and I've just been lucky or would this configuration not 
trigger the problems Stephan was warning about?
2. What is the "proper" fix for the patch collision between the raid 
patch and the ext3 patch in /include/linux/fs.h? I've just been 
changing the line
#define BH_LowPrio 8
to
#define BH_LowPrio 5
around line 198, it's been working but I don't know enough about the 
code to know if that might mess something else up or not work 
under some conditions.
Thanks,
JonH

Re: 2.2 kernel - Ext3 & Raid patches

[Andreas Dilger]

On May 21, 2002  16:40 -0500, Jon Hedlund wrote:
> Last September Stephan told someone on the linux-kernel list that 
> Ext3 and Raid 1 didn't work together on the 2.2 kernel. 
> Has this been fixed or have I just been lucky?

You've just been lucky.  I forget the exact scenario, but it is
something like if journal replay is happening while the RAID is being 
reconstructed after a crash you can get garbage written to your disk.

> Three times in the last 9 months one of the drives reported errors 
> and dropped offline, each time I have fdisked the bad drive, 
> formatted it, fsck'ed it and found no problems, fdisked it again, and 
> raidhotadd'ed it back in and it restored the array without problems.

This is probably a matter of block remapping replacing bad sectors when
you try to write to it.

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

Re: 2.2 kernel - Ext3 & Raid patches

[Stephen C. Tweedie]

Hi,

On Tue, May 21, 2002 at 07:21:33PM -0600, Andreas Dilger wrote:
> On May 21, 2002  16:40 -0500, Jon Hedlund wrote:
> > Last September Stephan told someone on the linux-kernel list that 
> > Ext3 and Raid 1 didn't work together on the 2.2 kernel. 
> > Has this been fixed or have I just been lucky?
> 
> You've just been lucky.  I forget the exact scenario, but it is
> something like if journal replay is happening while the RAID is being 
> reconstructed after a crash you can get garbage written to your disk.

Right --- the raid resync code in 2.2 uses the normal buffer cache,
which results in writes being scheduled for clean buffers, behind
ext3's back.  That's not allowed --- it violates the write ordering
requirements that make ext3 work, and trips up debugging assert
failures in the ext3 write checking code.

You might get away with it, but a raid resync on ext3 on 2.2 is
basically not safe.  If you wait until after the resync before
mounting the ext3 filesystem, you'll be OK.

It should work on 2.4.

Cheers,
 Stephen

Re: 2.2 kernel - Ext3 & Raid patches

[Mike Fedyk]

On Tue, May 21, 2002 at 04:40:06PM -0500, Jon Hedlund wrote:
> Last September Stephan told someone on the linux-kernel list that 
> Ext3 and Raid 1 didn't work together on the 2.2 kernel.

I believe that was me.

> Has this been fixed or have I just been lucky?  I've been using ext3 
> on a Raid 1 array of two IBM 75GB ide drives with kernel 2.2.19.  
> Three times in the last 9 months one of the drives reported errors 
> and dropped offline, each time I have fdisked the bad drive, 
> formatted it, fsck'ed it and found no problems, fdisked it again, and 
> raidhotadd'ed it back in and it restored the array without problems.

That test probably won't find the hard drive error, if there is one.

Take the drive offline, run a badblocks write test (the -w option -- danger
it will destroy the data on the disk, but you have a copy on the other RAID
mirror so only run it one one drive directly, not your /dev/md device file)

I doubt that it would be a remapping problem because the drive usually only
reports errors after all of the extra blocks are used up.  

Badblocks does a linear write four times.  If you hear the drive seeking
during those writes there is a good chance that it has remapped some blocks
and the drive will fail sooner than later.

> Two questions:
> 1. Besides the faulty drive, is my data in danger from this software 
> configuration and I've just been lucky or would this configuration not 
> trigger the problems Stephan was warning about?

Yes, avoid this configuration.

> 2. What is the "proper" fix for the patch collision between the raid
> patch and the ext3 patch in /include/linux/fs.h? 

Use 2.4.

> around line 198, it's been working but I don't know enough about the
> code to know if that might mess something else up or not work 
> under some conditions.

If I were you, I'd just test a 2.4 kernel on the configuration you want.
Unless there is some binary driver that use that doesn't support 2.4 there
isn't much use staying with 2.2.

This configuration is unsafe for 2.2, and I've used raid1 and raid5 with
ext3 without any trouble, even on degraded arrays (for as short a period as
possible of course).

Re: 2.2 kernel - Ext3 & Raid patches

[Tomas Szepe]

> > 2. What is the "proper" fix for the patch collision between the raid
> > patch and the ext3 patch in /include/linux/fs.h? 
> 
> Use 2.4.

Impossible on sparc32 on account of the lurking SRMMU bug.
(See yesterday's post by Joris Braakman <jorisb@nl.euro.net>.)


T.

Re: 2.2 kernel - Ext3 & Raid patches

[David S. Miller]

   From: Tomas Szepe <szepe@pinerecords.com>
   Date: Thu, 23 May 2002 09:02:44 +0200

   > > 2. What is the "proper" fix for the patch collision between the raid
   > > patch and the ext3 patch in /include/linux/fs.h? 
   > 
   > Use 2.4.
   
   Impossible on sparc32 on account of the lurking SRMMU bug.
   (See yesterday's post by Joris Braakman <jorisb@nl.euro.net>.)
   
There have been several patches posted to deal with that
problem, you can apply them yourself or grab Marcelo's
current 2.4.x BK tree.

Re: 2.2 kernel - Ext3 & Raid patches

[Tomas Szepe]

>    From: Tomas Szepe <szepe@pinerecords.com>
>    Date: Thu, 23 May 2002 09:02:44 +0200
> 
>    > > 2. What is the "proper" fix for the patch collision between the raid
>    > > patch and the ext3 patch in /include/linux/fs.h? 
>    > 
>    > Use 2.4.
>    
>    Impossible on sparc32 on account of the lurking SRMMU bug.
>    (See yesterday's post by Joris Braakman <jorisb@nl.euro.net>.)
>    
> There have been several patches posted to deal with that
> problem

Splendid. Where have they been posted?


T.

Re: 2.2 kernel - Ext3 & Raid patches

[Tomas Szepe]

[-- Attachment #1: Type: text/plain, Size: 945 bytes --]

Hi,


>    From: Tomas Szepe <szepe@pinerecords.com>
>    Date: Thu, 23 May 2002 09:02:44 +0200
> 
>    > > 2. What is the "proper" fix for the patch collision between the raid
>    > > patch and the ext3 patch in /include/linux/fs.h? 
>    > 
>    > Use 2.4.
>    
>    Impossible on sparc32 on account of the lurking SRMMU bug.
>    (See yesterday's post by Joris Braakman <jorisb@nl.euro.net>.)
>    
> There have been several patches posted to deal with that
> problem, you can apply them yourself or grab Marcelo's
> current 2.4.x BK tree.


Here comes for all sparc people who can't install BK:

All sparc32/sparc64 related changes since 2.4.19-pre8 in one diff
copied and fixed up by hand from
http://linux.bkbits.net:8080/linux-2.4/ChangeSet@-3w?nav=index.html

All I can claim as to the patched kernel's functionality --
it has compiled for me on sparc32. I'm going to try to boot
it next week when I'm changing disks in my server.

T.

[-- Attachment #2: patch-2.4.19-pre8-sparcfixes-upto020523-1.gz --]
[-- Type: application/x-gunzip, Size: 6688 bytes --]

Re: 2.2 kernel - Ext3 & Raid patches

[Stephen C. Tweedie]

Hi,

On Wed, May 22, 2002 at 06:11:44PM -0700, Mike Fedyk wrote:
> On Tue, May 21, 2002 at 04:40:06PM -0500, Jon Hedlund wrote:
> > 2. What is the "proper" fix for the patch collision between the raid
> > patch and the ext3 patch in /include/linux/fs.h? 
> 
> Use 2.4.

Actually, you just need to renumber one of the conflicting #defines to
something unused, and it will work fine.  Soft raid0 or linear mode
will work quite happily with ext3 on 2.2 after you do that, it's only
the resync after a crash that you get with raid1 or raid5 that is
dangerous.

Cheers,
 Stephen

Re: 2.2 kernel - Ext3 & Raid patches

[Herman Oosthuysen]

Stephen,

Does this mean that Ext3 is still not recommended for use with RAID1?

Thanks,
--
Herman Oosthuysen
Herman@WirelessNetworksInc.com
Suite 300, #3016, 5th Ave NE,
Calgary, Alberta, T2A 6K4, Canada
Phone: (403) 569-5687, Fax: (403) 235-3965
----- Original Message -----
From: Stephen C. Tweedie <sct@redhat.com>
To: Jon Hedlund <JH_ML@invtools.com>; <sct@redhat.com>; <akpm@zip.com.au>;
<linux-kernel@vger.kernel.org>
Sent: Thursday, May 23, 2002 2:49 AM
Subject: Re: 2.2 kernel - Ext3 & Raid patches


> Hi,
>
> On Wed, May 22, 2002 at 06:11:44PM -0700, Mike Fedyk wrote:
> > On Tue, May 21, 2002 at 04:40:06PM -0500, Jon Hedlund wrote:
> > > 2. What is the "proper" fix for the patch collision between the raid
> > > patch and the ext3 patch in /include/linux/fs.h?
> >
> > Use 2.4.
>
> Actually, you just need to renumber one of the conflicting #defines to
> something unused, and it will work fine.  Soft raid0 or linear mode
> will work quite happily with ext3 on 2.2 after you do that, it's only
> the resync after a crash that you get with raid1 or raid5 that is
> dangerous.
>
> Cheers,
>  Stephen
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: 2.2 kernel - Ext3 & Raid patches

[Stephen C. Tweedie]

Hi,

On Thu, May 23, 2002 at 08:52:04AM -0600, Herman Oosthuysen wrote:
 
> Does this mean that Ext3 is still not recommended for use with RAID1?

On 2.2, yes.  On 2.4, it should work just fine: the 2.4 raid layer is
much better behaved, and does not try to perform resync via the buffer
cache.

Cheers,
 Stephen