2 drive dropout (and raid 5), simultaneous, after 3 years

2 drive dropout (and raid 5), simultaneous, after 3 years

[Michael Stumpf]

I've got a an LVM cobbled together of 2 RAID-5 md's.  For the longest 
time I was running with 3 promise cards and surviving everything 
including the occasional drive failure, then suddenly I had double drive 
dropouts and the array would go into a degraded state.

10 drives in the system, Linux 2.4.22, Slackware 9, mdadm v1.2.0 (13 mar 
2003)

I started to diagnose; fdisk -l /dev/hdi  returned nothing for the two 
failed drives, but "dmesg" reports that the drives are happy, and that 
the md would have been automounted if not for a mismatch on the event 
counters (of the 2 failed drives).

I assumed that this had something to do with my semi-nonstandard 
application of a zillion (3) promise cards in 1 system, but I never had 
this problem before.  I ripped out the promise cards and stuck in 3ware 
5700s, cleaning it up a bit and also putting a single drive per ATA 
channel.  Two weeks later, the same problem crops up again.

The "problematic" drives are even mixed; 1 is WD, 1 is Maxtor (both 120gig).

Is this a known bug in 2.4.22 or mdadm 1.2.0?  Suggestions?


--------------------------------------------
My mailbox is spam-free with ChoiceMail, the leader in personal and corporate anti-spam solutions. Download your free copy of ChoiceMail from www.choicemailfree.com

RE: 2 drive dropout (and raid 5), simultaneous, after 3 years

[Guy]

What failure are you getting?  I assume a read error.  md will fail a drive
when it gets a read error from the drive.  It is "normal" to have a read
error once in a while, but more than 1 a year may indicate a drive going
bad.

I test my drives with this command:
dd if=/dev/hdi of=/dev/null bs=64k

You may look into using "smartd".  It monitors and tests disks for problems.
However, my dd test finds them first.  smartd has never told me anything
useful, but my drives are old, and are not smart enough for smartd.

Guy

-----Original Message-----
From: linux-raid-owner@vger.kernel.org
[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Michael Stumpf
Sent: Wednesday, December 08, 2004 4:03 PM
To: linux-raid@vger.kernel.org
Subject: 2 drive dropout (and raid 5), simultaneous, after 3 years


I've got a an LVM cobbled together of 2 RAID-5 md's.  For the longest 
time I was running with 3 promise cards and surviving everything 
including the occasional drive failure, then suddenly I had double drive 
dropouts and the array would go into a degraded state.

10 drives in the system, Linux 2.4.22, Slackware 9, mdadm v1.2.0 (13 mar 
2003)

I started to diagnose; fdisk -l /dev/hdi  returned nothing for the two 
failed drives, but "dmesg" reports that the drives are happy, and that 
the md would have been automounted if not for a mismatch on the event 
counters (of the 2 failed drives).

I assumed that this had something to do with my semi-nonstandard 
application of a zillion (3) promise cards in 1 system, but I never had 
this problem before.  I ripped out the promise cards and stuck in 3ware 
5700s, cleaning it up a bit and also putting a single drive per ATA 
channel.  Two weeks later, the same problem crops up again.

The "problematic" drives are even mixed; 1 is WD, 1 is Maxtor (both 120gig).

Is this a known bug in 2.4.22 or mdadm 1.2.0?  Suggestions?


--------------------------------------------
My mailbox is spam-free with ChoiceMail, the leader in personal and
corporate anti-spam solutions. Download your free copy of ChoiceMail from
www.choicemailfree.com

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: 2 drive dropout (and raid 5), simultaneous, after 3 years

[Michael Stumpf]

No idea what failure is occuring.  Your dd test, run from begin to end 
of each drive, completed fine.  Smartd had no info to report.

The fdisk weirdness was operator error; the /dev/sd* block nodes were 
missing (forgotten detail on age old upgrade).  Fixed with mknod.

So, I forced mdadm to assemble and it is reconstructing now.  
Troublesome, though, that 2 drives fail at once like this.  I think I 
should separate them to different raid-5s, just incase.



Guy wrote:

>What failure are you getting?  I assume a read error.  md will fail a drive
>when it gets a read error from the drive.  It is "normal" to have a read
>error once in a while, but more than 1 a year may indicate a drive going
>bad.
>
>I test my drives with this command:
>dd if=/dev/hdi of=/dev/null bs=64k
>
>You may look into using "smartd".  It monitors and tests disks for problems.
>However, my dd test finds them first.  smartd has never told me anything
>useful, but my drives are old, and are not smart enough for smartd.
>
>Guy
>
>-----Original Message-----
>From: linux-raid-owner@vger.kernel.org
>[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Michael Stumpf
>Sent: Wednesday, December 08, 2004 4:03 PM
>To: linux-raid@vger.kernel.org
>Subject: 2 drive dropout (and raid 5), simultaneous, after 3 years
>
>
>I've got a an LVM cobbled together of 2 RAID-5 md's.  For the longest 
>time I was running with 3 promise cards and surviving everything 
>including the occasional drive failure, then suddenly I had double drive 
>dropouts and the array would go into a degraded state.
>
>10 drives in the system, Linux 2.4.22, Slackware 9, mdadm v1.2.0 (13 mar 
>2003)
>
>I started to diagnose; fdisk -l /dev/hdi  returned nothing for the two 
>failed drives, but "dmesg" reports that the drives are happy, and that 
>the md would have been automounted if not for a mismatch on the event 
>counters (of the 2 failed drives).
>
>I assumed that this had something to do with my semi-nonstandard 
>application of a zillion (3) promise cards in 1 system, but I never had 
>this problem before.  I ripped out the promise cards and stuck in 3ware 
>5700s, cleaning it up a bit and also putting a single drive per ATA 
>channel.  Two weeks later, the same problem crops up again.
>
>The "problematic" drives are even mixed; 1 is WD, 1 is Maxtor (both 120gig).
>
>Is this a known bug in 2.4.22 or mdadm 1.2.0?  Suggestions?
>
>
>--------------------------------------------
>My mailbox is spam-free with ChoiceMail, the leader in personal and
>corporate anti-spam solutions. Download your free copy of ChoiceMail from
>www.choicemailfree.com
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>  
>


--------------------------------------------
My mailbox is spam-free with ChoiceMail, the leader in personal and corporate anti-spam solutions. Download your free copy of ChoiceMail from www.choicemailfree.com

RE: 2 drive dropout (and raid 5), simultaneous, after 3 years

[Guy]

You should have some sort of md error in your logs.  Try this command:
grep "md:" /var/log/messages*|more

Yes, they don't play well together, so separate them!  :)

Guy

-----Original Message-----
From: linux-raid-owner@vger.kernel.org
[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Michael Stumpf
Sent: Wednesday, December 08, 2004 11:46 PM
To: linux-raid@vger.kernel.org
Subject: Re: 2 drive dropout (and raid 5), simultaneous, after 3 years

No idea what failure is occuring.  Your dd test, run from begin to end 
of each drive, completed fine.  Smartd had no info to report.

The fdisk weirdness was operator error; the /dev/sd* block nodes were 
missing (forgotten detail on age old upgrade).  Fixed with mknod.

So, I forced mdadm to assemble and it is reconstructing now.  
Troublesome, though, that 2 drives fail at once like this.  I think I 
should separate them to different raid-5s, just incase.



Guy wrote:

>What failure are you getting?  I assume a read error.  md will fail a drive
>when it gets a read error from the drive.  It is "normal" to have a read
>error once in a while, but more than 1 a year may indicate a drive going
>bad.
>
>I test my drives with this command:
>dd if=/dev/hdi of=/dev/null bs=64k
>
>You may look into using "smartd".  It monitors and tests disks for
problems.
>However, my dd test finds them first.  smartd has never told me anything
>useful, but my drives are old, and are not smart enough for smartd.
>
>Guy
>
>-----Original Message-----
>From: linux-raid-owner@vger.kernel.org
>[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Michael Stumpf
>Sent: Wednesday, December 08, 2004 4:03 PM
>To: linux-raid@vger.kernel.org
>Subject: 2 drive dropout (and raid 5), simultaneous, after 3 years
>
>
>I've got a an LVM cobbled together of 2 RAID-5 md's.  For the longest 
>time I was running with 3 promise cards and surviving everything 
>including the occasional drive failure, then suddenly I had double drive 
>dropouts and the array would go into a degraded state.
>
>10 drives in the system, Linux 2.4.22, Slackware 9, mdadm v1.2.0 (13 mar 
>2003)
>
>I started to diagnose; fdisk -l /dev/hdi  returned nothing for the two 
>failed drives, but "dmesg" reports that the drives are happy, and that 
>the md would have been automounted if not for a mismatch on the event 
>counters (of the 2 failed drives).
>
>I assumed that this had something to do with my semi-nonstandard 
>application of a zillion (3) promise cards in 1 system, but I never had 
>this problem before.  I ripped out the promise cards and stuck in 3ware 
>5700s, cleaning it up a bit and also putting a single drive per ATA 
>channel.  Two weeks later, the same problem crops up again.
>
>The "problematic" drives are even mixed; 1 is WD, 1 is Maxtor (both
120gig).
>
>Is this a known bug in 2.4.22 or mdadm 1.2.0?  Suggestions?
>
>
>--------------------------------------------
>My mailbox is spam-free with ChoiceMail, the leader in personal and
>corporate anti-spam solutions. Download your free copy of ChoiceMail from
>www.choicemailfree.com
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>  
>


--------------------------------------------
My mailbox is spam-free with ChoiceMail, the leader in personal and
corporate anti-spam solutions. Download your free copy of ChoiceMail from
www.choicemailfree.com

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: 2 drive dropout (and raid 5), simultaneous, after 3 years

[Michael Stumpf]

All I see is this:

Apr 14 22:03:56 drown kernel: scsi: device set offline - not ready or 
command retry failed after host reset: host 1 channel 0 id 2 lun 0
Apr 14 22:03:56 drown kernel: scsi: device set offline - not ready or 
command retry failed after host reset: host 1 channel 0 id 3 lun 0
Apr 14 22:03:56 drown kernel: md: updating md1 RAID superblock on device
Apr 14 22:03:56 drown kernel: md: (skipping faulty sdj1 )
Apr 14 22:03:56 drown kernel: md: (skipping faulty sdi1 )
Apr 14 22:03:56 drown kernel: md: sdh1 [events: 000000b5]<6>(write) 
sdh1's sb offset: 117186944
Apr 14 22:03:56 drown kernel: md: sdg1 [events: 000000b5]<6>(write) 
sdg1's sb offset: 117186944
Apr 14 22:03:56 drown kernel: md: recovery thread got woken up ...
Apr 14 22:03:56 drown kernel: md: recovery thread finished ...

What the heck could that be?  Can that possibly be related to the fact 
that there weren't proper block device nodes sitting in the filesystem?!

I already ran WD's wonky tool to fix their "DMA timeout" problem, and 
one of the drives is a maxtor.  They're on separate ATA cables, and I've 
got about 5 drives per power supply.  I checked heat, and it wasn't very 
high.

Any other sources of information I could tap?  Maybe an "MD debug" 
setting in the kernel with a recompile?

Guy wrote:

>You should have some sort of md error in your logs.  Try this command:
>grep "md:" /var/log/messages*|more
>
>Yes, they don't play well together, so separate them!  :)
>
>Guy
>
>-----Original Message-----
>From: linux-raid-owner@vger.kernel.org
>[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Michael Stumpf
>Sent: Wednesday, December 08, 2004 11:46 PM
>To: linux-raid@vger.kernel.org
>Subject: Re: 2 drive dropout (and raid 5), simultaneous, after 3 years
>
>No idea what failure is occuring.  Your dd test, run from begin to end 
>of each drive, completed fine.  Smartd had no info to report.
>
>The fdisk weirdness was operator error; the /dev/sd* block nodes were 
>missing (forgotten detail on age old upgrade).  Fixed with mknod.
>
>So, I forced mdadm to assemble and it is reconstructing now.  
>Troublesome, though, that 2 drives fail at once like this.  I think I 
>should separate them to different raid-5s, just incase.
>
>
>
>Guy wrote:
>
>  
>
>>What failure are you getting?  I assume a read error.  md will fail a drive
>>when it gets a read error from the drive.  It is "normal" to have a read
>>error once in a while, but more than 1 a year may indicate a drive going
>>bad.
>>
>>I test my drives with this command:
>>dd if=/dev/hdi of=/dev/null bs=64k
>>
>>You may look into using "smartd".  It monitors and tests disks for
>>    
>>
>problems.
>  
>
>>However, my dd test finds them first.  smartd has never told me anything
>>useful, but my drives are old, and are not smart enough for smartd.
>>
>>Guy
>>
>>-----Original Message-----
>>From: linux-raid-owner@vger.kernel.org
>>[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Michael Stumpf
>>Sent: Wednesday, December 08, 2004 4:03 PM
>>To: linux-raid@vger.kernel.org
>>Subject: 2 drive dropout (and raid 5), simultaneous, after 3 years
>>
>>
>>I've got a an LVM cobbled together of 2 RAID-5 md's.  For the longest 
>>time I was running with 3 promise cards and surviving everything 
>>including the occasional drive failure, then suddenly I had double drive 
>>dropouts and the array would go into a degraded state.
>>
>>10 drives in the system, Linux 2.4.22, Slackware 9, mdadm v1.2.0 (13 mar 
>>2003)
>>
>>I started to diagnose; fdisk -l /dev/hdi  returned nothing for the two 
>>failed drives, but "dmesg" reports that the drives are happy, and that 
>>the md would have been automounted if not for a mismatch on the event 
>>counters (of the 2 failed drives).
>>
>>I assumed that this had something to do with my semi-nonstandard 
>>application of a zillion (3) promise cards in 1 system, but I never had 
>>this problem before.  I ripped out the promise cards and stuck in 3ware 
>>5700s, cleaning it up a bit and also putting a single drive per ATA 
>>channel.  Two weeks later, the same problem crops up again.
>>
>>The "problematic" drives are even mixed; 1 is WD, 1 is Maxtor (both
>>    
>>
>120gig).
>  
>
>>Is this a known bug in 2.4.22 or mdadm 1.2.0?  Suggestions?
>>
>>
>>--------------------------------------------
>>My mailbox is spam-free with ChoiceMail, the leader in personal and
>>corporate anti-spam solutions. Download your free copy of ChoiceMail from
>>www.choicemailfree.com
>>
>>-
>>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>the body of a message to majordomo@vger.kernel.org
>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>-
>>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>the body of a message to majordomo@vger.kernel.org
>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>
>> 
>>
>>    
>>
>
>
>--------------------------------------------
>My mailbox is spam-free with ChoiceMail, the leader in personal and
>corporate anti-spam solutions. Download your free copy of ChoiceMail from
>www.choicemailfree.com
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>  
>


--------------------------------------------
My mailbox is spam-free with ChoiceMail, the leader in personal and corporate anti-spam solutions. Download your free copy of ChoiceMail from www.choicemailfree.com

RE: 2 drive dropout (and raid 5), simultaneous, after 3 years

[Guy]

Since they both went off line at the same time, check the power cables.  Do
they share a common power cable, or doe each have a unique cable directly
from the power supply.

Switch power connections with another drive to see if the problem stays with
the power connection.

Guy

-----Original Message-----
From: linux-raid-owner@vger.kernel.org
[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Michael Stumpf
Sent: Thursday, December 09, 2004 9:45 AM
To: Guy; linux-raid@vger.kernel.org
Subject: Re: 2 drive dropout (and raid 5), simultaneous, after 3 years

All I see is this:

Apr 14 22:03:56 drown kernel: scsi: device set offline - not ready or 
command retry failed after host reset: host 1 channel 0 id 2 lun 0
Apr 14 22:03:56 drown kernel: scsi: device set offline - not ready or 
command retry failed after host reset: host 1 channel 0 id 3 lun 0
Apr 14 22:03:56 drown kernel: md: updating md1 RAID superblock on device
Apr 14 22:03:56 drown kernel: md: (skipping faulty sdj1 )
Apr 14 22:03:56 drown kernel: md: (skipping faulty sdi1 )
Apr 14 22:03:56 drown kernel: md: sdh1 [events: 000000b5]<6>(write) 
sdh1's sb offset: 117186944
Apr 14 22:03:56 drown kernel: md: sdg1 [events: 000000b5]<6>(write) 
sdg1's sb offset: 117186944
Apr 14 22:03:56 drown kernel: md: recovery thread got woken up ...
Apr 14 22:03:56 drown kernel: md: recovery thread finished ...

What the heck could that be?  Can that possibly be related to the fact 
that there weren't proper block device nodes sitting in the filesystem?!

I already ran WD's wonky tool to fix their "DMA timeout" problem, and 
one of the drives is a maxtor.  They're on separate ATA cables, and I've 
got about 5 drives per power supply.  I checked heat, and it wasn't very 
high.

Any other sources of information I could tap?  Maybe an "MD debug" 
setting in the kernel with a recompile?

Guy wrote:

>You should have some sort of md error in your logs.  Try this command:
>grep "md:" /var/log/messages*|more
>
>Yes, they don't play well together, so separate them!  :)
>
>Guy
>
>-----Original Message-----
>From: linux-raid-owner@vger.kernel.org
>[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Michael Stumpf
>Sent: Wednesday, December 08, 2004 11:46 PM
>To: linux-raid@vger.kernel.org
>Subject: Re: 2 drive dropout (and raid 5), simultaneous, after 3 years
>
>No idea what failure is occuring.  Your dd test, run from begin to end 
>of each drive, completed fine.  Smartd had no info to report.
>
>The fdisk weirdness was operator error; the /dev/sd* block nodes were 
>missing (forgotten detail on age old upgrade).  Fixed with mknod.
>
>So, I forced mdadm to assemble and it is reconstructing now.  
>Troublesome, though, that 2 drives fail at once like this.  I think I 
>should separate them to different raid-5s, just incase.
>
>
>
>Guy wrote:
>
>  
>
>>What failure are you getting?  I assume a read error.  md will fail a
drive
>>when it gets a read error from the drive.  It is "normal" to have a read
>>error once in a while, but more than 1 a year may indicate a drive going
>>bad.
>>
>>I test my drives with this command:
>>dd if=/dev/hdi of=/dev/null bs=64k
>>
>>You may look into using "smartd".  It monitors and tests disks for
>>    
>>
>problems.
>  
>
>>However, my dd test finds them first.  smartd has never told me anything
>>useful, but my drives are old, and are not smart enough for smartd.
>>
>>Guy
>>
>>-----Original Message-----
>>From: linux-raid-owner@vger.kernel.org
>>[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Michael Stumpf
>>Sent: Wednesday, December 08, 2004 4:03 PM
>>To: linux-raid@vger.kernel.org
>>Subject: 2 drive dropout (and raid 5), simultaneous, after 3 years
>>
>>
>>I've got a an LVM cobbled together of 2 RAID-5 md's.  For the longest 
>>time I was running with 3 promise cards and surviving everything 
>>including the occasional drive failure, then suddenly I had double drive 
>>dropouts and the array would go into a degraded state.
>>
>>10 drives in the system, Linux 2.4.22, Slackware 9, mdadm v1.2.0 (13 mar 
>>2003)
>>
>>I started to diagnose; fdisk -l /dev/hdi  returned nothing for the two 
>>failed drives, but "dmesg" reports that the drives are happy, and that 
>>the md would have been automounted if not for a mismatch on the event 
>>counters (of the 2 failed drives).
>>
>>I assumed that this had something to do with my semi-nonstandard 
>>application of a zillion (3) promise cards in 1 system, but I never had 
>>this problem before.  I ripped out the promise cards and stuck in 3ware 
>>5700s, cleaning it up a bit and also putting a single drive per ATA 
>>channel.  Two weeks later, the same problem crops up again.
>>
>>The "problematic" drives are even mixed; 1 is WD, 1 is Maxtor (both
>>    
>>
>120gig).
>  
>
>>Is this a known bug in 2.4.22 or mdadm 1.2.0?  Suggestions?
>>
>>
>>--------------------------------------------
>>My mailbox is spam-free with ChoiceMail, the leader in personal and
>>corporate anti-spam solutions. Download your free copy of ChoiceMail from
>>www.choicemailfree.com
>>
>>-
>>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>the body of a message to majordomo@vger.kernel.org
>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>-
>>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>the body of a message to majordomo@vger.kernel.org
>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>
>> 
>>
>>    
>>
>
>
>--------------------------------------------
>My mailbox is spam-free with ChoiceMail, the leader in personal and
>corporate anti-spam solutions. Download your free copy of ChoiceMail from
>www.choicemailfree.com
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>  
>


--------------------------------------------
My mailbox is spam-free with ChoiceMail, the leader in personal and
corporate anti-spam solutions. Download your free copy of ChoiceMail from
www.choicemailfree.com

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: 2 drive dropout (and raid 5), simultaneous, after 3 years

[Michael Stumpf]

Ahhhhhhh.. You're on to something here.  In all my years of ghetto raid 
one of the weakest things I've seen is the Y-molex-power-splitters.  Do 
you know where more solid ones can be found?  I'm to the point where I'd 
pay $10 or more for the bloody things if they didnt blink the power 
connection when moved a little bit.

I'll bet good money this is what happened.  Maybe I need to break out 
the soldering iron, but that's kind of an ugly, proprietary, and slow 
solution.



Guy wrote:

>Since they both went off line at the same time, check the power cables.  Do
>they share a common power cable, or doe each have a unique cable directly
>from the power supply.
>
>Switch power connections with another drive to see if the problem stays with
>the power connection.
>
>Guy
>
>-----Original Message-----
>From: linux-raid-owner@vger.kernel.org
>[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Michael Stumpf
>Sent: Thursday, December 09, 2004 9:45 AM
>To: Guy; linux-raid@vger.kernel.org
>Subject: Re: 2 drive dropout (and raid 5), simultaneous, after 3 years
>
>All I see is this:
>
>Apr 14 22:03:56 drown kernel: scsi: device set offline - not ready or 
>command retry failed after host reset: host 1 channel 0 id 2 lun 0
>Apr 14 22:03:56 drown kernel: scsi: device set offline - not ready or 
>command retry failed after host reset: host 1 channel 0 id 3 lun 0
>Apr 14 22:03:56 drown kernel: md: updating md1 RAID superblock on device
>Apr 14 22:03:56 drown kernel: md: (skipping faulty sdj1 )
>Apr 14 22:03:56 drown kernel: md: (skipping faulty sdi1 )
>Apr 14 22:03:56 drown kernel: md: sdh1 [events: 000000b5]<6>(write) 
>sdh1's sb offset: 117186944
>Apr 14 22:03:56 drown kernel: md: sdg1 [events: 000000b5]<6>(write) 
>sdg1's sb offset: 117186944
>Apr 14 22:03:56 drown kernel: md: recovery thread got woken up ...
>Apr 14 22:03:56 drown kernel: md: recovery thread finished ...
>
>What the heck could that be?  Can that possibly be related to the fact 
>that there weren't proper block device nodes sitting in the filesystem?!
>
>I already ran WD's wonky tool to fix their "DMA timeout" problem, and 
>one of the drives is a maxtor.  They're on separate ATA cables, and I've 
>got about 5 drives per power supply.  I checked heat, and it wasn't very 
>high.
>
>Any other sources of information I could tap?  Maybe an "MD debug" 
>setting in the kernel with a recompile?
>
>Guy wrote:
>
>  
>
>>You should have some sort of md error in your logs.  Try this command:
>>grep "md:" /var/log/messages*|more
>>
>>Yes, they don't play well together, so separate them!  :)
>>
>>Guy
>>
>>-----Original Message-----
>>From: linux-raid-owner@vger.kernel.org
>>[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Michael Stumpf
>>Sent: Wednesday, December 08, 2004 11:46 PM
>>To: linux-raid@vger.kernel.org
>>Subject: Re: 2 drive dropout (and raid 5), simultaneous, after 3 years
>>
>>No idea what failure is occuring.  Your dd test, run from begin to end 
>>of each drive, completed fine.  Smartd had no info to report.
>>
>>The fdisk weirdness was operator error; the /dev/sd* block nodes were 
>>missing (forgotten detail on age old upgrade).  Fixed with mknod.
>>
>>So, I forced mdadm to assemble and it is reconstructing now.  
>>Troublesome, though, that 2 drives fail at once like this.  I think I 
>>should separate them to different raid-5s, just incase.
>>
>>
>>
>>Guy wrote:
>>
>> 
>>
>>    
>>
>>>What failure are you getting?  I assume a read error.  md will fail a
>>>      
>>>
>drive
>  
>
>>>when it gets a read error from the drive.  It is "normal" to have a read
>>>error once in a while, but more than 1 a year may indicate a drive going
>>>bad.
>>>
>>>I test my drives with this command:
>>>dd if=/dev/hdi of=/dev/null bs=64k
>>>
>>>You may look into using "smartd".  It monitors and tests disks for
>>>   
>>>
>>>      
>>>
>>problems.
>> 
>>
>>    
>>
>>>However, my dd test finds them first.  smartd has never told me anything
>>>useful, but my drives are old, and are not smart enough for smartd.
>>>
>>>Guy
>>>
>>>-----Original Message-----
>>>From: linux-raid-owner@vger.kernel.org
>>>[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Michael Stumpf
>>>Sent: Wednesday, December 08, 2004 4:03 PM
>>>To: linux-raid@vger.kernel.org
>>>Subject: 2 drive dropout (and raid 5), simultaneous, after 3 years
>>>
>>>
>>>I've got a an LVM cobbled together of 2 RAID-5 md's.  For the longest 
>>>time I was running with 3 promise cards and surviving everything 
>>>including the occasional drive failure, then suddenly I had double drive 
>>>dropouts and the array would go into a degraded state.
>>>
>>>10 drives in the system, Linux 2.4.22, Slackware 9, mdadm v1.2.0 (13 mar 
>>>2003)
>>>
>>>I started to diagnose; fdisk -l /dev/hdi  returned nothing for the two 
>>>failed drives, but "dmesg" reports that the drives are happy, and that 
>>>the md would have been automounted if not for a mismatch on the event 
>>>counters (of the 2 failed drives).
>>>
>>>I assumed that this had something to do with my semi-nonstandard 
>>>application of a zillion (3) promise cards in 1 system, but I never had 
>>>this problem before.  I ripped out the promise cards and stuck in 3ware 
>>>5700s, cleaning it up a bit and also putting a single drive per ATA 
>>>channel.  Two weeks later, the same problem crops up again.
>>>
>>>The "problematic" drives are even mixed; 1 is WD, 1 is Maxtor (both
>>>   
>>>
>>>      
>>>
>>120gig).
>> 
>>
>>    
>>
>>>Is this a known bug in 2.4.22 or mdadm 1.2.0?  Suggestions?
>>>
>>>
>>>--------------------------------------------
>>>My mailbox is spam-free with ChoiceMail, the leader in personal and
>>>corporate anti-spam solutions. Download your free copy of ChoiceMail from
>>>www.choicemailfree.com
>>>
>>>-
>>>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>>the body of a message to majordomo@vger.kernel.org
>>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>>-
>>>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>>the body of a message to majordomo@vger.kernel.org
>>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>>
>>>
>>>
>>>   
>>>
>>>      
>>>
>>--------------------------------------------
>>My mailbox is spam-free with ChoiceMail, the leader in personal and
>>corporate anti-spam solutions. Download your free copy of ChoiceMail from
>>www.choicemailfree.com
>>
>>-
>>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>the body of a message to majordomo@vger.kernel.org
>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>-
>>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>the body of a message to majordomo@vger.kernel.org
>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>
>> 
>>
>>    
>>
>
>
>--------------------------------------------
>My mailbox is spam-free with ChoiceMail, the leader in personal and
>corporate anti-spam solutions. Download your free copy of ChoiceMail from
>www.choicemailfree.com
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>  
>


--------------------------------------------
My mailbox is spam-free with ChoiceMail, the leader in personal and corporate anti-spam solutions. Download your free copy of ChoiceMail from www.choicemailfree.com

Re: 2 drive dropout (and raid 5), simultaneous, after 3 years

[Doug Ledford]

On Thu, 2004-12-09 at 11:22 -0600, Michael Stumpf wrote:
> Ahhhhhhh.. You're on to something here.  In all my years of ghetto raid 
> one of the weakest things I've seen is the Y-molex-power-splitters.  Do 
> you know where more solid ones can be found?  I'm to the point where I'd 
> pay $10 or more for the bloody things if they didnt blink the power 
> connection when moved a little bit.
> 
> I'll bet good money this is what happened.  Maybe I need to break out 
> the soldering iron, but that's kind of an ugly, proprietary, and slow 
> solution.

Well, that is usually overkill anyway ;-)  I've solved this problem in
the past by simply getting out a pair of thin nose needle nose pliers
and crimping down on the connector's actual grip points.  Once I
tightened up the grip spots on the Y connector, the problem went away.

> 
> 
> Guy wrote:
> 
> >Since they both went off line at the same time, check the power cables.  Do
> >they share a common power cable, or doe each have a unique cable directly
> >from the power supply.
> >
> >Switch power connections with another drive to see if the problem stays with
> >the power connection.
> >
> >Guy
> >
> >-----Original Message-----
> >From: linux-raid-owner@vger.kernel.org
> >[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Michael Stumpf
> >Sent: Thursday, December 09, 2004 9:45 AM
> >To: Guy; linux-raid@vger.kernel.org
> >Subject: Re: 2 drive dropout (and raid 5), simultaneous, after 3 years
> >
> >All I see is this:
> >
> >Apr 14 22:03:56 drown kernel: scsi: device set offline - not ready or 
> >command retry failed after host reset: host 1 channel 0 id 2 lun 0
> >Apr 14 22:03:56 drown kernel: scsi: device set offline - not ready or 
> >command retry failed after host reset: host 1 channel 0 id 3 lun 0
> >Apr 14 22:03:56 drown kernel: md: updating md1 RAID superblock on device
> >Apr 14 22:03:56 drown kernel: md: (skipping faulty sdj1 )
> >Apr 14 22:03:56 drown kernel: md: (skipping faulty sdi1 )
> >Apr 14 22:03:56 drown kernel: md: sdh1 [events: 000000b5]<6>(write) 
> >sdh1's sb offset: 117186944
> >Apr 14 22:03:56 drown kernel: md: sdg1 [events: 000000b5]<6>(write) 
> >sdg1's sb offset: 117186944
> >Apr 14 22:03:56 drown kernel: md: recovery thread got woken up ...
> >Apr 14 22:03:56 drown kernel: md: recovery thread finished ...
> >
> >What the heck could that be?  Can that possibly be related to the fact 
> >that there weren't proper block device nodes sitting in the filesystem?!
> >
> >I already ran WD's wonky tool to fix their "DMA timeout" problem, and 
> >one of the drives is a maxtor.  They're on separate ATA cables, and I've 
> >got about 5 drives per power supply.  I checked heat, and it wasn't very 
> >high.
> >
> >Any other sources of information I could tap?  Maybe an "MD debug" 
> >setting in the kernel with a recompile?
> >
> >Guy wrote:
> >
> >  
> >
> >>You should have some sort of md error in your logs.  Try this command:
> >>grep "md:" /var/log/messages*|more
> >>
> >>Yes, they don't play well together, so separate them!  :)
> >>
> >>Guy
> >>
> >>-----Original Message-----
> >>From: linux-raid-owner@vger.kernel.org
> >>[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Michael Stumpf
> >>Sent: Wednesday, December 08, 2004 11:46 PM
> >>To: linux-raid@vger.kernel.org
> >>Subject: Re: 2 drive dropout (and raid 5), simultaneous, after 3 years
> >>
> >>No idea what failure is occuring.  Your dd test, run from begin to end 
> >>of each drive, completed fine.  Smartd had no info to report.
> >>
> >>The fdisk weirdness was operator error; the /dev/sd* block nodes were 
> >>missing (forgotten detail on age old upgrade).  Fixed with mknod.
> >>
> >>So, I forced mdadm to assemble and it is reconstructing now.  
> >>Troublesome, though, that 2 drives fail at once like this.  I think I 
> >>should separate them to different raid-5s, just incase.
> >>
> >>
> >>
> >>Guy wrote:
> >>
> >> 
> >>
> >>    
> >>
> >>>What failure are you getting?  I assume a read error.  md will fail a
> >>>      
> >>>
> >drive
> >  
> >
> >>>when it gets a read error from the drive.  It is "normal" to have a read
> >>>error once in a while, but more than 1 a year may indicate a drive going
> >>>bad.
> >>>
> >>>I test my drives with this command:
> >>>dd if=/dev/hdi of=/dev/null bs=64k
> >>>
> >>>You may look into using "smartd".  It monitors and tests disks for
> >>>   
> >>>
> >>>      
> >>>
> >>problems.
> >> 
> >>
> >>    
> >>
> >>>However, my dd test finds them first.  smartd has never told me anything
> >>>useful, but my drives are old, and are not smart enough for smartd.
> >>>
> >>>Guy
> >>>
> >>>-----Original Message-----
> >>>From: linux-raid-owner@vger.kernel.org
> >>>[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Michael Stumpf
> >>>Sent: Wednesday, December 08, 2004 4:03 PM
> >>>To: linux-raid@vger.kernel.org
> >>>Subject: 2 drive dropout (and raid 5), simultaneous, after 3 years
> >>>
> >>>
> >>>I've got a an LVM cobbled together of 2 RAID-5 md's.  For the longest 
> >>>time I was running with 3 promise cards and surviving everything 
> >>>including the occasional drive failure, then suddenly I had double drive 
> >>>dropouts and the array would go into a degraded state.
> >>>
> >>>10 drives in the system, Linux 2.4.22, Slackware 9, mdadm v1.2.0 (13 mar 
> >>>2003)
> >>>
> >>>I started to diagnose; fdisk -l /dev/hdi  returned nothing for the two 
> >>>failed drives, but "dmesg" reports that the drives are happy, and that 
> >>>the md would have been automounted if not for a mismatch on the event 
> >>>counters (of the 2 failed drives).
> >>>
> >>>I assumed that this had something to do with my semi-nonstandard 
> >>>application of a zillion (3) promise cards in 1 system, but I never had 
> >>>this problem before.  I ripped out the promise cards and stuck in 3ware 
> >>>5700s, cleaning it up a bit and also putting a single drive per ATA 
> >>>channel.  Two weeks later, the same problem crops up again.
> >>>
> >>>The "problematic" drives are even mixed; 1 is WD, 1 is Maxtor (both
> >>>   
> >>>
> >>>      
> >>>
> >>120gig).
> >> 
> >>
> >>    
> >>
> >>>Is this a known bug in 2.4.22 or mdadm 1.2.0?  Suggestions?
> >>>
> >>>
> >>>--------------------------------------------
> >>>My mailbox is spam-free with ChoiceMail, the leader in personal and
> >>>corporate anti-spam solutions. Download your free copy of ChoiceMail from
> >>>www.choicemailfree.com
> >>>
> >>>-
> >>>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> >>>the body of a message to majordomo@vger.kernel.org
> >>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >>>
> >>>-
> >>>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> >>>the body of a message to majordomo@vger.kernel.org
> >>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >>>
> >>>
> >>>
> >>>
> >>>   
> >>>
> >>>      
> >>>
> >>--------------------------------------------
> >>My mailbox is spam-free with ChoiceMail, the leader in personal and
> >>corporate anti-spam solutions. Download your free copy of ChoiceMail from
> >>www.choicemailfree.com
> >>
> >>-
> >>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> >>the body of a message to majordomo@vger.kernel.org
> >>More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >>
> >>-
> >>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> >>the body of a message to majordomo@vger.kernel.org
> >>More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >>
> >>
> >> 
> >>
> >>    
> >>
> >
> >
> >--------------------------------------------
> >My mailbox is spam-free with ChoiceMail, the leader in personal and
> >corporate anti-spam solutions. Download your free copy of ChoiceMail from
> >www.choicemailfree.com
> >
> >-
> >To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> >the body of a message to majordomo@vger.kernel.org
> >More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
> >
> >  
> >
> 
> 
> --------------------------------------------
> My mailbox is spam-free with ChoiceMail, the leader in personal and corporate anti-spam solutions. Download your free copy of ChoiceMail from www.choicemailfree.com
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
-- 
  Doug Ledford <dledford@redhat.com>
         Red Hat, Inc.
         1801 Varsity Dr.
         Raleigh, NC 27606

Re: 2 drive dropout (and raid 5), simultaneous, after 3 years

[Michael Stumpf]

Good advice.  I was actually about to post something I just found--at 
Fry's, Antec appears to be selling Y power connectors at the typical 
price, $2 each (2 per packet, $4/packet).  These however are different.  
They are more solid and connect more "rigidly" than anything I've bought 
in the last 4 years, and they are setup not so much as a "Y" but as an 
"M" in a "daisy chaining" fashion (  male --------female--------female ).

>
> Absolutely ideal for the kind of nonsense we are pursuing.
>
>
>
> Doug Ledford wrote:
>
>> On Thu, 2004-12-09 at 11:22 -0600, Michael Stumpf wrote:
>>  
>>
>>> Ahhhhhhh.. You're on to something here.  In all my years of ghetto 
>>> raid one of the weakest things I've seen is the 
>>> Y-molex-power-splitters.  Do you know where more solid ones can be 
>>> found?  I'm to the point where I'd pay $10 or more for the bloody 
>>> things if they didnt blink the power connection when moved a little 
>>> bit.
>>>
>>> I'll bet good money this is what happened.  Maybe I need to break 
>>> out the soldering iron, but that's kind of an ugly, proprietary, and 
>>> slow solution.
>>>   
>>
>>
>> Well, that is usually overkill anyway ;-)  I've solved this problem in
>> the past by simply getting out a pair of thin nose needle nose pliers
>> and crimping down on the connector's actual grip points.  Once I
>> tightened up the grip spots on the Y connector, the problem went away.
>>
>>  
>>
>>> Guy wrote:
>>>
>>>   
>>>
>>>> Since they both went off line at the same time, check the power 
>>>> cables.  Do
>>>> they share a common power cable, or doe each have a unique cable 
>>>> directly
>>>>     
>>>> from the power supply.
>>>
>>>   
>>>
>>>> Switch power connections with another drive to see if the problem 
>>>> stays with
>>>> the power connection.
>>>>
>>>> Guy
>>>>
>>>> -----Original Message-----
>>>> From: linux-raid-owner@vger.kernel.org
>>>> [mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Michael Stumpf
>>>> Sent: Thursday, December 09, 2004 9:45 AM
>>>> To: Guy; linux-raid@vger.kernel.org
>>>> Subject: Re: 2 drive dropout (and raid 5), simultaneous, after 3 years
>>>>
>>>> All I see is this:
>>>>
>>>> Apr 14 22:03:56 drown kernel: scsi: device set offline - not ready 
>>>> or command retry failed after host reset: host 1 channel 0 id 2 lun 0
>>>> Apr 14 22:03:56 drown kernel: scsi: device set offline - not ready 
>>>> or command retry failed after host reset: host 1 channel 0 id 3 lun 0
>>>> Apr 14 22:03:56 drown kernel: md: updating md1 RAID superblock on 
>>>> device
>>>> Apr 14 22:03:56 drown kernel: md: (skipping faulty sdj1 )
>>>> Apr 14 22:03:56 drown kernel: md: (skipping faulty sdi1 )
>>>> Apr 14 22:03:56 drown kernel: md: sdh1 [events: 000000b5]<6>(write) 
>>>> sdh1's sb offset: 117186944
>>>> Apr 14 22:03:56 drown kernel: md: sdg1 [events: 000000b5]<6>(write) 
>>>> sdg1's sb offset: 117186944
>>>> Apr 14 22:03:56 drown kernel: md: recovery thread got woken up ...
>>>> Apr 14 22:03:56 drown kernel: md: recovery thread finished ...
>>>>
>>>> What the heck could that be?  Can that possibly be related to the 
>>>> fact that there weren't proper block device nodes sitting in the 
>>>> filesystem?!
>>>>
>>>> I already ran WD's wonky tool to fix their "DMA timeout" problem, 
>>>> and one of the drives is a maxtor.  They're on separate ATA cables, 
>>>> and I've got about 5 drives per power supply.  I checked heat, and 
>>>> it wasn't very high.
>>>>
>>>> Any other sources of information I could tap?  Maybe an "MD debug" 
>>>> setting in the kernel with a recompile?
>>>>
>>>> Guy wrote:
>>>>
>>>>
>>>>
>>>>     
>>>>
>>>>> You should have some sort of md error in your logs.  Try this 
>>>>> command:
>>>>> grep "md:" /var/log/messages*|more
>>>>>
>>>>> Yes, they don't play well together, so separate them!  :)
>>>>>
>>>>> Guy
>>>>>
>>>>> -----Original Message-----
>>>>> From: linux-raid-owner@vger.kernel.org
>>>>> [mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Michael Stumpf
>>>>> Sent: Wednesday, December 08, 2004 11:46 PM
>>>>> To: linux-raid@vger.kernel.org
>>>>> Subject: Re: 2 drive dropout (and raid 5), simultaneous, after 3 
>>>>> years
>>>>>
>>>>> No idea what failure is occuring.  Your dd test, run from begin to 
>>>>> end of each drive, completed fine.  Smartd had no info to report.
>>>>>
>>>>> The fdisk weirdness was operator error; the /dev/sd* block nodes 
>>>>> were missing (forgotten detail on age old upgrade).  Fixed with 
>>>>> mknod.
>>>>>
>>>>> So, I forced mdadm to assemble and it is reconstructing now.  
>>>>> Troublesome, though, that 2 drives fail at once like this.  I 
>>>>> think I should separate them to different raid-5s, just incase.
>>>>>
>>>>>
>>>>>
>>>>> Guy wrote:
>>>>>
>>>>>
>>>>>
>>>>>  
>>>>>       
>>>>>
>>>>>> What failure are you getting?  I assume a read error.  md will 
>>>>>> fail a
>>>>>>    
>>>>>>         
>>>>>
>>>> drive
>>>>
>>>>
>>>>     
>>>>
>>>>>> when it gets a read error from the drive.  It is "normal" to have 
>>>>>> a read
>>>>>> error once in a while, but more than 1 a year may indicate a 
>>>>>> drive going
>>>>>> bad.
>>>>>>
>>>>>> I test my drives with this command:
>>>>>> dd if=/dev/hdi of=/dev/null bs=64k
>>>>>>
>>>>>> You may look into using "smartd".  It monitors and tests disks for
>>>>>>  
>>>>>>
>>>>>>    
>>>>>>         
>>>>>
>>>>> problems.
>>>>>
>>>>>
>>>>>  
>>>>>       
>>>>>
>>>>>> However, my dd test finds them first.  smartd has never told me 
>>>>>> anything
>>>>>> useful, but my drives are old, and are not smart enough for smartd.
>>>>>>
>>>>>> Guy
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: linux-raid-owner@vger.kernel.org
>>>>>> [mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Michael 
>>>>>> Stumpf
>>>>>> Sent: Wednesday, December 08, 2004 4:03 PM
>>>>>> To: linux-raid@vger.kernel.org
>>>>>> Subject: 2 drive dropout (and raid 5), simultaneous, after 3 years
>>>>>>
>>>>>>
>>>>>> I've got a an LVM cobbled together of 2 RAID-5 md's.  For the 
>>>>>> longest time I was running with 3 promise cards and surviving 
>>>>>> everything including the occasional drive failure, then suddenly 
>>>>>> I had double drive dropouts and the array would go into a 
>>>>>> degraded state.
>>>>>>
>>>>>> 10 drives in the system, Linux 2.4.22, Slackware 9, mdadm v1.2.0 
>>>>>> (13 mar 2003)
>>>>>>
>>>>>> I started to diagnose; fdisk -l /dev/hdi  returned nothing for 
>>>>>> the two failed drives, but "dmesg" reports that the drives are 
>>>>>> happy, and that the md would have been automounted if not for a 
>>>>>> mismatch on the event counters (of the 2 failed drives).
>>>>>>
>>>>>> I assumed that this had something to do with my semi-nonstandard 
>>>>>> application of a zillion (3) promise cards in 1 system, but I 
>>>>>> never had this problem before.  I ripped out the promise cards 
>>>>>> and stuck in 3ware 5700s, cleaning it up a bit and also putting a 
>>>>>> single drive per ATA channel.  Two weeks later, the same problem 
>>>>>> crops up again.
>>>>>>
>>>>>> The "problematic" drives are even mixed; 1 is WD, 1 is Maxtor (both
>>>>>>  
>>>>>>
>>>>>>    
>>>>>>         
>>>>>
>>>>> 120gig).
>>>>>
>>>>>
>>>>>  
>>>>>       
>>>>>
>>>>>> Is this a known bug in 2.4.22 or mdadm 1.2.0?  Suggestions?
>>>>>>
>>>>>>
>>>>>> --------------------------------------------
>>>>>> My mailbox is spam-free with ChoiceMail, the leader in personal and
>>>>>> corporate anti-spam solutions. Download your free copy of 
>>>>>> ChoiceMail from
>>>>>> www.choicemailfree.com
>>>>>>
>>>>>> -
>>>>>> To unsubscribe from this list: send the line "unsubscribe 
>>>>>> linux-raid" in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>>
>>>>>> -
>>>>>> To unsubscribe from this list: send the line "unsubscribe 
>>>>>> linux-raid" in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>  
>>>>>>
>>>>>>    
>>>>>>         
>>>>>
>>>>> --------------------------------------------
>>>>> My mailbox is spam-free with ChoiceMail, the leader in personal and
>>>>> corporate anti-spam solutions. Download your free copy of 
>>>>> ChoiceMail from
>>>>> www.choicemailfree.com
>>>>>
>>>>> -
>>>>> To unsubscribe from this list: send the line "unsubscribe 
>>>>> linux-raid" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>
>>>>> -
>>>>> To unsubscribe from this list: send the line "unsubscribe 
>>>>> linux-raid" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>  
>>>>>       
>>>>
>>>> --------------------------------------------
>>>> My mailbox is spam-free with ChoiceMail, the leader in personal and
>>>> corporate anti-spam solutions. Download your free copy of 
>>>> ChoiceMail from
>>>> www.choicemailfree.com
>>>>
>>>> -
>>>> To unsubscribe from this list: send the line "unsubscribe 
>>>> linux-raid" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>
>>>>
>>>>
>>>>
>>>>     
>>>
>>> --------------------------------------------
>>> My mailbox is spam-free with ChoiceMail, the leader in personal and 
>>> corporate anti-spam solutions. Download your free copy of ChoiceMail 
>>> from www.choicemailfree.com
>>>
>>> -
>>> To unsubscribe from this list: send the line "unsubscribe 
>>> linux-raid" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>   
>>
>
>


--------------------------------------------
My mailbox is spam-free with ChoiceMail, the leader in personal and corporate anti-spam solutions. Download your free copy of ChoiceMail from www.choicemailfree.com