Re:

[Phil Turmel <philip@turmel.org>]

On 06/09/2011 08:16 AM, Dragon wrote:
> Yes if all things get back to normal i will change to raid6. that was my idea for the future too.
> here the result of the script:
> 
> ./lsdrv
> **Warning** The following utility(ies) failed to execute:
>   pvs
>   lvs
> Some information may be missing.
> 
> PCI [pata_atiixp] 00:14.1 IDE interface: ATI Technologies Inc SB700/SB800 IDE Controller
>  ââscsi 0:0:0:0 ATA SAMSUNG HD154UI {S1XWJ1WZ401747}
>  â  ââsda: [8:0] MD raid5 (none/13) 1.36t md0 inactive spare {975d6eb2-285e-ed11-021d-f236c2d05073}
>  â     ââmd0: [9:0] Empty/Unknown 0.00k
>  ââscsi 0:0:1:0 ATA SAMSUNG HD154UI {S1XWJ1WZ405098}
>  â  ââsdb: [8:16] MD raid5 (none/13) 1.36t md0 inactive spare {975d6eb2-285e-ed11-021d-f236c2d05073}
>  ââscsi 1:0:0:0 ATA SAMSUNG SV2044D {0244J1BN626842}
>     ââsdc: [8:32] Partitioned (dos) 19.01g
>        ââsdc1: [8:33] (ext3) 18.17g {6858fc38-9fee-4ab5-8135-029f305b9198}
>        â  ââMounted as /dev/disk/by-uuid/6858fc38-9fee-4ab5-8135-029f305b9198 @ /
>        ââsdc2: [8:34] Partitioned (dos) 1.00k
>        ââsdc5: [8:37] (swap) 854.99m {f67c7f23-e5ac-4c05-992c-a9a494687026}
> PCI [sata_mv] 02:00.0 SCSI storage controller: Marvell Technology Group Ltd. 88SX7042 PCI-e 4-port SATA-II (rev 02)
>  ââscsi 2:0:0:0 ATA SAMSUNG HD154UI {S1XWJD2Z907626}
>  â  ââsdd: [8:48] MD raid5 (none/13) 1.36t md0 inactive spare {975d6eb2-285e-ed11-021d-f236c2d05073}
>  ââscsi 4:0:0:0 ATA SAMSUNG HD154UI {S1XWJ90ZA03442}
>  â  ââsde: [8:64] MD raid5 (none/13) 1.36t md0 inactive spare {975d6eb2-285e-ed11-021d-f236c2d05073}
>  ââscsi 6:0:0:0 ATA SAMSUNG HD154UI {S1XWJ9AB200390}
>  â  ââsdf: [8:80] MD raid5 (none/13) 1.36t md0 inactive spare {975d6eb2-285e-ed11-021d-f236c2d05073}
>  ââscsi 8:0:0:0 ATA SAMSUNG HD154UI {61833B761A63RP}
>     ââsdg: [8:96] MD raid5 (none/13) 1.36t md0 inactive spare {975d6eb2-285e-ed11-021d-f236c2d05073}
> PCI [sata_promise] 04:02.0 Mass storage controller: Promise Technology, Inc. PDC40718 (SATA 300 TX4) (rev 02)
>  ââscsi 3:0:0:0 ATA SAMSUNG HD154UI {S1XWJD5B201174}
>  â  ââsdh: [8:112] MD raid5 (none/13) 1.36t md0 inactive spare {975d6eb2-285e-ed11-021d-f236c2d05073}
>  ââscsi 5:0:0:0 ATA SAMSUNG HD154UI {S1XWJ9CB201815}
>  â  ââsdi: [8:128] MD raid5 (none/13) 1.36t md0 inactive spare {975d6eb2-285e-ed11-021d-f236c2d05073}
>  ââscsi 7:x:x:x [Empty]
>  ââscsi 9:0:0:0 ATA SAMSUNG HD154UI {A6311B761A3XPB}
>     ââsdj: [8:144] MD raid5 (none/13) 1.36t md0 inactive spare {975d6eb2-285e-ed11-021d-f236c2d05073}
> PCI [ahci] 00:11.0 SATA controller: ATI Technologies Inc SB700/SB800 SATA Controller [IDE mode]
>  ââscsi 10:0:0:0 ATA SAMSUNG HD154UI {S1XWJ1KS915803}
>  â  ââsdk: [8:160] MD raid5 (none/13) 1.36t md0 inactive spare {975d6eb2-285e-ed11-021d-f236c2d05073}
>  ââscsi 11:0:0:0 ATA SAMSUNG HD154UI {S1XWJ1KS915802}
>  â  ââsdl: [8:176] MD raid5 (none/13) 1.36t md0 inactive spare {975d6eb2-285e-ed11-021d-f236c2d05073}
>  ââscsi 12:0:0:0 ATA SAMSUNG HD154UI {S1XWJ1KSC08024}
>  â  ââsdm: [8:192] MD raid5 (none/13) 1.36t md0 inactive spare {975d6eb2-285e-ed11-021d-f236c2d05073}
>  ââscsi 13:0:0:0 ATA SAMSUNG HD154UI {S1XWJ1KS915804}
>     ââsdn: [8:208] MD raid5 (13) 1.36t inactive {975d6eb2-285e-ed11-021d-f236c2d05073}
> 

Very interesting.  You've exposed a limitation of my script.  I'll have to reconsider how I extract information from members of a partially started array.

Its also clear that you are using a fast-boot kernel with parallel probing of your scsi hosts.  That's why your device names sometimes change.

/dev/sdn is definitely the holdout, though.  Notice the "(13)" where the others are "(none/13)".

Before continuing, I've made the assumption that "mdadm --grow -n 12" was the last major operation attempted, and this is was put you in your current predicament?  If so, and you interrupted it, did you try to assemble the array with the --backup-file option from the shrink operation?  If you didn't, please stop the array, and retry the assemble (with all 13 devices) and the --backup-file option.  Try twice, if needed, adding "--force" the second time.

If that works, sit tight until the reshape is complete.

If that was already tried, or doesn't change the situation, here's what I recommend:

Stop the array: "mdadm -S /dev/md0"

Recreate the array "mdadm -C /dev/md0 -l 5 -n 13 -e 0.90 -c 64 --assume-clean /dev/sd{k,d,l,m,a,b,e,n,f,g,h,i,j}"

The order in {} matters! The option "--assume-clean" is vital!

You will be warned that the members appear to be part of another array.  Continue.

Do *NOT* mount the array!

Try a non-destructive fsck: "fsck -n /dev/md0"

If that has a huge number of errors, stop the array, and recreate again, swapping /dev/sdd and /dev/sdn, then repeat the fsck:

"mdadm -C /dev/md0 -l 5 -n 13 -e 0.90 -c 64 --assume-clean /dev/sd{k,n,l,m,a,b,e,d,f,g,h,i,j}"

If you get a good, or mostly good fsck, you've found the right combination, and you can try the shrink operations again.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html