* mdsadm -A won't assemble my array @ 2015-02-09 20:21 G. Michael Carter 2015-02-09 23:05 ` G. Michael Carter 0 siblings, 1 reply; 6+ messages in thread From: G. Michael Carter @ 2015-02-09 20:21 UTC (permalink / raw) To: linux-raid Some time last night my machine had a kernel panic. Two of the arrays didn't start up. One I managed to fix as a mdadm -E clued me in that three of the drives were ok. So I just reassembled the three and added the fourth. Then it just started no problem. My big array however I'm not so lucky. I've got a state of Raid level: 5 /dev/sdb: AA.. (state: clean) /dev/sdk: AAAA (state: active) /dev/sdo: A.AA (state: clean) /dev/sdp: A.AA (state: clean) Thus can only get two drives to match in any config. How do I get out of this mess? ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: mdsadm -A won't assemble my array 2015-02-09 20:21 mdsadm -A won't assemble my array G. Michael Carter @ 2015-02-09 23:05 ` G. Michael Carter 2015-02-10 0:13 ` Phil Turmel 0 siblings, 1 reply; 6+ messages in thread From: G. Michael Carter @ 2015-02-09 23:05 UTC (permalink / raw) To: linux-raid After doing a lot more reading... I think I'm getting down to running something like this. assemble force isn't doing much. mdadm --create --assume-clean --level=5 --verbose --chunk 512K --raid-devices=4 /dev/md3 /dev/sdb /dev/sdk /dev/sdo /dev/sdp But as per the big warning... says to check with you guys first, also need help writing the command (since it seems to be a one shot type thing) Here's the key information I think I need from the examine: Raid level: 5 Chunk Size: 512K Used Dev Size: 7813774336 /dev/sdb: AA.. (state: clean - active device 0) /dev/sdk: AAAA (state: active - active device 1) /dev/sdo: A.AA (state: clean - active device 2) /dev/sdp: A.AA (state: clean - active device 3) thanks On Mon, Feb 9, 2015 at 3:21 PM, G. Michael Carter <mikey@carterfamily.ca> wrote: > Some time last night my machine had a kernel panic. Two of the arrays > didn't start up. > > One I managed to fix as a mdadm -E clued me in that three of the > drives were ok. So I just reassembled the three and added the fourth. > Then it just started no problem. > > My big array however I'm not so lucky. > > I've got a state of > > > Raid level: 5 > /dev/sdb: AA.. (state: clean) > /dev/sdk: AAAA (state: active) > /dev/sdo: A.AA (state: clean) > /dev/sdp: A.AA (state: clean) > > Thus can only get two drives to match in any config. How do I get out > of this mess? -- G. Michael Carter Contact: H: 1-519-940-8935 | W: 1-905-267-8494 | M: 1-519-215-1869 | F: 1-519-941-0009 Google Talk: xmpp:mikeycarter1974@gmail.com ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: mdsadm -A won't assemble my array 2015-02-09 23:05 ` G. Michael Carter @ 2015-02-10 0:13 ` Phil Turmel 2015-02-10 1:36 ` G. Michael Carter 0 siblings, 1 reply; 6+ messages in thread From: Phil Turmel @ 2015-02-10 0:13 UTC (permalink / raw) To: G. Michael Carter, linux-raid Hi Michael, [Convention on kernel.org is reply-to-all, to trim replies and to bottom-post, or interleave your reply] On 02/09/2015 06:05 PM, G. Michael Carter wrote: > On Mon, Feb 9, 2015 at 3:21 PM, G. Michael Carter <mikey@carterfamily.ca> wrote: >> Some time last night my machine had a kernel panic. Two of the arrays >> didn't start up. >> >> One I managed to fix as a mdadm -E clued me in that three of the >> drives were ok. So I just reassembled the three and added the fourth. >> Then it just started no problem. >> >> My big array however I'm not so lucky. >> >> I've got a state of >> >> >> Raid level: 5 >> /dev/sdb: AA.. (state: clean) >> /dev/sdk: AAAA (state: active) >> /dev/sdo: A.AA (state: clean) >> /dev/sdp: A.AA (state: clean) Please show us *all* of your mdadm -E output for this array. Pasted inline is preferred. Also show a map of your device names versus drive serial numbers. An excerpt from "ls -l /dev/disk/by-id/" will do. You have many drives, and the kernel doesn't guarantee consistent naming. >> Thus can only get two drives to match in any config. How do I get out >> of this mess? > After doing a lot more reading... I think I'm getting down to running > something like this. assemble force isn't doing much. This throwaway line is critical. --assemble --force is the right answer to this situation, and if its not working, something else should be investigated. Do *not* use --create. Show your kernel and mdadm versions. Show the content of /proc/mdstat. Show the output of: mdadm --assemble --force --verbose /dev/mdX /dev/sd[bkop] and the tail of "dmesg" that corresponds to the above. > mdadm --create --assume-clean --level=5 --verbose --chunk 512K > --raid-devices=4 /dev/md3 /dev/sdb /dev/sdk /dev/sdo /dev/sdp > > But as per the big warning... says to check with you guys first, also > need help writing the command (since it seems to be a one shot type > thing) > > Here's the key information I think I need from the examine: > > Raid level: 5 > Chunk Size: 512K > Used Dev Size: 7813774336 > /dev/sdb: AA.. (state: clean - active device 0) > /dev/sdk: AAAA (state: active - active device 1) > /dev/sdo: A.AA (state: clean - active device 2) > /dev/sdp: A.AA (state: clean - active device 3) Oh, and this isn't nearly enough information to advise on --create, in the remote chance it turns out to be the right answer. Phil ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: mdsadm -A won't assemble my array 2015-02-10 0:13 ` Phil Turmel @ 2015-02-10 1:36 ` G. Michael Carter 2015-02-10 2:04 ` Phil Turmel 0 siblings, 1 reply; 6+ messages in thread From: G. Michael Carter @ 2015-02-10 1:36 UTC (permalink / raw) To: Phil Turmel; +Cc: linux-raid Was actually dreading the complete info question, as the machine has no CD/DVD, it is my netboot station and holds all my ISO mirrors. But turns out starting it's network interface from the emergency shell was easier than I thought. ---- uname Linux andromeda 3.16.6-203.fc20.x86_64 #1 SMP Sat Oct 25 12:44:32 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux ---- mdadm -V mdadm - v3.3 - 3rd September 2013 ---- disk-by-id total 0 lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-OCZ-AGILITY2_OCZ-12ENW740X6E8681U -> ../../sdc lrwxrwxrwx 1 root root 10 Feb 9 20:16 ata-OCZ-AGILITY2_OCZ-12ENW740X6E8681U-part1 -> ../../sdc1 lrwxrwxrwx 1 root root 10 Feb 9 20:16 ata-OCZ-AGILITY2_OCZ-12ENW740X6E8681U-part2 -> ../../sdc2 lrwxrwxrwx 1 root root 10 Feb 9 20:16 ata-OCZ-AGILITY2_OCZ-12ENW740X6E8681U-part3 -> ../../sdc3 lrwxrwxrwx 1 root root 10 Feb 9 20:16 ata-OCZ-AGILITY2_OCZ-12ENW740X6E8681U-part4 -> ../../sdc4 lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST2000DL003-9VT166_5YD5QSG3 -> ../../sda lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST2000DL003-9VT166_5YD604E0 -> ../../sde lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST3000DM001-1CH166_Z1F2H9YC -> ../../sdm lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST3000DM001-9YN166_S1F026CS -> ../../sdf lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST3000DM001-9YN166_W1F0GD7Y -> ../../sdn lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST3000DM001-9YN166_W1F0JSVP -> ../../sdh lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST33000651AS_9XK0A9AD -> ../../sdd lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST33000651AS_9XK0AV1G -> ../../sdl lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST33000651AS_9XK0N7GY -> ../../sdg lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST33000651AS_Z291009B -> ../../sdi lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST33000651AS_Z2911DKS -> ../../sdj lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST4000DM000-1F2168_W3009GE3 -> ../../sdp lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST4000DM000-1F2168_W300E08A -> ../../sdk lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST4000DM000-1F2168_Z300PYF2 -> ../../sdb lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST4000DM000-1F2168_Z300Q4YS -> ../../sdo lrwxrwxrwx 1 root root 9 Feb 9 20:16 lvm-pv-uuid-fIZX8P-yR3l-KR3t-b206-KiDM-PhpK-JwLevw -> ../../md0 lrwxrwxrwx 1 root root 9 Feb 9 20:16 lvm-pv-uuid-Uvrg7m-X8Hr-JJn9-CAlu-Lno6-K3rp-A2p6fj -> ../../md1 lrwxrwxrwx 1 root root 9 Feb 9 20:16 lvm-pv-uuid-XetEWl-bxcb-WOHF-jNIg-M10Q-xxz1-HDeCwH -> ../../md2 lrwxrwxrwx 1 root root 9 Feb 9 20:16 md-name-andromeda:0 -> ../../md0 lrwxrwxrwx 1 root root 9 Feb 9 20:16 md-name-andromeda:1 -> ../../md1 lrwxrwxrwx 1 root root 9 Feb 9 20:16 md-name-andromeda:2 -> ../../md2 lrwxrwxrwx 1 root root 9 Feb 9 20:16 md-uuid-569d52c7:91ba146a:2dc88abf:1dbd4f12 -> ../../md0 lrwxrwxrwx 1 root root 9 Feb 9 20:16 md-uuid-8b1dbda6:fc378fa5:774dcb4f:c273dca5 -> ../../md1 lrwxrwxrwx 1 root root 9 Feb 9 20:16 md-uuid-fb065d4d:c906243c:945b8291:73539d13 -> ../../md2 lrwxrwxrwx 1 root root 9 Feb 9 20:16 wwn-0x5000c5002da066d5 -> ../../sdd lrwxrwxrwx 1 root root 9 Feb 9 20:16 wwn-0x5000c5002dad7e24 -> ../../sdl lrwxrwxrwx 1 root root 9 Feb 9 20:16 wwn-0x5000c50036281493 -> ../../sdi lrwxrwxrwx 1 root root 9 Feb 9 20:16 wwn-0x5000c50036406c91 -> ../../sdj lrwxrwxrwx 1 root root 9 Feb 9 20:16 wwn-0x5000c50044338d40 -> ../../sde lrwxrwxrwx 1 root root 9 Feb 9 20:16 wwn-0x5000c50045add261 -> ../../sda lrwxrwxrwx 1 root root 9 Feb 9 20:16 wwn-0x5000c5004a1255d8 -> ../../sdf lrwxrwxrwx 1 root root 9 Feb 9 20:16 wwn-0x5000c5004ffa2772 -> ../../sdm lrwxrwxrwx 1 root root 9 Feb 9 20:16 wwn-0x5000c50050dd4721 -> ../../sdn lrwxrwxrwx 1 root root 9 Feb 9 20:16 wwn-0x5000c500510652c5 -> ../../sdh lrwxrwxrwx 1 root root 9 Feb 9 20:16 wwn-0x5000c500608925d1 -> ../../sdp lrwxrwxrwx 1 root root 9 Feb 9 20:16 wwn-0x5000c5006434f070 -> ../../sdb lrwxrwxrwx 1 root root 9 Feb 9 20:16 wwn-0x5000c50064360fff -> ../../sdo lrwxrwxrwx 1 root root 9 Feb 9 20:16 wwn-0x5e83a97f4233045f -> ../../sdc lrwxrwxrwx 1 root root 10 Feb 9 20:16 wwn-0x5e83a97f4233045f-part1 -> ../../sdc1 lrwxrwxrwx 1 root root 10 Feb 9 20:16 wwn-0x5e83a97f4233045f-part2 -> ../../sdc2 lrwxrwxrwx 1 root root 10 Feb 9 20:16 wwn-0x5e83a97f4233045f-part3 -> ../../sdc3 lrwxrwxrwx 1 root root 10 Feb 9 20:16 wwn-0x5e83a97f4233045f-part4 -> ../../sdc4 ---- mdadm -E /dev/sdb: Magic : a92b4efc Version : 1.2 Feature Map : 0x1 Array UUID : ce6de916:62aeda9c:b5688f54:f5a9249d Name : andromeda:3 (local to host andromeda) Creation Time : Tue Jul 22 16:02:30 2014 Raid Level : raid5 Raid Devices : 4 Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB) Array Size : 11720661504 (11177.69 GiB 12001.96 GB) Used Dev Size : 7813774336 (3725.90 GiB 4000.65 GB) Data Offset : 259072 sectors Super Offset : 8 sectors Unused Space : before=258984 sectors, after=3760 sectors State : clean Device UUID : e9e7af60:e1bc1c7f:107157b4:4099c48a Internal Bitmap : 8 sectors from superblock Update Time : Mon Feb 9 05:13:20 2015 Bad Block Log : 512 entries available at offset 72 sectors Checksum : 57eaf19 - correct Events : 158964 Layout : left-symmetric Chunk Size : 512K Device Role : Active device 0 Array State : AA.. ('A' == active, '.' == missing, 'R' == replacing) /dev/sdk: Magic : a92b4efc Version : 1.2 Feature Map : 0x1 Array UUID : ce6de916:62aeda9c:b5688f54:f5a9249d Name : andromeda:3 (local to host andromeda) Creation Time : Tue Jul 22 16:02:30 2014 Raid Level : raid5 Raid Devices : 4 Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB) Array Size : 11720661504 (11177.69 GiB 12001.96 GB) Used Dev Size : 7813774336 (3725.90 GiB 4000.65 GB) Data Offset : 259072 sectors Super Offset : 8 sectors Unused Space : before=258984 sectors, after=3760 sectors State : active Device UUID : ed09357e:36655c6d:b7430500:63d5e540 Internal Bitmap : 8 sectors from superblock Update Time : Mon Feb 9 05:10:53 2015 Bad Block Log : 512 entries available at offset 72 sectors Checksum : 131587ad - correct Events : 158964 Layout : left-symmetric Chunk Size : 512K Device Role : Active device 1 Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing) /dev/sdo: Magic : a92b4efc Version : 1.2 Feature Map : 0x1 Array UUID : ce6de916:62aeda9c:b5688f54:f5a9249d Name : andromeda:3 (local to host andromeda) Creation Time : Tue Jul 22 16:02:30 2014 Raid Level : raid5 Raid Devices : 4 Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB) Array Size : 11720661504 (11177.69 GiB 12001.96 GB) Used Dev Size : 7813774336 (3725.90 GiB 4000.65 GB) Data Offset : 259072 sectors Super Offset : 8 sectors Unused Space : before=258984 sectors, after=3760 sectors State : clean Device UUID : de849807:80d7f071:9909f3f2:78022d94 Internal Bitmap : 8 sectors from superblock Update Time : Mon Feb 9 05:11:55 2015 Bad Block Log : 512 entries available at offset 72 sectors Checksum : e73f681f - correct Events : 158962 Layout : left-symmetric Chunk Size : 512K Device Role : Active device 3 Array State : A.AA ('A' == active, '.' == missing, 'R' == replacing) /dev/sdp: Magic : a92b4efc Version : 1.2 Feature Map : 0x1 Array UUID : ce6de916:62aeda9c:b5688f54:f5a9249d Name : andromeda:3 (local to host andromeda) Creation Time : Tue Jul 22 16:02:30 2014 Raid Level : raid5 Raid Devices : 4 Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB) Array Size : 11720661504 (11177.69 GiB 12001.96 GB) Used Dev Size : 7813774336 (3725.90 GiB 4000.65 GB) Data Offset : 259072 sectors Super Offset : 8 sectors Unused Space : before=258984 sectors, after=3760 sectors State : clean Device UUID : 67fcd9a9:432b0c8b:178cc556:67b003b3 Internal Bitmap : 8 sectors from superblock Update Time : Mon Feb 9 05:11:55 2015 Bad Block Log : 512 entries available at offset 72 sectors Checksum : 254563d8 - correct Events : 158962 Layout : left-symmetric Chunk Size : 512K Device Role : Active device 2 Array State : A.AA ('A' == active, '.' == missing, 'R' == replacing) ---- blkid /dev/sda: UUID="8b1dbda6-fc37-8fa5-774d-cb4fc273dca5" UUID_SUB="82a7750b-74d9-c901-766e-7802b2737d3e" LABEL="andromeda:1" TYPE="linux_raid_member" /dev/sdb: UUID="ce6de916-62ae-da9c-b568-8f54f5a9249d" UUID_SUB="e9e7af60-e1bc-1c7f-1071-57b44099c48a" LABEL="andromeda:3" TYPE="linux_raid_member" /dev/sdc: PTUUID="99e8af68-ba3a-4343-9751-3217c4e3d9a1" PTTYPE="gpt" /dev/sdc1: PARTUUID="69de1013-cf8a-4131-bdab-03705decd42e" /dev/sdc2: UUID="944c019a-4fb0-434d-b998-361d90352230" TYPE="ext4" PARTLABEL="ext4" PARTUUID="cf7f5556-567e-450a-9712-e3d2bc735537" /dev/sdc3: UUID="9dbf5b08-afdd-47c5-b92b-5350dc26524c" TYPE="ext4" PARTUUID="46335bee-eea2-441a-8422-207b5c5fa44c" /dev/sdc4: UUID="7407b6b8-517f-49ca-aac0-01120d98fcdc" TYPE="swap" PARTUUID="f14bdff5-23f9-42f0-a6e3-b859076ca0bc" /dev/sdd: UUID="569d52c7-91ba-146a-2dc8-8abf1dbd4f12" UUID_SUB="9786712b-834a-47c1-4402-a4eba529e89e" LABEL="andromeda:0" TYPE="linux_raid_member" /dev/sde: UUID="8b1dbda6-fc37-8fa5-774d-cb4fc273dca5" UUID_SUB="44280e97-717e-09d5-92f5-3af95a5c7364" LABEL="andromeda:1" TYPE="linux_raid_member" /dev/sdf: UUID="569d52c7-91ba-146a-2dc8-8abf1dbd4f12" UUID_SUB="fdb635cc-f399-943e-bbb0-67809f0ac896" LABEL="andromeda:0" TYPE="linux_raid_member" /dev/sdg: UUID="fb065d4d-c906-243c-945b-829173539d13" UUID_SUB="08104eec-720d-4cae-ed0e-3ef06d4938ff" LABEL="andromeda:2" TYPE="linux_raid_member" /dev/sdh: UUID="fb065d4d-c906-243c-945b-829173539d13" UUID_SUB="49b3eafa-f4ab-2bb3-7faa-6996dfc4bb00" LABEL="andromeda:2" TYPE="linux_raid_member" /dev/sdi: UUID="fb065d4d-c906-243c-945b-829173539d13" UUID_SUB="a70e7fa1-6909-b030-e899-0f85555f0094" LABEL="andromeda:2" TYPE="linux_raid_member" /dev/sdj: UUID="fb065d4d-c906-243c-945b-829173539d13" UUID_SUB="c74f4f4f-0b4b-65d4-b8e9-90ce689678c8" LABEL="andromeda:2" TYPE="linux_raid_member" /dev/sdk: UUID="ce6de916-62ae-da9c-b568-8f54f5a9249d" UUID_SUB="ed09357e-3665-5c6d-b743-050063d5e540" LABEL="andromeda:3" TYPE="linux_raid_member" /dev/sdl: UUID="569d52c7-91ba-146a-2dc8-8abf1dbd4f12" UUID_SUB="bd04495d-ce3b-09c7-0cac-37b98f74b3a7" LABEL="andromeda:0" TYPE="linux_raid_member" /dev/sdm: UUID="569d52c7-91ba-146a-2dc8-8abf1dbd4f12" UUID_SUB="5c6e4d63-3f4d-a05e-d64e-62b339f4f767" LABEL="andromeda:0" TYPE="linux_raid_member" /dev/sdn: UUID="569d52c7-91ba-146a-2dc8-8abf1dbd4f12" UUID_SUB="1b2773d1-391b-3c78-42e0-fd405416d9e7" LABEL="andromeda:0" TYPE="linux_raid_member" /dev/sdo: UUID="ce6de916-62ae-da9c-b568-8f54f5a9249d" UUID_SUB="de849807-80d7-f071-9909-f3f278022d94" LABEL="andromeda:3" TYPE="linux_raid_member" /dev/sdp: UUID="ce6de916-62ae-da9c-b568-8f54f5a9249d" UUID_SUB="67fcd9a9-432b-0c8b-178c-c55667b003b3" LABEL="andromeda:3" TYPE="linux_raid_member" ---- dmesg [ 576.890380] md: md3 stopped. [ 576.891629] md: unbind<sdb> [ 576.900841] md: export_rdev(sdb) [ 576.902147] md: unbind<sdo> [ 576.906824] md: export_rdev(sdo) [ 576.908029] md: unbind<sdk> [ 576.911845] md: export_rdev(sdk) [ 576.913030] md: unbind<sdp> [ 576.916862] md: export_rdev(sdp) [ 585.164936] md: md3 stopped. [ 585.360571] md: bind<sdk> [ 585.360833] md: bind<sdp> [ 585.361050] md: bind<sdo> [ 585.361261] md: bind<sdb> [ 585.361294] md: md3 stopped. [ 585.361298] md: unbind<sdb> [ 585.386616] md: export_rdev(sdb) [ 585.387767] md: unbind<sdo> [ 585.398644] md: export_rdev(sdo) [ 585.399633] md: unbind<sdp> [ 585.404662] md: export_rdev(sdp) [ 585.405681] md: unbind<sdk> [ 585.410647] md: export_rdev(sdk) ---- mdadm -A output mdadm: looking for devices for /dev/md3 mdadm: /dev/sdb is identified as a member of /dev/md3, slot 0. mdadm: /dev/sdk is identified as a member of /dev/md3, slot 1. mdadm: /dev/sdo is identified as a member of /dev/md3, slot 3. mdadm: /dev/sdp is identified as a member of /dev/md3, slot 2. mdadm: added /dev/sdk to /dev/md3 as 1 mdadm: added /dev/sdp to /dev/md3 as 2 (possibly out of date) mdadm: added /dev/sdo to /dev/md3 as 3 (possibly out of date) mdadm: added /dev/sdb to /dev/md3 as 0 mdadm: /dev/md3 assembled from 2 drives - not enough to start the array. ---- /proc/mdstat Personalities : [raid6] [raid5] [raid4] [raid1] md1 : active raid1 sda[0] sde[1] 1953383488 blocks super 1.2 [2/2] [UU] bitmap: 0/15 pages [0KB], 65536KB chunk md0 : active raid5 sdl[0] sdn[3] sdd[5] sdm[1] 8790405120 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] [UUUU] bitmap: 0/22 pages [0KB], 65536KB chunk md2 : active raid6 sdj[0] sdi[1] sdg[3] sdh[2] 5860270080 blocks super 1.2 level 6, 512k chunk, algorithm 2 [4/4] [UUUU] bitmap: 0/22 pages [0KB], 65536KB chunk unused devices: <none> ** md3 is shutdown which is why I'm assuming it's not there. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: mdsadm -A won't assemble my array 2015-02-10 1:36 ` G. Michael Carter @ 2015-02-10 2:04 ` Phil Turmel 2015-02-10 2:48 ` G. Michael Carter 0 siblings, 1 reply; 6+ messages in thread From: Phil Turmel @ 2015-02-10 2:04 UTC (permalink / raw) To: G. Michael Carter; +Cc: linux-raid Hi Michael, On 02/09/2015 08:36 PM, G. Michael Carter wrote: > Was actually dreading the complete info question, as the machine has > no CD/DVD, it is my netboot station and holds all my ISO mirrors. But > turns out starting it's network interface from the emergency shell was > easier than I thought. Almost all good livecds can be put on a thumb drive to boot from, instead of using a real CD. I highly recommend sysrescuecd.org, FWIW. > ---- uname > Linux andromeda 3.16.6-203.fc20.x86_64 #1 SMP Sat Oct 25 12:44:32 UTC > 2014 x86_64 x86_64 x86_64 GNU/Linux Not too old, good. There been a steady stream of small bugfixes since 3.16. I'm not sure what fedora's been backporting. > ---- mdadm -V > mdadm - v3.3 - 3rd September 2013 Bugfixes to this, too. > ---- disk-by-id > lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST4000DM000-1F2168_W3009GE3 > -> ../../sdp > lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST4000DM000-1F2168_W300E08A > -> ../../sdk > lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST4000DM000-1F2168_Z300PYF2 > -> ../../sdb > lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST4000DM000-1F2168_Z300Q4YS > -> ../../sdo I think you missed the 'excerpt' part, but no harm done. Anyways, if I recall Seagate model numbering (past misery), these are green drives. Very bad for raid service. After we revive your array, you'll want to do some reading on 'timeout mismatch'.[1] (You are dangerously close to option "D" there, and option "C" is your only choice w/ green drives.) > ---- mdadm -E > /dev/sdb: > Magic : a92b4efc > Version : 1.2 > Feature Map : 0x1 > Array UUID : ce6de916:62aeda9c:b5688f54:f5a9249d > Name : andromeda:3 (local to host andromeda) > Creation Time : Tue Jul 22 16:02:30 2014 > Raid Level : raid5 > Raid Devices : 4 > > Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB) > Array Size : 11720661504 (11177.69 GiB 12001.96 GB) > Used Dev Size : 7813774336 (3725.90 GiB 4000.65 GB) > Data Offset : 259072 sectors > Super Offset : 8 sectors > Unused Space : before=258984 sectors, after=3760 sectors > State : clean > Device UUID : e9e7af60:e1bc1c7f:107157b4:4099c48a > > Internal Bitmap : 8 sectors from superblock > Update Time : Mon Feb 9 05:13:20 2015 > Bad Block Log : 512 entries available at offset 72 sectors > Checksum : 57eaf19 - correct > Events : 158964 > > Layout : left-symmetric > Chunk Size : 512K > > Device Role : Active device 0 > Array State : AA.. ('A' == active, '.' == missing, 'R' == replacing) > /dev/sdk: > Magic : a92b4efc > Version : 1.2 > Feature Map : 0x1 > Array UUID : ce6de916:62aeda9c:b5688f54:f5a9249d > Name : andromeda:3 (local to host andromeda) > Creation Time : Tue Jul 22 16:02:30 2014 > Raid Level : raid5 > Raid Devices : 4 > > Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB) > Array Size : 11720661504 (11177.69 GiB 12001.96 GB) > Used Dev Size : 7813774336 (3725.90 GiB 4000.65 GB) > Data Offset : 259072 sectors > Super Offset : 8 sectors > Unused Space : before=258984 sectors, after=3760 sectors > State : active > Device UUID : ed09357e:36655c6d:b7430500:63d5e540 > > Internal Bitmap : 8 sectors from superblock > Update Time : Mon Feb 9 05:10:53 2015 > Bad Block Log : 512 entries available at offset 72 sectors > Checksum : 131587ad - correct > Events : 158964 matching events. > Layout : left-symmetric > Chunk Size : 512K > > Device Role : Active device 1 > Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing) > /dev/sdo: > Magic : a92b4efc > Version : 1.2 > Feature Map : 0x1 > Array UUID : ce6de916:62aeda9c:b5688f54:f5a9249d > Name : andromeda:3 (local to host andromeda) > Creation Time : Tue Jul 22 16:02:30 2014 > Raid Level : raid5 > Raid Devices : 4 > > Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB) > Array Size : 11720661504 (11177.69 GiB 12001.96 GB) > Used Dev Size : 7813774336 (3725.90 GiB 4000.65 GB) > Data Offset : 259072 sectors > Super Offset : 8 sectors > Unused Space : before=258984 sectors, after=3760 sectors > State : clean > Device UUID : de849807:80d7f071:9909f3f2:78022d94 > > Internal Bitmap : 8 sectors from superblock > Update Time : Mon Feb 9 05:11:55 2015 > Bad Block Log : 512 entries available at offset 72 sectors > Checksum : e73f681f - correct > Events : 158962 Events off by two. Pretty minor. > Layout : left-symmetric > Chunk Size : 512K > > Device Role : Active device 3 > Array State : A.AA ('A' == active, '.' == missing, 'R' == replacing) > /dev/sdp: > Magic : a92b4efc > Version : 1.2 > Feature Map : 0x1 > Array UUID : ce6de916:62aeda9c:b5688f54:f5a9249d > Name : andromeda:3 (local to host andromeda) > Creation Time : Tue Jul 22 16:02:30 2014 > Raid Level : raid5 > Raid Devices : 4 > > Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB) > Array Size : 11720661504 (11177.69 GiB 12001.96 GB) > Used Dev Size : 7813774336 (3725.90 GiB 4000.65 GB) > Data Offset : 259072 sectors > Super Offset : 8 sectors > Unused Space : before=258984 sectors, after=3760 sectors > State : clean > Device UUID : 67fcd9a9:432b0c8b:178cc556:67b003b3 > > Internal Bitmap : 8 sectors from superblock > Update Time : Mon Feb 9 05:11:55 2015 > Bad Block Log : 512 entries available at offset 72 sectors > Checksum : 254563d8 - correct > Events : 158962 Also off by two. Again, minor. > Layout : left-symmetric > Chunk Size : 512K > > Device Role : Active device 2 > Array State : A.AA ('A' == active, '.' == missing, 'R' == replacing) > ---- dmesg > [ 576.890380] md: md3 stopped. > [ 576.891629] md: unbind<sdb> > [ 576.900841] md: export_rdev(sdb) > [ 576.902147] md: unbind<sdo> > [ 576.906824] md: export_rdev(sdo) > [ 576.908029] md: unbind<sdk> > [ 576.911845] md: export_rdev(sdk) > [ 576.913030] md: unbind<sdp> > [ 576.916862] md: export_rdev(sdp) > [ 585.164936] md: md3 stopped. > [ 585.360571] md: bind<sdk> > [ 585.360833] md: bind<sdp> > [ 585.361050] md: bind<sdo> > [ 585.361261] md: bind<sdb> > [ 585.361294] md: md3 stopped. > [ 585.361298] md: unbind<sdb> > [ 585.386616] md: export_rdev(sdb) > [ 585.387767] md: unbind<sdo> > [ 585.398644] md: export_rdev(sdo) > [ 585.399633] md: unbind<sdp> > [ 585.404662] md: export_rdev(sdp) > [ 585.405681] md: unbind<sdk> > [ 585.410647] md: export_rdev(sdk) > ---- mdadm -A output > mdadm: looking for devices for /dev/md3 > mdadm: /dev/sdb is identified as a member of /dev/md3, slot 0. > mdadm: /dev/sdk is identified as a member of /dev/md3, slot 1. > mdadm: /dev/sdo is identified as a member of /dev/md3, slot 3. > mdadm: /dev/sdp is identified as a member of /dev/md3, slot 2. > mdadm: added /dev/sdk to /dev/md3 as 1 > mdadm: added /dev/sdp to /dev/md3 as 2 (possibly out of date) > mdadm: added /dev/sdo to /dev/md3 as 3 (possibly out of date) > mdadm: added /dev/sdb to /dev/md3 as 0 > mdadm: /dev/md3 assembled from 2 drives - not enough to start the array. Please redo this with an explicit command line so we can see what's going on: mdadm --assemble --force --verbose /dev/md3 /dev/sd[bkop] > ---- /proc/mdstat > Personalities : [raid6] [raid5] [raid4] [raid1] > md1 : active raid1 sda[0] sde[1] > 1953383488 blocks super 1.2 [2/2] [UU] > bitmap: 0/15 pages [0KB], 65536KB chunk > > md0 : active raid5 sdl[0] sdn[3] sdd[5] sdm[1] > 8790405120 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] [UUUU] > bitmap: 0/22 pages [0KB], 65536KB chunk > > md2 : active raid6 sdj[0] sdi[1] sdg[3] sdh[2] > 5860270080 blocks super 1.2 level 6, 512k chunk, algorithm 2 [4/4] [UUUU] > bitmap: 0/22 pages [0KB], 65536KB chunk > > unused devices: <none> > > ** md3 is shutdown which is why I'm assuming it's not there. Yup. If --assemble --force doesn't work with your installed OS, temporarily boot from a recent system rescue cd and do the above over again (especially the /dev/disk/by-id excerpt). Phil [1] http://marc.info/?l=linux-raid&m=135811522817345&w=1 More history: http://marc.info/?l=linux-raid&m=133761065622164&w=2 http://marc.info/?l=linux-raid&m=135863964624202&w=2 http://marc.info/?l=linux-raid&m=139050322510249&w=2 You might want to read more from those threads than just the mails I've pointed out... ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: mdsadm -A won't assemble my array 2015-02-10 2:04 ` Phil Turmel @ 2015-02-10 2:48 ` G. Michael Carter 0 siblings, 0 replies; 6+ messages in thread From: G. Michael Carter @ 2015-02-10 2:48 UTC (permalink / raw) To: Phil Turmel; +Cc: linux-raid Well I'm pouring myself an ice wine and raising a glass to you. Downloaded fedora 21, ran the force assemble and it fixed my problem. Glad I did the sensible thing this time and waited for a response *wink* Thanks for your help. It's running a consistency check on those drives and then upgrading this server to fedora 21... was next on my list anyways. ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2015-02-10 2:48 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-02-09 20:21 mdsadm -A won't assemble my array G. Michael Carter 2015-02-09 23:05 ` G. Michael Carter 2015-02-10 0:13 ` Phil Turmel 2015-02-10 1:36 ` G. Michael Carter 2015-02-10 2:04 ` Phil Turmel 2015-02-10 2:48 ` G. Michael Carter
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox