* Guidance on reshape stuck at 0% after --grow
@ 2015-09-22 2:15 Guillaume Paumier
2015-09-22 10:29 ` Mikael Abrahamsson
0 siblings, 1 reply; 6+ messages in thread
From: Guillaume Paumier @ 2015-09-22 2:15 UTC (permalink / raw)
To: linux-raid
Hello,
First of all, my apologies if this is explained somewhere; I haven't
been able to find existing documentation about this exact situation.
Yesterday, I added a new (identical) disk to an existing, clean,
8-disk RAID6 array (with XFS), and attempted to grow the array:
# mdadm --add /dev/md0 /dev/sdb1
# mdadm --grow --raid-devices=9 /dev/md0
Over 24 hours later, the reshape is still in the same state as when it
started, i.e. at 0%. Looking at cat /proc/mdstat , no data seems to
have been moved, and the speed is shown as 0K/sec. The "finish"
estimate just keeps increasing. I don't see any process related to
mdadm in top. Unfortunately, I omitted the backup file in the --grow
command.
I'm looking for guidance regarding what to do next. I'm not sure
whether I should wait longer (how long?), or try to re-assemble the
seemingly-intact original array (how do I stop/restart the reshape
cleanly?), or something else. Any advice would be greatly appreciated.
Here's some additional information about the array after the --grow command:
______________________________________________
# mdadm -D /dev/md0
______________________________________________
/dev/md0:
Version : 1.0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Array Size : 23442098688 (22356.13 GiB 24004.71 GB)
Used Dev Size : 3907016448 (3726.02 GiB 4000.78 GB)
Raid Devices : 9
Total Devices : 9
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Sun Sep 20 16:48:15 2015
State : clean, reshaping
Active Devices : 9
Working Devices : 9
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 128K
Reshape Status : 0% complete
Delta Devices : 1, (8->9)
Name : (redacted):0
UUID : eea59047:120a0365:353da182:6787e030
Events : 35499
Number Major Minor RaidDevice State
0 8 33 0 active sync /dev/sdc1
1 8 49 1 active sync /dev/sdd1
2 8 97 2 active sync /dev/sdg1
3 8 113 3 active sync /dev/sdh1
4 8 129 4 active sync /dev/sdi1
5 8 145 5 active sync /dev/sdj1
7 8 81 6 active sync /dev/sdf1
6 8 65 7 active sync /dev/sde1
8 8 17 8 active sync /dev/sdb1
______________________________________________
cat /proc/mdstat
______________________________________________
Personalities : [raid6] [raid5] [raid4]
md0 : active raid6 sdb1[8] sdg1[2] sdj1[5] sdc1[0] sdf1[7] sde1[6]
sdh1[3] sdi1[4] sdd1[1]
23442098688 blocks super 1.0 level 6, 128k chunk, algorithm 2
[9/9] [UUUUUUUUU]
[>....................] reshape = 0.0% (0/3907016448)
finish=377198019201.6min speed=0K/sec
bitmap: 0/30 pages [0KB], 65536KB chunk
unused devices: <none>
______________________________________________
# dmesg | grep md
______________________________________________
<pre>
[ 4.468239] md: bind<sdd1>
[ 4.477539] md: bind<sdi1>
[ 4.489898] md: bind<sdh1>
[ 4.499959] md: bind<sde1>
[ 4.507908] md: bind<sdf1>
[ 4.534961] md: bind<sdc1>
[ 4.544582] md: bind<sdj1>
[ 4.550060] md: bind<sdg1>
[ 4.642375] md: raid6 personality registered for level 6
[ 4.642378] md: raid5 personality registered for level 5
[ 4.642407] md: raid4 personality registered for level 4
[ 4.644839] md/raid:md0: device sdg1 operational as raid disk 2
[ 4.644843] md/raid:md0: device sdj1 operational as raid disk 5
[ 4.644844] md/raid:md0: device sdc1 operational as raid disk 0
[ 4.644845] md/raid:md0: device sdf1 operational as raid disk 6
[ 4.644846] md/raid:md0: device sde1 operational as raid disk 7
[ 4.644847] md/raid:md0: device sdh1 operational as raid disk 3
[ 4.644848] md/raid:md0: device sdi1 operational as raid disk 4
[ 4.644850] md/raid:md0: device sdd1 operational as raid disk 1
[ 4.645566] md/raid:md0: allocated 0kB
[ 4.646409] md/raid:md0: raid level 6 active with 8 out of 8
devices, algorithm 2
[ 4.646575] created bitmap (30 pages) for device md0
[ 4.647292] md0: bitmap initialized from disk: read 2 pages, set 0
of 59617 bits
[ 4.699645] md0: detected capacity change from 0 to 24004709056512
[ 4.700805] md0: unknown partition table
[ 4.909274] XFS (md0): Mounting V4 Filesystem
[ 5.335425] XFS (md0): Ending clean mount
[ 1642.872936] md0: unknown partition table
[ 1665.183589] md: bind<sdb1>
[ 1665.217944] md0: unknown partition table
[ 1868.222033] md: reshape of RAID array md0
[ 1868.222037] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
[ 1868.222038] md: using maximum available idle IO bandwidth (but not
more than 200000 KB/sec) for reshape.
[ 1868.222047] md: using 128k window, over a total of 3907016448k.
[ 1868.276409] md0: unknown partition table
[ 2001.573583] md0: unknown partition table
______________________________________________
# mdadm --version
______________________________________________
mdadm - v3.3.1 - 5th June 2014
--
Guillaume Paumier
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Guidance on reshape stuck at 0% after --grow
2015-09-22 2:15 Guidance on reshape stuck at 0% after --grow Guillaume Paumier
@ 2015-09-22 10:29 ` Mikael Abrahamsson
2015-09-23 3:20 ` Guillaume Paumier
0 siblings, 1 reply; 6+ messages in thread
From: Mikael Abrahamsson @ 2015-09-22 10:29 UTC (permalink / raw)
To: Guillaume Paumier; +Cc: linux-raid
On Mon, 21 Sep 2015, Guillaume Paumier wrote:
> I'm looking for guidance regarding what to do next. I'm not sure
> whether I should wait longer (how long?), or try to re-assemble the
> seemingly-intact original array (how do I stop/restart the reshape
> cleanly?), or something else. Any advice would be greatly appreciated.
You're not the first one here with this problem.
I can't find it right now, but I believe someone issued --grow --continue
on the array to get it out of the state that you have right now.
I don't know if it's this:
http://www.spinics.net/lists/raid/msg49077.html
I do not recommend a reboot or anything else, others have had problems
with this. Also make sure you collect as much information as possible,
mdadm --examine, copy the superblocks in binary form using dd to somewhere
etc, so you can get back to the state where you are now.
Look through the mailing list archives from the past 6 months, you're not
alone in having this problem.
--
Mikael Abrahamsson email: swmike@swm.pp.se
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Guidance on reshape stuck at 0% after --grow
2015-09-22 10:29 ` Mikael Abrahamsson
@ 2015-09-23 3:20 ` Guillaume Paumier
2015-09-23 18:11 ` Wols Lists
0 siblings, 1 reply; 6+ messages in thread
From: Guillaume Paumier @ 2015-09-23 3:20 UTC (permalink / raw)
To: Mikael Abrahamsson; +Cc: linux-raid
Hello,
On 22 September 2015 at 03:29, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
>
> You're not the first one here with this problem.
>
> I can't find it right now, but I believe someone issued --grow --continue on
> the array to get it out of the state that you have right now.
>
> I don't know if it's this:
>
> http://www.spinics.net/lists/raid/msg49077.html
>
> I do not recommend a reboot or anything else, others have had problems with
> this. Also make sure you collect as much information as possible, mdadm
> --examine, copy the superblocks in binary form using dd to somewhere etc, so
> you can get back to the state where you are now.
>
> Look through the mailing list archives from the past 6 months, you're not
> alone in having this problem.
Thank you for your advice and pointers. I looked further into that
thread and others, and they helped to investigate the issue.
Unfortunately, it seems (of course) that I have only succeeded in
making the situation worse.
The thread you linked to was indeed similar to my issue. I first
looked into mdadm-grow-continue@md0.service, as suggested in
http://www.spinics.net/lists/raid/msg49042.html :
______________________________________________
# systemctl status mdadm-grow-continue@md0.service
-------------------------------------------------------------------------------
mdadm-grow-continue@md0.service - Manage MD Reshape on /dev/md0
Loaded: loaded (/usr/lib/systemd/system/mdadm-grow-continue@.service; static)
Active: failed (Result: exit-code) since dim. 2015-09-20 16:48:15
PDT; 1 day 20h ago
Process: 10989 ExecStart=/sbin/mdadm --grow --continue /dev/%I
(code=exited, status=1/FAILURE)
Main PID: 10989 (code=exited, status=1/FAILURE)
______________________________________________
The service was inactive, which was consistent with the fact that I
didn't see any activity in top. I tried to re-start it, but the result
was the same.
Per http://www.spinics.net/lists/raid/msg49098.html , I commented out
the StandardInput, StandardOutput and StandardError entries in the
service file, and reloaded the daemon. This gave me these additional
lines (bottom two lines of the output):
______________________________________________
# nano /usr/lib/systemd/system/mdadm-grow-continue@.service
# systemctl daemon-reload
# systemctl start mdadm-grow-continue@md0.service
# systemctl status mdadm-grow-continue@md0.service
-------------------------------------------------------------------------------
mdadm-grow-continue@md0.service - Manage MD Reshape on /dev/md0
Loaded: loaded (/usr/lib/systemd/system/mdadm-grow-continue@.service; static)
Active: failed (Result: exit-code) since mar. 2015-09-22 13:03:26 PDT; 2s ago
Process: 14688 ExecStart=/sbin/mdadm --grow --continue /dev/%I
(code=exited, status=1/FAILURE)
Main PID: 14688 (code=exited, status=1/FAILURE)
sept. 22 13:03:26 mdadm[14688]: mdadm: Need to backup 5376K of
critical section..
sept. 22 13:03:26 mdadm[14688]: mdadm: array: Cannot grow - need a
spare or backup-file to backup critical section
______________________________________________
So apparently, the array couldn't be grown because there was no backup
file originally specified.
At that point, I figured I would stop the array and try to re-assemble
it in its prior state. In retrospect, it was probably a bad idea, but
I didn't realize at the time that stopping the array would be so
problematic.
I attempted to re-assemble using the "--update=revert-reshape" option
as suggested in http://www.spinics.net/lists/raid/msg49596.html , but
that didn't work:
______________________________________________
# mdadm --assemble --force --update=revert-reshape /dev/md0 /dev/sdc1
/dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1 /dev/sdi1 /dev/sdj1
--backup-file=/root/backup.bak --invalid-backup --verbose
-------------------------------------------------------------------------------
mdadm: looking for devices for /dev/md0
mdadm: /dev/sdc1 is identified as a member of /dev/md0, slot 0.
mdadm: /dev/sdd1 is identified as a member of /dev/md0, slot 1.
mdadm: /dev/sde1 is identified as a member of /dev/md0, slot 7.
mdadm: /dev/sdf1 is identified as a member of /dev/md0, slot 6.
mdadm: /dev/sdg1 is identified as a member of /dev/md0, slot 2.
mdadm: /dev/sdh1 is identified as a member of /dev/md0, slot 3.
mdadm: /dev/sdi1 is identified as a member of /dev/md0, slot 4.
mdadm: /dev/sdj1 is identified as a member of /dev/md0, slot 5.
mdadm: :/dev/md0 has an active reshape - checking if critical section
needs to be restored
mdadm: Cannot read from /root/backup.bak
mdadm: Failed to find backup of critical section
mdadm: continuing without restoring backup
mdadm: added /dev/sdd1 to /dev/md0 as 1
mdadm: added /dev/sdg1 to /dev/md0 as 2
mdadm: added /dev/sdh1 to /dev/md0 as 3
mdadm: added /dev/sdi1 to /dev/md0 as 4
mdadm: added /dev/sdj1 to /dev/md0 as 5
mdadm: added /dev/sdf1 to /dev/md0 as 6
mdadm: added /dev/sde1 to /dev/md0 as 7
mdadm: no uptodate device for slot 16 of /dev/md0
mdadm: added /dev/sdc1 to /dev/md0 as 0
mdadm: failed to RUN_ARRAY /dev/md0: Invalid argument
______________________________________________
This was the output of mdadm -D afterwards:
______________________________________________
# mdadm -D /dev/md0
-------------------------------------------------------------------------------
/dev/md0:
Version : 1.0
Raid Level : raid0
Total Devices : 8
Persistence : Superblock is persistent
State : inactive
Delta Devices : -1, (1->0)
New Level : raid6
New Layout : left-symmetric
New Chunksize : 128K
Name : (redacted):0
UUID : eea59047:120a0365:353da182:6787e030
Events : 35500
Number Major Minor RaidDevice
- 8 33 - /dev/sdc1
- 8 49 - /dev/sdd1
- 8 65 - /dev/sde1
- 8 81 - /dev/sdf1
- 8 97 - /dev/sdg1
- 8 113 - /dev/sdh1
- 8 129 - /dev/sdi1
- 8 145 - /dev/sdj1
______________________________________________
dmesg provided some additional information, saying "reshape_position
too early for auto-recovery":
______________________________________________
# dmesg | grep md
-------------------------------------------------------------------------------
[164758.944492] md: bind<sdd1>
[164758.944819] md: bind<sdg1>
[164758.944977] md: bind<sdh1>
[164758.945152] md: bind<sdi1>
[164758.945339] md: bind<sdj1>
[164758.945472] md: bind<sdf1>
[164758.945610] md: bind<sde1>
[164758.945785] md: bind<sdc1>
[164758.952353] md/raid:md0: reshape_position too early for
auto-recovery - aborting.
[164758.952357] md: pers->run() failed ...
______________________________________________
I think this is the same situation as the one described by Daniel Koch
in the thread "Failed to grow" from August 2015:
http://www.spinics.net/lists/raid/msg49643.html . The signs look very
similar.
I have a version of mdadm --examine before stopping the array, and one
of it now (both are posted at the end of this message). I've also
posted a diff at
https://gist.github.com/gpaumier/da3200b357f35288021d/revisions to
make it easier to see the differences.
As far as I can tell, the only difference is the additional disk I
initially tried to add to the array. When trying to re-assemble, mdadm
complained that that disk (/dev/sdb1) had a different superblock
("mdadm: superblock on /dev/sdb1 doesn't match others - assembly
aborted") so I left it out when trying to re-assemble.
I now realize (per http://www.spinics.net/lists/raid/msg49090.html )
that I should have tried to manually do "mdadm --grow --continue" with
an empty backup file before stopping the array, unfortunately that
wasn't clear to me at the time.
According to http://www.spinics.net/lists/raid/msg49669.html , my last
option seems to be to re-create the array, which worked for the
poster. However, that operation (described at
https://raid.wiki.kernel.org/index.php/RAID_Recovery#Restore_array_by_recreating_.28after_multiple_device_failure.29
) seems very dangerous so I wanted to ask again here before doing
anything else.
This is how I see the current situation:
* The good news is that all the data and metadata still appear clean,
so if I tread carefully I should be able to recover my array with no
data loss (and then re-attempt to grow)
* The bad news is that my only option at this point seems to be to try
to re-create the array manually, which would basically cause all data
to be lost if if doesn't work.
I figure I have a 50% chance of restoring the array, and 50% chance of
making it completely unusable. I'm not very comfortable with those
odds, so I would appreciate if anyone had any advice regarding what to
do now, and alternatives to manually re-creating the array, if there
are any.
Many thanks for your help.
______________________________________________
Appendix 1: Output of mdadm --examine /dev/sd[b-j]1 before stopping the array
-------------------------------------------------------------------------------
# mdadm --examine /dev/sd[b-j]1
/dev/sdb1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 9
Avail Dev Size : 7814033128 (3726.02 GiB 4000.78 GB)
Array Size : 27349115136 (26082.15 GiB 28005.49 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : bde6e5ab:5f4a56e9:10ad82fc:301d477b
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : 1 (8->9)
Update Time : Sun Sep 20 16:48:15 2015
Bad Block Log : 512 entries available at offset -8 sectors
Checksum : 28369459 - correct
Events : 35499
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 8
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdc1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 9
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 27349115136 (26082.15 GiB 28005.49 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : e1b689b5:b4a2c5a7:56057b69:a9101af0
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : 1 (8->9)
Update Time : Sun Sep 20 16:48:15 2015
Checksum : d20d0897 - correct
Events : 35499
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 0
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdd1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 9
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 27349115136 (26082.15 GiB 28005.49 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : 1d8e74d3:9abd37f8:f2cf0ab8:02fdcfd6
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : 1 (8->9)
Update Time : Sun Sep 20 16:48:15 2015
Checksum : 75afb1b0 - correct
Events : 35499
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 1
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sde1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 9
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 27349115136 (26082.15 GiB 28005.49 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : ddf17d3d:ea944bfb:6886cc91:3366f55f
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : 1 (8->9)
Update Time : Sun Sep 20 16:48:15 2015
Checksum : 45b40c6b - correct
Events : 35499
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 7
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdf1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 9
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 27349115136 (26082.15 GiB 28005.49 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : 38675f59:ea412b1f:67d6ed9a:a33fc5dd
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : 1 (8->9)
Update Time : Sun Sep 20 16:48:15 2015
Checksum : c665836 - correct
Events : 35499
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 6
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdg1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 9
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 27349115136 (26082.15 GiB 28005.49 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : b24758e6:042412c5:9b5a3c06:f167aedf
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : 1 (8->9)
Update Time : Sun Sep 20 16:48:15 2015
Checksum : ac7dc747 - correct
Events : 35499
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 2
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdh1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 9
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 27349115136 (26082.15 GiB 28005.49 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : 00e47d82:b49c3905:3ed961fe:40a5f259
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : 1 (8->9)
Update Time : Sun Sep 20 16:48:15 2015
Checksum : fb349837 - correct
Events : 35499
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 3
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdi1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 9
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 27349115136 (26082.15 GiB 28005.49 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : a7e34040:fa12382f:c2ef3d85:9c95b1d0
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : 1 (8->9)
Update Time : Sun Sep 20 16:48:15 2015
Checksum : e0911505 - correct
Events : 35499
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 4
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdj1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 9
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 27349115136 (26082.15 GiB 28005.49 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : 9d89c55d:9f4a2181:6b87922f:0681d580
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : 1 (8->9)
Update Time : Sun Sep 20 16:48:15 2015
Checksum : aa777dfc - correct
Events : 35499
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 5
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
______________________________________________
______________________________________________
Appendix 2: Output of mdadm --examine /dev/sd[b-j]1 currently.
-------------------------------------------------------------------------------
# mdadm --examine /dev/sd[b-j]1
/dev/sdb1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 7814033128 (3726.02 GiB 4000.78 GB)
Array Size : 23442098688 (22356.13 GiB 24004.71 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : bde6e5ab:5f4a56e9:10ad82fc:301d477b
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : -1 (9->8)
Update Time : Tue Sep 22 13:25:43 2015
Bad Block Log : 512 entries available at offset -8 sectors
Checksum : 283907e0 - correct
Events : 35500
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 8
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdc1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 23442098688 (22356.13 GiB 24004.71 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : e1b689b5:b4a2c5a7:56057b69:a9101af0
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : -1 (9->8)
Update Time : Tue Sep 22 13:25:43 2015
Checksum : d20f7c1e - correct
Events : 35500
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 0
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdd1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 23442098688 (22356.13 GiB 24004.71 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : 1d8e74d3:9abd37f8:f2cf0ab8:02fdcfd6
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : -1 (9->8)
Update Time : Tue Sep 22 13:25:43 2015
Checksum : 75b22537 - correct
Events : 35500
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 1
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sde1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 23442098688 (22356.13 GiB 24004.71 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : ddf17d3d:ea944bfb:6886cc91:3366f55f
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : -1 (9->8)
Update Time : Tue Sep 22 13:25:43 2015
Checksum : 45b67ff2 - correct
Events : 35500
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 7
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdf1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 23442098688 (22356.13 GiB 24004.71 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : 38675f59:ea412b1f:67d6ed9a:a33fc5dd
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : -1 (9->8)
Update Time : Tue Sep 22 13:25:43 2015
Checksum : c68cbbd - correct
Events : 35500
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 6
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdg1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 23442098688 (22356.13 GiB 24004.71 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : b24758e6:042412c5:9b5a3c06:f167aedf
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : -1 (9->8)
Update Time : Tue Sep 22 13:25:43 2015
Checksum : ac803ace - correct
Events : 35500
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 2
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdh1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 23442098688 (22356.13 GiB 24004.71 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : 00e47d82:b49c3905:3ed961fe:40a5f259
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : -1 (9->8)
Update Time : Tue Sep 22 13:25:43 2015
Checksum : fb370bbe - correct
Events : 35500
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 3
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdi1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 23442098688 (22356.13 GiB 24004.71 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : a7e34040:fa12382f:c2ef3d85:9c95b1d0
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : -1 (9->8)
Update Time : Tue Sep 22 13:25:43 2015
Checksum : e093888c - correct
Events : 35500
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 4
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdj1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 23442098688 (22356.13 GiB 24004.71 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : 9d89c55d:9f4a2181:6b87922f:0681d580
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : -1 (9->8)
Update Time : Tue Sep 22 13:25:43 2015
Checksum : aa79f183 - correct
Events : 35500
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 5
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
______________________________________________
--
Guillaume Paumier
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Guidance on reshape stuck at 0% after --grow
2015-09-23 3:20 ` Guillaume Paumier
@ 2015-09-23 18:11 ` Wols Lists
2015-09-27 0:59 ` Guillaume Paumier
0 siblings, 1 reply; 6+ messages in thread
From: Wols Lists @ 2015-09-23 18:11 UTC (permalink / raw)
To: Guillaume Paumier, Mikael Abrahamsson; +Cc: linux-raid
On 23/09/15 04:20, Guillaume Paumier wrote:
> I figure I have a 50% chance of restoring the array, and 50% chance of
> making it completely unusable. I'm not very comfortable with those
> odds, so I would appreciate if anyone had any advice regarding what to
> do now, and alternatives to manually re-creating the array, if there
> are any.
Read up on loopback devices. Sorry, never done it, can't help
personally, but it sticks a ram-disk layer between mdadm and the actual
disk.
So the disks themselves are read-only, and you can try things out to see
what works. If it fails, you throw the loopback layer away, if it works
you can run it on the disk itself.
Cheers,
Wol
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Guidance on reshape stuck at 0% after --grow
2015-09-23 18:11 ` Wols Lists
@ 2015-09-27 0:59 ` Guillaume Paumier
2015-09-27 23:53 ` Guillaume Paumier
0 siblings, 1 reply; 6+ messages in thread
From: Guillaume Paumier @ 2015-09-27 0:59 UTC (permalink / raw)
To: Wols Lists; +Cc: Mikael Abrahamsson, linux-raid
Hello,
On 23 September 2015 at 11:11, Wols Lists <antlists@youngman.org.uk> wrote:
>
> Read up on loopback devices. Sorry, never done it, can't help
> personally, but it sticks a ram-disk layer between mdadm and the actual
> disk.
>
> So the disks themselves are read-only, and you can try things out to see
> what works. If it fails, you throw the loopback layer away, if it works
> you can run it on the disk itself.
Thank you; that seems similar to what is described at
https://raid.wiki.kernel.org/index.php/Recovering_a_failed_software_RAID#Making_the_harddisks_read-only_using_an_overlay_file
, and was also suggested in
http://www.spinics.net/lists/raid/msg49669.html
It seems I'm running into another issue, described at the bottom of
https://raid.wiki.kernel.org/index.php/RAID_Recovery#Restore_array_by_recreating_.28after_multiple_device_failure.29
:
"If upon running the above with the --size parameter you get, as one
of the authors of this page did, an error such as: "mdadm: /dev/sdb1
is smaller than given size. xxxK < yyyK + metadata", you may have
stumbled upon a problem where the array was initially created with an
earlier version of mdadm that reserved less device space. The solution
seems to be to find an earlier version of mdadm to run with the
creation command above"
This seems consistent with the errors I'm getting, in the form:
mdadm: /dev/sdc1 is smaller than given size. 3906885632K < 3907016448K
+ metadata
I found the list of previous versions of mdadm at
https://www.kernel.org/pub/linux/utils/raid/mdadm/ , but I'm not sure
which one to pick, and how easy it is to use those earlier versions.
According to my logs, I originally created the array in late July
2013. I'm not afraid of compiling it from source, but any additional
information would be helpful.
--
Guillaume Paumier
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Guidance on reshape stuck at 0% after --grow
2015-09-27 0:59 ` Guillaume Paumier
@ 2015-09-27 23:53 ` Guillaume Paumier
0 siblings, 0 replies; 6+ messages in thread
From: Guillaume Paumier @ 2015-09-27 23:53 UTC (permalink / raw)
To: Wols Lists; +Cc: Mikael Abrahamsson, linux-raid
Hello,
After a discussion on IRC, I rebooted the machine to boot using
SystemRescueCD, in order to see if a newer version would work better.
It didn't, but after rebooting into the regular OS of the machine, the
array was assembled. It's still not running, though:
mdadm -D gives the following output:
___________________________________________
# mdadm -D /dev/md0
---------------------------------------------------------------------------
/dev/md0:
Version : 1.0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Used Dev Size : -1
Raid Devices : 8
Total Devices : 9
Persistence : Superblock is persistent
Update Time : Tue Sep 22 13:25:43 2015
State : active, Not Started
Active Devices : 9
Working Devices : 9
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 128K
Delta Devices : -1, (9->8)
Name : (redacted):0
UUID : eea59047:120a0365:353da182:6787e030
Events : 35500
Number Major Minor RaidDevice State
0 8 33 0 active sync /dev/sdc1
1 8 49 1 active sync /dev/sdd1
2 8 97 2 active sync /dev/sdg1
3 8 113 3 active sync /dev/sdh1
4 8 129 4 active sync /dev/sdi1
5 8 145 5 active sync /dev/sdj1
7 8 81 6 active sync /dev/sdf1
6 8 65 7 active sync /dev/sde1
8 8 17 8 active sync /dev/sdb1
___________________________________________
Note that "Raid Devices : 8", "Total Devices : 9" but /dev/sdb1 is
still listed as RaidDevice #8 (i.e. the 9th device) and in "active
sync". I'm not sure how this works.
dmesg still indicates that the reshape can't proceed:
___________________________________________
# dmesg | grep md
---------------------------------------------------------------------------
[ 4.366341] md: bind<sdg1>
[ 4.368685] md: bind<sdd1>
[ 4.370618] md: bind<sdf1>
[ 4.387630] md: bind<sdh1>
[ 4.392266] md: bind<sdc1>
[ 4.397084] md: bind<sdb1>
[ 4.411668] md: bind<sdj1>
[ 4.453465] md: bind<sde1>
[ 4.489803] md: bind<sdi1>
[ 39.129640] md: raid6 personality registered for level 6
[ 39.129643] md: raid5 personality registered for level 5
[ 39.129644] md: raid4 personality registered for level 4
[ 39.133609] md/raid:md0: reshape_position too early for
auto-recovery - aborting.
[ 39.133613] md: pers->run() failed ...
___________________________________________
At this point, I'm wondering if it would help to remove sdb1 from the
array. Does that sound like a good idea? If so, should I attempt to
"fail" it and format it? Or try to shrink to 8 devices instead of 9?
For reference:
___________________________________________
mdadm --examine /dev/sd[b-j]1
-------------------------------------------------------------------------------
mdadm --examine /dev/sd[b-j]1
/dev/sdb1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 7814033128 (3726.02 GiB 4000.78 GB)
Array Size : 23442098688 (22356.13 GiB 24004.71 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : bde6e5ab:5f4a56e9:10ad82fc:301d477b
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : -1 (9->8)
Update Time : Tue Sep 22 13:25:43 2015
Bad Block Log : 512 entries available at offset -8 sectors
Checksum : 283907e0 - correct
Events : 35500
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 8
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdc1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 23442098688 (22356.13 GiB 24004.71 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : e1b689b5:b4a2c5a7:56057b69:a9101af0
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : -1 (9->8)
Update Time : Tue Sep 22 13:25:43 2015
Checksum : d20f7c1e - correct
Events : 35500
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 0
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdd1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 23442098688 (22356.13 GiB 24004.71 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : 1d8e74d3:9abd37f8:f2cf0ab8:02fdcfd6
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : -1 (9->8)
Update Time : Tue Sep 22 13:25:43 2015
Checksum : 75b22537 - correct
Events : 35500
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 1
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sde1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 23442098688 (22356.13 GiB 24004.71 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : ddf17d3d:ea944bfb:6886cc91:3366f55f
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : -1 (9->8)
Update Time : Tue Sep 22 13:25:43 2015
Checksum : 45b67ff2 - correct
Events : 35500
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 7
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdf1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 23442098688 (22356.13 GiB 24004.71 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : 38675f59:ea412b1f:67d6ed9a:a33fc5dd
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : -1 (9->8)
Update Time : Tue Sep 22 13:25:43 2015
Checksum : c68cbbd - correct
Events : 35500
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 6
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdg1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 23442098688 (22356.13 GiB 24004.71 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : b24758e6:042412c5:9b5a3c06:f167aedf
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : -1 (9->8)
Update Time : Tue Sep 22 13:25:43 2015
Checksum : ac803ace - correct
Events : 35500
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 2
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdh1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 23442098688 (22356.13 GiB 24004.71 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : 00e47d82:b49c3905:3ed961fe:40a5f259
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : -1 (9->8)
Update Time : Tue Sep 22 13:25:43 2015
Checksum : fb370bbe - correct
Events : 35500
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 3
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdi1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 23442098688 (22356.13 GiB 24004.71 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : a7e34040:fa12382f:c2ef3d85:9c95b1d0
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : -1 (9->8)
Update Time : Tue Sep 22 13:25:43 2015
Checksum : e093888c - correct
Events : 35500
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 4
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdj1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x5
Array UUID : eea59047:120a0365:353da182:6787e030
Name : (redacted):0
Creation Time : Thu Aug 1 12:23:07 2013
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 7814033136 (3726.02 GiB 4000.78 GB)
Array Size : 23442098688 (22356.13 GiB 24004.71 GB)
Used Dev Size : 7814032896 (3726.02 GiB 4000.78 GB)
Super Offset : 7814033392 sectors
Unused Space : before=0 sectors, after=480 sectors
State : clean
Device UUID : 9d89c55d:9f4a2181:6b87922f:0681d580
Internal Bitmap : -16 sectors from superblock
Reshape pos'n : 0
Delta Devices : -1 (9->8)
Update Time : Tue Sep 22 13:25:43 2015
Checksum : aa79f183 - correct
Events : 35500
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 5
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
___________________________________________
--
Guillaume Paumier
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2015-09-27 23:53 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-09-22 2:15 Guidance on reshape stuck at 0% after --grow Guillaume Paumier
2015-09-22 10:29 ` Mikael Abrahamsson
2015-09-23 3:20 ` Guillaume Paumier
2015-09-23 18:11 ` Wols Lists
2015-09-27 0:59 ` Guillaume Paumier
2015-09-27 23:53 ` Guillaume Paumier
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).