* Help recovering an interrupted raid0 reshape
@ 2015-04-04 2:22 Jonathan Harker (Jesusaurus)
2015-04-06 23:46 ` NeilBrown
0 siblings, 1 reply; 12+ messages in thread
From: Jonathan Harker (Jesusaurus) @ 2015-04-04 2:22 UTC (permalink / raw)
To: linux-raid
I have a raid0 array whose component devices are raid1 arrays. In an
attempt to expand the pre-existing raid0 array, I created a new raid1
device and then added it, growing the raid0 array.
But then the system lost power shortly after the reshaping began.
After rebooting the original two components are listed as spares in an
inactive raid4 array and the new component is not listed in
/proc/mdstat:
Personalities : [raid6] [raid5] [raid4] [raid1] [raid10] [raid0]
[linear] [multipath]
md124 : inactive md126[0](S) md127[1](S)
3907022200 blocks super 1.2
md0 : active raid1 sda5[0] sdb2[1]
107652416 blocks [2/2] [UU]
bitmap: 1/1 pages [4KB], 65536KB chunk
md125 : active raid1 sdh1[0] sdg1[1]
2930134016 blocks super 1.2 [2/2] [UU]
bitmap: 0/22 pages [0KB], 65536KB chunk
md126 : active raid1 sdc1[0] sdd1[1]
1953512312 blocks super 1.2 [2/2] [UU]
md127 : active raid1 sde1[2] sdf1[1]
1953512312 blocks super 1.2 [2/2] [UU]
unused devices: <none>
Looking at the details of the inactive array shows that it is in a
reshape between raid0 and raid4:
/dev/md124:
Version : 1.2
Raid Level : raid0
Total Devices : 2
Persistence : Superblock is persistent
State : inactive
Delta Devices : -1, (1->0)
New Level : raid4
New Chunksize : 512K
Name : hordern:hordern1 (local to host hordern)
UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
Events : 14013
Number Major Minor RaidDevice
- 9 126 - /dev/md/beta
- 9 127 - /dev/md/alpha
And examining each component shows that they have a consistent view of
where in that reshape they are (based on the reshape position), but
not consistent in the size of the array:
/dev/md/alpha:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x4
Array UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
Name : hordern:hordern1 (local to host hordern)
Creation Time : Fri Jan 2 09:59:40 2009
Raid Level : raid4
Raid Devices : 3
Avail Dev Size : 3907021824 (1863.01 GiB 2000.40 GB)
Array Size : 3907021824 (3726.03 GiB 4000.79 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1968 sectors, after=752 sectors
State : active
Device UUID : 63aaa2e4:2a09f495:8372c7f9:eb2f2773
Reshape pos'n : 129067008 (123.09 GiB 132.16 GB)
Delta Devices : -1 (4->3)
Update Time : Sun Mar 29 15:11:35 2015
Checksum : 8be5e0e6 - correct
Events : 14013
Chunk Size : 512K
Device Role : Active device 1
Array State : AA.. ('A' == active, '.' == missing, 'R' == replacing)
/dev/md/beta:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x4
Array UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
Name : hordern:hordern1 (local to host hordern)
Creation Time : Fri Jan 2 09:59:40 2009
Raid Level : raid4
Raid Devices : 3
Avail Dev Size : 3907022576 (1863.01 GiB 2000.40 GB)
Array Size : 3907021824 (3726.03 GiB 4000.79 GB)
Used Dev Size : 3907021824 (1863.01 GiB 2000.40 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1968 sectors, after=752 sectors
State : clean
Device UUID : 6e6dce14:3ebb2bb5:187aa292:403a55f6
Reshape pos'n : 129067008 (123.09 GiB 132.16 GB)
Delta Devices : -1 (4->3)
Update Time : Sun Mar 29 15:11:35 2015
Checksum : f7526add - correct
Events : 14013
Chunk Size : 512K
Device Role : Active device 0
Array State : AA.. ('A' == active, '.' == missing, 'R' == replacing)
/dev/md/gamma:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x6
Array UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
Name : hordern:hordern1 (local to host hordern)
Creation Time : Fri Jan 2 09:59:40 2009
Raid Level : raid4
Raid Devices : 4
Avail Dev Size : 5860265984 (2794.39 GiB 3000.46 GB)
Array Size : 5860532736 (5589.04 GiB 6001.19 GB)
Used Dev Size : 3907021824 (1863.01 GiB 2000.40 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Recovery Offset : 86403072 sectors
Unused Space : before=1960 sectors, after=1953244160 sectors
State : active
Device UUID : 782873ea:e265ecd4:5cc80ddf:035ba2b4
Reshape pos'n : 129067008 (123.09 GiB 132.16 GB)
Delta Devices : 1 (3->4)
Update Time : Sun Mar 29 00:05:29 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 710dc078 - correct
Events : 673
Chunk Size : 512K
Device Role : Active device 2
Array State : AAA. ('A' == active, '.' == missing, 'R' == replacing)
When I stop the inactive array and try to assemble it from all three
components, I get an error about the superblock on the third component
not matching the other two components (which makes sense since the
array sizes are different):
hordern ~ # mdadm --verbose --verbose --assemble /dev/md/hordern1
/dev/md/alpha /dev/md/beta /dev/md/gamma
mdadm: looking for devices for /dev/md/hordern1
mdadm: UUID differs from /dev/md0.
mdadm: UUID differs from /dev/md/alpha.
mdadm: UUID differs from /dev/md/beta.
mdadm: UUID differs from /dev/md/gamma.
mdadm: UUID differs from /dev/md0.
mdadm: UUID differs from /dev/md/alpha.
mdadm: UUID differs from /dev/md/beta.
mdadm: UUID differs from /dev/md/gamma.
mdadm: UUID differs from /dev/md0.
mdadm: UUID differs from /dev/md/alpha.
mdadm: UUID differs from /dev/md/beta.
mdadm: UUID differs from /dev/md/gamma.
mdadm: superblock on /dev/md/gamma doesn't match others - assembly aborted
First, what could cause the initial two components to have a different
superblock than the newly added component? And can I convince them to
be the same?
Second, is there documentation anywhere about the internal process of
growing a raid0 array? Why does it convert to a raid4 array? And what
do the Delta Devices lines mean?
Third, is it possible to resume the reshape? If not, can it be reverted?
--
Jon
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Help recovering an interrupted raid0 reshape
2015-04-04 2:22 Help recovering an interrupted raid0 reshape Jonathan Harker (Jesusaurus)
@ 2015-04-06 23:46 ` NeilBrown
2015-04-07 1:50 ` NeilBrown
0 siblings, 1 reply; 12+ messages in thread
From: NeilBrown @ 2015-04-06 23:46 UTC (permalink / raw)
To: Jonathan Harker (Jesusaurus); +Cc: linux-raid
[-- Attachment #1: Type: text/plain, Size: 7552 bytes --]
On Fri, 3 Apr 2015 19:22:15 -0700 "Jonathan Harker (Jesusaurus)"
<jesusaurus@gentlydownthe.net> wrote:
> I have a raid0 array whose component devices are raid1 arrays. In an
> attempt to expand the pre-existing raid0 array, I created a new raid1
> device and then added it, growing the raid0 array.
> But then the system lost power shortly after the reshaping began.
>
> After rebooting the original two components are listed as spares in an
> inactive raid4 array and the new component is not listed in
> /proc/mdstat:
>
>
> Personalities : [raid6] [raid5] [raid4] [raid1] [raid10] [raid0]
> [linear] [multipath]
> md124 : inactive md126[0](S) md127[1](S)
> 3907022200 blocks super 1.2
>
> md0 : active raid1 sda5[0] sdb2[1]
> 107652416 blocks [2/2] [UU]
> bitmap: 1/1 pages [4KB], 65536KB chunk
>
> md125 : active raid1 sdh1[0] sdg1[1]
> 2930134016 blocks super 1.2 [2/2] [UU]
> bitmap: 0/22 pages [0KB], 65536KB chunk
>
> md126 : active raid1 sdc1[0] sdd1[1]
> 1953512312 blocks super 1.2 [2/2] [UU]
>
> md127 : active raid1 sde1[2] sdf1[1]
> 1953512312 blocks super 1.2 [2/2] [UU]
>
> unused devices: <none>
>
>
> Looking at the details of the inactive array shows that it is in a
> reshape between raid0 and raid4:
>
>
> /dev/md124:
> Version : 1.2
> Raid Level : raid0
> Total Devices : 2
> Persistence : Superblock is persistent
>
> State : inactive
>
> Delta Devices : -1, (1->0)
> New Level : raid4
> New Chunksize : 512K
>
> Name : hordern:hordern1 (local to host hordern)
> UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
> Events : 14013
>
> Number Major Minor RaidDevice
>
> - 9 126 - /dev/md/beta
> - 9 127 - /dev/md/alpha
>
>
> And examining each component shows that they have a consistent view of
> where in that reshape they are (based on the reshape position), but
> not consistent in the size of the array:
>
> /dev/md/alpha:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x4
> Array UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
> Name : hordern:hordern1 (local to host hordern)
> Creation Time : Fri Jan 2 09:59:40 2009
> Raid Level : raid4
> Raid Devices : 3
>
> Avail Dev Size : 3907021824 (1863.01 GiB 2000.40 GB)
> Array Size : 3907021824 (3726.03 GiB 4000.79 GB)
> Data Offset : 2048 sectors
> Super Offset : 8 sectors
> Unused Space : before=1968 sectors, after=752 sectors
> State : active
> Device UUID : 63aaa2e4:2a09f495:8372c7f9:eb2f2773
>
> Reshape pos'n : 129067008 (123.09 GiB 132.16 GB)
> Delta Devices : -1 (4->3)
>
> Update Time : Sun Mar 29 15:11:35 2015
> Checksum : 8be5e0e6 - correct
> Events : 14013
>
> Chunk Size : 512K
>
> Device Role : Active device 1
> Array State : AA.. ('A' == active, '.' == missing, 'R' == replacing)
> /dev/md/beta:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x4
> Array UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
> Name : hordern:hordern1 (local to host hordern)
> Creation Time : Fri Jan 2 09:59:40 2009
> Raid Level : raid4
> Raid Devices : 3
>
> Avail Dev Size : 3907022576 (1863.01 GiB 2000.40 GB)
> Array Size : 3907021824 (3726.03 GiB 4000.79 GB)
> Used Dev Size : 3907021824 (1863.01 GiB 2000.40 GB)
> Data Offset : 2048 sectors
> Super Offset : 8 sectors
> Unused Space : before=1968 sectors, after=752 sectors
> State : clean
> Device UUID : 6e6dce14:3ebb2bb5:187aa292:403a55f6
>
> Reshape pos'n : 129067008 (123.09 GiB 132.16 GB)
> Delta Devices : -1 (4->3)
>
> Update Time : Sun Mar 29 15:11:35 2015
> Checksum : f7526add - correct
> Events : 14013
>
> Chunk Size : 512K
>
> Device Role : Active device 0
> Array State : AA.. ('A' == active, '.' == missing, 'R' == replacing)
> /dev/md/gamma:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x6
> Array UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
> Name : hordern:hordern1 (local to host hordern)
> Creation Time : Fri Jan 2 09:59:40 2009
> Raid Level : raid4
> Raid Devices : 4
>
> Avail Dev Size : 5860265984 (2794.39 GiB 3000.46 GB)
> Array Size : 5860532736 (5589.04 GiB 6001.19 GB)
> Used Dev Size : 3907021824 (1863.01 GiB 2000.40 GB)
> Data Offset : 2048 sectors
> Super Offset : 8 sectors
> Recovery Offset : 86403072 sectors
> Unused Space : before=1960 sectors, after=1953244160 sectors
> State : active
> Device UUID : 782873ea:e265ecd4:5cc80ddf:035ba2b4
>
> Reshape pos'n : 129067008 (123.09 GiB 132.16 GB)
> Delta Devices : 1 (3->4)
>
> Update Time : Sun Mar 29 00:05:29 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 710dc078 - correct
> Events : 673
>
> Chunk Size : 512K
>
> Device Role : Active device 2
> Array State : AAA. ('A' == active, '.' == missing, 'R' == replacing)
>
>
> When I stop the inactive array and try to assemble it from all three
> components, I get an error about the superblock on the third component
> not matching the other two components (which makes sense since the
> array sizes are different):
>
> hordern ~ # mdadm --verbose --verbose --assemble /dev/md/hordern1
> /dev/md/alpha /dev/md/beta /dev/md/gamma
> mdadm: looking for devices for /dev/md/hordern1
> mdadm: UUID differs from /dev/md0.
> mdadm: UUID differs from /dev/md/alpha.
> mdadm: UUID differs from /dev/md/beta.
> mdadm: UUID differs from /dev/md/gamma.
> mdadm: UUID differs from /dev/md0.
> mdadm: UUID differs from /dev/md/alpha.
> mdadm: UUID differs from /dev/md/beta.
> mdadm: UUID differs from /dev/md/gamma.
> mdadm: UUID differs from /dev/md0.
> mdadm: UUID differs from /dev/md/alpha.
> mdadm: UUID differs from /dev/md/beta.
> mdadm: UUID differs from /dev/md/gamma.
> mdadm: superblock on /dev/md/gamma doesn't match others - assembly aborted
>
>
> First, what could cause the initial two components to have a different
> superblock than the newly added component? And can I convince them to
> be the same?
Good question. The important difference is this line:
> Delta Devices : -1 (4->3)
> Delta Devices : 1 (3->4)
Also "Raid Devices" are different. This is what leads to the array size
being reported differently.
I don't know why this happened. I'll have to hunt through the code.
>
> Second, is there documentation anywhere about the internal process of
> growing a raid0 array? Why does it convert to a raid4 array? And what
> do the Delta Devices lines mean?
- no documentation like that
- the "raid0" module does support reshape. The "raid5" module does.
So the array is converted to a RAID4 with a missing parity device, then
reshaped, then converted back. It is just an internal detail really.
- "Delta devices" means that there is reshape happening that is changing
then number of devices - "delta" is a difference.
It is clearly confused.
>
> Third, is it possible to resume the reshape? If not, can it be reverted?
>
I'll have to get back to you on that. Hopefully you can be patient a little
longer.
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Help recovering an interrupted raid0 reshape
2015-04-06 23:46 ` NeilBrown
@ 2015-04-07 1:50 ` NeilBrown
2015-04-07 5:14 ` Jonathan Harker (Jesusaurus)
0 siblings, 1 reply; 12+ messages in thread
From: NeilBrown @ 2015-04-07 1:50 UTC (permalink / raw)
To: Jonathan Harker (Jesusaurus); +Cc: linux-raid
[-- Attachment #1: Type: text/plain, Size: 642 bytes --]
On Tue, 7 Apr 2015 09:46:08 +1000 NeilBrown <neilb@suse.de> wrote:
> On Fri, 3 Apr 2015 19:22:15 -0700 "Jonathan Harker (Jesusaurus)"
> <jesusaurus@gentlydownthe.net> wrote:
>
> >
> > Third, is it possible to resume the reshape? If not, can it be reverted?
> >
>
> I'll have to get back to you on that. Hopefully you can be patient a little
> longer.
>
> NeilBrown
>
>
Hi again Jonathan,
please report what version of mdadm and of Linux that you are running.
Also is there any chance that you tried
mdadm --assemble .... --update=revert-reshape
??
And if you did, why didn't you say so :-)
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Help recovering an interrupted raid0 reshape
2015-04-07 1:50 ` NeilBrown
@ 2015-04-07 5:14 ` Jonathan Harker (Jesusaurus)
2015-04-07 6:30 ` NeilBrown
0 siblings, 1 reply; 12+ messages in thread
From: Jonathan Harker (Jesusaurus) @ 2015-04-07 5:14 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
On Mon, Apr 6, 2015 at 6:50 PM, NeilBrown <neilb@suse.de> wrote:
> On Tue, 7 Apr 2015 09:46:08 +1000 NeilBrown <neilb@suse.de> wrote:
>
>> On Fri, 3 Apr 2015 19:22:15 -0700 "Jonathan Harker (Jesusaurus)"
>> <jesusaurus@gentlydownthe.net> wrote:
>>
>> >
>> > Third, is it possible to resume the reshape? If not, can it be reverted?
>> >
>>
>> I'll have to get back to you on that. Hopefully you can be patient a little
>> longer.
>>
>> NeilBrown
>>
Thanks for taking the time to look into this. I've managed to be
patient with the problem for about a week now (I was cautiously poking
and prodding the array for a few days before reaching out to this
list) so I think I can manage a little more patience.
>>
>
> Hi again Jonathan,
> please report what version of mdadm and of Linux that you are running.
My mdadm version is "mdadm - v3.3.1 - 5th June 2014" and my linux
version (according to `uname -r`) is 3.16.2-3.
>
> Also is there any chance that you tried
> mdadm --assemble .... --update=revert-reshape
> ??
> And if you did, why didn't you say so :-)
During those first couple days I tried a couple assemble and
incremental commands to try to get the reshape to restart, though I
don't remember exactly what they were. It's possible I tried something
as scary and irreversible as --update=revert-reshape in some desperate
haze, so let's assume I did, but I was trying hard to avoid any advice
that included --force in the command.
>
> NeilBrown
Thanks again for looking into this, and let me know if there is any
other useful information I can provide.
--
Jonathan
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Help recovering an interrupted raid0 reshape
2015-04-07 5:14 ` Jonathan Harker (Jesusaurus)
@ 2015-04-07 6:30 ` NeilBrown
2015-04-07 17:02 ` Jonathan Harker (Jesusaurus)
0 siblings, 1 reply; 12+ messages in thread
From: NeilBrown @ 2015-04-07 6:30 UTC (permalink / raw)
To: Jonathan Harker (Jesusaurus); +Cc: linux-raid
[-- Attachment #1: Type: text/plain, Size: 1915 bytes --]
On Mon, 6 Apr 2015 22:14:56 -0700 "Jonathan Harker (Jesusaurus)"
<jesusaurus@gentlydownthe.net> wrote:
> On Mon, Apr 6, 2015 at 6:50 PM, NeilBrown <neilb@suse.de> wrote:
> > On Tue, 7 Apr 2015 09:46:08 +1000 NeilBrown <neilb@suse.de> wrote:
> >
> >> On Fri, 3 Apr 2015 19:22:15 -0700 "Jonathan Harker (Jesusaurus)"
> >> <jesusaurus@gentlydownthe.net> wrote:
> >>
> >> >
> >> > Third, is it possible to resume the reshape? If not, can it be reverted?
> >> >
> >>
> >> I'll have to get back to you on that. Hopefully you can be patient a little
> >> longer.
> >>
> >> NeilBrown
> >>
>
> Thanks for taking the time to look into this. I've managed to be
> patient with the problem for about a week now (I was cautiously poking
> and prodding the array for a few days before reaching out to this
> list) so I think I can manage a little more patience.
>
> >>
> >
> > Hi again Jonathan,
> > please report what version of mdadm and of Linux that you are running.
>
> My mdadm version is "mdadm - v3.3.1 - 5th June 2014" and my linux
> version (according to `uname -r`) is 3.16.2-3.
>
> >
> > Also is there any chance that you tried
> > mdadm --assemble .... --update=revert-reshape
> > ??
> > And if you did, why didn't you say so :-)
>
> During those first couple days I tried a couple assemble and
> incremental commands to try to get the reshape to restart, though I
> don't remember exactly what they were. It's possible I tried something
> as scary and irreversible as --update=revert-reshape in some desperate
> haze, so let's assume I did, but I was trying hard to avoid any advice
> that included --force in the command.
Try:
mdadm -S /dev/md124
mdadm -A /dev/md124 --update=revert-reshape /dev/md/alpha /dev/md/beta
mdadm -S /dev/md124
mdadm -A /dev/md124 -vvv /dev/md/alpha /dev/md/beta /dev/md/gamma
What does that report?
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Help recovering an interrupted raid0 reshape
2015-04-07 6:30 ` NeilBrown
@ 2015-04-07 17:02 ` Jonathan Harker (Jesusaurus)
2015-04-07 21:13 ` NeilBrown
0 siblings, 1 reply; 12+ messages in thread
From: Jonathan Harker (Jesusaurus) @ 2015-04-07 17:02 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
On Mon, Apr 6, 2015 at 11:30 PM, NeilBrown <neilb@suse.de> wrote:
>
> Try:
> mdadm -S /dev/md124
> mdadm -A /dev/md124 --update=revert-reshape /dev/md/alpha /dev/md/beta
> mdadm -S /dev/md124
> mdadm -A /dev/md124 -vvv /dev/md/alpha /dev/md/beta /dev/md/gamma
>
> What does that report?
>
> NeilBrown
>
# mdadm --stop /dev/md124
mdadm: stopped /dev/md124
# mdadm -A /dev/md124 --update=revert-reshape /dev/md/alpha /dev/md/beta
mdadm: /dev/md124 assembled from 2 drives - not enough to start the array.
# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [raid1] [raid10] [raid0]
[linear] [multipath]
md124 : inactive md126[0](S) md127[1](S)
3907022200 blocks super 1.2
md0 : active raid1 sda5[0] sdb2[1]
107652416 blocks [2/2] [UU]
bitmap: 1/1 pages [4KB], 65536KB chunk
md125 : active raid1 sdh1[0] sdg1[1]
2930134016 blocks super 1.2 [2/2] [UU]
bitmap: 0/22 pages [0KB], 65536KB chunk
md126 : active raid1 sdc1[0] sdd1[1]
1953512312 blocks super 1.2 [2/2] [UU]
md127 : active raid1 sde1[2] sdf1[1]
1953512312 blocks super 1.2 [2/2] [UU]
unused devices: <none>
# mdadm --stop /dev/md124
mdadm: stopped /dev/md124
# mdadm -A /dev/md124 -vvv /dev/md/alpha /dev/md/beta /dev/md/gamma
mdadm: looking for devices for /dev/md124
mdadm: UUID differs from /dev/md0.
mdadm: UUID differs from /dev/md/alpha.
mdadm: UUID differs from /dev/md/beta.
mdadm: UUID differs from /dev/md/gamma.
mdadm: UUID differs from /dev/md0.
mdadm: UUID differs from /dev/md/alpha.
mdadm: UUID differs from /dev/md/beta.
mdadm: UUID differs from /dev/md/gamma.
mdadm: UUID differs from /dev/md0.
mdadm: UUID differs from /dev/md/alpha.
mdadm: UUID differs from /dev/md/beta.
mdadm: UUID differs from /dev/md/gamma.
mdadm: /dev/md/alpha is identified as a member of /dev/md124, slot 1.
mdadm: /dev/md/beta is identified as a member of /dev/md124, slot 0.
mdadm: /dev/md/gamma is identified as a member of /dev/md124, slot 2.
mdadm: :/dev/md124 has an active reshape - checking if critical
section needs to be restored
mdadm: added /dev/md/alpha to /dev/md124 as 1
mdadm: added /dev/md/gamma to /dev/md124 as 2 (possibly out of date)
mdadm: no uptodate device for slot 6 of /dev/md124
mdadm: added /dev/md/beta to /dev/md124 as 0
mdadm: /dev/md124 assembled from 2 drives - not enough to start the array.
# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [raid1] [raid10] [raid0]
[linear] [multipath]
md124 : inactive md125[3](S) md127[1](S) md126[0](S)
6837155192 blocks super 1.2
md0 : active raid1 sda5[0] sdb2[1]
107652416 blocks [2/2] [UU]
bitmap: 0/1 pages [0KB], 65536KB chunk
md125 : active raid1 sdh1[0] sdg1[1]
2930134016 blocks super 1.2 [2/2] [UU]
bitmap: 0/22 pages [0KB], 65536KB chunk
md126 : active raid1 sdc1[0] sdd1[1]
1953512312 blocks super 1.2 [2/2] [UU]
md127 : active raid1 sde1[2] sdf1[1]
1953512312 blocks super 1.2 [2/2] [UU]
unused devices: <none>
# mdadm --examine /dev/md/alpha
/dev/md/alpha:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x4
Array UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
Name : hordern:hordern1 (local to host hordern)
Creation Time : Fri Jan 2 09:59:40 2009
Raid Level : raid4
Raid Devices : 4
Avail Dev Size : 3907021824 (1863.01 GiB 2000.40 GB)
Array Size : 5860532736 (5589.04 GiB 6001.19 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1968 sectors, after=752 sectors
State : active
Device UUID : 63aaa2e4:2a09f495:8372c7f9:eb2f2773
Reshape pos'n : 129067008 (123.09 GiB 132.16 GB)
Delta Devices : 1 (3->4)
Update Time : Sun Mar 29 15:11:35 2015
Checksum : 8be5e0e8 - correct
Events : 14013
Chunk Size : 512K
Device Role : Active device 1
Array State : AA.. ('A' == active, '.' == missing, 'R' == replacing)
# mdadm --examine /dev/md/beta
/dev/md/beta:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x4
Array UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
Name : hordern:hordern1 (local to host hordern)
Creation Time : Fri Jan 2 09:59:40 2009
Raid Level : raid4
Raid Devices : 4
Avail Dev Size : 3907022576 (1863.01 GiB 2000.40 GB)
Array Size : 5860532736 (5589.04 GiB 6001.19 GB)
Used Dev Size : 3907021824 (1863.01 GiB 2000.40 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1968 sectors, after=752 sectors
State : clean
Device UUID : 6e6dce14:3ebb2bb5:187aa292:403a55f6
Reshape pos'n : 129067008 (123.09 GiB 132.16 GB)
Delta Devices : 1 (3->4)
Update Time : Sun Mar 29 15:11:35 2015
Checksum : f7526adf - correct
Events : 14013
Chunk Size : 512K
Device Role : Active device 0
Array State : AA.. ('A' == active, '.' == missing, 'R' == replacing)
# mdadm --examine /dev/md/gamma
/dev/md/gamma:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x6
Array UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
Name : hordern:hordern1 (local to host hordern)
Creation Time : Fri Jan 2 09:59:40 2009
Raid Level : raid4
Raid Devices : 4
Avail Dev Size : 5860265984 (2794.39 GiB 3000.46 GB)
Array Size : 5860532736 (5589.04 GiB 6001.19 GB)
Used Dev Size : 3907021824 (1863.01 GiB 2000.40 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Recovery Offset : 86403072 sectors
Unused Space : before=1960 sectors, after=1953244160 sectors
State : active
Device UUID : 782873ea:e265ecd4:5cc80ddf:035ba2b4
Reshape pos'n : 129067008 (123.09 GiB 132.16 GB)
Delta Devices : 1 (3->4)
Update Time : Sun Mar 29 00:05:29 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 710dc078 - correct
Events : 673
Chunk Size : 512K
Device Role : Active device 2
Array State : AAA. ('A' == active, '.' == missing, 'R' == replacing)
# mdadm --detail /dev/md124
/dev/md124:
Version : 1.2
Raid Level : raid0
Total Devices : 3
Persistence : Superblock is persistent
State : inactive
Delta Devices : 1, (-1->0)
New Level : raid4
New Chunksize : 512K
Name : hordern:hordern1 (local to host hordern)
UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
Events : 673
Number Major Minor RaidDevice
- 9 125 - /dev/md/gamma
- 9 126 - /dev/md/beta
- 9 127 - /dev/md/alpha
So it looks like all three component devices have consistent
superblocks now, awesome! But the raid0 array is still inactive with
all three components listed as spares. It looks like /dev/md/gamma has
a much lower event count, I'm guessing that is what causes the disk to
be marked as possibly out of date.
Is an "uptodate device" a specific thing, or does that simply mean
that some component devices are out of date? The lack of spaces makes
me think that uptodate is some keyword I'm not recognizing.
--
Jonathan
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Help recovering an interrupted raid0 reshape
2015-04-07 17:02 ` Jonathan Harker (Jesusaurus)
@ 2015-04-07 21:13 ` NeilBrown
2015-04-07 22:31 ` Jonathan Harker (Jesusaurus)
0 siblings, 1 reply; 12+ messages in thread
From: NeilBrown @ 2015-04-07 21:13 UTC (permalink / raw)
To: Jonathan Harker (Jesusaurus); +Cc: linux-raid
[-- Attachment #1: Type: text/plain, Size: 8564 bytes --]
On Tue, 7 Apr 2015 10:02:13 -0700 "Jonathan Harker (Jesusaurus)"
<jesusaurus@gentlydownthe.net> wrote:
> On Mon, Apr 6, 2015 at 11:30 PM, NeilBrown <neilb@suse.de> wrote:
> >
> > Try:
> > mdadm -S /dev/md124
> > mdadm -A /dev/md124 --update=revert-reshape /dev/md/alpha /dev/md/beta
> > mdadm -S /dev/md124
> > mdadm -A /dev/md124 -vvv /dev/md/alpha /dev/md/beta /dev/md/gamma
> >
> > What does that report?
> >
> > NeilBrown
> >
>
> # mdadm --stop /dev/md124
> mdadm: stopped /dev/md124
> # mdadm -A /dev/md124 --update=revert-reshape /dev/md/alpha /dev/md/beta
> mdadm: /dev/md124 assembled from 2 drives - not enough to start the array.
> # cat /proc/mdstat
> Personalities : [raid6] [raid5] [raid4] [raid1] [raid10] [raid0]
> [linear] [multipath]
> md124 : inactive md126[0](S) md127[1](S)
> 3907022200 blocks super 1.2
>
> md0 : active raid1 sda5[0] sdb2[1]
> 107652416 blocks [2/2] [UU]
> bitmap: 1/1 pages [4KB], 65536KB chunk
>
> md125 : active raid1 sdh1[0] sdg1[1]
> 2930134016 blocks super 1.2 [2/2] [UU]
> bitmap: 0/22 pages [0KB], 65536KB chunk
>
> md126 : active raid1 sdc1[0] sdd1[1]
> 1953512312 blocks super 1.2 [2/2] [UU]
>
> md127 : active raid1 sde1[2] sdf1[1]
> 1953512312 blocks super 1.2 [2/2] [UU]
>
> unused devices: <none>
> # mdadm --stop /dev/md124
> mdadm: stopped /dev/md124
> # mdadm -A /dev/md124 -vvv /dev/md/alpha /dev/md/beta /dev/md/gamma
> mdadm: looking for devices for /dev/md124
> mdadm: UUID differs from /dev/md0.
> mdadm: UUID differs from /dev/md/alpha.
> mdadm: UUID differs from /dev/md/beta.
> mdadm: UUID differs from /dev/md/gamma.
> mdadm: UUID differs from /dev/md0.
> mdadm: UUID differs from /dev/md/alpha.
> mdadm: UUID differs from /dev/md/beta.
> mdadm: UUID differs from /dev/md/gamma.
> mdadm: UUID differs from /dev/md0.
> mdadm: UUID differs from /dev/md/alpha.
> mdadm: UUID differs from /dev/md/beta.
> mdadm: UUID differs from /dev/md/gamma.
> mdadm: /dev/md/alpha is identified as a member of /dev/md124, slot 1.
> mdadm: /dev/md/beta is identified as a member of /dev/md124, slot 0.
> mdadm: /dev/md/gamma is identified as a member of /dev/md124, slot 2.
> mdadm: :/dev/md124 has an active reshape - checking if critical
> section needs to be restored
> mdadm: added /dev/md/alpha to /dev/md124 as 1
> mdadm: added /dev/md/gamma to /dev/md124 as 2 (possibly out of date)
> mdadm: no uptodate device for slot 6 of /dev/md124
> mdadm: added /dev/md/beta to /dev/md124 as 0
> mdadm: /dev/md124 assembled from 2 drives - not enough to start the array.
> # cat /proc/mdstat
> Personalities : [raid6] [raid5] [raid4] [raid1] [raid10] [raid0]
> [linear] [multipath]
> md124 : inactive md125[3](S) md127[1](S) md126[0](S)
> 6837155192 blocks super 1.2
>
> md0 : active raid1 sda5[0] sdb2[1]
> 107652416 blocks [2/2] [UU]
> bitmap: 0/1 pages [0KB], 65536KB chunk
>
> md125 : active raid1 sdh1[0] sdg1[1]
> 2930134016 blocks super 1.2 [2/2] [UU]
> bitmap: 0/22 pages [0KB], 65536KB chunk
>
> md126 : active raid1 sdc1[0] sdd1[1]
> 1953512312 blocks super 1.2 [2/2] [UU]
>
> md127 : active raid1 sde1[2] sdf1[1]
> 1953512312 blocks super 1.2 [2/2] [UU]
>
> unused devices: <none>
>
> # mdadm --examine /dev/md/alpha
> /dev/md/alpha:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x4
> Array UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
> Name : hordern:hordern1 (local to host hordern)
> Creation Time : Fri Jan 2 09:59:40 2009
> Raid Level : raid4
> Raid Devices : 4
>
> Avail Dev Size : 3907021824 (1863.01 GiB 2000.40 GB)
> Array Size : 5860532736 (5589.04 GiB 6001.19 GB)
> Data Offset : 2048 sectors
> Super Offset : 8 sectors
> Unused Space : before=1968 sectors, after=752 sectors
> State : active
> Device UUID : 63aaa2e4:2a09f495:8372c7f9:eb2f2773
>
> Reshape pos'n : 129067008 (123.09 GiB 132.16 GB)
> Delta Devices : 1 (3->4)
>
> Update Time : Sun Mar 29 15:11:35 2015
> Checksum : 8be5e0e8 - correct
> Events : 14013
>
> Chunk Size : 512K
>
> Device Role : Active device 1
> Array State : AA.. ('A' == active, '.' == missing, 'R' == replacing)
>
> # mdadm --examine /dev/md/beta
> /dev/md/beta:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x4
> Array UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
> Name : hordern:hordern1 (local to host hordern)
> Creation Time : Fri Jan 2 09:59:40 2009
> Raid Level : raid4
> Raid Devices : 4
>
> Avail Dev Size : 3907022576 (1863.01 GiB 2000.40 GB)
> Array Size : 5860532736 (5589.04 GiB 6001.19 GB)
> Used Dev Size : 3907021824 (1863.01 GiB 2000.40 GB)
> Data Offset : 2048 sectors
> Super Offset : 8 sectors
> Unused Space : before=1968 sectors, after=752 sectors
> State : clean
> Device UUID : 6e6dce14:3ebb2bb5:187aa292:403a55f6
>
> Reshape pos'n : 129067008 (123.09 GiB 132.16 GB)
> Delta Devices : 1 (3->4)
>
> Update Time : Sun Mar 29 15:11:35 2015
> Checksum : f7526adf - correct
> Events : 14013
>
> Chunk Size : 512K
>
> Device Role : Active device 0
> Array State : AA.. ('A' == active, '.' == missing, 'R' == replacing)
>
> # mdadm --examine /dev/md/gamma
> /dev/md/gamma:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x6
> Array UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
> Name : hordern:hordern1 (local to host hordern)
> Creation Time : Fri Jan 2 09:59:40 2009
> Raid Level : raid4
> Raid Devices : 4
>
> Avail Dev Size : 5860265984 (2794.39 GiB 3000.46 GB)
> Array Size : 5860532736 (5589.04 GiB 6001.19 GB)
> Used Dev Size : 3907021824 (1863.01 GiB 2000.40 GB)
> Data Offset : 2048 sectors
> Super Offset : 8 sectors
> Recovery Offset : 86403072 sectors
> Unused Space : before=1960 sectors, after=1953244160 sectors
> State : active
> Device UUID : 782873ea:e265ecd4:5cc80ddf:035ba2b4
>
> Reshape pos'n : 129067008 (123.09 GiB 132.16 GB)
> Delta Devices : 1 (3->4)
>
> Update Time : Sun Mar 29 00:05:29 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 710dc078 - correct
> Events : 673
>
> Chunk Size : 512K
>
> Device Role : Active device 2
> Array State : AAA. ('A' == active, '.' == missing, 'R' == replacing)
>
> # mdadm --detail /dev/md124
> /dev/md124:
> Version : 1.2
> Raid Level : raid0
> Total Devices : 3
> Persistence : Superblock is persistent
>
> State : inactive
>
> Delta Devices : 1, (-1->0)
> New Level : raid4
> New Chunksize : 512K
>
> Name : hordern:hordern1 (local to host hordern)
> UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
> Events : 673
>
> Number Major Minor RaidDevice
>
> - 9 125 - /dev/md/gamma
> - 9 126 - /dev/md/beta
> - 9 127 - /dev/md/alpha
>
> So it looks like all three component devices have consistent
> superblocks now, awesome! But the raid0 array is still inactive with
> all three components listed as spares. It looks like /dev/md/gamma has
> a much lower event count, I'm guessing that is what causes the disk to
> be marked as possibly out of date.
>
> Is an "uptodate device" a specific thing, or does that simply mean
> that some component devices are out of date? The lack of spaces makes
> me think that uptodate is some keyword I'm not recognizing.
>
Looks good. Nearly there.
The difference in event counts is probably due to you trying lots of things
out, and them only affecting two devices.
If you
# mdadm --stop /dev/md124
# mdadm -A --force /dev/md124 -vvv /dev/md/alpha /dev/md/beta /dev/md/gamma
i.e. just add --force, it should ignored the difference in event count and
assemble the array.
For RAID0, the event count isn't really relevant to the data as there is no
possibility for inconsistency between data and parity on different devices.
As the reshape position is the same on all devices, I don't think there is
any risk at all in just using --force.
Of course, perform an fsck afterwards just to build confidence.
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Help recovering an interrupted raid0 reshape
2015-04-07 21:13 ` NeilBrown
@ 2015-04-07 22:31 ` Jonathan Harker (Jesusaurus)
2015-04-07 22:56 ` NeilBrown
0 siblings, 1 reply; 12+ messages in thread
From: Jonathan Harker (Jesusaurus) @ 2015-04-07 22:31 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
On Tue, Apr 7, 2015 at 2:13 PM, NeilBrown <neilb@suse.de> wrote:
> On Tue, 7 Apr 2015 10:02:13 -0700 "Jonathan Harker (Jesusaurus)"
> <jesusaurus@gentlydownthe.net> wrote:
>
>> On Mon, Apr 6, 2015 at 11:30 PM, NeilBrown <neilb@suse.de> wrote:
>> >
>> > Try:
>> > mdadm -S /dev/md124
>> > mdadm -A /dev/md124 --update=revert-reshape /dev/md/alpha /dev/md/beta
>> > mdadm -S /dev/md124
>> > mdadm -A /dev/md124 -vvv /dev/md/alpha /dev/md/beta /dev/md/gamma
>> >
>> > What does that report?
>> >
>> > NeilBrown
>> >
>>
>> # mdadm --stop /dev/md124
>> mdadm: stopped /dev/md124
>> # mdadm -A /dev/md124 --update=revert-reshape /dev/md/alpha /dev/md/beta
>> mdadm: /dev/md124 assembled from 2 drives - not enough to start the array.
>> # cat /proc/mdstat
>> Personalities : [raid6] [raid5] [raid4] [raid1] [raid10] [raid0]
>> [linear] [multipath]
>> md124 : inactive md126[0](S) md127[1](S)
>> 3907022200 blocks super 1.2
>>
>> md0 : active raid1 sda5[0] sdb2[1]
>> 107652416 blocks [2/2] [UU]
>> bitmap: 1/1 pages [4KB], 65536KB chunk
>>
>> md125 : active raid1 sdh1[0] sdg1[1]
>> 2930134016 blocks super 1.2 [2/2] [UU]
>> bitmap: 0/22 pages [0KB], 65536KB chunk
>>
>> md126 : active raid1 sdc1[0] sdd1[1]
>> 1953512312 blocks super 1.2 [2/2] [UU]
>>
>> md127 : active raid1 sde1[2] sdf1[1]
>> 1953512312 blocks super 1.2 [2/2] [UU]
>>
>> unused devices: <none>
>> # mdadm --stop /dev/md124
>> mdadm: stopped /dev/md124
>> # mdadm -A /dev/md124 -vvv /dev/md/alpha /dev/md/beta /dev/md/gamma
>> mdadm: looking for devices for /dev/md124
>> mdadm: UUID differs from /dev/md0.
>> mdadm: UUID differs from /dev/md/alpha.
>> mdadm: UUID differs from /dev/md/beta.
>> mdadm: UUID differs from /dev/md/gamma.
>> mdadm: UUID differs from /dev/md0.
>> mdadm: UUID differs from /dev/md/alpha.
>> mdadm: UUID differs from /dev/md/beta.
>> mdadm: UUID differs from /dev/md/gamma.
>> mdadm: UUID differs from /dev/md0.
>> mdadm: UUID differs from /dev/md/alpha.
>> mdadm: UUID differs from /dev/md/beta.
>> mdadm: UUID differs from /dev/md/gamma.
>> mdadm: /dev/md/alpha is identified as a member of /dev/md124, slot 1.
>> mdadm: /dev/md/beta is identified as a member of /dev/md124, slot 0.
>> mdadm: /dev/md/gamma is identified as a member of /dev/md124, slot 2.
>> mdadm: :/dev/md124 has an active reshape - checking if critical
>> section needs to be restored
>> mdadm: added /dev/md/alpha to /dev/md124 as 1
>> mdadm: added /dev/md/gamma to /dev/md124 as 2 (possibly out of date)
>> mdadm: no uptodate device for slot 6 of /dev/md124
>> mdadm: added /dev/md/beta to /dev/md124 as 0
>> mdadm: /dev/md124 assembled from 2 drives - not enough to start the array.
>> # cat /proc/mdstat
>> Personalities : [raid6] [raid5] [raid4] [raid1] [raid10] [raid0]
>> [linear] [multipath]
>> md124 : inactive md125[3](S) md127[1](S) md126[0](S)
>> 6837155192 blocks super 1.2
>>
>> md0 : active raid1 sda5[0] sdb2[1]
>> 107652416 blocks [2/2] [UU]
>> bitmap: 0/1 pages [0KB], 65536KB chunk
>>
>> md125 : active raid1 sdh1[0] sdg1[1]
>> 2930134016 blocks super 1.2 [2/2] [UU]
>> bitmap: 0/22 pages [0KB], 65536KB chunk
>>
>> md126 : active raid1 sdc1[0] sdd1[1]
>> 1953512312 blocks super 1.2 [2/2] [UU]
>>
>> md127 : active raid1 sde1[2] sdf1[1]
>> 1953512312 blocks super 1.2 [2/2] [UU]
>>
>> unused devices: <none>
>>
>> # mdadm --examine /dev/md/alpha
>> /dev/md/alpha:
>> Magic : a92b4efc
>> Version : 1.2
>> Feature Map : 0x4
>> Array UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
>> Name : hordern:hordern1 (local to host hordern)
>> Creation Time : Fri Jan 2 09:59:40 2009
>> Raid Level : raid4
>> Raid Devices : 4
>>
>> Avail Dev Size : 3907021824 (1863.01 GiB 2000.40 GB)
>> Array Size : 5860532736 (5589.04 GiB 6001.19 GB)
>> Data Offset : 2048 sectors
>> Super Offset : 8 sectors
>> Unused Space : before=1968 sectors, after=752 sectors
>> State : active
>> Device UUID : 63aaa2e4:2a09f495:8372c7f9:eb2f2773
>>
>> Reshape pos'n : 129067008 (123.09 GiB 132.16 GB)
>> Delta Devices : 1 (3->4)
>>
>> Update Time : Sun Mar 29 15:11:35 2015
>> Checksum : 8be5e0e8 - correct
>> Events : 14013
>>
>> Chunk Size : 512K
>>
>> Device Role : Active device 1
>> Array State : AA.. ('A' == active, '.' == missing, 'R' == replacing)
>>
>> # mdadm --examine /dev/md/beta
>> /dev/md/beta:
>> Magic : a92b4efc
>> Version : 1.2
>> Feature Map : 0x4
>> Array UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
>> Name : hordern:hordern1 (local to host hordern)
>> Creation Time : Fri Jan 2 09:59:40 2009
>> Raid Level : raid4
>> Raid Devices : 4
>>
>> Avail Dev Size : 3907022576 (1863.01 GiB 2000.40 GB)
>> Array Size : 5860532736 (5589.04 GiB 6001.19 GB)
>> Used Dev Size : 3907021824 (1863.01 GiB 2000.40 GB)
>> Data Offset : 2048 sectors
>> Super Offset : 8 sectors
>> Unused Space : before=1968 sectors, after=752 sectors
>> State : clean
>> Device UUID : 6e6dce14:3ebb2bb5:187aa292:403a55f6
>>
>> Reshape pos'n : 129067008 (123.09 GiB 132.16 GB)
>> Delta Devices : 1 (3->4)
>>
>> Update Time : Sun Mar 29 15:11:35 2015
>> Checksum : f7526adf - correct
>> Events : 14013
>>
>> Chunk Size : 512K
>>
>> Device Role : Active device 0
>> Array State : AA.. ('A' == active, '.' == missing, 'R' == replacing)
>>
>> # mdadm --examine /dev/md/gamma
>> /dev/md/gamma:
>> Magic : a92b4efc
>> Version : 1.2
>> Feature Map : 0x6
>> Array UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
>> Name : hordern:hordern1 (local to host hordern)
>> Creation Time : Fri Jan 2 09:59:40 2009
>> Raid Level : raid4
>> Raid Devices : 4
>>
>> Avail Dev Size : 5860265984 (2794.39 GiB 3000.46 GB)
>> Array Size : 5860532736 (5589.04 GiB 6001.19 GB)
>> Used Dev Size : 3907021824 (1863.01 GiB 2000.40 GB)
>> Data Offset : 2048 sectors
>> Super Offset : 8 sectors
>> Recovery Offset : 86403072 sectors
>> Unused Space : before=1960 sectors, after=1953244160 sectors
>> State : active
>> Device UUID : 782873ea:e265ecd4:5cc80ddf:035ba2b4
>>
>> Reshape pos'n : 129067008 (123.09 GiB 132.16 GB)
>> Delta Devices : 1 (3->4)
>>
>> Update Time : Sun Mar 29 00:05:29 2015
>> Bad Block Log : 512 entries available at offset 72 sectors
>> Checksum : 710dc078 - correct
>> Events : 673
>>
>> Chunk Size : 512K
>>
>> Device Role : Active device 2
>> Array State : AAA. ('A' == active, '.' == missing, 'R' == replacing)
>>
>> # mdadm --detail /dev/md124
>> /dev/md124:
>> Version : 1.2
>> Raid Level : raid0
>> Total Devices : 3
>> Persistence : Superblock is persistent
>>
>> State : inactive
>>
>> Delta Devices : 1, (-1->0)
>> New Level : raid4
>> New Chunksize : 512K
>>
>> Name : hordern:hordern1 (local to host hordern)
>> UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
>> Events : 673
>>
>> Number Major Minor RaidDevice
>>
>> - 9 125 - /dev/md/gamma
>> - 9 126 - /dev/md/beta
>> - 9 127 - /dev/md/alpha
>>
>> So it looks like all three component devices have consistent
>> superblocks now, awesome! But the raid0 array is still inactive with
>> all three components listed as spares. It looks like /dev/md/gamma has
>> a much lower event count, I'm guessing that is what causes the disk to
>> be marked as possibly out of date.
>>
>> Is an "uptodate device" a specific thing, or does that simply mean
>> that some component devices are out of date? The lack of spaces makes
>> me think that uptodate is some keyword I'm not recognizing.
>>
>
> Looks good. Nearly there.
>
> The difference in event counts is probably due to you trying lots of things
> out, and them only affecting two devices.
>
> If you
> # mdadm --stop /dev/md124
> # mdadm -A --force /dev/md124 -vvv /dev/md/alpha /dev/md/beta /dev/md/gamma
>
> i.e. just add --force, it should ignored the difference in event count and
> assemble the array.
> For RAID0, the event count isn't really relevant to the data as there is no
> possibility for inconsistency between data and parity on different devices.
> As the reshape position is the same on all devices, I don't think there is
> any risk at all in just using --force.
> Of course, perform an fsck afterwards just to build confidence.
>
> NeilBrown
>
Unfortunately, adding --force didn't seem to make any difference:
# mdadm --stop /dev/md124
mdadm: stopped /dev/md124
# mdadm -A --force /dev/md124 -vvv /dev/md/alpha /dev/md/beta /dev/md/gamma
mdadm: looking for devices for /dev/md124
mdadm: UUID differs from /dev/md0.
mdadm: UUID differs from /dev/md/alpha.
mdadm: UUID differs from /dev/md/beta.
mdadm: UUID differs from /dev/md/gamma.
mdadm: UUID differs from /dev/md0.
mdadm: UUID differs from /dev/md/alpha.
mdadm: UUID differs from /dev/md/beta.
mdadm: UUID differs from /dev/md/gamma.
mdadm: UUID differs from /dev/md0.
mdadm: UUID differs from /dev/md/alpha.
mdadm: UUID differs from /dev/md/beta.
mdadm: UUID differs from /dev/md/gamma.
mdadm: /dev/md/alpha is identified as a member of /dev/md124, slot 1.
mdadm: /dev/md/beta is identified as a member of /dev/md124, slot 0.
mdadm: /dev/md/gamma is identified as a member of /dev/md124, slot 2.
mdadm: :/dev/md124 has an active reshape - checking if critical
section needs to be restored
mdadm: added /dev/md/alpha to /dev/md124 as 1
mdadm: added /dev/md/gamma to /dev/md124 as 2 (possibly out of date)
mdadm: no uptodate device for slot 6 of /dev/md124
mdadm: added /dev/md/beta to /dev/md124 as 0
mdadm: /dev/md124 assembled from 2 drives - not enough to start the array.
# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [raid1] [raid10] [raid0]
[linear] [multipath]
md124 : inactive md125[3](S) md127[1](S) md126[0](S)
6837155192 blocks super 1.2
md0 : active raid1 sda5[0] sdb2[1]
107652416 blocks [2/2] [UU]
bitmap: 0/1 pages [0KB], 65536KB chunk
md125 : active raid1 sdh1[0] sdg1[1]
2930134016 blocks super 1.2 [2/2] [UU]
bitmap: 0/22 pages [0KB], 65536KB chunk
md126 : active raid1 sdc1[0] sdd1[1]
1953512312 blocks super 1.2 [2/2] [UU]
md127 : active raid1 sde1[2] sdf1[1]
1953512312 blocks super 1.2 [2/2] [UU]
unused devices: <none>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Help recovering an interrupted raid0 reshape
2015-04-07 22:31 ` Jonathan Harker (Jesusaurus)
@ 2015-04-07 22:56 ` NeilBrown
2015-04-07 23:24 ` Jonathan Harker (Jesusaurus)
0 siblings, 1 reply; 12+ messages in thread
From: NeilBrown @ 2015-04-07 22:56 UTC (permalink / raw)
To: Jonathan Harker (Jesusaurus); +Cc: linux-raid
[-- Attachment #1: Type: text/plain, Size: 12018 bytes --]
On Tue, 7 Apr 2015 15:31:32 -0700 "Jonathan Harker (Jesusaurus)"
<jesusaurus@gentlydownthe.net> wrote:
> On Tue, Apr 7, 2015 at 2:13 PM, NeilBrown <neilb@suse.de> wrote:
> > On Tue, 7 Apr 2015 10:02:13 -0700 "Jonathan Harker (Jesusaurus)"
> > <jesusaurus@gentlydownthe.net> wrote:
> >
> >> On Mon, Apr 6, 2015 at 11:30 PM, NeilBrown <neilb@suse.de> wrote:
> >> >
> >> > Try:
> >> > mdadm -S /dev/md124
> >> > mdadm -A /dev/md124 --update=revert-reshape /dev/md/alpha /dev/md/beta
> >> > mdadm -S /dev/md124
> >> > mdadm -A /dev/md124 -vvv /dev/md/alpha /dev/md/beta /dev/md/gamma
> >> >
> >> > What does that report?
> >> >
> >> > NeilBrown
> >> >
> >>
> >> # mdadm --stop /dev/md124
> >> mdadm: stopped /dev/md124
> >> # mdadm -A /dev/md124 --update=revert-reshape /dev/md/alpha /dev/md/beta
> >> mdadm: /dev/md124 assembled from 2 drives - not enough to start the array.
> >> # cat /proc/mdstat
> >> Personalities : [raid6] [raid5] [raid4] [raid1] [raid10] [raid0]
> >> [linear] [multipath]
> >> md124 : inactive md126[0](S) md127[1](S)
> >> 3907022200 blocks super 1.2
> >>
> >> md0 : active raid1 sda5[0] sdb2[1]
> >> 107652416 blocks [2/2] [UU]
> >> bitmap: 1/1 pages [4KB], 65536KB chunk
> >>
> >> md125 : active raid1 sdh1[0] sdg1[1]
> >> 2930134016 blocks super 1.2 [2/2] [UU]
> >> bitmap: 0/22 pages [0KB], 65536KB chunk
> >>
> >> md126 : active raid1 sdc1[0] sdd1[1]
> >> 1953512312 blocks super 1.2 [2/2] [UU]
> >>
> >> md127 : active raid1 sde1[2] sdf1[1]
> >> 1953512312 blocks super 1.2 [2/2] [UU]
> >>
> >> unused devices: <none>
> >> # mdadm --stop /dev/md124
> >> mdadm: stopped /dev/md124
> >> # mdadm -A /dev/md124 -vvv /dev/md/alpha /dev/md/beta /dev/md/gamma
> >> mdadm: looking for devices for /dev/md124
> >> mdadm: UUID differs from /dev/md0.
> >> mdadm: UUID differs from /dev/md/alpha.
> >> mdadm: UUID differs from /dev/md/beta.
> >> mdadm: UUID differs from /dev/md/gamma.
> >> mdadm: UUID differs from /dev/md0.
> >> mdadm: UUID differs from /dev/md/alpha.
> >> mdadm: UUID differs from /dev/md/beta.
> >> mdadm: UUID differs from /dev/md/gamma.
> >> mdadm: UUID differs from /dev/md0.
> >> mdadm: UUID differs from /dev/md/alpha.
> >> mdadm: UUID differs from /dev/md/beta.
> >> mdadm: UUID differs from /dev/md/gamma.
> >> mdadm: /dev/md/alpha is identified as a member of /dev/md124, slot 1.
> >> mdadm: /dev/md/beta is identified as a member of /dev/md124, slot 0.
> >> mdadm: /dev/md/gamma is identified as a member of /dev/md124, slot 2.
> >> mdadm: :/dev/md124 has an active reshape - checking if critical
> >> section needs to be restored
> >> mdadm: added /dev/md/alpha to /dev/md124 as 1
> >> mdadm: added /dev/md/gamma to /dev/md124 as 2 (possibly out of date)
> >> mdadm: no uptodate device for slot 6 of /dev/md124
> >> mdadm: added /dev/md/beta to /dev/md124 as 0
> >> mdadm: /dev/md124 assembled from 2 drives - not enough to start the array.
> >> # cat /proc/mdstat
> >> Personalities : [raid6] [raid5] [raid4] [raid1] [raid10] [raid0]
> >> [linear] [multipath]
> >> md124 : inactive md125[3](S) md127[1](S) md126[0](S)
> >> 6837155192 blocks super 1.2
> >>
> >> md0 : active raid1 sda5[0] sdb2[1]
> >> 107652416 blocks [2/2] [UU]
> >> bitmap: 0/1 pages [0KB], 65536KB chunk
> >>
> >> md125 : active raid1 sdh1[0] sdg1[1]
> >> 2930134016 blocks super 1.2 [2/2] [UU]
> >> bitmap: 0/22 pages [0KB], 65536KB chunk
> >>
> >> md126 : active raid1 sdc1[0] sdd1[1]
> >> 1953512312 blocks super 1.2 [2/2] [UU]
> >>
> >> md127 : active raid1 sde1[2] sdf1[1]
> >> 1953512312 blocks super 1.2 [2/2] [UU]
> >>
> >> unused devices: <none>
> >>
> >> # mdadm --examine /dev/md/alpha
> >> /dev/md/alpha:
> >> Magic : a92b4efc
> >> Version : 1.2
> >> Feature Map : 0x4
> >> Array UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
> >> Name : hordern:hordern1 (local to host hordern)
> >> Creation Time : Fri Jan 2 09:59:40 2009
> >> Raid Level : raid4
> >> Raid Devices : 4
> >>
> >> Avail Dev Size : 3907021824 (1863.01 GiB 2000.40 GB)
> >> Array Size : 5860532736 (5589.04 GiB 6001.19 GB)
> >> Data Offset : 2048 sectors
> >> Super Offset : 8 sectors
> >> Unused Space : before=1968 sectors, after=752 sectors
> >> State : active
> >> Device UUID : 63aaa2e4:2a09f495:8372c7f9:eb2f2773
> >>
> >> Reshape pos'n : 129067008 (123.09 GiB 132.16 GB)
> >> Delta Devices : 1 (3->4)
> >>
> >> Update Time : Sun Mar 29 15:11:35 2015
> >> Checksum : 8be5e0e8 - correct
> >> Events : 14013
> >>
> >> Chunk Size : 512K
> >>
> >> Device Role : Active device 1
> >> Array State : AA.. ('A' == active, '.' == missing, 'R' == replacing)
> >>
> >> # mdadm --examine /dev/md/beta
> >> /dev/md/beta:
> >> Magic : a92b4efc
> >> Version : 1.2
> >> Feature Map : 0x4
> >> Array UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
> >> Name : hordern:hordern1 (local to host hordern)
> >> Creation Time : Fri Jan 2 09:59:40 2009
> >> Raid Level : raid4
> >> Raid Devices : 4
> >>
> >> Avail Dev Size : 3907022576 (1863.01 GiB 2000.40 GB)
> >> Array Size : 5860532736 (5589.04 GiB 6001.19 GB)
> >> Used Dev Size : 3907021824 (1863.01 GiB 2000.40 GB)
> >> Data Offset : 2048 sectors
> >> Super Offset : 8 sectors
> >> Unused Space : before=1968 sectors, after=752 sectors
> >> State : clean
> >> Device UUID : 6e6dce14:3ebb2bb5:187aa292:403a55f6
> >>
> >> Reshape pos'n : 129067008 (123.09 GiB 132.16 GB)
> >> Delta Devices : 1 (3->4)
> >>
> >> Update Time : Sun Mar 29 15:11:35 2015
> >> Checksum : f7526adf - correct
> >> Events : 14013
> >>
> >> Chunk Size : 512K
> >>
> >> Device Role : Active device 0
> >> Array State : AA.. ('A' == active, '.' == missing, 'R' == replacing)
> >>
> >> # mdadm --examine /dev/md/gamma
> >> /dev/md/gamma:
> >> Magic : a92b4efc
> >> Version : 1.2
> >> Feature Map : 0x6
> >> Array UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
> >> Name : hordern:hordern1 (local to host hordern)
> >> Creation Time : Fri Jan 2 09:59:40 2009
> >> Raid Level : raid4
> >> Raid Devices : 4
> >>
> >> Avail Dev Size : 5860265984 (2794.39 GiB 3000.46 GB)
> >> Array Size : 5860532736 (5589.04 GiB 6001.19 GB)
> >> Used Dev Size : 3907021824 (1863.01 GiB 2000.40 GB)
> >> Data Offset : 2048 sectors
> >> Super Offset : 8 sectors
> >> Recovery Offset : 86403072 sectors
> >> Unused Space : before=1960 sectors, after=1953244160 sectors
> >> State : active
> >> Device UUID : 782873ea:e265ecd4:5cc80ddf:035ba2b4
> >>
> >> Reshape pos'n : 129067008 (123.09 GiB 132.16 GB)
> >> Delta Devices : 1 (3->4)
> >>
> >> Update Time : Sun Mar 29 00:05:29 2015
> >> Bad Block Log : 512 entries available at offset 72 sectors
> >> Checksum : 710dc078 - correct
> >> Events : 673
> >>
> >> Chunk Size : 512K
> >>
> >> Device Role : Active device 2
> >> Array State : AAA. ('A' == active, '.' == missing, 'R' == replacing)
> >>
> >> # mdadm --detail /dev/md124
> >> /dev/md124:
> >> Version : 1.2
> >> Raid Level : raid0
> >> Total Devices : 3
> >> Persistence : Superblock is persistent
> >>
> >> State : inactive
> >>
> >> Delta Devices : 1, (-1->0)
> >> New Level : raid4
> >> New Chunksize : 512K
> >>
> >> Name : hordern:hordern1 (local to host hordern)
> >> UUID : 1f4979ba:c49a77c0:59e689c2:bcc21c0a
> >> Events : 673
> >>
> >> Number Major Minor RaidDevice
> >>
> >> - 9 125 - /dev/md/gamma
> >> - 9 126 - /dev/md/beta
> >> - 9 127 - /dev/md/alpha
> >>
> >> So it looks like all three component devices have consistent
> >> superblocks now, awesome! But the raid0 array is still inactive with
> >> all three components listed as spares. It looks like /dev/md/gamma has
> >> a much lower event count, I'm guessing that is what causes the disk to
> >> be marked as possibly out of date.
> >>
> >> Is an "uptodate device" a specific thing, or does that simply mean
> >> that some component devices are out of date? The lack of spaces makes
> >> me think that uptodate is some keyword I'm not recognizing.
> >>
> >
> > Looks good. Nearly there.
> >
> > The difference in event counts is probably due to you trying lots of things
> > out, and them only affecting two devices.
> >
> > If you
> > # mdadm --stop /dev/md124
> > # mdadm -A --force /dev/md124 -vvv /dev/md/alpha /dev/md/beta /dev/md/gamma
> >
> > i.e. just add --force, it should ignored the difference in event count and
> > assemble the array.
> > For RAID0, the event count isn't really relevant to the data as there is no
> > possibility for inconsistency between data and parity on different devices.
> > As the reshape position is the same on all devices, I don't think there is
> > any risk at all in just using --force.
> > Of course, perform an fsck afterwards just to build confidence.
> >
> > NeilBrown
> >
>
> Unfortunately, adding --force didn't seem to make any difference:
>
> # mdadm --stop /dev/md124
> mdadm: stopped /dev/md124
> # mdadm -A --force /dev/md124 -vvv /dev/md/alpha /dev/md/beta /dev/md/gamma
> mdadm: looking for devices for /dev/md124
> mdadm: UUID differs from /dev/md0.
> mdadm: UUID differs from /dev/md/alpha.
> mdadm: UUID differs from /dev/md/beta.
> mdadm: UUID differs from /dev/md/gamma.
> mdadm: UUID differs from /dev/md0.
> mdadm: UUID differs from /dev/md/alpha.
> mdadm: UUID differs from /dev/md/beta.
> mdadm: UUID differs from /dev/md/gamma.
> mdadm: UUID differs from /dev/md0.
> mdadm: UUID differs from /dev/md/alpha.
> mdadm: UUID differs from /dev/md/beta.
> mdadm: UUID differs from /dev/md/gamma.
> mdadm: /dev/md/alpha is identified as a member of /dev/md124, slot 1.
> mdadm: /dev/md/beta is identified as a member of /dev/md124, slot 0.
> mdadm: /dev/md/gamma is identified as a member of /dev/md124, slot 2.
> mdadm: :/dev/md124 has an active reshape - checking if critical
> section needs to be restored
> mdadm: added /dev/md/alpha to /dev/md124 as 1
> mdadm: added /dev/md/gamma to /dev/md124 as 2 (possibly out of date)
> mdadm: no uptodate device for slot 6 of /dev/md124
> mdadm: added /dev/md/beta to /dev/md124 as 0
> mdadm: /dev/md124 assembled from 2 drives - not enough to start the array.
> # cat /proc/mdstat
> Personalities : [raid6] [raid5] [raid4] [raid1] [raid10] [raid0]
> [linear] [multipath]
> md124 : inactive md125[3](S) md127[1](S) md126[0](S)
> 6837155192 blocks super 1.2
>
> md0 : active raid1 sda5[0] sdb2[1]
> 107652416 blocks [2/2] [UU]
> bitmap: 0/1 pages [0KB], 65536KB chunk
>
> md125 : active raid1 sdh1[0] sdg1[1]
> 2930134016 blocks super 1.2 [2/2] [UU]
> bitmap: 0/22 pages [0KB], 65536KB chunk
>
> md126 : active raid1 sdc1[0] sdd1[1]
> 1953512312 blocks super 1.2 [2/2] [UU]
>
> md127 : active raid1 sde1[2] sdf1[1]
> 1953512312 blocks super 1.2 [2/2] [UU]
>
> unused devices: <none>
Hmm... I think I see the bug. It should be easy enough to fix, but I'd like
to be able to test it.
Could you please:
mkdir /tmp/md.metadata
mdadm --dump /tmp/md.metadata /dev/md/alpha /dev/md/beta /dev/md/gamma
tar czSf /tmp/md.tgz /tmp/md.metadata
and then send me /tmp/md.tgz, which should be tiny and contain just the
metadata from the array.
[[the patch which introduced the problem has a description which starts
"This is a bit of a hack and ..."
Never accept hacks!
]]
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Help recovering an interrupted raid0 reshape
2015-04-07 22:56 ` NeilBrown
@ 2015-04-07 23:24 ` Jonathan Harker (Jesusaurus)
2015-04-08 2:09 ` NeilBrown
0 siblings, 1 reply; 12+ messages in thread
From: Jonathan Harker (Jesusaurus) @ 2015-04-07 23:24 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
On Tue, Apr 7, 2015 at 3:56 PM, NeilBrown <neilb@suse.de> wrote:
>
>
> Hmm... I think I see the bug. It should be easy enough to fix, but I'd like
> to be able to test it.
> Could you please:
>
> mkdir /tmp/md.metadata
> mdadm --dump /tmp/md.metadata /dev/md/alpha /dev/md/beta /dev/md/gamma
> tar czSf /tmp/md.tgz /tmp/md.metadata
>
> and then send me /tmp/md.tgz, which should be tiny and contain just the
> metadata from the array.
I'll send that to you once tar finishes.
>
> [[the patch which introduced the problem has a description which starts
> "This is a bit of a hack and ..."
> Never accept hacks!
> ]]
I've reviewed and accepted a few hacks in my day as well. Hopefully
the cleanup isn't too ugly.
Thanks again for looking into this.
--
Jonathan
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Help recovering an interrupted raid0 reshape
2015-04-07 23:24 ` Jonathan Harker (Jesusaurus)
@ 2015-04-08 2:09 ` NeilBrown
2015-04-08 17:00 ` Jonathan Harker (Jesusaurus)
0 siblings, 1 reply; 12+ messages in thread
From: NeilBrown @ 2015-04-08 2:09 UTC (permalink / raw)
To: Jonathan Harker (Jesusaurus); +Cc: linux-raid
[-- Attachment #1: Type: text/plain, Size: 1411 bytes --]
On Tue, 7 Apr 2015 16:24:35 -0700 "Jonathan Harker (Jesusaurus)"
<jesusaurus@gentlydownthe.net> wrote:
> On Tue, Apr 7, 2015 at 3:56 PM, NeilBrown <neilb@suse.de> wrote:
> >
> >
> > Hmm... I think I see the bug. It should be easy enough to fix, but I'd like
> > to be able to test it.
> > Could you please:
> >
> > mkdir /tmp/md.metadata
> > mdadm --dump /tmp/md.metadata /dev/md/alpha /dev/md/beta /dev/md/gamma
> > tar czSf /tmp/md.tgz /tmp/md.metadata
> >
> > and then send me /tmp/md.tgz, which should be tiny and contain just the
> > metadata from the array.
>
> I'll send that to you once tar finishes.
Thanks. And just as well I tested as there was another problem.
I think it is all fixed now.
If you:
cd /tmp
git clone git://neil.brown.name/mdadm
cd mdadm
make
./mdadm -A /dev/md124 --force -vvvv /dev/md/{alpha,beta,gamma}
it should assemble the array and continue the reshape.
When it finishes it will leave the array as degraded RAID4.
Simply:
mdadm /dev/md125 --grow --level=0
and it will instantly become RAID0.
NeilBrown
>
> >
> > [[the patch which introduced the problem has a description which starts
> > "This is a bit of a hack and ..."
> > Never accept hacks!
> > ]]
>
> I've reviewed and accepted a few hacks in my day as well. Hopefully
> the cleanup isn't too ugly.
> Thanks again for looking into this.
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Help recovering an interrupted raid0 reshape
2015-04-08 2:09 ` NeilBrown
@ 2015-04-08 17:00 ` Jonathan Harker (Jesusaurus)
0 siblings, 0 replies; 12+ messages in thread
From: Jonathan Harker (Jesusaurus) @ 2015-04-08 17:00 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
On Tue, Apr 7, 2015 at 7:09 PM, NeilBrown <neilb@suse.de> wrote:
> On Tue, 7 Apr 2015 16:24:35 -0700 "Jonathan Harker (Jesusaurus)"
> <jesusaurus@gentlydownthe.net> wrote:
>
>> On Tue, Apr 7, 2015 at 3:56 PM, NeilBrown <neilb@suse.de> wrote:
>> >
>> >
>> > Hmm... I think I see the bug. It should be easy enough to fix, but I'd like
>> > to be able to test it.
>> > Could you please:
>> >
>> > mkdir /tmp/md.metadata
>> > mdadm --dump /tmp/md.metadata /dev/md/alpha /dev/md/beta /dev/md/gamma
>> > tar czSf /tmp/md.tgz /tmp/md.metadata
>> >
>> > and then send me /tmp/md.tgz, which should be tiny and contain just the
>> > metadata from the array.
>>
>> I'll send that to you once tar finishes.
>
> Thanks. And just as well I tested as there was another problem.
> I think it is all fixed now.
> If you:
>
> cd /tmp
> git clone git://neil.brown.name/mdadm
> cd mdadm
> make
> ./mdadm -A /dev/md124 --force -vvvv /dev/md/{alpha,beta,gamma}
>
> it should assemble the array and continue the reshape.
>
> When it finishes it will leave the array as degraded RAID4.
> Simply:
> mdadm /dev/md125 --grow --level=0
>
> and it will instantly become RAID0.
>
> NeilBrown
>
>>
>> >
>> > [[the patch which introduced the problem has a description which starts
>> > "This is a bit of a hack and ..."
>> > Never accept hacks!
>> > ]]
>>
>> I've reviewed and accepted a few hacks in my day as well. Hopefully
>> the cleanup isn't too ugly.
>> Thanks again for looking into this.
>>
>
Awesome, the interrupted reshape has restarted! And in about 900 short
minutes I should be able to reshape back to a raid0.
Thanks again for all your help.
--
Jonathan
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2015-04-08 17:00 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-04-04 2:22 Help recovering an interrupted raid0 reshape Jonathan Harker (Jesusaurus)
2015-04-06 23:46 ` NeilBrown
2015-04-07 1:50 ` NeilBrown
2015-04-07 5:14 ` Jonathan Harker (Jesusaurus)
2015-04-07 6:30 ` NeilBrown
2015-04-07 17:02 ` Jonathan Harker (Jesusaurus)
2015-04-07 21:13 ` NeilBrown
2015-04-07 22:31 ` Jonathan Harker (Jesusaurus)
2015-04-07 22:56 ` NeilBrown
2015-04-07 23:24 ` Jonathan Harker (Jesusaurus)
2015-04-08 2:09 ` NeilBrown
2015-04-08 17:00 ` Jonathan Harker (Jesusaurus)
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox