linux-raid.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Can't expand linear RAID on top of 2 x RAID1
@ 2012-06-22 10:41 Adam Goryachev
  2012-06-22 23:05 ` Stan Hoeppner
  0 siblings, 1 reply; 7+ messages in thread
From: Adam Goryachev @ 2012-06-22 10:41 UTC (permalink / raw)
  To: linux-raid

I have expanded my system over time, I started with 2 x 2TB drives in
RAID1 (md2)

I then added 2 x 750GB drives, configured as RAID1 (md1)
Then created a third raid (md3) as linear with the md2 + md1

Finally, I've upgraded the 2 x 750G to 2 x 1TB drives (one at a time).

I then did a mdadm --grow to expand the RAID1 from 750G to 1TB

The problem I am having is that I can't expand the linear (md3) array to
grow the extra 250G of space.

Could anyone suggest how I might be able to get the extra 250G of space
to become available?

keep:~# cat /proc/mdstat
Personalities : [raid1] [linear]
md3 : active linear md2[0] md1[1]
      2686081193 blocks super 1.2 0k rounding
     
md1 : active (auto-read-only) raid1 sdb1[2] sdd1[3]
      976758977 blocks super 1.2 [2/2] [UU]
     
md2 : active (auto-read-only) raid1 sda1[0] sdc1[2]
      1953510841 blocks super 1.2 [2/2] [UU]
     
unused devices: <none>

keep:~# mdadm --grow /dev/md3 --size=max
mdadm: component size of /dev/md3 has been set to 0K

There is nothing logged for dmesg after the above command, no change to
/proc/mdstat.

I have tried a reboot with no change either.

Thanks for any assistance, I'd really appreciate it.

Regards,
Adam

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Can't expand linear RAID on top of 2 x RAID1
  2012-06-22 10:41 Can't expand linear RAID on top of 2 x RAID1 Adam Goryachev
@ 2012-06-22 23:05 ` Stan Hoeppner
  2012-06-23  2:53   ` Adam Goryachev
  0 siblings, 1 reply; 7+ messages in thread
From: Stan Hoeppner @ 2012-06-22 23:05 UTC (permalink / raw)
  To: Adam Goryachev; +Cc: linux-raid

On 6/22/2012 5:41 AM, Adam Goryachev wrote:
> I have expanded my system over time, I started with 2 x 2TB drives in
> RAID1 (md2)
> 
> I then added 2 x 750GB drives, configured as RAID1 (md1)
> Then created a third raid (md3) as linear with the md2 + md1
> 
> Finally, I've upgraded the 2 x 750G to 2 x 1TB drives (one at a time).
> 
> I then did a mdadm --grow to expand the RAID1 from 750G to 1TB
> 
> The problem I am having is that I can't expand the linear (md3) array to
> grow the extra 250G of space.
> 
> Could anyone suggest how I might be able to get the extra 250G of space
> to become available?

If you think about this for a few minutes more, and re-read how md
--linear works, and thus how growing a linear array works, you'll surely
understand why you can't do what you're attempting to do.

As for seeing that extra 250GB, I don't have an answer.  Typically
linear arrays are used in lieu of growing constituent member arrays.
That's kinda the whole point of linear (concatenation).

You could try deleting the linear array and simply creating a new one.
But surely the changed offsets would wreak havoc on the filesystem
currently spanning this mess.

-- 
Stan

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Can't expand linear RAID on top of 2 x RAID1
  2012-06-22 23:05 ` Stan Hoeppner
@ 2012-06-23  2:53   ` Adam Goryachev
  2012-06-23 12:18     ` Stan Hoeppner
  2012-06-25  2:52     ` NeilBrown
  0 siblings, 2 replies; 7+ messages in thread
From: Adam Goryachev @ 2012-06-23  2:53 UTC (permalink / raw)
  To: stan; +Cc: linux-raid

Stan Hoeppner <stan@hardwarefreak.com> wrote:

>On 6/22/2012 5:41 AM, Adam Goryachev wrote:
>> I have expanded my system over time, I started with 2 x 2TB drives in
>> RAID1 (md2)
>> 
>> I then added 2 x 750GB drives, configured as RAID1 (md1)
>> Then created a third raid (md3) as linear with the md2 + md1
>> 
>> Finally, I've upgraded the 2 x 750G to 2 x 1TB drives (one at a
>time).
>> 
>> I then did a mdadm --grow to expand the RAID1 from 750G to 1TB
>> 
>> The problem I am having is that I can't expand the linear (md3) array
>to
>> grow the extra 250G of space.
>> 
>> Could anyone suggest how I might be able to get the extra 250G of
>space
>> to become available?
>
>If you think about this for a few minutes more, and re-read how md
>--linear works, and thus how growing a linear array works, you'll
>surely
>understand why you can't do what you're attempting to do.

Actually, no I don't understand the problem... From my (probably limited) understanding, linear simply appends each drive to the array. I understand that you would not be able to increase the size of any constituent device which is not the last component of the array, but I don't see any reason why it is not possible to increase the size of the last component. (Which is what I am attempting to do)

>As for seeing that extra 250GB, I don't have an answer.  Typically
>linear arrays are used in lieu of growing constituent member arrays.
>That's kinda the whole point of linear (concatenation).

Well, the only other alternative I can think of is to reduce the size of the array back to 750G, reduce the size of the partitions for that array to 750G. Then create a new 250G partition on the two 1TB drives, create a fourth RAID1 array, and then add that 4th RAID1 array to the end of the existing linear array. This just didn't seem like a very good solution (ie, not clean).

>You could try deleting the linear array and simply creating a new one.
>But surely the changed offsets would wreak havoc on the filesystem
>currently spanning this mess.

Perhaps this would work. If I delete the linear array and re-create, with assume-clean (or whatever the right flag is, I'll read the man page later when I'm doing it), then like I said, the first component device will be the same, the second component device is the same, but just happens to be bigger. Any comments on whether this should work?

What should I do to ensure that the component devices are ordered the way I think they are? Apparently the numbers in /proc/mdstat just tell me what order the devices were added, not what order they are in the array?

Thank you for your advice.

Regards,
Adam




^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Can't expand linear RAID on top of 2 x RAID1
  2012-06-23  2:53   ` Adam Goryachev
@ 2012-06-23 12:18     ` Stan Hoeppner
  2012-06-24 14:17       ` Stan Hoeppner
  2012-06-25  2:52     ` NeilBrown
  1 sibling, 1 reply; 7+ messages in thread
From: Stan Hoeppner @ 2012-06-23 12:18 UTC (permalink / raw)
  To: Adam Goryachev; +Cc: linux-raid

On 6/22/2012 9:53 PM, Adam Goryachev wrote:
> Stan Hoeppner <stan@hardwarefreak.com> wrote:
> 
>> On 6/22/2012 5:41 AM, Adam Goryachev wrote:
>>> I have expanded my system over time, I started with 2 x 2TB drives in
>>> RAID1 (md2)
>>>
>>> I then added 2 x 750GB drives, configured as RAID1 (md1)
>>> Then created a third raid (md3) as linear with the md2 + md1
>>>
>>> Finally, I've upgraded the 2 x 750G to 2 x 1TB drives (one at a
>> time).
>>>
>>> I then did a mdadm --grow to expand the RAID1 from 750G to 1TB
>>>
>>> The problem I am having is that I can't expand the linear (md3) array
>> to
>>> grow the extra 250G of space.
>>>
>>> Could anyone suggest how I might be able to get the extra 250G of
>> space
>>> to become available?
>>
>> If you think about this for a few minutes more, and re-read how md
>> --linear works, and thus how growing a linear array works, you'll
>> surely
>> understand why you can't do what you're attempting to do.
> 
> Actually, no I don't understand the problem... From my (probably limited) understanding, linear simply appends each drive to the array. I understand that you would not be able to increase the size of any constituent device which is not the last component of the array, but I don't see any reason why it is not possible to increase the size of the last component. (Which is what I am attempting to do)
> 
>> As for seeing that extra 250GB, I don't have an answer.  Typically
>> linear arrays are used in lieu of growing constituent member arrays.
>> That's kinda the whole point of linear (concatenation).
> 
> Well, the only other alternative I can think of is to reduce the size of the array back to 750G, reduce the size of the partitions for that array to 750G. Then create a new 250G partition on the two 1TB drives, create a fourth RAID1 array, and then add that 4th RAID1 array to the end of the existing linear array. This just didn't seem like a very good solution (ie, not clean).

This is the only sure fire way I see ATM to do this.  But I agree, it's
not very clean.  I'm a big proponent of only one array per physical
disk, as I stated recently, and the reasons for it.  Given what I'm
guessing your workload profile is, it wouldn't make a difference.

>> You could try deleting the linear array and simply creating a new one.
>> But surely the changed offsets would wreak havoc on the filesystem
>> currently spanning this mess.
> 
> Perhaps this would work. If I delete the linear array and re-create, with assume-clean (or whatever the right flag is, I'll read the man page later when I'm doing it), then like I said, the first component device will be the same, the second component device is the same, but just happens to be bigger. Any comments on whether this should work?
> 
> What should I do to ensure that the component devices are ordered the way I think they are? Apparently the numbers in /proc/mdstat just tell me what order the devices were added, not what order they are in the array?
> 
> Thank you for your advice.

The best advice I can give you is to wait for Neil Brown to weigh in
before you make any such changes.

-- 
Stan


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Can't expand linear RAID on top of 2 x RAID1
  2012-06-23 12:18     ` Stan Hoeppner
@ 2012-06-24 14:17       ` Stan Hoeppner
  0 siblings, 0 replies; 7+ messages in thread
From: Stan Hoeppner @ 2012-06-24 14:17 UTC (permalink / raw)
  To: stan; +Cc: Adam Goryachev, linux-raid, NeilBrown

Hay Neil,

Could you please take a look at this one?  I'm at the limits of my
knowledge in this case, not exactly sure what the OP's next step should be.

Thanks.

-- 
Stan


On 6/23/2012 7:18 AM, Stan Hoeppner wrote:
> On 6/22/2012 9:53 PM, Adam Goryachev wrote:
>> Stan Hoeppner <stan@hardwarefreak.com> wrote:
>>
>>> On 6/22/2012 5:41 AM, Adam Goryachev wrote:
>>>> I have expanded my system over time, I started with 2 x 2TB drives in
>>>> RAID1 (md2)
>>>>
>>>> I then added 2 x 750GB drives, configured as RAID1 (md1)
>>>> Then created a third raid (md3) as linear with the md2 + md1
>>>>
>>>> Finally, I've upgraded the 2 x 750G to 2 x 1TB drives (one at a
>>> time).
>>>>
>>>> I then did a mdadm --grow to expand the RAID1 from 750G to 1TB
>>>>
>>>> The problem I am having is that I can't expand the linear (md3) array
>>> to
>>>> grow the extra 250G of space.
>>>>
>>>> Could anyone suggest how I might be able to get the extra 250G of
>>> space
>>>> to become available?
>>>
>>> If you think about this for a few minutes more, and re-read how md
>>> --linear works, and thus how growing a linear array works, you'll
>>> surely
>>> understand why you can't do what you're attempting to do.
>>
>> Actually, no I don't understand the problem... From my (probably limited) understanding, linear simply appends each drive to the array. I understand that you would not be able to increase the size of any constituent device which is not the last component of the array, but I don't see any reason why it is not possible to increase the size of the last component. (Which is what I am attempting to do)
>>
>>> As for seeing that extra 250GB, I don't have an answer.  Typically
>>> linear arrays are used in lieu of growing constituent member arrays.
>>> That's kinda the whole point of linear (concatenation).
>>
>> Well, the only other alternative I can think of is to reduce the size of the array back to 750G, reduce the size of the partitions for that array to 750G. Then create a new 250G partition on the two 1TB drives, create a fourth RAID1 array, and then add that 4th RAID1 array to the end of the existing linear array. This just didn't seem like a very good solution (ie, not clean).
> 
> This is the only sure fire way I see ATM to do this.  But I agree, it's
> not very clean.  I'm a big proponent of only one array per physical
> disk, as I stated recently, and the reasons for it.  Given what I'm
> guessing your workload profile is, it wouldn't make a difference.
> 
>>> You could try deleting the linear array and simply creating a new one.
>>> But surely the changed offsets would wreak havoc on the filesystem
>>> currently spanning this mess.
>>
>> Perhaps this would work. If I delete the linear array and re-create, with assume-clean (or whatever the right flag is, I'll read the man page later when I'm doing it), then like I said, the first component device will be the same, the second component device is the same, but just happens to be bigger. Any comments on whether this should work?
>>
>> What should I do to ensure that the component devices are ordered the way I think they are? Apparently the numbers in /proc/mdstat just tell me what order the devices were added, not what order they are in the array?
>>
>> Thank you for your advice.
> 
> The best advice I can give you is to wait for Neil Brown to weigh in
> before you make any such changes.
> 



^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Can't expand linear RAID on top of 2 x RAID1
  2012-06-23  2:53   ` Adam Goryachev
  2012-06-23 12:18     ` Stan Hoeppner
@ 2012-06-25  2:52     ` NeilBrown
  2012-06-29  7:53       ` Adam Goryachev
  1 sibling, 1 reply; 7+ messages in thread
From: NeilBrown @ 2012-06-25  2:52 UTC (permalink / raw)
  To: Adam Goryachev; +Cc: stan, linux-raid

[-- Attachment #1: Type: text/plain, Size: 3925 bytes --]

On Sat, 23 Jun 2012 12:53:12 +1000 Adam Goryachev
<mailinglists@websitemanagers.com.au> wrote:

> Stan Hoeppner <stan@hardwarefreak.com> wrote:
> 
> >On 6/22/2012 5:41 AM, Adam Goryachev wrote:
> >> I have expanded my system over time, I started with 2 x 2TB drives in
> >> RAID1 (md2)
> >> 
> >> I then added 2 x 750GB drives, configured as RAID1 (md1)
> >> Then created a third raid (md3) as linear with the md2 + md1
> >> 
> >> Finally, I've upgraded the 2 x 750G to 2 x 1TB drives (one at a
> >time).
> >> 
> >> I then did a mdadm --grow to expand the RAID1 from 750G to 1TB
> >> 
> >> The problem I am having is that I can't expand the linear (md3) array
> >to
> >> grow the extra 250G of space.
> >> 
> >> Could anyone suggest how I might be able to get the extra 250G of
> >space
> >> to become available?
> >
> >If you think about this for a few minutes more, and re-read how md
> >--linear works, and thus how growing a linear array works, you'll
> >surely
> >understand why you can't do what you're attempting to do.
> 
> Actually, no I don't understand the problem... From my (probably limited) understanding, linear simply appends each drive to the array. I understand that you would not be able to increase the size of any constituent device which is not the last component of the array, but I don't see any reason why it is not possible to increase the size of the last component. (Which is what I am attempting to do)
> 
> >As for seeing that extra 250GB, I don't have an answer.  Typically
> >linear arrays are used in lieu of growing constituent member arrays.
> >That's kinda the whole point of linear (concatenation).
> 
> Well, the only other alternative I can think of is to reduce the size of the array back to 750G, reduce the size of the partitions for that array to 750G. Then create a new 250G partition on the two 1TB drives, create a fourth RAID1 array, and then add that 4th RAID1 array to the end of the existing linear array. This just didn't seem like a very good solution (ie, not clean).
> 
> >You could try deleting the linear array and simply creating a new one.
> >But surely the changed offsets would wreak havoc on the filesystem
> >currently spanning this mess.
> 
> Perhaps this would work. If I delete the linear array and re-create, with assume-clean (or whatever the right flag is, I'll read the man page later when I'm doing it), then like I said, the first component device will be the same, the second component device is the same, but just happens to be bigger. Any comments on whether this should work?

The is no '--assume-clean' for linear.  Linear arrays cannot be clean or
dirty as md stores no redundant information.

Just re-creating the array would probably work, but there is an easier way.

md doesn't record the size of the array for Linear and RAID0.  It just adds
together the sizes of the devices that it finds.

With 1.x metadata, md does record the effective size of each devices, so adds
those together.
You can update this effective size by assembling with 
   --update=devicesize

I just checked this will loop devices and it works as expected (assuming you
have 1.1 or 1.2 metadata).
So:
  mdadm -S /dev/md2
  mdadm -A --update=devicesize /dev/md0 /dev/md1

(order doesn't matter with assemble).

> 
> What should I do to ensure that the component devices are ordered the way I think they are? Apparently the numbers in /proc/mdstat just tell me what order the devices were added, not what order they are in the array?

mdadm --detail  or mdadm --examine would tell you what you want to know.

NeilBrown


> 
> Thank you for your advice.
> 
> Regards,
> Adam
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Can't expand linear RAID on top of 2 x RAID1
  2012-06-25  2:52     ` NeilBrown
@ 2012-06-29  7:53       ` Adam Goryachev
  0 siblings, 0 replies; 7+ messages in thread
From: Adam Goryachev @ 2012-06-29  7:53 UTC (permalink / raw)
  To: linux-raid

On 25/06/12 12:52, NeilBrown wrote:
> The is no '--assume-clean' for linear.  Linear arrays cannot be
> clean or dirty as md stores no redundant information.
> 
> Just re-creating the array would probably work, but there is an
> easier way.
> 
> md doesn't record the size of the array for Linear and RAID0.  It
> just adds together the sizes of the devices that it finds.
> 
> With 1.x metadata, md does record the effective size of each
> devices, so adds those together. You can update this effective size
> by assembling with --update=devicesize
> 
> I just checked this will loop devices and it works as expected
> (assuming you have 1.1 or 1.2 metadata). So: mdadm -S /dev/md2 
> mdadm -A --update=devicesize /dev/md0 /dev/md1
> 
> (order doesn't matter with assemble).

Hi,

Just a followup email on this to say Thank You very much for your
expert assistance, it was greatly appreciated.

> mdadm -S /dev/md2

I had some issues with this command, df showed that it was unmounted,
lsof showed nothing looking at the device nor filesystem where it was
mounted, etc... Eventually, I ran "mount" and realised I had bind
mounted a sub-directory elsewhere... Once I umounted that I could stop
the array... (Duh moment :)

> mdadm -A --update=devicesize /dev/md0 /dev/md1
I got an error that the device was in use, thankfully mdadm has the
smarts to stop me from idly shooting myself in the foot when doing
something as silly as copy & paste instructions from others without
stopping and thinking first.

I adjusted the command to the correct:
mdadm -A --update=devicesize /dev/md2 /dev/md0 /dev/md1

and got:
Size was 3907019634
Size is 3907019634
Size was 1465142752
Size is 1953515906

I then checked the array, was told the filesystem was clean, and then
resized that, and now I have the extra 250G space (just in time, I
only had 6% free space there).

So, again, thank you for your assistance, and thanks for all the work
you put into the drivers etc.

Regards,
Adam


^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2012-06-29  7:53 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-06-22 10:41 Can't expand linear RAID on top of 2 x RAID1 Adam Goryachev
2012-06-22 23:05 ` Stan Hoeppner
2012-06-23  2:53   ` Adam Goryachev
2012-06-23 12:18     ` Stan Hoeppner
2012-06-24 14:17       ` Stan Hoeppner
2012-06-25  2:52     ` NeilBrown
2012-06-29  7:53       ` Adam Goryachev

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).