Re: RAID1 growth of version 1.1/1.2 metadata violates least surprise.

[NeilBrown <neilb@suse.de>]

[-- Attachment #1: Type: text/plain, Size: 2858 bytes --]

On Sun, 9 Jun 2013 09:33:05 -0500 "Dr. Greg Wettstein"
<greg@wind.enjellic.com> wrote:

> Hi, hope the week is starting out well for everyone.
> 
> We ran into an interesting issue secondary to the growth of a RAID1
> volume with version 1.1 metadata.  We are interested in reflections
> from Neil and/or others as to whether the problem can be addressed.
> 
> We grew a RAID1 mirror which had its two individual legs provided by
> SAN volumes from an SCST based target server.  We provisioned
> additional storage on the targets and then updated the advertised size
> of the two block devices.
> 
> Following that we carried out a rescan of the block devices on the
> initiator and then used mdadm to 'grow' the size of the RAID1 mirror.
> That was about 4-5 months ago and we ended up running into issues when
> the machine was just recently rebooted.
> 
> The RAID1 mirror refused to assemble secondary to the superblocks on
> the individual devices having a different idea of the volume size as
> compared to the actual volume size.  The mdadm manpage makes reference
> to this problem in the section on the '-U' (update) functionality.
> 
> It would seem to violate the notion of 'least surprise' for a grow
> command to work only to end up with a device which would not assemble
> without external intervention at some distant time in the future.  It
> isn't uncommon to have systems up for a year or more which tends to
> result in this issue being overlooked on systems which have been
> resized.
> 
> I'm assuming there is no technical reason why the superblocks on the
> individual devices cannot be updated as part of the grow.  Events such
> as adding/removing persistent bitmaps suggest this is a possibility.
> If its an issue of needing code we could investigate doing the
> legwork since it is an issue we would like to see addressed.
> 
> This behavior was experienced on a largely stock RHEL5 box with the
> following versions:
> 
> 	Kernel: 2.6.18-348.6.1.el5
> 	mdadm:	v2.6.9
> 
> Obviously running Oracle.... :-)
> 
> Any thoughts/suggestions would be appreciated.
> 
> Have a good week.

(sorry for the delay - I guess I was having too much fun that week....)

Your description of events doesn't fit with my understanding of how things
should work.
It would help a lot to include actual error messages and "mdadm --examine"
details of devices.

Certainly if you provision underlying devices to be larger and then "grow"
the array, then the superblocks should be updated and everything should work
after a reboot.
If you were using 0.90 or 1.0 metadata which is at the end of the device
there could be confusion when reprovisioning devices (that is mainly with
--update=devicesize is for), but 1.1 metadata should not be affected.

So: I need details - sorry.

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]