Re: Growing RAID1 array with bitmaps enabled

[Bryan Mesich <bryan.mesich@ndsu.edu>]

[-- Attachment #1: Type: text/plain, Size: 3307 bytes --]

On Fri, Dec 19, 2008 at 03:54:33PM +1100, Neil Brown wrote:

[snip...]
>  
> > 1) Does the superblock increase in size when the array is grown 
> > to make room for a larger bitmap?  (This doesn't seem possible 
> > to me, particularly if using a v1.1 or v1.2 superblock).
> 
> The bitmap does not live in the superblock.  It lives near the
> superblock. For 0.90, the superblock is 4K in size, and lives at 
> least 60K from the end of the device.  The bitmap lives in that 60K.
> 
> For 1.0, the superblock is 1K in size and lives between 8K and 12K
> from the end of the device.  There is usually some space reserved
> before the superblock which is not used for data.  The bitmap lives in
> one of these spaces.
> 
> For 1.1 and 1.2, the superblock is 1K in size and lives near the start
> of the device.  There is usually some space reserved after the
> superblock and before the data.  The bitmap is stored there.
>
 
Thanks for the explanation and correction. 

[snip...]

> > With this in mind (and the adage that patches are welcome), has there 
> > been any interest in being able to use a write-intent bitmap for this 
> > kind of use case?  The benefit of this functionality would keep me 
> > from eating two _full_ re-syncs on the array when growing the block 
> > devices.
> 
> I think you misunderstand.  There is no need for a full resync.
> You simply
> 
>   mdadm --grow /dev/md0 --bitmap none
>   mdadm --grow /dev/md0 --size max
>   mdadm --wait /dev/md0
>   mdadm --grow /dev/md0 --bitmap internal
> 
> The "--wait" will wait while the array syncs the data from 40GB to
> 100GB.

Before I can grow the array (using your example), I need to fail
and remove the devices from the array so they can be grown on the
FC target.  Something like this:

    # Fail and remove the device that is to be grown
    mdadm /dev/md0 --fail /dev/sdc1
    mdadm /dev/md0 --remove /dev/sdc1

    # Remove the device from the system
    echo '1' >| /sys/block/sdc/device/delete

Now that the exported device has been removed from the initiator,
I can grow the volume on the target.

    # Scan for the block device that is now 100GB
    echo "- - -" >| /sys/class/scsi_host/host3/scan

With the system now seeing the larger block device, I can resize
the partition (I'll need to use v1.1 or v1.2 suberblock to
prevent the superblock from getting clobbered).

    # Add device back into the array
    mdadm /dev/md0 --add /dev/sdc1

This is where I would like to use a write-intent bitmap.  When the 
re-sync finishes, I would do the same to the other device (and
hopefully use the bitmap for its re-sync).   

Your email about the bitmap not growing has me a bit gun shy.  Can 
I use a write-intent bitmap in this situation without shooting 
myself in the foot?

After both devices have been grown, I would grow the array just
as you pointed out in your example:

   mdadm --grow /dev/md0 --bitmap none
   mdadm --grow /dev/md0 --size max
   mdadm --wait /dev/md0
   mdadm --grow /dev/md0 --bitmap internal

If I understand you correctly, its the "mdadm --grow /dev/md0
--size max" that will bite me if I have a bitmap enabled.  Thus,
you removed the bitmap from the array while growing.

Thanks in advance,

Bryan

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]