Re: [patch 3/4] fastboot: make the raid autodetect code wait for all devices to init

Re: [patch 3/4] fastboot: make the raid autodetect code wait for all devices to init

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 748 bytes --]

also sprach Neil Brown <neilb@suse.de> [2008.07.21.0106 +0200]:
> The "real" solution here involves assembling arrays in userspace using
> "mdadm --incremental" from udevd, and using write-intent-bitmaps so
> that writing to an array before all the component devices are
> available can be done without requiring a full resync.  There is still
> a bit more code needed to make that work really smoothly.

It was my understanding that write-intent bitmaps slow down all
operations and are not suggested on e.g. workstations. No?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"i think, therefore i'm single"
                                                      -- lizz winstead
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [patch 3/4] fastboot: make the raid autodetect code wait for all devices to init

[Neil Brown]

On Monday July 21, madduck@madduck.net wrote:
> also sprach Neil Brown <neilb@suse.de> [2008.07.21.0106 +0200]:
> > The "real" solution here involves assembling arrays in userspace using
> > "mdadm --incremental" from udevd, and using write-intent-bitmaps so
> > that writing to an array before all the component devices are
> > available can be done without requiring a full resync.  There is still
> > a bit more code needed to make that work really smoothly.
> 
> It was my understanding that write-intent bitmaps slow down all
> operations and are not suggested on e.g. workstations. No?

Well, they don't slow down reads.
If you have a separate root filesystem (i.e. /home and /var are
elsewhere), it is likely to be read-mostly, so bitmaps probably won't
hurt much.
And an external bitmap on a dedicated device has minimal performance
cost.

However I neither suggest having nor not-having bitmaps.  The choice to
use them involves a trade-off which I cannot make for other people.

They would, however, be very useful to cover the gap when assembling
arrays incrementally.
If, for example, you have a 6 disk raid5 array and 5 disks have been
found, what do you do?  
  - wait for the 6th, that might never arrive
  - start degraded and if a write happens before the 6th disk arrives,
    have to rebuild the 6th disk completely.

Neither is a good option.
An alternate is
  - add an internal bitmap, and remove it after the 6th disk has
    arrived, or after we are sure there are no more disks to find.

Doing this means that if a recovery is needed when the 6th disk
arrives, it will be very fast.

It's not hard to notice that the bitmap proposed here does not need to
be on stable storage.  It is not protecting against a crash, just
against a window when the array is degraded.  So if we could support
bitmaps on a tmpfs, we could use an external bitmap in /tmp instead of
an internal bitmap.
Or even - we could enhance the md code to always use a bitmap, but
simply not write it to storage if no such was configured.

(If a crash happens during that window between writing to the degraded
array and recovering the few blocks needed on the final device, then
you would be in an unfortunate position.  For raid1/10 you would just
need a full resync, which you would have needed anyway, so no loss.
For raid4/5/6, you have a potential for dataloss, so I probably would
not make this behaviour the default for those levels...)

NeilBrown

Re: [patch 3/4] fastboot: make the raid autodetect code wait for all devices to init

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 2777 bytes --]

[462617@bugs.debian.org added to Cc]

also sprach Neil Brown <neilb@suse.de> [2008.07.22.0124 +0200]:
> > It was my understanding that write-intent bitmaps slow down all
> > operations and are not suggested on e.g. workstations. No?
> 
> Well, they don't slow down reads.
> If you have a separate root filesystem (i.e. /home and /var are
> elsewhere), it is likely to be read-mostly, so bitmaps probably won't
> hurt much.
> And an external bitmap on a dedicated device has minimal performance
> cost.
> 
> However I neither suggest having nor not-having bitmaps.  The choice to
> use them involves a trade-off which I cannot make for other people.

Ok, in the light of http://bugs.debian.org/462617, this answers my
question and Debian will not make them default. It would be nice to
have the option though, during installation.

Including the rest of your reply for the sake of the bug record,
which is on Cc:

> They would, however, be very useful to cover the gap when assembling
> arrays incrementally.
> If, for example, you have a 6 disk raid5 array and 5 disks have been
> found, what do you do?  
>   - wait for the 6th, that might never arrive
>   - start degraded and if a write happens before the 6th disk arrives,
>     have to rebuild the 6th disk completely.
> 
> Neither is a good option.
> An alternate is
>   - add an internal bitmap, and remove it after the 6th disk has
>     arrived, or after we are sure there are no more disks to find.
> 
> Doing this means that if a recovery is needed when the 6th disk
> arrives, it will be very fast.
> 
> It's not hard to notice that the bitmap proposed here does not need to
> be on stable storage.  It is not protecting against a crash, just
> against a window when the array is degraded.  So if we could support
> bitmaps on a tmpfs, we could use an external bitmap in /tmp instead of
> an internal bitmap.
> Or even - we could enhance the md code to always use a bitmap, but
> simply not write it to storage if no such was configured.

This would be very nice. In fact, everything that makes md better
and requires less complexity in the mdadm packaging is nice. :)

> (If a crash happens during that window between writing to the degraded
> array and recovering the few blocks needed on the final device, then
> you would be in an unfortunate position.  For raid1/10 you would just
> need a full resync, which you would have needed anyway, so no loss.
> For raid4/5/6, you have a potential for dataloss, so I probably would
> not make this behaviour the default for those levels...)

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
drink canada dry! you might not succeed, but it *is* fun trying.
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 197 bytes --]