raid1 + writemostly

raid1 + writemostly

[Jon Nelson]

According to the manpage for mdadm, --write-mostly can only be used
add build, create, or add time.  How does one set write-mostly devices
*after* they've been added?

mdadm /dev/mdX --write-mostly /dev/someDevice
succeeds but does nothing.

According to Documentation/md.txt, writing writemostly into
/sys/block/mdX/md/dev-someDevice/state
should do it, and it sort of appears to:

turnip:~ # mdadm --detail /dev/md11
/dev/md11:
        Version : 01.00.03
  Creation Time : Mon Dec 15 07:06:13 2008
     Raid Level : raid1
     Array Size : 78123988 (74.50 GiB 80.00 GB)
  Used Dev Size : 156247976 (149.01 GiB 160.00 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 11
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Mon Dec 15 12:21:20 2008
          State : active
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

           Name : turnip:11
           UUID : cf24d099:9e174a79:2a2f6797:dcff1420
         Events : 3840

    Number   Major   Minor   RaidDevice State
       2      43        0        0      active sync   /dev/nbd0
       3       8        0        1      active sync   /dev/sda
turnip:~ # echo writemostly > /sys/block/md11/md/dev-nbd0/state
turnip:~ # mdadm --detail /dev/md11
/dev/md11:
        Version : 01.00.03
  Creation Time : Mon Dec 15 07:06:13 2008
     Raid Level : raid1
     Array Size : 78123988 (74.50 GiB 80.00 GB)
  Used Dev Size : 156247976 (149.01 GiB 160.00 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 11
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Mon Dec 15 12:23:05 2008
          State : active
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

           Name : turnip:11
           UUID : cf24d099:9e174a79:2a2f6797:dcff1420
         Events : 3840

    Number   Major   Minor   RaidDevice State
       2      43        0        0      active sync writemostly   /dev/nbd0
       3       8        0        1      active sync   /dev/sda
turnip:~ # mdadm --examine-bitmap /dev/nbd0
        Filename : /dev/nbd0
           Magic : 6d746962
         Version : 4
            UUID : cf24d099:9e174a79:2a2f6797:dcff1420
          Events : 3840
  Events Cleared : 3840
           State : OK
       Chunksize : 4 MB
          Daemon : 5s flush period
      Write Mode : Normal
       Sync Size : 78123988 (74.50 GiB 80.00 GB)
          Bitmap : 19074 bits (chunks), 0 dirty (0.0%)
turnip:~ #


Why doesn't --examine-bitmap show the right mode?

Also, removing the bitmap from the array does not zero out the bitmap
on the actual device. Shouldn't it do that?
turnip:~ # mdadm --grow /dev/md11 --bitmap=none
turnip:~ # mdadm --examine-bitmap /dev/nbd0
        Filename : /dev/nbd0
           Magic : 6d746962
         Version : 4
            UUID : cf24d099:9e174a79:2a2f6797:dcff1420
          Events : 3841
  Events Cleared : 3841
           State : OK
       Chunksize : 4 MB
          Daemon : 5s flush period
      Write Mode : Normal
       Sync Size : 78123988 (74.50 GiB 80.00 GB)
          Bitmap : 19074 bits (chunks), 0 dirty (0.0%)
turnip:~ #

This is with mdadm 2.6.4 on 2.6.25.18-0.2-default
I can try mdadm 3.0 if you like.

-- 
Jon

Re: raid1 + writemostly

[Neil Brown]

On Monday December 15, jnelson-linux-raid@jamponi.net wrote:
> According to the manpage for mdadm, --write-mostly can only be used
> add build, create, or add time.  How does one set write-mostly devices
> *after* they've been added?

You cannot.
The easiest approach is to remove it and re-add it with the
write-mostly flag.

> 
> mdadm /dev/mdX --write-mostly /dev/someDevice
> succeeds but does nothing.
> 
> According to Documentation/md.txt, writing writemostly into
> /sys/block/mdX/md/dev-someDevice/state
> should do it, and it sort of appears to:
> 
> turnip:~ # mdadm --detail /dev/md11
> /dev/md11:
...
>     Number   Major   Minor   RaidDevice State
>        2      43        0        0      active sync   /dev/nbd0
>        3       8        0        1      active sync   /dev/sda
> turnip:~ # echo writemostly > /sys/block/md11/md/dev-nbd0/state
> turnip:~ # mdadm --detail /dev/md11
> /dev/md11:
..
>     Number   Major   Minor   RaidDevice State
>        2      43        0        0      active sync writemostly   /dev/nbd0
>        3       8        0        1      active sync   /dev/sda
> turnip:~ # mdadm --examine-bitmap /dev/nbd0
>         Filename : /dev/nbd0
>            Magic : 6d746962
>          Version : 4
>             UUID : cf24d099:9e174a79:2a2f6797:dcff1420
>           Events : 3840
>   Events Cleared : 3840
>            State : OK
>        Chunksize : 4 MB
>           Daemon : 5s flush period
>       Write Mode : Normal
>        Sync Size : 78123988 (74.50 GiB 80.00 GB)
>           Bitmap : 19074 bits (chunks), 0 dirty (0.0%)
> turnip:~ #
> 
> 
> Why doesn't --examine-bitmap show the right mode?

What were you expecting?  The bitmap never reports anything about
writemostly, only write-behind.

> 
> Also, removing the bitmap from the array does not zero out the bitmap
> on the actual device. Shouldn't it do that?

No.  It simply records in the array metadata that there is no bitmap.
It might be sensible to get "--examine-bitmap" to report that the
bitmap is not active in some way.

NeilBrown


> turnip:~ # mdadm --grow /dev/md11 --bitmap=none
> turnip:~ # mdadm --examine-bitmap /dev/nbd0
>         Filename : /dev/nbd0
>            Magic : 6d746962
>          Version : 4
>             UUID : cf24d099:9e174a79:2a2f6797:dcff1420
>           Events : 3841
>   Events Cleared : 3841
>            State : OK
>        Chunksize : 4 MB
>           Daemon : 5s flush period
>       Write Mode : Normal
>        Sync Size : 78123988 (74.50 GiB 80.00 GB)
>           Bitmap : 19074 bits (chunks), 0 dirty (0.0%)
> turnip:~ #
> 
> This is with mdadm 2.6.4 on 2.6.25.18-0.2-default
> I can try mdadm 3.0 if you like.
> 
> -- 
> Jon
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: raid1 + writemostly

[Jon Nelson]

On Mon, Dec 15, 2008 at 3:26 PM, Neil Brown <neilb@suse.de> wrote:
> On Monday December 15, jnelson-linux-raid@jamponi.net wrote:
>> According to the manpage for mdadm, --write-mostly can only be used
>> add build, create, or add time.  How does one set write-mostly devices
>> *after* they've been added?
>
> You cannot.
> The easiest approach is to remove it and re-add it with the
> write-mostly flag.

OK. However, why does fiddling with the
/sys/block/mdX/dev-someDevice/state seem to do it?


>> Why doesn't --examine-bitmap show the right mode?
>
> What were you expecting?  The bitmap never reports anything about
> writemostly, only write-behind.

Yes, I was expecting write-behind. D'oh!

>> Also, removing the bitmap from the array does not zero out the bitmap
>> on the actual device. Shouldn't it do that?
>
> No.  It simply records in the array metadata that there is no bitmap.
> It might be sensible to get "--examine-bitmap" to report that the
> bitmap is not active in some way.

I'd like to offer my perspective on that - I feel as though when
bitmaps are removed from arrays that the actual bitmap data should be
zeroed (or marked as invalid somehow). What we have is a situation
wherein a device that has never had a bitmap will say more or less
just that when queried. However, a device that has had a bitmap at
*any* point in the past (provided it's not been overwritten) will
display that bitmap, no matter how crazy it is.

-- 
Jon