Re: Incorrect in-kernel bitmap on raid10

NeilBrown
2009-05-01T21:36:45Z

On Sat, May 2, 2009 3:55 am, Mario 'BitKoenig' Holbe wrote:
> On Fri, May 01, 2009 at 12:11:43PM +1000, Neil Brown wrote:
>> There some other places
>> where are are overflowing on a shift. One of those (in
>> bitmap_dirty_bits) can cause the problem you see.
>> This patch should fix it. Please confirm.
>
> Together with the small syntax-fix attached this patch fixes the
> allocation of half of the available pages only. Now, all pages are
> allocated when I set all bits and they all get cleaned in-kernel as well
> as on-disk.

Good. Thanks for the confirmation (and fix).

>
> However, can you confirm that the bitmap is really used in raid10
> resync? I removed half of the disks (a correctly removable subset, of
> course :)), copied 100G to the degraded array, got about 7k bit set in
> the bitmap, (re-)added the removed devices (mdadm correctly states
> re-add as well), but the resync looks *very* sequential.
> Moreover: I stopped and re-assembled the array with about 2k bit left
> set and the resync starts from the beginning, I can see no skip to the
> previous position in the resync process.
> I'll try to watch this and will trigger you again when I have more
> stable evidence, but perhaps you have some faster test-cases, I have to
> wait for at least 5 hours now :)

I just did some testing and it does seem to honour the bitmap during
recovery. However there are some caveats.

1/ it processes the whole array from start to finish in chunk-sized blocks
and simply doesn't generate IO where it isn't needed. This is different
to e.g. raid1 where it can skip over a whole bitmap-chunk at at time.
So it does use more CPU
2/ With raid1, when it skips a whole bitmap chunk, that chunk is not
included in the speed calculation. With raid10, everything is included.
So I found the resync was hitting the limit of 200M/sec and backing off.
I increased the limited (Added a few more zeros) and it sped up.
3/ I found a bug. If you have two devices missing and add just one,
then after the recovery it might clear the bitmap even though
there is another missing device. When that device is re-added, it will
be added with no recovery. This is bad. I'll post a patch shortly.

What speed are (were) you getting for resync. If it was around 200M/sec,
then point 2 would explain it. If it was closer to the device speed,
then there must be something else going wrong.

NeilBrown