Re: [PATCH -next v3 6/7] md: factor out a helper rdev_addable() from remove_and_add_spares()

[Yu Kuai <yukuai1@huaweicloud.com>]

Hi,

在 2023/08/23 13:26, Song Liu 写道:
> On Tue, Aug 22, 2023 at 8:04 PM Yu Kuai <yukuai1@huaweicloud.com> wrote:
>>
>> Hi,
>>
>> 在 2023/08/22 10:17, Yu Kuai 写道:
>>> Hi,
>>>
>>> 在 2023/08/22 7:22, Song Liu 写道:
>>>> On Sun, Aug 20, 2023 at 2:13 AM Yu Kuai <yukuai1@huaweicloud.com> wrote:
>>>>>
>>>>> From: Yu Kuai <yukuai3@huawei.com>
>>>>>
>>>>> There are no functional changes, just to make the code simpler and
>>>>> prepare to delay remove_and_add_spares() to md_start_sync().
>>>>>
>>>>> Signed-off-by: Yu Kuai <yukuai3@huawei.com>
>>>>> ---
>>>>>    drivers/md/md.c | 28 ++++++++++++++++------------
>>>>>    1 file changed, 16 insertions(+), 12 deletions(-)
>>>>>
>>>>> diff --git a/drivers/md/md.c b/drivers/md/md.c
>>>>> index 11d27c934fdd..cdc361c521d4 100644
>>>>> --- a/drivers/md/md.c
>>>>> +++ b/drivers/md/md.c
>>>>> @@ -9177,6 +9177,20 @@ static bool rdev_is_spare(struct md_rdev *rdev)
>>>>>                  !test_bit(Faulty, &rdev->flags);
>>>>>    }
>>>>>
>>>>> +static bool rdev_addable(struct md_rdev *rdev)
>>>>> +{
>>>>> +       if (test_bit(Candidate, &rdev->flags) || rdev->raid_disk >= 0 ||
>>>>> +           test_bit(Faulty, &rdev->flags))
>>>>> +               return false;
>>>>> +
>>>>> +       if (!test_bit(Journal, &rdev->flags) &&
>>>>> !md_is_rdwr(rdev->mddev) &&
>>>>
>>>> Instead of straightforward refactoring, I hope we can make these rdev_*
>>>> helpers more meaningful, and hopefullly reusable. For example, let's
>>>> define
>>>> the meaning of "addable", and write the function to match that
>>>> meaning. In
>>>> this case, I think we shouldn't check md_is_rdwr() inside rdev_addable().
>>>>
>>>> Does this make sense?
>>>
>>> Yes, this make sense, rdev can be added to read-only array.
>>>
>>> There are total three callers of pers->hot_add_sisk, I'll try to find if
>>> they have common conditions.
>>
>> Unfortunately, the conditions is quite different, and It's difficult to
>> factor out a common helper for them to use.
>>
>> In this case, !md_is_rdwr() is one of the four conditions, which means
>> if the array is read-only, there is a special case that rdev can't be
>> added to the configuration. Perhaps it's okay to keep this?
> 
> My main concern is that rdev_addable() is not making the code easier to
> understand. We have a few different cases at this point:
> 
> 1. rdev is not suitable for add (Faulty, raid_disk>=0, Candidate);
> 2. rdev is Journal;
> 3. Re-add rdev to RO array;
> 4. Non-re-add rdev to RO array;
> 5. Other cases.
> 
> Current rdev_addable() handles more or less all of this, which is
> confusing. Maybe we can do something along similar to the
> following (not tested). Does this look more clear?
> 
> Thanks,
> Song
> 
> diff --git i/drivers/md/md.c w/drivers/md/md.c
> index 78be7811a89f..8cb855d03e0a 100644
> --- i/drivers/md/md.c
> +++ w/drivers/md/md.c
> @@ -9117,6 +9117,20 @@ void md_do_sync(struct md_thread *thread)
>   }
>   EXPORT_SYMBOL_GPL(md_do_sync);
> 
> +static bool rdev_addable(struct md_rdev *rdev)
> +{
> +       if (test_bit(Candidate, &rdev->flags) || rdev->raid_disk >= 0 ||
> +           test_bit(Faulty, &rdev->flags))
> +               return false;
> +       return true;
> +}
> +
> +static bool rdev_is_readd(struct md_rdev *rdev)
> +{
> +       return rdev->saved_raid_disk >= 0 ||
> +               !test_bit(Bitmap_sync, &rdev->flags);
This should use '&&' instead of '||' ?

> +}
> +
>   static int remove_and_add_spares(struct mddev *mddev,
>                                   struct md_rdev *this)
>   {
> @@ -9176,25 +9190,24 @@ static int remove_and_add_spares(struct mddev *mddev,
>          rdev_for_each(rdev, mddev) {
>                  if (this && this != rdev)
>                          continue;
> -               if (test_bit(Candidate, &rdev->flags))
> -                       continue;
>                  if (rdev->raid_disk >= 0 &&
>                      !test_bit(In_sync, &rdev->flags) &&
>                      !test_bit(Journal, &rdev->flags) &&
>                      !test_bit(Faulty, &rdev->flags))
>                          spares++;
> -               if (rdev->raid_disk >= 0)
> +
> +               if (!rdev_addable(rdev))
>                          continue;
> -               if (test_bit(Faulty, &rdev->flags))
> +
> +               if (test_bit(Journal, &rdev->flags))
> +                       goto hot_add_disk;
> +

I understand what you mean now, but I must use the exact same judgement
in the new helper md_spares_need_change() in patch 7, there will be 
redundant code this way.

How about this, rework rdev_addable():

   static bool rdev_addable(struct md_rdev *rdev)
   {
+         /* rdev is already used, don't add it again. */
           if (test_bit(Candidate, &rdev->flags) || rdev->raid_disk >= 0 ||
               test_bit(Faulty, &rdev->flags))
                   return false;

~         /* Allow to add journal disk. */
~         if (test_bit(Journal, &rdev->flags))
~_                return true;

~         /* Allow to add if array is read-write. */
+         if (md_is_rdwr(rdev->mddev))
+                 return true;
+
+         /*
+          * For read-only array, only allow to readd a rdev. And if 
bitmap is
+          * used, don't allow to readd a rdev that is too old.
+          */
+         if (rdev->saved_raid_disk >=0 && !test_bit(Bitmap_sync, 
&rdev->flags))
+                 return true;
+
+         return false;
   }


Thanks,
Kuai

> +               if (!md_is_rdwr(mddev) && !rdev_is_readd(rdev))
>                          continue;
> -               if (!test_bit(Journal, &rdev->flags)) {
> -                       if (!md_is_rdwr(mddev) &&
> -                           !(rdev->saved_raid_disk >= 0 &&
> -                             !test_bit(Bitmap_sync, &rdev->flags)))
> -                               continue;
> 
> -                       rdev->recovery_offset = 0;
> -               }
> +               rdev->recovery_offset = 0;
> +
> +       hot_add_disk:
>                  if (mddev->pers->hot_add_disk(mddev, rdev) == 0) {
>                          /* failure here is OK */
>                          sysfs_link_rdev(mddev, rdev);
> .
>