Patch to fix boot from RAID-1 partitioned arrays

Patch to fix boot from RAID-1 partitioned arrays

[Geoff Back]

Good morning.

I have had problems in all recent kernels with booting directly from MD 
RAID-1 partitioned arrays (i.e. without using an initrd).
All the usual requirements - building md and raid1 into the kernel, 
correct partition types, etc - are correct.

Deep investigation has led me to conclude that the issue is caused by 
boot-time assembly of the array not reading the partition table, meaning 
that the partitions are not visible and cannot be mounted as root 
filesystem.

The change to drivers/md/md-autodetect.c in commit 
a1d6bc018911390274e3904bdd28240cd96ddc54 appears to be related, although 
this only covers some cases not all, so I have not been able to 
determine exactly what commit led to this problem.

I have implemented a patch that causes the partition table to be read 
immediately after the array is started, for the two boot-time assembly 
paths (from autodetect with raid=part, and from assembly with md=d0,...) 
which is included below.

Regards,

Geoff.

diff -Naur linux-5.12.2.orig/drivers/md/md-autodetect.c 
linux-5.12.2/drivers/md/md-autodetect.c
--- linux-5.12.2.orig/drivers/md/md-autodetect.c    2021-05-12 
09:14:07.096442083 +0100
+++ linux-5.12.2/drivers/md/md-autodetect.c    2021-05-12 
09:22:07.734653840 +0100
@@ -232,8 +232,24 @@
      mddev_unlock(mddev);
  out_blkdev_put:
      blkdev_put(bdev, FMODE_READ);
-}

+    /*
+     * Need to force read of partition table in order for partitioned
+     * arrays to be bootable.  Deliberately done after all cleanup,
+     * and only for successfully loaded arrays.
+     */
+    if (err == 0)
+    {
+        struct block_device *bd;
+
+        bd = blkdev_get_by_dev(mdev, FMODE_READ, NULL);
+        if (IS_ERR(bd))
+            pr_err("md: failed to get md device\n");
+        else
+            blkdev_put(bd, FMODE_READ);
+    }
+}
+
  static int __init raid_setup(char *str)
  {
      int len, pos;
diff -Naur linux-5.12.2.orig/drivers/md/md.c linux-5.12.2/drivers/md/md.c
--- linux-5.12.2.orig/drivers/md/md.c    2021-05-12 09:14:07.127441838 +0100
+++ linux-5.12.2/drivers/md/md.c    2021-05-12 09:34:26.960827487 +0100
@@ -6467,6 +6467,20 @@
          pr_warn("md: do_md_run() returned %d\n", err);
          do_md_stop(mddev, 0, NULL);
      }
+    else
+    {
+        /*
+         * Need to force read of partition table in order for partitioned
+         * arrays to be bootable.
+         */
+        struct block_device *bd;
+
+        bd = blkdev_get_by_dev(mdev, FMODE_READ, NULL);
+        if (IS_ERR(bd))
+            pr_err("md: failed to get md device\n");
+        else
+            blkdev_put(bd, FMODE_READ);
+    }
  }

  /*



-- 
Geoff Back
What if we're all just characters in someone's nightmares?

Re: Patch to fix boot from RAID-1 partitioned arrays

[Geoff Back]

Hello,

Apologies, just spotted a typo in my patch.  Updated patch below.
Regards,

Geoff.


diff -Naur linux-5.12.2.orig/drivers/md/md-autodetect.c 
linux-5.12.2/drivers/md/md-autodetect.c
--- linux-5.12.2.orig/drivers/md/md-autodetect.c    2021-05-12 
09:14:07.096442083 +0100
+++ linux-5.12.2/drivers/md/md-autodetect.c    2021-05-12 
09:22:07.734653840 +0100
@@ -232,8 +232,24 @@
      mddev_unlock(mddev);
  out_blkdev_put:
      blkdev_put(bdev, FMODE_READ);
-}

+    /*
+     * Need to force read of partition table in order for partitioned
+     * arrays to be bootable.  Deliberately done after all cleanup,
+     * and only for successfully loaded arrays.
+     */
+    if (err == 0)
+    {
+        struct block_device *bd;
+
+        bd = blkdev_get_by_dev(mdev, FMODE_READ, NULL);
+        if (IS_ERR(bd))
+            pr_err("md: failed to get md device\n");
+        else
+            blkdev_put(bd, FMODE_READ);
+    }
+}
+
  static int __init raid_setup(char *str)
  {
      int len, pos;
diff -Naur linux-5.12.2.orig/drivers/md/md.c linux-5.12.2/drivers/md/md.c
--- linux-5.12.2.orig/drivers/md/md.c    2021-05-12 09:14:07.127441838 
+0100
+++ linux-5.12.2/drivers/md/md.c    2021-05-12 09:34:26.960827487 +0100
@@ -6467,6 +6467,20 @@
          pr_warn("md: do_md_run() returned %d\n", err);
          do_md_stop(mddev, 0, NULL);
      }
+    else
+    {
+        /*
+         * Need to force read of partition table in order for partitioned
+         * arrays to be bootable.
+         */
+        struct block_device *bd;
+
+        bd = blkdev_get_by_dev(mddev->unit, FMODE_READ, NULL);
+        if (IS_ERR(bd))
+            pr_err("md: failed to get md device\n");
+        else
+            blkdev_put(bd, FMODE_READ);
+    }
  }

  /*



--

Geoff Back
What if we're all just characters in someone's nightmares?

Re: Patch to fix boot from RAID-1 partitioned arrays

[Wols Lists]

On 12/05/21 09:41, Geoff Back wrote:
> Good morning.
> 
> I have had problems in all recent kernels with booting directly from MD
> RAID-1 partitioned arrays (i.e. without using an initrd).
> All the usual requirements - building md and raid1 into the kernel,
> correct partition types, etc - are correct.
> 
> Deep investigation has led me to conclude that the issue is caused by
> boot-time assembly of the array not reading the partition table, meaning
> that the partitions are not visible and cannot be mounted as root
> filesystem.

The other thing is, what superblock are you using? Sounds to me like
you're trying to use an unsupported and bit-rotting option.

Standard procedure today is that you MUST run mdadm to assemble the
array, which means having a functioning user-space, which I believe
means initrd or some such to create the array before you can make it root.

You saying that you need to read the partition table even when you have
a successfully assembled array makes me think something is weird here ...

If you can give us a bit more detail, we can then decide whether what
you're doing is supposed to work or not.

Basically, as I understand what you're doing, you need a 0.9
(unsupported) superblock, and also (unsupported) in-kernel raid assembly.

Cheers,
Wol

Re: Patch to fix boot from RAID-1 partitioned arrays

[Geoff Back]

On 12/05/2021 12:07, Wols Lists wrote:
> On 12/05/21 09:41, Geoff Back wrote:
>> Good morning.
>>
>> I have had problems in all recent kernels with booting directly from MD
>> RAID-1 partitioned arrays (i.e. without using an initrd).
>> All the usual requirements - building md and raid1 into the kernel,
>> correct partition types, etc - are correct.
>>
>> Deep investigation has led me to conclude that the issue is caused by
>> boot-time assembly of the array not reading the partition table, meaning
>> that the partitions are not visible and cannot be mounted as root
>> filesystem.
> 
> The other thing is, what superblock are you using? Sounds to me like
> you're trying to use an unsupported and bit-rotting option.

Yes, I am using 0.9 superblocks, because in-kernel auto detection does
not support the newer 1.x superblock formats.

> 
> Standard procedure today is that you MUST run mdadm to assemble the
> array, which means having a functioning user-space, which I believe
> means initrd or some such to create the array before you can make it root.

Using an initrd would add considerable additional complexity and
resource consumption to a configuration that until recently (some time
since linux 5.5.3) just worked.  Yes, I know that the general approach
for common distros is to use initrd, but for my use case it is not
appropriate.

> 
> You saying that you need to read the partition table even when you have
> a successfully assembled array makes me think something is weird here ...

The problem is not with in-kernel assembly and starting of the array -
that works perfectly well.  However, when the 'md' driver assembles and
runs the partitionable array device (typically /dev/md_d0) it only
causes the array device itself to get registered with the block layer.
The assembled array is not then scanned for partitions until something
accesses the device, at which point the pending GD_NEED_PART_SCAN flag
in the array's block-device structure causes the probe for a partition
table to take place and the partitions to become accessible.

Without my patch, there does not appear to be anything between array
assembly and root mount that will access the array and cause the
partition probe operation.

To be clear, the situation is that the array has been fully assembled
and activated but the partitions it contains remain inaccessible.

> 
> If you can give us a bit more detail, we can then decide whether what
> you're doing is supposed to work or not.
> 
> Basically, as I understand what you're doing, you need a 0.9
> (unsupported) superblock, and also (unsupported) in-kernel raid assembly.

Neither the 0.9 superblock format, nor the in-kernel array detection
process, are marked as deprecated in the kernel Kconfig - indeed
automatic raid detection still defaults to enabled when 'md' is enabled.

My patch is intended as a minimally-invasive fix to a functional
regression - this used to work and the kernel sources/documentation say
it should still work, but in practice (without the patch) it doesn't.

There was a relatively recent change that removed code which
deliberately performed partition probing after assembly in one of the
two code paths, but according to the patch it was only removed on the
basis that it was unclear why the probing was done; I suspect that this
regression may be partly due to that change. My patch effectively
reinstates the same functionality in that code path and adds the same
functionality to the other code path.


> 
> Cheers,
> Wol
> 

Thanks,

Geoff.

-- 
Geoff Back
What if we're all just characters in someone's nightmares?

Re: Patch to fix boot from RAID-1 partitioned arrays

[J. Brian Kelley]

Heh, sounds analogous to a quirk in BTRFS where a additional 'hook' is 
required or "btrfs" included in MODULES= (apparently triggering some 
(e)udev rule) in the initrd configuration file. Most linux distros take 
care of this , but not ARCH (or derivatives).  Hate to even think of 
incorporating this into a kernel....

On 2021-05-12 6:56 a.m., Geoff Back wrote:
>
> The problem is not with in-kernel assembly and starting of the array -
> that works perfectly well.  However, when the 'md' driver assembles and
> runs the partitionable array device (typically /dev/md_d0) it only
> causes the array device itself to get registered with the block layer.
> The assembled array is not then scanned for partitions until something
> accesses the device, at which point the pending GD_NEED_PART_SCAN flag
> in the array's block-device structure causes the probe for a partition
> table to take place and the partitions to become accessible.
>
> Without my patch, there does not appear to be anything between array
> assembly and root mount that will access the array and cause the
> partition probe operation.
>
> To be clear, the situation is that the array has been fully assembled
> and activated but the partitions it contains remain inaccessible.
>
> Thanks,
>
> Geoff.
>