[PATCH 0/4] dracut: fix various issues with newly degraded md arrays

[PATCH 0/4] dracut: fix various issues with newly degraded md arrays

[NeilBrown]

When an md array is newly degraded at boot it will not be assembled
until mdraid_start.sh is run.
There are a few problem with the timeout handling that causes this
not to work properly in all circumstances.
These patches fix the bug I had reported.

Thanks,
NeilBrown


---

NeilBrown (4):
      mdraid/mdraid_start.sh: tell mainloop if we make progress.
      systemd/dracut-iniqueue - continue waiting if any timeout script makes progress
      dracut-systemd/rootfs-generator.sh: always create generated files.
      crypt/parse-crypt.sh: hide encrypted devices from systemd timeout warnings.


 modules.d/90crypt/parse-crypt.sh               |    4 ++
 modules.d/90mdraid/mdraid_start.sh             |    1 
 modules.d/98dracut-systemd/dracut-initqueue.sh |    1 
 modules.d/98dracut-systemd/rootfs-generator.sh |   15 ++++---
 modules.d/99base/dracut-lib.sh                 |   53 +++++++++++++++---------
 5 files changed, 48 insertions(+), 26 deletions(-)

--
Signature

[PATCH 1/4] mdraid/mdraid_start.sh: tell mainloop if we make progress.

[NeilBrown]

mdraid_start is a number of scripts which run after a timeout.
If it makes progress, it should tell the main loop so that it
knows that it is worth waiting a bit longer.

So in that case, create the initqueue/work file which the main loop
checks for.

Signed-off-by: NeilBrown <neilb-l3A5Bk7waGM@public.gmane.org>
---
 modules.d/90mdraid/mdraid_start.sh |    1 +
 1 file changed, 1 insertion(+)

diff --git a/modules.d/90mdraid/mdraid_start.sh b/modules.d/90mdraid/mdraid_start.sh
index 761e64f312d3..400ab5dc46c7 100755
--- a/modules.d/90mdraid/mdraid_start.sh
+++ b/modules.d/90mdraid/mdraid_start.sh
@@ -27,6 +27,7 @@ _md_force_run() {
 
         _path_d="${_path_s%/*}/degraded"
         [ ! -r "$_path_d" ] && continue
+        > $hookdir/initqueue/work
     done
 }
 

[PATCH 2/4] systemd/dracut-iniqueue - continue waiting if any timeout script makes progress

[NeilBrown]

A number of timeout scripts can be registered.  If any one of them
makes progress - e.g. assembles a degraded md array - then
the main loop should wait a bit longer rather than pressing forward.

This is particularly important is resume-from-hibernate requires a
degraded md array.  Both the script to forcibly assemble the md array
and the script to abort hibernation if the device doesn't appear
are 'timeout' scripts.  There needs to be a reasonable delay between
these running.

So: if any script has indicated that progress was made, break of out
the loop and go back to normal waiting.

Signed-off-by: NeilBrown <neilb-l3A5Bk7waGM@public.gmane.org>
---
 modules.d/98dracut-systemd/dracut-initqueue.sh |    1 +
 1 file changed, 1 insertion(+)

diff --git a/modules.d/98dracut-systemd/dracut-initqueue.sh b/modules.d/98dracut-systemd/dracut-initqueue.sh
index 88cd1e056ed7..af9cec2c5b8c 100755
--- a/modules.d/98dracut-systemd/dracut-initqueue.sh
+++ b/modules.d/98dracut-systemd/dracut-initqueue.sh
@@ -60,6 +60,7 @@ while :; do
             job=$job . $job
             udevadm settle --timeout=0 >/dev/null 2>&1 || main_loop=0
             [ -f $hookdir/initqueue/work ] && main_loop=0
+            [ $main_loop -eq 0 ] && break
         done
     fi
 

[PATCH 3/4] dracut-systemd/rootfs-generator.sh: always create generated files.

[NeilBrown]

When 'systemctl daemon-reload' is run, systemd will clean out
/run/systemd/generator and re-run all the generators.
So it is important that the generators always create the required
files.

rootfs-generator.sh currently does *not* create the desired files
if $hookdir/initqueue/finished/devexists-${_name}.sh
exists.

This is not removed by "systectl daemon-reload" so the first time this
generator is run it will do the right thing.  Subsequent times it
won't.

This results in incorrect timeouts after "daemon-reload" is run.

So let the existence of each file only guard the creation that file.

Signed-off-by: NeilBrown <neilb-l3A5Bk7waGM@public.gmane.org>
---
 modules.d/98dracut-systemd/rootfs-generator.sh |   15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/modules.d/98dracut-systemd/rootfs-generator.sh b/modules.d/98dracut-systemd/rootfs-generator.sh
index f3c7d1f237df..97512c07ab06 100755
--- a/modules.d/98dracut-systemd/rootfs-generator.sh
+++ b/modules.d/98dracut-systemd/rootfs-generator.sh
@@ -11,14 +11,15 @@ generator_wait_for_dev()
     _timeout=$(getarg rd.timeout)
     _timeout=${_timeout:-0}
 
-    [ -e "$hookdir/initqueue/finished/devexists-${_name}.sh" ] && return 0
+    if ! [ -e "$hookdir/initqueue/finished/devexists-${_name}.sh" ]; then
 
-    printf '[ -e "%s" ]\n' $1 \
-        >> "$hookdir/initqueue/finished/devexists-${_name}.sh"
-    {
-        printf '[ -e "%s" ] || ' $1
-        printf 'warn "\"%s\" does not exist"\n' $1
-    } >> "$hookdir/emergency/80-${_name}.sh"
+        printf '[ -e "%s" ]\n' $1 \
+            >> "$hookdir/initqueue/finished/devexists-${_name}.sh"
+        {
+            printf '[ -e "%s" ] || ' $1
+            printf 'warn "\"%s\" does not exist"\n' $1
+        } >> "$hookdir/emergency/80-${_name}.sh"
+    fi
 
     _name=$(dev_unit_name "$1")
     if ! [ -L /run/systemd/generator/initrd.target.wants/${_name}.device ]; then

[PATCH 4/4] crypt/parse-crypt.sh: hide encrypted devices from systemd timeout warnings.

[NeilBrown]

When systemd's crypttab generator parsed crypttab, it tells
systemd about several devices which may not appear until later
in the boot sequence, and which are not needed while dract is running.

This can particularly happen when an md array is encrypted,
and the array is newly degraded so that it doesn't appear until
dracut runs mdraid_start.sh.

This can result in systemd printing warning messages which are
inappropriate.

So tell systemd that the timeout for each of these is zero.

This is involves splitting some functionality out of wait_for_dev()

That function does two things:
 - creates 'finished' hooks so that dracut will wait for the device,
   and
 - sets the systemd timeout for the device to zero, so systemd doesn't
   wait.

We only want the second of these for most encrypted devices.
So split that out into a new function set_systemd_timeout_for_dev(),
and call it from parse-crypt.sh

Signed-off-by: NeilBrown <neilb-l3A5Bk7waGM@public.gmane.org>
---
 modules.d/90crypt/parse-crypt.sh |    4 +++
 modules.d/99base/dracut-lib.sh   |   53 ++++++++++++++++++++++++--------------
 2 files changed, 38 insertions(+), 19 deletions(-)

diff --git a/modules.d/90crypt/parse-crypt.sh b/modules.d/90crypt/parse-crypt.sh
index 94ad1f63ae6f..5a64652cc51c 100755
--- a/modules.d/90crypt/parse-crypt.sh
+++ b/modules.d/90crypt/parse-crypt.sh
@@ -14,6 +14,10 @@ else
     LUKS=$(getargs rd.luks.uuid -d rd_LUKS_UUID)
     tout=$(getarg rd.luks.key.tout)
 
+    while read _dev _uuid ; do
+        set_systemd_timeout_for_dev $_dev
+    done
+
     if [ -n "$LUKS" ]; then
         for luksid in $LUKS; do
 
diff --git a/modules.d/99base/dracut-lib.sh b/modules.d/99base/dracut-lib.sh
index 259e2057b67e..c8ff3c1a32d1 100755
--- a/modules.d/99base/dracut-lib.sh
+++ b/modules.d/99base/dracut-lib.sh
@@ -895,12 +895,10 @@ dev_unit_name()
     printf -- "%s" "$dev"
 }
 
-# wait_for_dev <dev>
-#
-# Installs a initqueue-finished script,
-# which will cause the main loop only to exit,
-# if the device <dev> is recognized by the system.
-wait_for_dev()
+# set_systemd_timeout_for_dev <dev>
+# Set 'rd.timeout' as the systemd timeout for <dev>
+
+set_systemd_timeout_for_dev()
 {
     local _name
     local _needreload
@@ -915,19 +913,6 @@ wait_for_dev()
     _timeout=$(getarg rd.timeout)
     _timeout=${_timeout:-0}
 
-    _name="$(str_replace "$1" '/' '\x2f')"
-
-    type mark_hostonly >/dev/null 2>&1 && mark_hostonly "$hookdir/initqueue/finished/devexists-${_name}.sh"
-
-    [ -e "${PREFIX}$hookdir/initqueue/finished/devexists-${_name}.sh" ] && return 0
-
-    printf '[ -e "%s" ]\n' $1 \
-        >> "${PREFIX}$hookdir/initqueue/finished/devexists-${_name}.sh"
-    {
-        printf '[ -e "%s" ] || ' $1
-        printf 'warn "\"%s\" does not exist"\n' $1
-    } >> "${PREFIX}$hookdir/emergency/80-${_name}.sh"
-
     if [ -n "$DRACUT_SYSTEMD" ]; then
         _name=$(dev_unit_name "$1")
         if ! [ -L ${PREFIX}/etc/systemd/system/initrd.target.wants/${_name}.device ]; then
@@ -952,6 +937,36 @@ wait_for_dev()
         fi
     fi
 }
+# wait_for_dev <dev>
+#
+# Installs a initqueue-finished script,
+# which will cause the main loop only to exit,
+# if the device <dev> is recognized by the system.
+wait_for_dev()
+{
+    local _name
+    local _noreload
+
+    if [ "$1" = "-n" ]; then
+        _noreload=-n
+        shift
+    fi
+
+    _name="$(str_replace "$1" '/' '\x2f')"
+
+    type mark_hostonly >/dev/null 2>&1 && mark_hostonly "$hookdir/initqueue/finished/devexists-${_name}.sh"
+
+    [ -e "${PREFIX}$hookdir/initqueue/finished/devexists-${_name}.sh" ] && return 0
+
+    printf '[ -e "%s" ]\n' $1 \
+        >> "${PREFIX}$hookdir/initqueue/finished/devexists-${_name}.sh"
+    {
+        printf '[ -e "%s" ] || ' $1
+        printf 'warn "\"%s\" does not exist"\n' $1
+    } >> "${PREFIX}$hookdir/emergency/80-${_name}.sh"
+
+    set_systemd_timeout_for_dev $_noreload $1
+}
 
 cancel_wait_for_dev()
 {

Re: [PATCH 4/4] crypt/parse-crypt.sh: hide encrypted devices from systemd timeout warnings.

[Harald Hoyer]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 26.03.2015 05:41, NeilBrown wrote:
> When systemd's crypttab generator parsed crypttab, it tells systemd about 
> several devices which may not appear until later in the boot sequence,
> and which are not needed while dract is running.
> 
> This can particularly happen when an md array is encrypted, and the array 
> is newly degraded so that it doesn't appear until dracut runs 
> mdraid_start.sh.
> 
> This can result in systemd printing warning messages which are 
> inappropriate.
> 
> So tell systemd that the timeout for each of these is zero.
> 
> This is involves splitting some functionality out of wait_for_dev()
> 
> That function does two things: - creates 'finished' hooks so that dracut 
> will wait for the device, and - sets the systemd timeout for the device
> to zero, so systemd doesn't wait.
> 
> We only want the second of these for most encrypted devices. So split
> that out into a new function set_systemd_timeout_for_dev(), and call it
> from parse-crypt.sh
> 
> Signed-off-by: NeilBrown <neilb-l3A5Bk7waGM@public.gmane.org> --- 
> modules.d/90crypt/parse-crypt.sh |    4 +++
> modules.d/99base/dracut-lib.sh |   53
> ++++++++++++++++++++++++-------------- 2 files changed, 38 insertions(+),
> 19 deletions(-)
> 
> diff --git a/modules.d/90crypt/parse-crypt.sh 
> b/modules.d/90crypt/parse-crypt.sh index 94ad1f63ae6f..5a64652cc51c 100755
> --- a/modules.d/90crypt/parse-crypt.sh +++ 
> b/modules.d/90crypt/parse-crypt.sh @@ -14,6 +14,10 @@ else LUKS=$(getargs 
> rd.luks.uuid -d rd_LUKS_UUID) tout=$(getarg rd.luks.key.tout)
> 
> +    while read _dev _uuid ; do +        set_systemd_timeout_for_dev $_dev
> +    done +

missing redirector from /etc/crypttab I guess

[Dracut GitHub] Patchset imported to github

[Dracut GitHub Import Bot]

Patchset imported to github.
Pull request:
<https://github.com/haraldh/dracut/compare/master...dracut-mailing-devs:20150326044148.10121.15286.stgit@notabene.brown>

Re: [PATCH 4/4] crypt/parse-crypt.sh: hide encrypted devices from systemd timeout warnings.

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 758 bytes --]


[I was alerted that you had replied to the list, but not to me.
 I'm not on the list..]

> > 
> > +    while read _dev _uuid ; do 
> > +        set_systemd_timeout_for_dev $_dev
> > +    done
> 
> missing redirector from /etc/crypttab I guess

Sorry about that.  The missing redirector is actually from 
   /usr/lib/dracut/modules.d/90crypt/block_uuid.map
but that looks like an openSUSE specific thing...

https://build.opensuse.org/package/view_file/openSUSE:13.2/dracut/0144-90crypt-Fixed-crypttab_contains-to-also-work-with-de.patch?expand=1

It should just use /etc/crypttab, but then the "read" needs to change

 while read _mapper _dev _rest; do
    set_systemd_timeout_for_dev $_dev
 done < /etc/crypttab

Thanks,
NeilBrown

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]