[PATCH 0/2] initramfs-framework enhancements

[PATCH 0/2] initramfs-framework enhancements

[Patrick Ohly]

These are two enhancements that we've used for a while now in Ostro OS:
- a retry loop that we found necessary for some slow USB hardware
- finding the root partition by partition UUID (only filesystem UUID
  supported before)

The following changes since commit 646c366c2566bd8dd6f73681cea9f5b021589a56:

  gst-player: upgrade to latest HEAD (2016-06-27 14:08:37 +0100)

are available in the git repository at:

  git://github.com/pohly/openembedded-core initramfs-framework
  https://github.com/pohly/openembedded-core/tree/initramfs-framework

Igor Stoppa (1):
  meta-ostro-fixes: initramfs-framework: Add support for PartUUIDs

Patrick Ohly (1):
  meta-ostro-fixes: initramfs-framework: add retry loop for slow boot
    devices (like USB)

 .../initrdscripts/initramfs-framework/finish       | 33 ------------
 .../initrdscripts/initramfs-framework/rootfs       | 62 ++++++++++++++++++++++
 .../initrdscripts/initramfs-framework_1.0.bb       |  4 +-
 3 files changed, 65 insertions(+), 34 deletions(-)
 create mode 100644 meta/recipes-core/initrdscripts/initramfs-framework/rootfs

-- 
2.1.4

[PATCH 1/2] meta-ostro-fixes: initramfs-framework: add retry loop for slow boot devices (like USB)

[Patrick Ohly]

On some hardware platforms (Gigabyte, qemu), detection of USB devices
by the kernel is slow enough such that it happens only after the first
attempt to mount the rootfs. We need to keep trying for a while
(default: 5s seconds, controlled by roottimeout=<seconds>) and sleep
between each attempt (default: one second, rootdelay=<seconds>).

This change intentionally splits finding the rootfs (in the new
"rootfs") and switching to it ("finish"). That is needed to keep udev
running while waiting for the rootfs, because it shuts down before
"finish" starts. It is also the direction that was discussed on the OE
mailing list for future changes to initramfs-framework (like
supporting a "live CD" module, which would replace or further augment
mounting of the rootfs).

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
---
 .../initrdscripts/initramfs-framework/finish       | 33 -------------
 .../initrdscripts/initramfs-framework/rootfs       | 57 ++++++++++++++++++++++
 .../initrdscripts/initramfs-framework_1.0.bb       |  4 +-
 3 files changed, 60 insertions(+), 34 deletions(-)
 create mode 100644 meta/recipes-core/initrdscripts/initramfs-framework/rootfs

diff --git a/meta/recipes-core/initrdscripts/initramfs-framework/finish b/meta/recipes-core/initrdscripts/initramfs-framework/finish
index d09bbb8..717383e 100755
--- a/meta/recipes-core/initrdscripts/initramfs-framework/finish
+++ b/meta/recipes-core/initrdscripts/initramfs-framework/finish
@@ -8,39 +8,6 @@ finish_enabled() {
 
 finish_run() {
 	if [ -n "$ROOTFS_DIR" ]; then
-		if [ -n "$bootparam_rootdelay" ]; then
-			debug "Sleeping for $rootdelay second(s) to wait root to settle..."
-			sleep $bootparam_rootdelay
-		fi
-
-		if [ -n "$bootparam_root" ]; then
-			debug "No e2fs compatible filesystem has been mounted, mounting $bootparam_root..."
-
-			if [ "`echo ${bootparam_root} | cut -c1-5`" = "UUID=" ]; then
-				root_uuid=`echo $bootparam_root | cut -c6-`
-				bootparam_root="/dev/disk/by-uuid/$root_uuid"
-			fi
-
-			if [ -e "$bootparam_root" ]; then
-				flags=""
-				if [ -n "$bootparam_ro" ]; then
-					if [  -n "$bootparam_rootflags" ]; then
-						bootparam_rootflags="$bootparam_rootflags,"
-					fi
-					bootparam_rootflags="${bootparam_rootflags}ro"
-				fi
-				if [ -n "$bootparam_rootflags" ]; then
-					flags="$flags -o$bootparam_rootflags"
-				fi
-				if [ -n "$bootparam_rootfstype" ]; then
-					flags="$flags -t$bootparam_rootfstype"
-				fi
-				mount $flags $bootparam_root $ROOTFS_DIR
-			else
-				msg "root '$bootparam_root' doesn't exist."
-			fi
-		fi
-
 		if [ ! -d $ROOTFS_DIR/dev ]; then
 			fatal "ERROR: There's no '/dev' on rootfs."
 		fi
diff --git a/meta/recipes-core/initrdscripts/initramfs-framework/rootfs b/meta/recipes-core/initrdscripts/initramfs-framework/rootfs
new file mode 100644
index 0000000..5790d8c
--- /dev/null
+++ b/meta/recipes-core/initrdscripts/initramfs-framework/rootfs
@@ -0,0 +1,57 @@
+#!/bin/sh
+# Copyright (C) 2011 O.S. Systems Software LTDA.
+# Licensed on MIT
+
+rootfs_enabled() {
+	return 0
+}
+
+rootfs_run() {
+        if [ -z "$ROOTFS_DIR" ]; then
+		return
+        fi
+	C=0
+	delay=${bootparam_rootdelay:-1}
+	timeout=${bootparam_roottimeout:-5}
+	while [ ! -d $ROOTFS_DIR/dev ]; do
+		if [ $(( $C * $delay )) -gt $timeout ]; then
+			fatal "root '$bootparam_root' doesn't exist or does not contain a /dev."
+		fi
+
+		if [ -n "$bootparam_root" ]; then
+			debug "No e2fs compatible filesystem has been mounted, mounting $bootparam_root..."
+
+			if [ "`echo ${bootparam_root} | cut -c1-5`" = "UUID=" ]; then
+				root_uuid=`echo $bootparam_root | cut -c6-`
+				bootparam_root="/dev/disk/by-uuid/$root_uuid"
+			fi
+
+			if [ -e "$bootparam_root" ]; then
+				flags=""
+				if [ -n "$bootparam_ro" ] && ! echo "$bootparam_rootflags" | grep -w -q "ro"; then
+					if [  -n "$bootparam_rootflags" ]; then
+						bootparam_rootflags="$bootparam_rootflags,"
+					fi
+					bootparam_rootflags="${bootparam_rootflags}ro"
+				fi
+				if [ -n "$bootparam_rootflags" ]; then
+					flags="$flags -o$bootparam_rootflags"
+				fi
+				if [ -n "$bootparam_rootfstype" ]; then
+					flags="$flags -t$bootparam_rootfstype"
+				fi
+				mount $flags $bootparam_root $ROOTFS_DIR
+				if [ -d $ROOTFS_DIR/dev ]; then
+					break
+				else
+					# It is unlikely to change, but keep trying anyway.
+					# Perhaps we pick a different device next time.
+					umount $ROOTFS_DIR
+					fi
+				fi
+		fi
+		debug "Sleeping for $delay second(s) to wait root to settle..."
+		sleep $delay
+		C=$(( $C + 1 ))
+	done
+}
diff --git a/meta/recipes-core/initrdscripts/initramfs-framework_1.0.bb b/meta/recipes-core/initrdscripts/initramfs-framework_1.0.bb
index e5cf9cb..89e153d 100644
--- a/meta/recipes-core/initrdscripts/initramfs-framework_1.0.bb
+++ b/meta/recipes-core/initrdscripts/initramfs-framework_1.0.bb
@@ -8,6 +8,7 @@ PR = "r2"
 inherit allarch
 
 SRC_URI = "file://init \
+           file://rootfs \
            file://finish \
            file://mdev \
            file://udev \
@@ -21,6 +22,7 @@ do_install() {
 
     # base
     install -m 0755 ${WORKDIR}/init ${D}/init
+    install -m 0755 ${WORKDIR}/rootfs ${D}/init.d/90-rootfs
     install -m 0755 ${WORKDIR}/finish ${D}/init.d/99-finish
 
     # mdev
@@ -47,7 +49,7 @@ PACKAGES = "${PN}-base \
             initramfs-module-e2fs \
             initramfs-module-debug"
 
-FILES_${PN}-base = "/init /init.d/99-finish /dev"
+FILES_${PN}-base = "/init /init.d/90-rootfs /init.d/99-finish /dev"
 
 SUMMARY_initramfs-module-mdev = "initramfs support for mdev"
 RDEPENDS_initramfs-module-mdev = "${PN}-base busybox-mdev"
-- 
2.1.4

[PATCH 2/2] meta-ostro-fixes: initramfs-framework: Add support for PartUUIDs

[Patrick Ohly]

From: Igor Stoppa <igor.stoppa@intel.com>

The rootfs can be addressed also by referring to the PartUUID
value from the GPT.
This patch enables such type of reference.

Signed-off-by: Igor Stoppa <igor.stoppa@intel.com>
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
---
 meta/recipes-core/initrdscripts/initramfs-framework/rootfs | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/meta/recipes-core/initrdscripts/initramfs-framework/rootfs b/meta/recipes-core/initrdscripts/initramfs-framework/rootfs
index 5790d8c..14768f1 100644
--- a/meta/recipes-core/initrdscripts/initramfs-framework/rootfs
+++ b/meta/recipes-core/initrdscripts/initramfs-framework/rootfs
@@ -26,6 +26,11 @@ rootfs_run() {
 				bootparam_root="/dev/disk/by-uuid/$root_uuid"
 			fi
 
+			if [ "`echo ${bootparam_root} | cut -c1-9`" = "PARTUUID=" ]; then
+				root_uuid=`echo $bootparam_root | cut -c10-`
+				bootparam_root="/dev/disk/by-partuuid/$root_uuid"
+			fi
+
 			if [ -e "$bootparam_root" ]; then
 				flags=""
 				if [ -n "$bootparam_ro" ] && ! echo "$bootparam_rootflags" | grep -w -q "ro"; then
-- 
2.1.4

Re: [PATCH 1/2] meta-ostro-fixes: initramfs-framework: add retry loop for slow boot devices (like USB)

[Otavio Salvador]

On Fri, Jul 1, 2016 at 10:53 AM, Patrick Ohly <patrick.ohly@intel.com> wrote:
> On some hardware platforms (Gigabyte, qemu), detection of USB devices
> by the kernel is slow enough such that it happens only after the first
> attempt to mount the rootfs. We need to keep trying for a while
> (default: 5s seconds, controlled by roottimeout=<seconds>) and sleep
> between each attempt (default: one second, rootdelay=<seconds>).
>
> This change intentionally splits finding the rootfs (in the new
> "rootfs") and switching to it ("finish"). That is needed to keep udev
> running while waiting for the rootfs, because it shuts down before
> "finish" starts. It is also the direction that was discussed on the OE
> mailing list for future changes to initramfs-framework (like
> supporting a "live CD" module, which would replace or further augment
> mounting of the rootfs).
>
> Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>

Agreed but please split the rootfs in another module; so we don't
force it to be included. The e2fs can rdepend on it.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 1/2] meta-ostro-fixes: initramfs-framework: add retry loop for slow boot devices (like USB)

[Otavio Salvador]

Hello Ross,

On Fri, Jul 1, 2016 at 2:50 PM, Otavio Salvador <otavio@ossystems.com.br> wrote:
> On Fri, Jul 1, 2016 at 10:53 AM, Patrick Ohly <patrick.ohly@intel.com> wrote:
>> On some hardware platforms (Gigabyte, qemu), detection of USB devices
>> by the kernel is slow enough such that it happens only after the first
>> attempt to mount the rootfs. We need to keep trying for a while
>> (default: 5s seconds, controlled by roottimeout=<seconds>) and sleep
>> between each attempt (default: one second, rootdelay=<seconds>).
>>
>> This change intentionally splits finding the rootfs (in the new
>> "rootfs") and switching to it ("finish"). That is needed to keep udev
>> running while waiting for the rootfs, because it shuts down before
>> "finish" starts. It is also the direction that was discussed on the OE
>> mailing list for future changes to initramfs-framework (like
>> supporting a "live CD" module, which would replace or further augment
>> mounting of the rootfs).
>>
>> Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
>
> Agreed but please split the rootfs in another module; so we don't
> force it to be included. The e2fs can rdepend on it.

I noticed this has been merged but my comment here was totally ignored. Why?

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 1/2] meta-ostro-fixes: initramfs-framework: add retry loop for slow boot devices (like USB)

[Patrick Ohly]

On Mon, 2016-07-11 at 11:35 -0300, Otavio Salvador wrote:
> Hello Ross,
> 
> On Fri, Jul 1, 2016 at 2:50 PM, Otavio Salvador <otavio@ossystems.com.br> wrote:
> > Agreed but please split the rootfs in another module; so we don't
> > force it to be included. The e2fs can rdepend on it.
> 
> I noticed this has been merged but my comment here was totally ignored. Why?

I can't speak for Ross, but perhaps he felt that it was already a
worthwhile improvement. I wanted to address your comment (and still do),
but didn't find the time last week. I'll do it via another patch now,
okay?

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: [PATCH 1/2] meta-ostro-fixes: initramfs-framework: add retry loop for slow boot devices (like USB)

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 553 bytes --]

On 11 July 2016 at 16:10, Patrick Ohly <patrick.ohly@intel.com> wrote:

> > I noticed this has been merged but my comment here was totally ignored.
> Why?
>
> I can't speak for Ross, but perhaps he felt that it was already a
> worthwhile improvement. I wanted to address your comment (and still do),
> but didn't find the time last week. I'll do it via another patch now,
> okay?


Because I managed to remove it and then reintroduce it with a subsequent
rebase, my fault entirely.

Thanks Patrick for offering a follow-up patch.

Ross

[-- Attachment #2: Type: text/html, Size: 1008 bytes --]

Re: [PATCH 1/2] meta-ostro-fixes: initramfs-framework: add retry loop for slow boot devices (like USB)

[Otavio Salvador]

On Mon, Jul 11, 2016 at 12:10 PM, Patrick Ohly <patrick.ohly@intel.com> wrote:
> On Mon, 2016-07-11 at 11:35 -0300, Otavio Salvador wrote:
>> Hello Ross,
>>
>> On Fri, Jul 1, 2016 at 2:50 PM, Otavio Salvador <otavio@ossystems.com.br> wrote:
>> > Agreed but please split the rootfs in another module; so we don't
>> > force it to be included. The e2fs can rdepend on it.
>>
>> I noticed this has been merged but my comment here was totally ignored. Why?
>
> I can't speak for Ross, but perhaps he felt that it was already a
> worthwhile improvement. I wanted to address your comment (and still do),
> but didn't find the time last week. I'll do it via another patch now,
> okay?

Ok but Ross action is wrong in my opinion, in worse case he should
have asked before. If comments are ignored there is no point in people
spend time doing so.

I wanted the change to be atomic and not two changes which may
required more rework for same logic change. :-(

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 1/2] meta-ostro-fixes: initramfs-framework: add retry loop for slow boot devices (like USB)

[Patrick Ohly]

On Mon, 2016-07-11 at 12:17 -0300, Otavio Salvador wrote:
> On Mon, Jul 11, 2016 at 12:10 PM, Patrick Ohly <patrick.ohly@intel.com> wrote:
> > On Mon, 2016-07-11 at 11:35 -0300, Otavio Salvador wrote:
> >> Hello Ross,
> >>
> >> On Fri, Jul 1, 2016 at 2:50 PM, Otavio Salvador <otavio@ossystems.com.br> wrote:
> >> > Agreed but please split the rootfs in another module; so we don't
> >> > force it to be included. The e2fs can rdepend on it.
> >>
> >> I noticed this has been merged but my comment here was totally ignored. Why?
> >
> > I can't speak for Ross, but perhaps he felt that it was already a
> > worthwhile improvement. I wanted to address your comment (and still do),
> > but didn't find the time last week. I'll do it via another patch now,
> > okay?
> 
> Ok but Ross action is wrong in my opinion, in worse case he should
> have asked before. If comments are ignored there is no point in people
> spend time doing so.
> 
> I wanted the change to be atomic and not two changes which may
> required more rework for same logic change. :-(

It turned out to be just a mistake that it was already merge.

On the other hand, one can also argue that these are two separate
logical changes - adding the retry loop, and then making that code just
the optional default implementation for mounting.

Anyway, see "initramfs-framework: make rootfs module optional" for that
second part. Note that I did not make e2fs depend on rootfs, because
there's no real connection between the two (for example, the rootfs
could also be btrfs). Instead it is the finish script which depends on
some kind of rootfs mounting.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.