[PATCH v2 0/9] Fixes for a distro with a merged /usr

[PATCH v2 0/9] Fixes for a distro with a merged /usr

[Joshua Lock]

I've been playing around with building a distro with a merged /usr[1] and
the following series fixes several issues I ran across in OE core metadata.

These have been build tested by building a core-image-sato image with
ipk packaging for: i) qemux86 with a default configuration, ii) qemux86-64
with a mutlilib configuration where the image includes lib32-ncurses and
iii) for my merged /usr test distro.

I used buildhistory to compare core-image-sato before and after this series
and the only change is:

Changes to images/qemux86/glibc/core-image-sato (files-in-image.txt):
  /bin/lsmod.kmod changed symlink target from ../bin/kmod to kmod

Changes since v1:
* Fix busybox typo/thinko which lead to installing busybox links into /usr/bin
* Fix systemd recipe to use nonarch_base_libdir for udev files

Regards,

Joshua

The following changes since commit aba3ef50d65e0dc8659a48bf98d0fb00dd44a6fc:

  debianutils: create package for run-parts (2015-09-02 23:51:16 +0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib joshuagl/merged-usr
  http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=joshuagl/merged-usr

Joshua Lock (9):
  kmod: fix link creation when base_bindir != /bin
  busybox: fixes for when base_bindir != /bin
  systemd: remove hard-coded paths in FILES entries
  alsa-utils: handle udev dir being 2 levels below /
  pulseaudio: handle udev dir being 2 levels below /
  bluez: handle udev dir being 2 levels below /
  pcmciautils: handle udev dir being 2 levels below /
  nfs-utils: don't force use of /sbin as sbindir
  procps: fix for base_sbindir == sbindir

 meta/recipes-bsp/pcmciautils/pcmciautils_018.bb    |  4 +-
 meta/recipes-connectivity/bluez5/bluez5.inc        |  1 +
 .../nfs-utils/nfs-utils_1.3.1.bb                   |  8 ++++
 meta/recipes-core/busybox/busybox.inc              | 49 ++++++++++++----------
 meta/recipes-core/systemd/systemd_219.bb           | 10 ++---
 meta/recipes-extended/procps/procps_3.3.10.bb      |  5 ++-
 meta/recipes-kernel/kmod/kmod_git.bb               |  4 +-
 meta/recipes-multimedia/alsa/alsa-utils_1.0.29.bb  |  2 +-
 meta/recipes-multimedia/pulseaudio/pulseaudio.inc  |  2 +-
 9 files changed, 49 insertions(+), 36 deletions(-)

-- 
2.1.4

[PATCH v2 1/9] kmod: fix link creation when base_bindir != /bin

[Joshua Lock]

If base_bindir is not a direct child of / the link creation in
do_install_append creates incorrect relative links.

Instead pass a full path to the link source and use the -r flag
to ln to ensure the link is relative.

Signed-off-by: Joshua Lock <joshua.lock@collabora.co.uk>
---
 meta/recipes-kernel/kmod/kmod_git.bb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-kernel/kmod/kmod_git.bb b/meta/recipes-kernel/kmod/kmod_git.bb
index ba4d85e..120fd87 100644
--- a/meta/recipes-kernel/kmod/kmod_git.bb
+++ b/meta/recipes-kernel/kmod/kmod_git.bb
@@ -21,9 +21,9 @@ do_install_append () {
         install -dm755 ${D}${base_bindir}
         install -dm755 ${D}${base_sbindir}
         # add symlinks to kmod
-        ln -s ..${base_bindir}/kmod ${D}${base_bindir}/lsmod
+        ln -sr ${D}${base_bindir}/kmod ${D}${base_bindir}/lsmod
         for tool in insmod rmmod depmod modinfo modprobe; do
-                ln -s ..${base_bindir}/kmod ${D}${base_sbindir}/${tool}
+                ln -sr ${D}${base_bindir}/kmod ${D}${base_sbindir}/${tool}
         done
         # configuration directories
         install -dm755 ${D}${base_libdir}/depmod.d
-- 
2.1.4

[PATCH v2 2/9] busybox: fixes for when base_bindir != /bin

[Joshua Lock]

* Replace all hard-coded paths with variables
* Run sed over busybox.links.* to replace /bin with ${base_bindir}

Signed-off-by: Joshua Lock <joshua.lock@collabora.co.uk>
---
 meta/recipes-core/busybox/busybox.inc | 49 +++++++++++++++++++----------------
 1 file changed, 26 insertions(+), 23 deletions(-)

diff --git a/meta/recipes-core/busybox/busybox.inc b/meta/recipes-core/busybox/busybox.inc
index 0769d92..ed8f9fe 100644
--- a/meta/recipes-core/busybox/busybox.inc
+++ b/meta/recipes-core/busybox/busybox.inc
@@ -178,6 +178,9 @@ do_install () {
 	if [ "${base_sbindir}" != "/sbin" ]; then
 		sed -i "s:^/sbin/:${base_sbindir}/:" busybox.links*
 	fi
+	if [ "${base_bindir}" != "/bin" ]; then
+		sed -i "s:^/bin/:${base_bindir}/:" busybox.links*
+	fi
 
 	install -d ${D}${sysconfdir}/init.d
 
@@ -338,10 +341,10 @@ python do_package_prepend () {
         return
 
     if os.path.exists('%s/etc/busybox.links' % (dvar)):
-        set_alternative_vars("/etc/busybox.links", "/bin/busybox")
+        set_alternative_vars("${sysconfdir}/busybox.links", "${base_bindir}/busybox")
     else:
-        set_alternative_vars("/etc/busybox.links.nosuid", "/bin/busybox.nosuid")
-        set_alternative_vars("/etc/busybox.links.suid", "/bin/busybox.suid")
+        set_alternative_vars("${sysconfdir}/busybox.links.nosuid", "${base_bindir}/busybox.nosuid")
+        set_alternative_vars("${sysconfdir}/busybox.links.suid", "${base_bindir}/busybox.suid")
 }
 
 pkg_postinst_${PN} () {
@@ -353,22 +356,22 @@ pkg_postinst_${PN} () {
 	if test "x$D" = "x"; then
 		# Remove busybox.nosuid if it's a symlink, because this situation indicates
 		# that we're installing or upgrading to a one-binary busybox.
-		if test -h /bin/busybox.nosuid; then
-			rm -f /bin/busybox.nosuid
+		if test -h ${base_bindir}/busybox.nosuid; then
+			rm -f ${base_bindir}/busybox.nosuid
 		fi
 		for suffix in "" ".nosuid" ".suid"; do
-			if test -e /etc/busybox.links$suffix; then
+			if test -e ${sysconfdir}/busybox.links$suffix; then
 				while read link; do
 					if test ! -e "$link"; then
 						case "$link" in
 							/*/*/*)
-								to="../../bin/busybox$suffix"
+								to="../..${base_bindir}/busybox$suffix"
 								;;
 							/bin/*)
 								to="busybox$suffix"
 								;;
 							/*/*)
-								to="../bin/busybox$suffix"
+								to="..${base_bindir}/busybox$suffix"
 								;;
 						esac
 						# we can use busybox here because even if we are using splitted busybox
@@ -376,7 +379,7 @@ pkg_postinst_${PN} () {
 						busybox rm -f $link
 						busybox ln -s $to $link
 					fi
-				done < /etc/busybox.links$suffix
+				done < ${sysconfdir}/busybox.links$suffix
 			fi
 		done
 	fi
@@ -387,19 +390,19 @@ pkg_prerm_${PN} () {
 	# providing its files, this will make update-alternatives work, but the update-rc.d part
 	# for syslog, httpd and/or udhcpd will fail if there is no other package providing sh
 	tmpdir=`mktemp -d /tmp/busyboxrm-XXXXXX`
-	ln -s /bin/busybox $tmpdir/[
-	ln -s /bin/busybox $tmpdir/test
-	ln -s /bin/busybox $tmpdir/head
-	ln -s /bin/busybox $tmpdir/sh
-	ln -s /bin/busybox $tmpdir/basename
-	ln -s /bin/busybox $tmpdir/echo
-	ln -s /bin/busybox $tmpdir/mv
-	ln -s /bin/busybox $tmpdir/ln
-	ln -s /bin/busybox $tmpdir/dirname
-	ln -s /bin/busybox $tmpdir/rm
-	ln -s /bin/busybox $tmpdir/sed
-	ln -s /bin/busybox $tmpdir/sort
-	ln -s /bin/busybox $tmpdir/grep
+	ln -s ${base_bindir}/busybox $tmpdir/[
+	ln -s ${base_bindir}/busybox $tmpdir/test
+	ln -s ${base_bindir}/busybox $tmpdir/head
+	ln -s ${base_bindir}/busybox $tmpdir/sh
+	ln -s ${base_bindir}/busybox $tmpdir/basename
+	ln -s ${base_bindir}/busybox $tmpdir/echo
+	ln -s ${base_bindir}/busybox $tmpdir/mv
+	ln -s ${base_bindir}/busybox $tmpdir/ln
+	ln -s ${base_bindir}/busybox $tmpdir/dirname
+	ln -s ${base_bindir}/busybox $tmpdir/rm
+	ln -s ${base_bindir}/busybox $tmpdir/sed
+	ln -s ${base_bindir}/busybox $tmpdir/sort
+	ln -s ${base_bindir}/busybox $tmpdir/grep
 	export PATH=$PATH:$tmpdir
 }
 
@@ -407,7 +410,7 @@ pkg_prerm_${PN}-syslog () {
 	# remove syslog
 	if test "x$D" = "x"; then
 		if test "$1" = "upgrade" -o "$1" = "remove"; then
-			/etc/init.d/syslog stop
+			${sysconfdir}/init.d/syslog stop
 		fi
 	fi
 }
-- 
2.1.4

[PATCH v2 3/9] systemd: remove hard-coded paths in FILES entries

[Joshua Lock]

Signed-off-by: Joshua Lock <joshua.lock@collabora.co.uk>
---
 meta/recipes-core/systemd/systemd_219.bb | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-core/systemd/systemd_219.bb b/meta/recipes-core/systemd/systemd_219.bb
index 4d32fd5..a64dbac 100644
--- a/meta/recipes-core/systemd/systemd_219.bb
+++ b/meta/recipes-core/systemd/systemd_219.bb
@@ -306,10 +306,10 @@ FILES_${PN} = " ${base_bindir}/* \
                 ${exec_prefix}/lib/sysctl.d \
                 ${exec_prefix}/lib/sysusers.d \
                 ${localstatedir} \
-                /lib/udev/rules.d/70-uaccess.rules \
-                /lib/udev/rules.d/71-seat.rules \
-                /lib/udev/rules.d/73-seat-late.rules \
-                /lib/udev/rules.d/99-systemd.rules \
+                ${nonarch_base_libdir}/udev/rules.d/70-uaccess.rules \
+                ${nonarch_base_libdir}/udev/rules.d/71-seat.rules \
+                ${nonarch_base_libdir}/udev/rules.d/73-seat-late.rules \
+                ${nonarch_base_libdir}/udev/rules.d/99-systemd.rules \
                "
 
 FILES_${PN}-dbg += "${rootlibdir}/.debug ${systemd_unitdir}/.debug ${systemd_unitdir}/*/.debug ${base_libdir}/security/.debug/"
@@ -327,7 +327,7 @@ RRECOMMENDS_${PN} += "systemd-serialgetty systemd-vconsole-setup \
 
 PACKAGES =+ "udev-dbg udev udev-hwdb"
 
-FILES_udev-dbg += "/lib/udev/.debug"
+FILES_udev-dbg += "${nonarch_base_libdir}/udev/.debug"
 
 RPROVIDES_udev = "hotplug"
 
-- 
2.1.4

[PATCH v2 4/9] alsa-utils: handle udev dir being 2 levels below /

[Joshua Lock]

When building with a merged /usr dir the udev directory
lives at /usr/lib/udev - update the FILES pattern to also
pick up udev files installed two levels below the / to
ensure a merged /usr works.

Signed-off-by: Joshua Lock <joshua.lock@collabora.co.uk>
---
 meta/recipes-multimedia/alsa/alsa-utils_1.0.29.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-multimedia/alsa/alsa-utils_1.0.29.bb b/meta/recipes-multimedia/alsa/alsa-utils_1.0.29.bb
index 263f38a..97fe4b2 100644
--- a/meta/recipes-multimedia/alsa/alsa-utils_1.0.29.bb
+++ b/meta/recipes-multimedia/alsa/alsa-utils_1.0.29.bb
@@ -58,7 +58,7 @@ FILES_alsa-utils-midi        = "${bindir}/aplaymidi ${bindir}/arecordmidi ${bind
 FILES_alsa-utils-aconnect    = "${bindir}/aconnect"
 FILES_alsa-utils-aseqnet     = "${bindir}/aseqnet"
 FILES_alsa-utils-iecset      = "${bindir}/iecset"
-FILES_alsa-utils-alsactl     = "${sbindir}/alsactl */udev/rules.d ${systemd_unitdir} ${localstatedir}/lib/alsa ${datadir}/alsa/init/"
+FILES_alsa-utils-alsactl     = "${sbindir}/alsactl */udev/rules.d */*/udev/rules.d ${systemd_unitdir} ${localstatedir}/lib/alsa ${datadir}/alsa/init/"
 FILES_alsa-utils-aseqdump    = "${bindir}/aseqdump"
 FILES_alsa-utils-alsaloop    = "${bindir}/alsaloop"
 FILES_alsa-utils-alsaucm     = "${bindir}/alsaucm"
-- 
2.1.4

[PATCH v2 5/9] pulseaudio: handle udev dir being 2 levels below /

[Joshua Lock]

When building with a merged /usr dir the udev directory
lives at /usr/lib/udev - update the FILES pattern to also
pick up udev files installed two levels below the / to
ensure a merged /usr works.

Signed-off-by: Joshua Lock <joshua.lock@collabora.co.uk>
---
 meta/recipes-multimedia/pulseaudio/pulseaudio.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-multimedia/pulseaudio/pulseaudio.inc b/meta/recipes-multimedia/pulseaudio/pulseaudio.inc
index 27e05d3..5b806d7 100644
--- a/meta/recipes-multimedia/pulseaudio/pulseaudio.inc
+++ b/meta/recipes-multimedia/pulseaudio/pulseaudio.inc
@@ -115,7 +115,7 @@ FILES_${PN}-dbg += "${libexecdir}/pulse/.debug \
 FILES_${PN}-dev += "${libdir}/pulse-${PV}/modules/*.la ${datadir}/vala ${libdir}/cmake"   
 FILES_${PN}-conf = "${sysconfdir}"
 FILES_${PN}-bin += "${sysconfdir}/default/volatiles/volatiles.04_pulse"
-FILES_${PN}-server = "${bindir}/pulseaudio ${bindir}/start-* ${sysconfdir} ${bindir}/pactl */udev/rules.d/*.rules ${systemd_user_unitdir}/*"
+FILES_${PN}-server = "${bindir}/pulseaudio ${bindir}/start-* ${sysconfdir} ${bindir}/pactl */udev/rules.d/*.rules */*/udev/rules.d/*.rules ${systemd_user_unitdir}/*"
 
 #SYSTEMD_PACKAGES = "${PN}-server"
 SYSTEMD_SERVICE_${PN}-server = "pulseaudio.service"
-- 
2.1.4

[PATCH v2 6/9] bluez: handle udev dir being 2 levels below /

[Joshua Lock]

When building with a merged /usr dir the udev directory
lives at /usr/lib/udev - update the FILES pattern to also
pick up udev files installed two levels below the / to
ensure a merged /usr works.

Signed-off-by: Joshua Lock <joshua.lock@collabora.co.uk>
---
 meta/recipes-connectivity/bluez5/bluez5.inc | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-connectivity/bluez5/bluez5.inc b/meta/recipes-connectivity/bluez5/bluez5.inc
index 0fe5be5..039c443 100644
--- a/meta/recipes-connectivity/bluez5/bluez5.inc
+++ b/meta/recipes-connectivity/bluez5/bluez5.inc
@@ -95,6 +95,7 @@ FILES_${PN}-dbg += "\
   ${libdir}/bluetooth/plugins/.debug \
   ${libdir}/*/.debug \
   */udev/.debug \
+  */*/udev/.debug \
   "
 
 RDEPENDS_${PN}-testtools += "python python-dbus python-pygobject"
-- 
2.1.4

[PATCH v2 7/9] pcmciautils: handle udev dir being 2 levels below /

[Joshua Lock]

When building with a merged /usr dir the udev directory
lives at /usr/lib/udev - update the FILES pattern to also
pick up udev files installed two levels below the / to
ensure a merged /usr works.

Signed-off-by: Joshua Lock <joshua.lock@collabora.co.uk>
---
 meta/recipes-bsp/pcmciautils/pcmciautils_018.bb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-bsp/pcmciautils/pcmciautils_018.bb b/meta/recipes-bsp/pcmciautils/pcmciautils_018.bb
index 24ceed8..857bd07 100644
--- a/meta/recipes-bsp/pcmciautils/pcmciautils_018.bb
+++ b/meta/recipes-bsp/pcmciautils/pcmciautils_018.bb
@@ -9,5 +9,5 @@ SRC_URI[sha256sum] = "79e6ae441278e178c07501d492394ed2c0326fdb66894f6d040ec811b0
 
 PR = "r1"
 
-FILES_${PN}-dbg += "*/udev/.debug"
-FILES_${PN} += "*/udev"
+FILES_${PN}-dbg += "*/udev/.debug */*/udev/.debug"
+FILES_${PN} += "*/udev */*/udev"
-- 
2.1.4

[PATCH v2 8/9] nfs-utils: don't force use of /sbin as sbindir

[Joshua Lock]

The Makefile for mount and osd_login utilities forces
/sbin as asbindir, however on a merged /usr system this directory
might not exist. Instead sed in the system sbindir.

Signed-off-by: Joshua Lock <joshua.lock@collabora.co.uk>
---
 meta/recipes-connectivity/nfs-utils/nfs-utils_1.3.1.bb | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/meta/recipes-connectivity/nfs-utils/nfs-utils_1.3.1.bb b/meta/recipes-connectivity/nfs-utils/nfs-utils_1.3.1.bb
index 6da8509..42101de 100644
--- a/meta/recipes-connectivity/nfs-utils/nfs-utils_1.3.1.bb
+++ b/meta/recipes-connectivity/nfs-utils/nfs-utils_1.3.1.bb
@@ -90,6 +90,14 @@ RDEPENDS_${PN}-stats = "python"
 
 FILES_${PN} += "${systemd_unitdir}"
 
+do_configure_prepend() {
+        sed -i -e 's,sbindir = /sbin,sbindir = ${base_sbindir},g' \
+            ${S}/utils/mount/Makefile.am
+
+        sed -i -e 's,sbindir = /sbin,sbindir = ${base_sbindir},g' \
+            ${S}/utils/osd_login/Makefile.am
+}
+
 # Make clean needed because the package comes with
 # precompiled 64-bit objects that break the build
 do_compile_prepend() {
-- 
2.1.4

[PATCH v2 9/9] procps: fix for base_sbindir == sbindir

[Joshua Lock]

An rmdir call in do_install_append was trying to remove the
sbindir, however in a system with a merged usr this directory
is not empty and therefore failing to rm it causes an ERROR.

Instead check that sbindir != base_sbindir before trying to
remove the directory.

Signed-off-by: Joshua Lock <joshua.lock@collabora.co.uk>
---
 meta/recipes-extended/procps/procps_3.3.10.bb | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-extended/procps/procps_3.3.10.bb b/meta/recipes-extended/procps/procps_3.3.10.bb
index 65d64ec..dcfaba7 100644
--- a/meta/recipes-extended/procps/procps_3.3.10.bb
+++ b/meta/recipes-extended/procps/procps_3.3.10.bb
@@ -31,8 +31,9 @@ do_install_append () {
 	[ "${bindir}" != "${base_bindir}" ] && for i in ${base_bindir_progs}; do mv ${D}${bindir}/$i ${D}${base_bindir}/$i; done
 	install -d ${D}${base_sbindir}
 	[ "${sbindir}" != "${base_sbindir}" ] && for i in ${base_sbindir_progs}; do mv ${D}${sbindir}/$i ${D}${base_sbindir}/$i; done
-        # Remove now empty dir
-	rmdir ${D}/${sbindir}
+        if [ "${base_sbindir}" != "${sbindir}" ]; then
+                rmdir ${D}${sbindir}
+        fi
 
         install -d ${D}${sysconfdir}
         install -m 0644 ${WORKDIR}/sysctl.conf ${D}${sysconfdir}/sysctl.conf
-- 
2.1.4

Re: [PATCH v2 1/9] kmod: fix link creation when base_bindir != /bin

[Richard Purdie]

On Fri, 2015-09-04 at 15:59 +0100, Joshua Lock wrote:
> If base_bindir is not a direct child of / the link creation in
> do_install_append creates incorrect relative links.
> 
> Instead pass a full path to the link source and use the -r flag
> to ln to ensure the link is relative.
> 
> Signed-off-by: Joshua Lock <joshua.lock@collabora.co.uk>
> ---
>  meta/recipes-kernel/kmod/kmod_git.bb | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/recipes-kernel/kmod/kmod_git.bb b/meta/recipes-kernel/kmod/kmod_git.bb
> index ba4d85e..120fd87 100644
> --- a/meta/recipes-kernel/kmod/kmod_git.bb
> +++ b/meta/recipes-kernel/kmod/kmod_git.bb
> @@ -21,9 +21,9 @@ do_install_append () {
>          install -dm755 ${D}${base_bindir}
>          install -dm755 ${D}${base_sbindir}
>          # add symlinks to kmod
> -        ln -s ..${base_bindir}/kmod ${D}${base_bindir}/lsmod
> +        ln -sr ${D}${base_bindir}/kmod ${D}${base_bindir}/lsmod
>          for tool in insmod rmmod depmod modinfo modprobe; do
> -                ln -s ..${base_bindir}/kmod ${D}${base_sbindir}/${tool}
> +                ln -sr ${D}${base_bindir}/kmod ${D}${base_sbindir}/${tool}
>          done
>          # configuration directories
>          install -dm755 ${D}${base_libdir}/depmod.d

Not all our build machines appear to support ln -r, e.g.:

https://autobuilder.yoctoproject.org/main/builders/nightly-x32/builds/479/steps/BuildImages/logs/stdio

(debian7)

We do however have a python script called "lnr" in scripts which might
help...

Cheers,

Richard