[PATCH 0/5] misc enhancements

[PATCH 0/5] misc enhancements

[Patrick Ohly]

I've worked on making Ostro OS support Toybox and "stateless"
configuration, i.e. running with empty /etc and reserving that
for local configuration changes.

That work depends on several enhancements in
OE-core. "package_manager.py: better error handling in opkg's package
listing" is not needed anymore, but helped debug a packaging problem.

Please pick whatever you deem suitable for OE-core at this time, I'll
keep maintaining the rest and can re-submit again later.

The following changes since commit d60806e56aed2f62f6a0e030a564f7fdc4a1314d:

  build-appliance-image: Exclude DDATETIME from task signature (2016-04-03 15:51:10 +0100)

are available in the git repository at:

  git://github.com/pohly/openembedded-core stateless
  https://github.com/pohly/openembedded-core/tree/stateless

Patrick Ohly (5):
  bluez5.inc: remove obsolete workaround
  ncurses: reorder PACKAGES
  systemd: make systemd-serialgetty optional
  package_manager.py: better error handling in opkg's package listing
  ca-certificates: support Toybox

 meta/lib/oe/package_manager.py                     | 15 ++++++----
 meta/recipes-connectivity/bluez5/bluez5.inc        |  2 --
 meta/recipes-core/ncurses/ncurses.inc              |  2 +-
 meta/recipes-core/systemd/systemd_229.bb           | 14 +++++++--
 .../update-ca-certificates-support-Toybox.patch    | 35 ++++++++++++++++++++++
 .../ca-certificates/ca-certificates_20160104.bb    |  1 +
 6 files changed, 57 insertions(+), 12 deletions(-)
 create mode 100644 meta/recipes-support/ca-certificates/ca-certificates/update-ca-certificates-support-Toybox.patch

-- 
2.1.4

[PATCH 1/5] bluez5.inc: remove obsolete workaround

[Patrick Ohly]

Bluez 5.37 itself correctly installs bluetooth.conf, and honors
the path settings in dbus-1.pc.

Removing the obsolete workaround is necessary for compiling
"stateless" (= read-only system configuration moved out of /etc).

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
---
 meta/recipes-connectivity/bluez5/bluez5.inc | 2 --
 1 file changed, 2 deletions(-)

diff --git a/meta/recipes-connectivity/bluez5/bluez5.inc b/meta/recipes-connectivity/bluez5/bluez5.inc
index 5232523..a508229 100644
--- a/meta/recipes-connectivity/bluez5/bluez5.inc
+++ b/meta/recipes-connectivity/bluez5/bluez5.inc
@@ -60,8 +60,6 @@ do_install_append() {
 	    install -m 0644 ${S}/profiles/input/input.conf ${D}/${sysconfdir}/bluetooth/
 	fi
 
-	install -m 0644 ${S}/src/bluetooth.conf ${D}/${sysconfdir}/dbus-1/system.d/
-
 	# Install desired tools that upstream leaves in build area
         for f in ${NOINST_TOOLS} ; do
 	    install -m 755 ${B}/$f ${D}/${bindir}
-- 
2.1.4

[PATCH 2/5] ncurses: reorder PACKAGES

[Patrick Ohly]

Having ncurses-terminfo-base before ncurses-terminfo is currently irrelevant
because the current file lists are completely disjunct. However, when building
"stateless" via a .bbappend, the content of curses-terminfo-base also needs
to live under /usr/share and then it becomes important that files
belonging to ncurses-terminfo-base are checked first.

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
---
 meta/recipes-core/ncurses/ncurses.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/ncurses/ncurses.inc b/meta/recipes-core/ncurses/ncurses.inc
index acd2579..ff0117b 100644
--- a/meta/recipes-core/ncurses/ncurses.inc
+++ b/meta/recipes-core/ncurses/ncurses.inc
@@ -269,8 +269,8 @@ BBCLASSEXTEND = "native nativesdk"
 
 PACKAGES += " \
   ${PN}-tools \
-  ${PN}-terminfo \
   ${PN}-terminfo-base \
+  ${PN}-terminfo \
 "
 
 FILES_${PN} = "\
-- 
2.1.4

[PATCH 3/5] systemd: make systemd-serialgetty optional

[Patrick Ohly]

Some distros may prefer to use the upstream systemd support for
starting getty on serial ports. This is now possible by adding
"serial-getty-generator" to PACKAGECONFIG.

The default is unchanged, i.e. systemd's own serial-getty@.service
file does not get packaged and instead systemd-serialgetty is pulled
into images via RRECOMMENDS.

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
---
 meta/recipes-core/systemd/systemd_229.bb | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-core/systemd/systemd_229.bb b/meta/recipes-core/systemd/systemd_229.bb
index 309a4c3..c23c749 100644
--- a/meta/recipes-core/systemd/systemd_229.bb
+++ b/meta/recipes-core/systemd/systemd_229.bb
@@ -96,6 +96,11 @@ PACKAGECONFIG ??= "xz \
 PACKAGECONFIG_remove_libc-musl = "selinux"
 PACKAGECONFIG_remove_libc-musl = "smack"
 
+# Use the upstream systemd serial-getty@.service and rely on
+# systemd-getty-generator instead of using the OE-core specific
+# systemd-serialgetty.bb - not enabled by default.
+PACKAGECONFIG[serial-getty-generator] = ""
+
 PACKAGECONFIG[journal-upload] = "--enable-libcurl,--disable-libcurl,curl"
 # Sign the journal for anti-tampering
 PACKAGECONFIG[gcrypt] = "--enable-gcrypt,--disable-gcrypt,libgcrypt"
@@ -204,8 +209,10 @@ do_configure_prepend() {
 do_install() {
 	autotools_do_install
 	install -d ${D}/${base_sbindir}
-	# Provided by a separate recipe
-	rm ${D}${systemd_unitdir}/system/serial-getty* -f
+	if ${@bb.utils.contains('PACKAGECONFIG', 'serial-getty-generator', 'false', 'true', d)}; then
+		# Provided by a separate recipe
+		rm ${D}${systemd_unitdir}/system/serial-getty* -f
+	fi
 
 	# Provide support for initramfs
 	[ ! -e ${D}/init ] && ln -s ${rootlibexecdir}/systemd/systemd ${D}/init
@@ -455,7 +462,8 @@ FILES_${PN}-dev += "${base_libdir}/security/*.la ${datadir}/dbus-1/interfaces/ $
 RDEPENDS_${PN} += "kmod dbus util-linux-mount udev (= ${EXTENDPKGV})"
 RDEPENDS_${PN} += "volatile-binds update-rc.d"
 
-RRECOMMENDS_${PN} += "systemd-serialgetty systemd-vconsole-setup \
+RRECOMMENDS_${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'serial-getty-generator', '', 'systemd-serialgetty', d)} \
+                      systemd-vconsole-setup \
                       systemd-extra-utils \
                       systemd-compat-units udev-hwdb \
                       util-linux-agetty  util-linux-fsck e2fsprogs-e2fsck \
-- 
2.1.4

[PATCH 4/5] package_manager.py: better error handling in opkg's package listing

[Patrick Ohly]

opkg does not return a non-zero exit code even if it found
errors. When that happens, parsing the output leads to strange
follow-up errors.

To avoid this we need to check explicitly for non-empty
stderr. Reporting only that on a failure also leads to shorter error
messages (stdout may be very large).

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
---
 meta/lib/oe/package_manager.py | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 0d23d8b..b4b359a 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -468,13 +468,16 @@ class OpkgPkgsList(PkgsList):
     def list_pkgs(self, format=None):
         cmd = "%s %s status" % (self.opkg_cmd, self.opkg_args)
 
-        try:
-            # bb.note(cmd)
-            cmd_output = subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=True).strip()
-
-        except subprocess.CalledProcessError as e:
+        # opkg returns success even when it printed some
+        # "Collected errors:" report to stderr. Mixing stderr into
+        # stdout then leads to random failures later on when
+        # parsing the output. To avoid this we need to collect both
+        # output streams separately and check for empty stderr.
+        p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
+        cmd_output, cmd_stderr = p.communicate()
+        if p.returncode or cmd_stderr:
             bb.fatal("Cannot get the installed packages list. Command '%s' "
-                     "returned %d:\n%s" % (cmd, e.returncode, e.output))
+                     "returned %d and stderr:\n%s" % (cmd, p.returncode, cmd_stderr))
 
         return self.opkg_query(cmd_output)
 
-- 
2.1.4

[PATCH 5/5] ca-certificates: support Toybox

[Patrick Ohly]

"mktemp -t" is deprecated and does not work when using Toybox. Replace
with something that works also with Toybox.

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
---
 .../update-ca-certificates-support-Toybox.patch    | 35 ++++++++++++++++++++++
 .../ca-certificates/ca-certificates_20160104.bb    |  1 +
 2 files changed, 36 insertions(+)
 create mode 100644 meta/recipes-support/ca-certificates/ca-certificates/update-ca-certificates-support-Toybox.patch

diff --git a/meta/recipes-support/ca-certificates/ca-certificates/update-ca-certificates-support-Toybox.patch b/meta/recipes-support/ca-certificates/ca-certificates/update-ca-certificates-support-Toybox.patch
new file mode 100644
index 0000000..b3b21e2
--- /dev/null
+++ b/meta/recipes-support/ca-certificates/ca-certificates/update-ca-certificates-support-Toybox.patch
@@ -0,0 +1,35 @@
+From 30378026d136efa779732e3f6664e2ecf461e458 Mon Sep 17 00:00:00 2001
+From: Patrick Ohly <patrick.ohly@intel.com>
+Date: Thu, 17 Mar 2016 12:38:09 +0100
+Subject: [PATCH] update-ca-certificates: support Toybox
+
+"mktemp -t" is deprecated and does not work when using Toybox. Replace
+with something that works also with Toybox.
+
+Upstream-Status: Inappropriate [embedded specific]
+
+Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
+---
+ sbin/update-ca-certificates | 6 +++---
+ 1 file changed, 3 insertions(+), 3 deletions(-)
+
+diff --git a/sbin/update-ca-certificates b/sbin/update-ca-certificates
+index 79c41bb..ae9e3f1 100755
+--- a/sbin/update-ca-certificates
++++ b/sbin/update-ca-certificates
+@@ -113,9 +113,9 @@ trap cleanup 0
+ 
+ # Helper files.  (Some of them are not simple arrays because we spawn
+ # subshells later on.)
+-TEMPBUNDLE="$(mktemp -t "${CERTBUNDLE}.tmp.XXXXXX")"
+-ADDED="$(mktemp -t "ca-certificates.tmp.XXXXXX")"
+-REMOVED="$(mktemp -t "ca-certificates.tmp.XXXXXX")"
++TEMPBUNDLE="$(mktemp -p${TMPDIR:-/tmp} "${CERTBUNDLE}.tmp.XXXXXX")"
++ADDED="$(mktemp -p${TMPDIR:-/tmp} "ca-certificates.tmp.XXXXXX")"
++REMOVED="$(mktemp -p${TMPDIR:-/tmp} "ca-certificates.tmp.XXXXXX")"
+ 
+ # Adds a certificate to the list of trusted ones.  This includes a symlink
+ # in /etc/ssl/certs to the certificate file and its inclusion into the
+-- 
+2.1.4
+
diff --git a/meta/recipes-support/ca-certificates/ca-certificates_20160104.bb b/meta/recipes-support/ca-certificates/ca-certificates_20160104.bb
index 4266926..e0f1939 100644
--- a/meta/recipes-support/ca-certificates/ca-certificates_20160104.bb
+++ b/meta/recipes-support/ca-certificates/ca-certificates_20160104.bb
@@ -17,6 +17,7 @@ SRC_URI = "git://anonscm.debian.org/collab-maint/ca-certificates.git \
            file://0001-update-ca-certificates-remove-c-rehash.patch \
            file://0002-update-ca-certificates-use-SYSROOT.patch \
            file://0001-update-ca-certificates-don-t-use-Debianisms-in-run-p.patch \
+           file://update-ca-certificates-support-Toybox.patch \
            file://default-sysroot.patch \
            file://sbindir.patch"
 
-- 
2.1.4

Re: [PATCH 0/5] misc enhancements

[Otavio Salvador]

On Mon, Apr 4, 2016 at 10:41 AM, Patrick Ohly <patrick.ohly@intel.com> wrote:
> I've worked on making Ostro OS support Toybox and "stateless"
> configuration, i.e. running with empty /etc and reserving that
> for local configuration changes.
>
> That work depends on several enhancements in
> OE-core. "package_manager.py: better error handling in opkg's package
> listing" is not needed anymore, but helped debug a packaging problem.
>
> Please pick whatever you deem suitable for OE-core at this time, I'll
> keep maintaining the rest and can re-submit again later.

I looked at all patches and all looks sane and generic for OE-Core acceptance.


-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 5/5] ca-certificates: support Toybox

[Richard Purdie]

On Mon, 2016-04-04 at 15:41 +0200, Patrick Ohly wrote:
> "mktemp -t" is deprecated and does not work when using Toybox.
> Replace
> with something that works also with Toybox.
> 
> Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
> ---
>  .../update-ca-certificates-support-Toybox.patch    | 35
> ++++++++++++++++++++++
>  .../ca-certificates/ca-certificates_20160104.bb    |  1 +
>  2 files changed, 36 insertions(+)
>  create mode 100644 meta/recipes-support/ca-certificates/ca
> -certificates/update-ca-certificates-support-Toybox.patch
> 
> diff --git a/meta/recipes-support/ca-certificates/ca
> -certificates/update-ca-certificates-support-Toybox.patch
> b/meta/recipes-support/ca-certificates/ca-certificates/update-ca
> -certificates-support-Toybox.patch
> new file mode 100644
> index 0000000..b3b21e2
> --- /dev/null
> +++ b/meta/recipes-support/ca-certificates/ca-certificates/update-ca
> -certificates-support-Toybox.patch
> @@ -0,0 +1,35 @@
> +From 30378026d136efa779732e3f6664e2ecf461e458 Mon Sep 17 00:00:00
> 2001
> +From: Patrick Ohly <patrick.ohly@intel.com>
> +Date: Thu, 17 Mar 2016 12:38:09 +0100
> +Subject: [PATCH] update-ca-certificates: support Toybox
> +
> +"mktemp -t" is deprecated and does not work when using Toybox.
> Replace
> +with something that works also with Toybox.
> +
> +Upstream-Status: Inappropriate [embedded specific]

If its deprecated, is it really Inappropriate?

Cheers,

Richard