[RFC PATCH 0/1] classes: add class for retaining build results

[RFC PATCH 0/1] classes: add class for retaining build results

[Paul Eggleton]

This is a renamed, reworked and polished version of the workdir_save class
that I sent previously. I'll send documentation patches separately if folks
think this looks good.


Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

The following changes since commit 8d78b819c2ec33fce3a34254fa90864ee5fa7617:

  IMAGE_LOCALES_ARCHIVE: add option to prevent locale archive creation (2020-10-13 09:41:51 +0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib paule/retain
  http://cgit.openembedded.org/openembedded-core-contrib/log/?h=paule/retain

Paul Eggleton (1):
  classes: add class for retaining build results

 meta/classes/retain.bbclass | 103 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 103 insertions(+)
 create mode 100644 meta/classes/retain.bbclass

-- 
1.8.3.1

[RFC PATCH 1/1] classes: add class for retaining build results

[Paul Eggleton]

From: Paul Eggleton <paul.eggleton@microsoft.com>

If you are running your builds inside an environment where you don't
have access to the build tree (e.g. an autobuilder where you can only
download final artifacts such as images), then debugging build failures
can be difficult - you can't examine log files, the source tree or
output files. When enabled, this class does two things:

1) Triggers on task failure and saves a tarball of the work directory
   for the task's recipe
2) Optionally saves tarballs of a list of nominated directories

It puts these tarballs in a configurable location, where they can be
picked up by a separate process and made available as downloadable
artifacts.

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
---
 meta/classes/retain.bbclass | 103 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 103 insertions(+)
 create mode 100644 meta/classes/retain.bbclass

diff --git a/meta/classes/retain.bbclass b/meta/classes/retain.bbclass
new file mode 100644
index 0000000..e98765f
--- /dev/null
+++ b/meta/classes/retain.bbclass
@@ -0,0 +1,103 @@
+# Creates a tarball of the work directory for a recipe when one of its
+# tasks fails, as well as (optionally) other nominated directories.
+# Useful in cases where the environment in which builds are run is
+# ephemeral or otherwise inaccessible for examination during
+# debugging.
+#
+# To enable, simply add the following to your configuration:
+#
+# INHERIT += "retain"
+#
+# You can also specify extra directories to save at the end of the build
+# upon failure or always (space-separated) e.g.:
+#
+# RETAIN_EXTRADIRS_FAILURE = "${LOG_DIR} ${TMPDIR}/pkgdata"
+# RETAIN_EXTRADIRS_ALWAYS = "${BUILDSTATS_BASE}"
+#
+# If you wish to use a different tarball name prefix you can do so by
+# adding a : followed by the desired prefix (no spaces) e.g. to use
+# "buildlogs" for the tarball of ${LOG_DIR} you would do this:
+#
+# RETAIN_EXTRADIRS_FAILURE = "${LOG_DIR}:buildlogs ${TMPDIR}/pkgdata"
+#
+# Notes:
+# * For this to be useful you also need corresponding logic in your build
+#   orchestration tool to pick up any files written out to RETAIN_OUTDIR
+#   (with the other assumption being that no files are present there at
+#   the start of the build).
+# * Work directories can be quite large, so saving them can take some time
+#   and of course space.
+# * Extra directories must naturally be populated at the time the retain
+#   goes to save them (build completion); to try ensure this for things
+#   that are also saved on build completion (e.g. buildstats), put the
+#   INHERIT += "retain" after the INHERIT += lines for the class that
+#   is writing out the data that you wish to save.
+# * The tarballs have the tarball name as a top-level directory so that
+#   multiple tarballs can be extracted side-by-side easily.
+#
+# Copyright (c) 2020 Microsoft Corporation
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+
+RETAIN_OUTDIR ?= "${TMPDIR}/retained"
+RETAIN_EXTRADIRS_FAILURE ?= ""
+RETAIN_EXTRADIRS_ALWAYS ?= ""
+RETAIN_ENABLED ?= "1"
+
+
+def retain_retain_dir(desc, tarprefix, path, tarbasepath, d):
+    import datetime
+
+    outdir = d.getVar('RETAIN_OUTDIR')
+    bb.utils.mkdirhier(outdir)
+    tstamp = datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
+    tarname = '%s_%s' % (tarprefix, tstamp)
+    tarfp = os.path.join(outdir, '%s.tar.gz' % tarname)
+    tardir = os.path.relpath(path, tarbasepath)
+    cmdargs = ['tar', 'czf', tarfp]
+    # Prefix paths within the tarball with the tarball name so that
+    # multiple tarballs can be extracted side-by-side
+    cmdargs += ['--transform', 's:^:%s/:' % tarname]
+    cmdargs += [tardir]
+    bb.plain('NOTE: retain: saving %s to %s' % (desc, tarfp))
+    try:
+        bb.process.run(cmdargs, cwd=tarbasepath)
+    except bb.process.ExecutionError as e:
+        # It is possible for other tasks to be writing to the workdir
+        # while we are tarring it up, in which case tar will return 1,
+        # but we don't care in this situation (tar returns 2 for other
+        # errors so we we will see those)
+        if e.exitcode != 1:
+            bb.warn('retain: error saving %s: %s' % (desc, str(e)))
+
+
+addhandler retain_workdir_handler
+retain_workdir_handler[eventmask] = "bb.build.TaskFailed bb.event.BuildCompleted"
+
+python retain_workdir_handler() {
+    if d.getVar('RETAIN_ENABLED') != '1':
+        return
+
+    if isinstance(e, bb.build.TaskFailed):
+        pn = d.getVar('PN')
+        workdir = d.getVar('WORKDIR')
+        base_workdir = d.getVar('BASE_WORKDIR')
+        taskname = d.getVar('BB_CURRENTTASK')
+        desc = 'workdir for failed task %s.do_%s' % (pn, taskname)
+        retain_retain_dir(desc, 'workdir_%s' % pn, workdir, base_workdir, d)
+    elif isinstance(e, bb.event.BuildCompleted):
+        paths = d.getVar('RETAIN_EXTRADIRS_ALWAYS').split()
+        if e._failures:
+            paths += d.getVar('RETAIN_EXTRADIRS_FAILURE').split()
+
+        for path in list(set(paths)):
+            if ':' in path:
+                path, itemname = path.rsplit(':', 1)
+            else:
+                itemname = os.path.basename(path)
+            if os.path.exists(path):
+                retain_retain_dir(itemname, itemname, path, os.path.dirname(path), d)
+            else:
+                bb.warn('retain: extra directory %s does not currently exist' % path)
+}
-- 
1.8.3.1