[OE-core][RFC PATCH 2/2] image.bbclass: deploy image artifacts in stages

["Bartosz Golaszewski" <brgl@bgdev.pl>]

From: Bartosz Golaszewski <bgolaszewski@baylibre.com>

Make each IMAGE_CMD task an sstate task with its own IMGDEPLOYDIR
override. This way each generated set of artifacts is deployed as soon
as it's ready instead of the do_image_complete task handling the entire
deployement. This allows us to better fine-tune dependencies e.g. we
can make do_image_wic depend on fitImage task which can in turn depend
on do_image_ext4.

We need to completely delete the do_package task in order to avoid
problems with task hash changes as well as delete the IMGDEPLOYDIR
variable from the data object passed to each image task so that it's
expanded with the correct override.

Signed-off-by: Bartosz Golaszewski <bgolaszewski@baylibre.com>
---
 meta/classes/image.bbclass | 26 +++++++++++++++++++++++++-
 1 file changed, 25 insertions(+), 1 deletion(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 07aa1f1fa5..3487e2abe4 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -428,10 +428,12 @@ python () {
         # date/time values. It will get expanded at execution time.
         # Similarly TMPDIR since otherwise we see QA stamp comparision problems
         # Expand PV else it can trigger get_srcrev which can fail due to these variables being unset
+        # Delete IMGDEPLOYDIR so that each task gets its own, overriden value
         localdata.setVar('PV', d.getVar('PV'))
         localdata.delVar('DATETIME')
         localdata.delVar('DATE')
         localdata.delVar('TMPDIR')
+        localdata.delVar('IMGDEPLOYDIR')
         vardepsexclude = (d.getVarFlag('IMAGE_CMD_' + realt, 'vardepsexclude', True) or '').split()
         for dep in vardepsexclude:
             localdata.delVar(dep)
@@ -502,6 +504,28 @@ python () {
 
         bb.debug(2, "Adding task %s before %s, after %s" % (task, 'do_image_complete', after))
         bb.build.addtask(task, 'do_image_complete', after, d)
+
+        imgdeploydir = d.getVar('IMGDEPLOYDIR')
+        task_override = task[3:].replace('_', '-') # 'do_image_ext4' becomes 'image-ext4'
+        taskdeploydir = '{}/deploy-{}-{}'.format(os.path.dirname(imgdeploydir),
+                                                 d.getVar('PN'), task_override)
+
+        # Each image task gets its own IMGDEPLOYDIR and becomes part of SSTATETASKS
+        # so that each set of artifacts be deployed right after the task completes.
+        d.setVar('IMGDEPLOYDIR_task-{}'.format(task_override), taskdeploydir)
+        d.appendVar('SSTATETASKS', ' {}'.format(task))
+        d.setVarFlag(task, 'sstate-inputdirs', taskdeploydir)
+        d.setVarFlag(task, 'sstate-outputdirs', d.getVar('DEPLOY_DIR_IMAGE'))
+        d.setVarFlag(task, 'cleandirs', taskdeploydir)
+        d.setVar('SSTATE_SKIP_CREATION_task-{}'.format(task_override), '1')
+
+        # FIXME We need to do this manually if we want to set the sstate flags from
+        # python code. The reason for this is that sstate.bbclass does this from an
+        # annonymous python function too and we can't guarantee correct ordering.
+        # This isn't a problem for most recipes as they define sstate tasks statically
+        # with bitbake variables.
+        d.prependVarFlag(task, 'prefuncs', 'sstate_task_prefunc ')
+        d.appendVarFlag(task, 'postfuncs', ' sstate_task_postfunc')
 }
 
 #
@@ -611,7 +635,7 @@ do_compile[noexec] = "1"
 do_install[noexec] = "1"
 deltask do_populate_lic
 deltask do_populate_sysroot
-do_package[noexec] = "1"
+deltask do_package
 deltask do_package_qa
 do_packagedata[noexec] = "1"
 deltask do_package_write_ipk
-- 
2.19.1