All of lore.kernel.org
 help / color / mirror / Atom feed
From: Joshua Watt <jpewhacker@gmail.com>
To: openembedded-core@lists.openembedded.org
Cc: Joshua Watt <JPEWhacker@gmail.com>
Subject: [OE-core][PATCH 1/6] classes-recipe/image: Add image file manifest
Date: Mon, 10 Jun 2024 15:41:47 -0600	[thread overview]
Message-ID: <20240610214456.2757397-2-JPEWhacker@gmail.com> (raw)
In-Reply-To: <20240610214456.2757397-1-JPEWhacker@gmail.com>

Downstream tasks may want to know what image files were written by the
do_image family of tasks (e.g. SPDX) so have each task write out a
manifest file that describes the files it produced, then aggregate them
in do_image_complete

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
---
 meta/classes-recipe/image.bbclass | 61 +++++++++++++++++++++++++++++++
 1 file changed, 61 insertions(+)

diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass
index 28be6c63623..122af2b0328 100644
--- a/meta/classes-recipe/image.bbclass
+++ b/meta/classes-recipe/image.bbclass
@@ -88,6 +88,11 @@ PACKAGE_INSTALL_ATTEMPTONLY ?= "${FEATURE_INSTALL_OPTIONAL}"
 
 IMGDEPLOYDIR = "${WORKDIR}/deploy-${PN}-image-complete"
 
+IMGMANIFESTDIR = "${WORKDIR}/image-task-manifest"
+
+IMAGE_FILE_MANIFEST_DIR = "${WORKDIR}/deploy-image-file-manifest"
+IMAGE_FILE_MANIFEST = "${IMAGE_FILE_MANIFEST_DIR}/manifest.json"
+
 # Images are generally built explicitly, do not need to be part of world.
 EXCLUDE_FROM_WORLD = "1"
 
@@ -277,14 +282,31 @@ fakeroot python do_image () {
     execute_pre_post_process(d, pre_process_cmds)
 }
 do_image[dirs] = "${TOPDIR}"
+do_image[cleandirs] = "${IMGMANIFESTDIR}"
 addtask do_image after do_rootfs
 
 fakeroot python do_image_complete () {
     from oe.utils import execute_pre_post_process
+    from pathlib import Path
+    import json
 
     post_process_cmds = d.getVar("IMAGE_POSTPROCESS_COMMAND")
 
     execute_pre_post_process(d, post_process_cmds)
+
+    image_manifest_dir = Path(d.getVar('IMGMANIFESTDIR'))
+
+    data = []
+
+    for taskname in bb.build.preceedtask("do_image_complete", False, d):
+        manifest_path = image_manifest_dir / (taskname + ".json")
+        if not manifest_path.exists():
+            continue
+        with manifest_path.open("r") as f:
+            data.extend(json.load(f))
+
+    with open(d.getVar("IMAGE_FILE_MANIFEST"), "w") as f:
+        json.dump(data, f)
 }
 do_image_complete[dirs] = "${TOPDIR}"
 SSTATETASKS += "do_image_complete"
@@ -292,6 +314,8 @@ SSTATE_SKIP_CREATION:task-image-complete = '1'
 do_image_complete[sstate-inputdirs] = "${IMGDEPLOYDIR}"
 do_image_complete[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"
 do_image_complete[stamp-extra-info] = "${MACHINE_ARCH}"
+do_image_complete[sstate-plaindirs] =  "${IMAGE_FILE_MANIFEST_DIR}"
+do_image_complete[dirs] = "${IMAGE_FILE_MANIFEST_DIR}"
 addtask do_image_complete after do_image before do_build
 python do_image_complete_setscene () {
     sstate_setscene(d)
@@ -513,6 +537,8 @@ python () {
         d.appendVarFlag(task, 'subimages', ' ' + ' '.join(subimages))
         d.appendVarFlag(task, 'vardeps', ' ' + ' '.join(vardeps))
         d.appendVarFlag(task, 'vardepsexclude', ' DATETIME DATE ' + ' '.join(vardepsexclude))
+        d.appendVarFlag(task, 'postfuncs', ' write_image_file_manifest')
+        d.appendVarFlag(task, 'dirs', ' ' + d.getVar('IMGMANIFESTDIR'))
 
         bb.debug(2, "Adding task %s before %s, after %s" % (task, 'do_image_complete', after))
         bb.build.addtask(task, 'do_image_complete', after, d)
@@ -610,6 +636,41 @@ python create_symlinks() {
             bb.note("Skipping symlink, source does not exist: %s -> %s" % (dst, src))
 }
 
+python write_image_file_manifest() {
+    import json
+    from pathlib import Path
+
+    taskname = d.getVar("BB_CURRENTTASK")
+    image_deploy_dir = Path(d.getVar('IMGDEPLOYDIR'))
+    image_manifest_dir = Path(d.getVar('IMGMANIFESTDIR'))
+    manifest_path = image_manifest_dir / ("do_" + d.getVar("BB_CURRENTTASK") + ".json")
+
+    image_name = d.getVar("IMAGE_NAME")
+    image_basename = d.getVar("IMAGE_BASENAME")
+    machine = d.getVar("MACHINE")
+
+    subimages = (d.getVarFlag("do_" + taskname, 'subimages', False) or "").split()
+    imagetype = d.getVarFlag("do_" + taskname, 'imagetype', False)
+
+    data = {
+        "taskname": taskname,
+        "imagetype": imagetype,
+        "images": []
+    }
+
+    for type in subimages:
+        image_filename = image_name + "." + type
+        image_path = image_deploy_dir / image_filename
+        if not image_path.exists():
+            continue
+        data["images"].append({
+            "filename": image_filename,
+        })
+
+    with manifest_path.open("w") as f:
+        json.dump([data], f)
+}
+
 MULTILIBRE_ALLOW_REP += "${base_bindir} ${base_sbindir} ${bindir} ${sbindir} ${libexecdir} ${sysconfdir} ${nonarch_base_libdir}/udev /lib/modules/[^/]*/modules.*"
 MULTILIB_CHECK_FILE = "${WORKDIR}/multilib_check.py"
 MULTILIB_TEMP_ROOTFS = "${WORKDIR}/multilib"
-- 
2.43.2



  reply	other threads:[~2024-06-10 21:45 UTC|newest]

Thread overview: 93+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-06-10 21:41 [OE-core][PATCH 0/6] Add SPDX 3.0 support Joshua Watt
2024-06-10 21:41 ` Joshua Watt [this message]
2024-06-11  8:50   ` [OE-core][PATCH 1/6] classes-recipe/image: Add image file manifest Martin Hundebøll
2024-06-10 21:41 ` [OE-core][PATCH 2/6] classes/spdx-common: Move common SPDX to new class Joshua Watt
2024-07-17 16:44   ` Adrian Freihofer
2024-06-10 21:41 ` [OE-core][PATCH 3/6] classes/spdx-common: Add SPDX version to path Joshua Watt
2024-06-10 21:41 ` [OE-core][PATCH 4/6] classes/spdx-common: Return empty list from extract_licenses Joshua Watt
2024-06-10 21:41 ` [OE-core][PATCH 5/6] classes/create-spdx-3.0: Add class Joshua Watt
2024-06-10 22:11   ` Patchtest results for " patchtest
2024-06-18 14:48   ` Marta Rybczynska
2024-06-18 15:22     ` Joshua Watt
2024-06-10 21:41 ` [OE-core][PATCH 6/6] classes-recipe/image_types: Add SPDX_IMAGE_PURPOSE to images Joshua Watt
2024-06-11  8:50 ` [OE-core][PATCH 0/6] Add SPDX 3.0 support Richard Purdie
2024-06-11 10:40 ` Richard Purdie
2024-06-11 14:42   ` Joshua Watt
2024-06-19 22:13 ` [OE-core][PATCH v2 0/7] " Joshua Watt
2024-06-19 22:13   ` [OE-core][PATCH v2 1/7] classes-recipe/image: Add image file manifest Joshua Watt
2024-06-19 22:13   ` [OE-core][PATCH v2 2/7] classes/create-spdx-3.0: Add classs Joshua Watt
2024-06-19 22:45     ` Patchtest results for " patchtest
2024-06-19 22:13   ` [OE-core][PATCH v2 3/7] classes-recipe/image_types: Add SPDX_IMAGE_PURPOSE to images Joshua Watt
2024-06-19 22:13   ` [OE-core][PATCH v2 4/7] selftest: spdx: Add SPDX 3.0 test cases Joshua Watt
2024-06-19 22:13   ` [OE-core][PATCH v2 5/7] classes-recipe: nospdx: Add class Joshua Watt
2024-06-19 22:13   ` [OE-core][PATCH v2 6/7] classes/spdx-common: Move SPDX_SUPPLIER Joshua Watt
2024-06-19 22:13   ` [OE-core][PATCH v2 7/7] Switch default spdx version to 3.0 Joshua Watt
2024-06-19 22:45     ` Patchtest results for " patchtest
2024-06-21  4:15   ` [OE-core][PATCH v2 0/7] Add SPDX 3.0 support Khem Raj
2024-06-21  6:24   ` Alexandre Belloni
2024-06-21 14:24     ` Joshua Watt
2024-06-21 17:21     ` Joshua Watt
2024-06-24 15:20   ` [OE-core][PATCH v3 00/10] " Joshua Watt
2024-06-24 15:20     ` [OE-core][PATCH v3 01/10] classes-recipe/image: Add image file manifest Joshua Watt
2024-06-24 15:20     ` [OE-core][PATCH v3 02/10] classes-recipe/baremetal-image: " Joshua Watt
2024-06-24 15:20     ` [OE-core][PATCH v3 03/10] classes/create-spdx-3.0: Add classes Joshua Watt
2024-06-24 15:20     ` [OE-core][PATCH v3 04/10] classes-global/staging: Exclude do_create_spdx from automatic sysroot extension Joshua Watt
2024-06-24 15:20     ` [OE-core][PATCH v3 05/10] binutils-cross-testsuite: Rename to binutils-testsuite Joshua Watt
2024-06-24 15:20     ` [OE-core][PATCH v3 06/10] classes-recipe/image_types: Add SPDX_IMAGE_PURPOSE to images Joshua Watt
2024-06-24 15:20     ` [OE-core][PATCH v3 07/10] selftest: spdx: Add SPDX 3.0 test cases Joshua Watt
2024-06-24 15:20     ` [OE-core][PATCH v3 08/10] classes-recipe: nospdx: Add class Joshua Watt
2024-06-24 15:20     ` [OE-core][PATCH v3 09/10] classes/spdx-common: Move SPDX_SUPPLIER Joshua Watt
2024-06-24 15:20     ` [OE-core][PATCH v3 10/10] Switch default spdx version to 3.0 Joshua Watt
2024-06-24 19:10   ` [OE-core][PATCH v4 00/10] Add SPDX 3.0 support Joshua Watt
2024-06-24 19:10     ` [OE-core][PATCH v4 01/10] classes-recipe/image: Add image file manifest Joshua Watt
2024-06-24 19:10     ` [OE-core][PATCH v4 02/10] classes-recipe/baremetal-image: " Joshua Watt
2024-06-25 10:24       ` Ernst Persson
2024-06-24 19:10     ` [OE-core][PATCH v4 03/10] classes/create-spdx-3.0: Add classes Joshua Watt
2024-06-25 14:44       ` Richard Purdie
2024-06-25 18:40       ` Mark Hatle
2024-06-27 16:33         ` Joshua Watt
2024-06-27 16:47           ` Joshua Watt
2024-06-24 19:10     ` [OE-core][PATCH v4 04/10] classes-global/staging: Exclude do_create_spdx from automatic sysroot extension Joshua Watt
2024-06-24 19:10     ` [OE-core][PATCH v4 05/10] binutils-cross-testsuite: Rename to binutils-testsuite Joshua Watt
2024-06-24 19:10     ` [OE-core][PATCH v4 06/10] classes-recipe/image_types: Add SPDX_IMAGE_PURPOSE to images Joshua Watt
2024-06-24 19:10     ` [OE-core][PATCH v4 07/10] selftest: spdx: Add SPDX 3.0 test cases Joshua Watt
2024-06-24 19:10     ` [OE-core][PATCH v4 08/10] classes-recipe: nospdx: Add class Joshua Watt
2024-06-24 19:10     ` [OE-core][PATCH v4 09/10] classes/spdx-common: Move SPDX_SUPPLIER Joshua Watt
2024-06-24 19:11     ` [OE-core][PATCH v4 10/10] Switch default spdx version to 3.0 Joshua Watt
2024-06-25 15:08     ` [OE-core][PATCH v4 00/10] Add SPDX 3.0 support Alexandre Belloni
2024-06-25 15:43       ` Richard Purdie
2024-07-03 13:59     ` [OE-core][PATCH v5 0/8] " Joshua Watt
2024-07-03 13:59       ` [OE-core][PATCH v5 1/8] classes-recipe/image: Add image file manifest Joshua Watt
2024-07-03 13:59       ` [OE-core][PATCH v5 2/8] classes-recipe/baremetal-image: " Joshua Watt
2024-07-11  9:56         ` Richard Purdie
2024-07-03 13:59       ` [OE-core][PATCH v5 3/8] classes/create-spdx-3.0: Add classes Joshua Watt
2024-07-03 13:59       ` [OE-core][PATCH v5 4/8] classes-global/staging: Exclude do_create_spdx from automatic sysroot extension Joshua Watt
2024-07-03 13:59       ` [OE-core][PATCH v5 5/8] classes-recipe/image_types: Add SPDX_IMAGE_PURPOSE to images Joshua Watt
2024-07-03 13:59       ` [OE-core][PATCH v5 6/8] selftest: spdx: Add SPDX 3.0 test cases Joshua Watt
2024-07-03 13:59       ` [OE-core][PATCH v5 7/8] classes-recipe: nospdx: Add class Joshua Watt
2024-07-03 13:59       ` [OE-core][PATCH v5 8/8] Switch default spdx version to 3.0 Joshua Watt
2024-07-05  7:17       ` [OE-core][PATCH v5 0/8] Add SPDX 3.0 support Richard Purdie
     [not found]       ` <17DF3FE80C22BC48.23364@lists.openembedded.org>
2024-07-10 12:18         ` Richard Purdie
2024-07-10 14:03           ` Joshua Watt
2024-07-10 14:22             ` Richard Purdie
2024-07-12 15:58       ` [OE-core][PATCH v6 00/12] " Joshua Watt
2024-07-12 15:58         ` [OE-core][PATCH v6 01/12] classes-recipe/image: Add image file manifest Joshua Watt
2024-07-12 15:58         ` [OE-core][PATCH v6 02/12] classes-recipe/baremetal-image: " Joshua Watt
2024-07-12 15:58         ` [OE-core][PATCH v6 03/12] classes/create-spdx-3.0: Add classes Joshua Watt
2024-07-12 15:58         ` [OE-core][PATCH v6 04/12] classes-global/staging: Exclude do_create_spdx from automatic sysroot extension Joshua Watt
2024-07-12 15:58         ` [OE-core][PATCH v6 05/12] classes-recipe/image_types: Add SPDX_IMAGE_PURPOSE to images Joshua Watt
2024-07-12 15:58         ` [OE-core][PATCH v6 06/12] selftest: spdx: Add SPDX 3.0 test cases Joshua Watt
2024-07-12 15:58         ` [OE-core][PATCH v6 07/12] classes-recipe: nospdx: Add class Joshua Watt
2024-07-12 15:58         ` [OE-core][PATCH v6 08/12] selftest: sstatetests: Exclude all SPDX tasks Joshua Watt
2024-07-12 15:58         ` [OE-core][PATCH v6 09/12] classes/spdx-common: Move to library Joshua Watt
2024-07-12 15:58         ` [OE-core][PATCH v6 10/12] classes/create-spdx-3.0: Move tasks " Joshua Watt
2024-07-12 15:58         ` [OE-core][PATCH v6 11/12] classes/create-spdx-2.2: Handle empty packages Joshua Watt
2024-07-12 15:58         ` [OE-core][PATCH v6 12/12] Switch default spdx version to 3.0 Joshua Watt
2024-07-13  6:44         ` [OE-core][PATCH v6 00/12] Add SPDX 3.0 support Richard Purdie
2024-07-15 20:40           ` Joshua Watt
2024-07-15 21:07             ` Richard Purdie
2024-07-15 21:26               ` Joshua Watt
2024-07-15 23:00                 ` Richard Purdie
     [not found]                 ` <17E2852F1C219F3B.14505@lists.openembedded.org>
2024-07-16 13:18                   ` Richard Purdie
2024-07-16 13:46                     ` Joshua Watt
     [not found]                   ` <17E2B3F7B69CE314.18588@lists.openembedded.org>
2024-07-16 14:14                     ` Richard Purdie

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240610214456.2757397-2-JPEWhacker@gmail.com \
    --to=jpewhacker@gmail.com \
    --cc=openembedded-core@lists.openembedded.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.