From: Joshua Watt <jpewhacker@gmail.com>
To: openembedded-core@lists.openembedded.org
Cc: Joshua Watt <JPEWhacker@gmail.com>
Subject: [OE-core][PATCH 1/6] classes-recipe/image: Add image file manifest
Date: Mon, 10 Jun 2024 15:41:47 -0600 [thread overview]
Message-ID: <20240610214456.2757397-2-JPEWhacker@gmail.com> (raw)
In-Reply-To: <20240610214456.2757397-1-JPEWhacker@gmail.com>
Downstream tasks may want to know what image files were written by the
do_image family of tasks (e.g. SPDX) so have each task write out a
manifest file that describes the files it produced, then aggregate them
in do_image_complete
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
---
meta/classes-recipe/image.bbclass | 61 +++++++++++++++++++++++++++++++
1 file changed, 61 insertions(+)
diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass
index 28be6c63623..122af2b0328 100644
--- a/meta/classes-recipe/image.bbclass
+++ b/meta/classes-recipe/image.bbclass
@@ -88,6 +88,11 @@ PACKAGE_INSTALL_ATTEMPTONLY ?= "${FEATURE_INSTALL_OPTIONAL}"
IMGDEPLOYDIR = "${WORKDIR}/deploy-${PN}-image-complete"
+IMGMANIFESTDIR = "${WORKDIR}/image-task-manifest"
+
+IMAGE_FILE_MANIFEST_DIR = "${WORKDIR}/deploy-image-file-manifest"
+IMAGE_FILE_MANIFEST = "${IMAGE_FILE_MANIFEST_DIR}/manifest.json"
+
# Images are generally built explicitly, do not need to be part of world.
EXCLUDE_FROM_WORLD = "1"
@@ -277,14 +282,31 @@ fakeroot python do_image () {
execute_pre_post_process(d, pre_process_cmds)
}
do_image[dirs] = "${TOPDIR}"
+do_image[cleandirs] = "${IMGMANIFESTDIR}"
addtask do_image after do_rootfs
fakeroot python do_image_complete () {
from oe.utils import execute_pre_post_process
+ from pathlib import Path
+ import json
post_process_cmds = d.getVar("IMAGE_POSTPROCESS_COMMAND")
execute_pre_post_process(d, post_process_cmds)
+
+ image_manifest_dir = Path(d.getVar('IMGMANIFESTDIR'))
+
+ data = []
+
+ for taskname in bb.build.preceedtask("do_image_complete", False, d):
+ manifest_path = image_manifest_dir / (taskname + ".json")
+ if not manifest_path.exists():
+ continue
+ with manifest_path.open("r") as f:
+ data.extend(json.load(f))
+
+ with open(d.getVar("IMAGE_FILE_MANIFEST"), "w") as f:
+ json.dump(data, f)
}
do_image_complete[dirs] = "${TOPDIR}"
SSTATETASKS += "do_image_complete"
@@ -292,6 +314,8 @@ SSTATE_SKIP_CREATION:task-image-complete = '1'
do_image_complete[sstate-inputdirs] = "${IMGDEPLOYDIR}"
do_image_complete[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"
do_image_complete[stamp-extra-info] = "${MACHINE_ARCH}"
+do_image_complete[sstate-plaindirs] = "${IMAGE_FILE_MANIFEST_DIR}"
+do_image_complete[dirs] = "${IMAGE_FILE_MANIFEST_DIR}"
addtask do_image_complete after do_image before do_build
python do_image_complete_setscene () {
sstate_setscene(d)
@@ -513,6 +537,8 @@ python () {
d.appendVarFlag(task, 'subimages', ' ' + ' '.join(subimages))
d.appendVarFlag(task, 'vardeps', ' ' + ' '.join(vardeps))
d.appendVarFlag(task, 'vardepsexclude', ' DATETIME DATE ' + ' '.join(vardepsexclude))
+ d.appendVarFlag(task, 'postfuncs', ' write_image_file_manifest')
+ d.appendVarFlag(task, 'dirs', ' ' + d.getVar('IMGMANIFESTDIR'))
bb.debug(2, "Adding task %s before %s, after %s" % (task, 'do_image_complete', after))
bb.build.addtask(task, 'do_image_complete', after, d)
@@ -610,6 +636,41 @@ python create_symlinks() {
bb.note("Skipping symlink, source does not exist: %s -> %s" % (dst, src))
}
+python write_image_file_manifest() {
+ import json
+ from pathlib import Path
+
+ taskname = d.getVar("BB_CURRENTTASK")
+ image_deploy_dir = Path(d.getVar('IMGDEPLOYDIR'))
+ image_manifest_dir = Path(d.getVar('IMGMANIFESTDIR'))
+ manifest_path = image_manifest_dir / ("do_" + d.getVar("BB_CURRENTTASK") + ".json")
+
+ image_name = d.getVar("IMAGE_NAME")
+ image_basename = d.getVar("IMAGE_BASENAME")
+ machine = d.getVar("MACHINE")
+
+ subimages = (d.getVarFlag("do_" + taskname, 'subimages', False) or "").split()
+ imagetype = d.getVarFlag("do_" + taskname, 'imagetype', False)
+
+ data = {
+ "taskname": taskname,
+ "imagetype": imagetype,
+ "images": []
+ }
+
+ for type in subimages:
+ image_filename = image_name + "." + type
+ image_path = image_deploy_dir / image_filename
+ if not image_path.exists():
+ continue
+ data["images"].append({
+ "filename": image_filename,
+ })
+
+ with manifest_path.open("w") as f:
+ json.dump([data], f)
+}
+
MULTILIBRE_ALLOW_REP += "${base_bindir} ${base_sbindir} ${bindir} ${sbindir} ${libexecdir} ${sysconfdir} ${nonarch_base_libdir}/udev /lib/modules/[^/]*/modules.*"
MULTILIB_CHECK_FILE = "${WORKDIR}/multilib_check.py"
MULTILIB_TEMP_ROOTFS = "${WORKDIR}/multilib"
--
2.43.2
next prev parent reply other threads:[~2024-06-10 21:45 UTC|newest]
Thread overview: 93+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-06-10 21:41 [OE-core][PATCH 0/6] Add SPDX 3.0 support Joshua Watt
2024-06-10 21:41 ` Joshua Watt [this message]
2024-06-11 8:50 ` [OE-core][PATCH 1/6] classes-recipe/image: Add image file manifest Martin Hundebøll
2024-06-10 21:41 ` [OE-core][PATCH 2/6] classes/spdx-common: Move common SPDX to new class Joshua Watt
2024-07-17 16:44 ` Adrian Freihofer
2024-06-10 21:41 ` [OE-core][PATCH 3/6] classes/spdx-common: Add SPDX version to path Joshua Watt
2024-06-10 21:41 ` [OE-core][PATCH 4/6] classes/spdx-common: Return empty list from extract_licenses Joshua Watt
2024-06-10 21:41 ` [OE-core][PATCH 5/6] classes/create-spdx-3.0: Add class Joshua Watt
2024-06-10 22:11 ` Patchtest results for " patchtest
2024-06-18 14:48 ` Marta Rybczynska
2024-06-18 15:22 ` Joshua Watt
2024-06-10 21:41 ` [OE-core][PATCH 6/6] classes-recipe/image_types: Add SPDX_IMAGE_PURPOSE to images Joshua Watt
2024-06-11 8:50 ` [OE-core][PATCH 0/6] Add SPDX 3.0 support Richard Purdie
2024-06-11 10:40 ` Richard Purdie
2024-06-11 14:42 ` Joshua Watt
2024-06-19 22:13 ` [OE-core][PATCH v2 0/7] " Joshua Watt
2024-06-19 22:13 ` [OE-core][PATCH v2 1/7] classes-recipe/image: Add image file manifest Joshua Watt
2024-06-19 22:13 ` [OE-core][PATCH v2 2/7] classes/create-spdx-3.0: Add classs Joshua Watt
2024-06-19 22:45 ` Patchtest results for " patchtest
2024-06-19 22:13 ` [OE-core][PATCH v2 3/7] classes-recipe/image_types: Add SPDX_IMAGE_PURPOSE to images Joshua Watt
2024-06-19 22:13 ` [OE-core][PATCH v2 4/7] selftest: spdx: Add SPDX 3.0 test cases Joshua Watt
2024-06-19 22:13 ` [OE-core][PATCH v2 5/7] classes-recipe: nospdx: Add class Joshua Watt
2024-06-19 22:13 ` [OE-core][PATCH v2 6/7] classes/spdx-common: Move SPDX_SUPPLIER Joshua Watt
2024-06-19 22:13 ` [OE-core][PATCH v2 7/7] Switch default spdx version to 3.0 Joshua Watt
2024-06-19 22:45 ` Patchtest results for " patchtest
2024-06-21 4:15 ` [OE-core][PATCH v2 0/7] Add SPDX 3.0 support Khem Raj
2024-06-21 6:24 ` Alexandre Belloni
2024-06-21 14:24 ` Joshua Watt
2024-06-21 17:21 ` Joshua Watt
2024-06-24 15:20 ` [OE-core][PATCH v3 00/10] " Joshua Watt
2024-06-24 15:20 ` [OE-core][PATCH v3 01/10] classes-recipe/image: Add image file manifest Joshua Watt
2024-06-24 15:20 ` [OE-core][PATCH v3 02/10] classes-recipe/baremetal-image: " Joshua Watt
2024-06-24 15:20 ` [OE-core][PATCH v3 03/10] classes/create-spdx-3.0: Add classes Joshua Watt
2024-06-24 15:20 ` [OE-core][PATCH v3 04/10] classes-global/staging: Exclude do_create_spdx from automatic sysroot extension Joshua Watt
2024-06-24 15:20 ` [OE-core][PATCH v3 05/10] binutils-cross-testsuite: Rename to binutils-testsuite Joshua Watt
2024-06-24 15:20 ` [OE-core][PATCH v3 06/10] classes-recipe/image_types: Add SPDX_IMAGE_PURPOSE to images Joshua Watt
2024-06-24 15:20 ` [OE-core][PATCH v3 07/10] selftest: spdx: Add SPDX 3.0 test cases Joshua Watt
2024-06-24 15:20 ` [OE-core][PATCH v3 08/10] classes-recipe: nospdx: Add class Joshua Watt
2024-06-24 15:20 ` [OE-core][PATCH v3 09/10] classes/spdx-common: Move SPDX_SUPPLIER Joshua Watt
2024-06-24 15:20 ` [OE-core][PATCH v3 10/10] Switch default spdx version to 3.0 Joshua Watt
2024-06-24 19:10 ` [OE-core][PATCH v4 00/10] Add SPDX 3.0 support Joshua Watt
2024-06-24 19:10 ` [OE-core][PATCH v4 01/10] classes-recipe/image: Add image file manifest Joshua Watt
2024-06-24 19:10 ` [OE-core][PATCH v4 02/10] classes-recipe/baremetal-image: " Joshua Watt
2024-06-25 10:24 ` Ernst Persson
2024-06-24 19:10 ` [OE-core][PATCH v4 03/10] classes/create-spdx-3.0: Add classes Joshua Watt
2024-06-25 14:44 ` Richard Purdie
2024-06-25 18:40 ` Mark Hatle
2024-06-27 16:33 ` Joshua Watt
2024-06-27 16:47 ` Joshua Watt
2024-06-24 19:10 ` [OE-core][PATCH v4 04/10] classes-global/staging: Exclude do_create_spdx from automatic sysroot extension Joshua Watt
2024-06-24 19:10 ` [OE-core][PATCH v4 05/10] binutils-cross-testsuite: Rename to binutils-testsuite Joshua Watt
2024-06-24 19:10 ` [OE-core][PATCH v4 06/10] classes-recipe/image_types: Add SPDX_IMAGE_PURPOSE to images Joshua Watt
2024-06-24 19:10 ` [OE-core][PATCH v4 07/10] selftest: spdx: Add SPDX 3.0 test cases Joshua Watt
2024-06-24 19:10 ` [OE-core][PATCH v4 08/10] classes-recipe: nospdx: Add class Joshua Watt
2024-06-24 19:10 ` [OE-core][PATCH v4 09/10] classes/spdx-common: Move SPDX_SUPPLIER Joshua Watt
2024-06-24 19:11 ` [OE-core][PATCH v4 10/10] Switch default spdx version to 3.0 Joshua Watt
2024-06-25 15:08 ` [OE-core][PATCH v4 00/10] Add SPDX 3.0 support Alexandre Belloni
2024-06-25 15:43 ` Richard Purdie
2024-07-03 13:59 ` [OE-core][PATCH v5 0/8] " Joshua Watt
2024-07-03 13:59 ` [OE-core][PATCH v5 1/8] classes-recipe/image: Add image file manifest Joshua Watt
2024-07-03 13:59 ` [OE-core][PATCH v5 2/8] classes-recipe/baremetal-image: " Joshua Watt
2024-07-11 9:56 ` Richard Purdie
2024-07-03 13:59 ` [OE-core][PATCH v5 3/8] classes/create-spdx-3.0: Add classes Joshua Watt
2024-07-03 13:59 ` [OE-core][PATCH v5 4/8] classes-global/staging: Exclude do_create_spdx from automatic sysroot extension Joshua Watt
2024-07-03 13:59 ` [OE-core][PATCH v5 5/8] classes-recipe/image_types: Add SPDX_IMAGE_PURPOSE to images Joshua Watt
2024-07-03 13:59 ` [OE-core][PATCH v5 6/8] selftest: spdx: Add SPDX 3.0 test cases Joshua Watt
2024-07-03 13:59 ` [OE-core][PATCH v5 7/8] classes-recipe: nospdx: Add class Joshua Watt
2024-07-03 13:59 ` [OE-core][PATCH v5 8/8] Switch default spdx version to 3.0 Joshua Watt
2024-07-05 7:17 ` [OE-core][PATCH v5 0/8] Add SPDX 3.0 support Richard Purdie
[not found] ` <17DF3FE80C22BC48.23364@lists.openembedded.org>
2024-07-10 12:18 ` Richard Purdie
2024-07-10 14:03 ` Joshua Watt
2024-07-10 14:22 ` Richard Purdie
2024-07-12 15:58 ` [OE-core][PATCH v6 00/12] " Joshua Watt
2024-07-12 15:58 ` [OE-core][PATCH v6 01/12] classes-recipe/image: Add image file manifest Joshua Watt
2024-07-12 15:58 ` [OE-core][PATCH v6 02/12] classes-recipe/baremetal-image: " Joshua Watt
2024-07-12 15:58 ` [OE-core][PATCH v6 03/12] classes/create-spdx-3.0: Add classes Joshua Watt
2024-07-12 15:58 ` [OE-core][PATCH v6 04/12] classes-global/staging: Exclude do_create_spdx from automatic sysroot extension Joshua Watt
2024-07-12 15:58 ` [OE-core][PATCH v6 05/12] classes-recipe/image_types: Add SPDX_IMAGE_PURPOSE to images Joshua Watt
2024-07-12 15:58 ` [OE-core][PATCH v6 06/12] selftest: spdx: Add SPDX 3.0 test cases Joshua Watt
2024-07-12 15:58 ` [OE-core][PATCH v6 07/12] classes-recipe: nospdx: Add class Joshua Watt
2024-07-12 15:58 ` [OE-core][PATCH v6 08/12] selftest: sstatetests: Exclude all SPDX tasks Joshua Watt
2024-07-12 15:58 ` [OE-core][PATCH v6 09/12] classes/spdx-common: Move to library Joshua Watt
2024-07-12 15:58 ` [OE-core][PATCH v6 10/12] classes/create-spdx-3.0: Move tasks " Joshua Watt
2024-07-12 15:58 ` [OE-core][PATCH v6 11/12] classes/create-spdx-2.2: Handle empty packages Joshua Watt
2024-07-12 15:58 ` [OE-core][PATCH v6 12/12] Switch default spdx version to 3.0 Joshua Watt
2024-07-13 6:44 ` [OE-core][PATCH v6 00/12] Add SPDX 3.0 support Richard Purdie
2024-07-15 20:40 ` Joshua Watt
2024-07-15 21:07 ` Richard Purdie
2024-07-15 21:26 ` Joshua Watt
2024-07-15 23:00 ` Richard Purdie
[not found] ` <17E2852F1C219F3B.14505@lists.openembedded.org>
2024-07-16 13:18 ` Richard Purdie
2024-07-16 13:46 ` Joshua Watt
[not found] ` <17E2B3F7B69CE314.18588@lists.openembedded.org>
2024-07-16 14:14 ` Richard Purdie
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240610214456.2757397-2-JPEWhacker@gmail.com \
--to=jpewhacker@gmail.com \
--cc=openembedded-core@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox