* [PATCH 1/2] toaster.bbclass: remove directory scan logic for detecting artifacts
2016-07-12 22:54 [PATCH 0/2] toaster: Better SDK artifacts and fstypes V2 bavery
@ 2016-07-12 22:54 ` bavery
2016-07-12 22:54 ` [PATCH 2/2] toaster.bbclass: only scan files-in-image.txt if it exists bavery
2016-07-19 8:04 ` [PATCH 0/2] toaster: Better SDK artifacts and fstypes V2 Richard Purdie
2 siblings, 0 replies; 4+ messages in thread
From: bavery @ 2016-07-12 22:54 UTC (permalink / raw)
To: openembedded-core
From: Elliot Smith <elliot.smith@intel.com>
toaster.bbclass does a scan of the image deploy and SDK directories
when a build finishes. However, this brings no benefit and could be
better managed and made easier to modify if moved to toasterui and
carried out when the BuildCompleted event occurs.
Remove the image scan code from toaster.bbclass, prior to moving it
to toasterui and buildinfohelper.
Also remove the license manifest update code, as this can also be
done from toasterui.
The postfuncs for do_populate_sdk are retained, but no longer
do the directory scan for SDK artifacts. Instead, they fire
an event with the value of the TOOLCHAIN_OUTPUTNAME variable,
as this is only accessible at the point when the do_populate_sdk
and do_populate_sdk_ext tasks are run. The value of this can then
be used by buildinfohelper to find the SDK artifacts produced by a
target.
[YOCTO #9002]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
---
classes/toaster.bbclass | 74 ++++++-------------------------------------------
1 file changed, 8 insertions(+), 66 deletions(-)
diff --git a/classes/toaster.bbclass b/classes/toaster.bbclass
index 1878fe0..90ea563 100644
--- a/classes/toaster.bbclass
+++ b/classes/toaster.bbclass
@@ -136,60 +136,16 @@ python toaster_package_dumpdata() {
# 2. Dump output image files information
-python toaster_image_dumpdata() {
- """
- Image filename for output images is not standardized.
- image_types.bbclass will spell out IMAGE_CMD_xxx variables that actually
- have hardcoded ways to create image file names in them.
- So we look for files starting with the set name.
-
- We also look for other files in the images/ directory which don't
- match IMAGE_NAME, such as the kernel bzImage, modules tarball etc.
- """
-
- dir_to_walk = d.getVar('DEPLOY_DIR_IMAGE', True);
- image_name = d.getVar('IMAGE_NAME', True);
- image_info_data = {}
- artifact_info_data = {}
-
- # collect all images and artifacts in the images directory
- for dirpath, dirnames, filenames in os.walk(dir_to_walk):
- for filename in filenames:
- full_path = os.path.join(dirpath, filename)
- try:
- if filename.startswith(image_name):
- # image
- image_info_data[full_path] = os.stat(full_path).st_size
- else:
- # other non-image artifact
- if not os.path.islink(full_path):
- artifact_info_data[full_path] = os.stat(full_path).st_size
- except OSError as e:
- bb.event.fire(bb.event.MetadataEvent("OSErrorException", e), d)
-
- bb.event.fire(bb.event.MetadataEvent("ImageFileSize", image_info_data), d)
- bb.event.fire(bb.event.MetadataEvent("ArtifactFileSize", artifact_info_data), d)
-}
-
python toaster_artifact_dumpdata() {
"""
- Dump data about artifacts in the SDK_DEPLOY directory
+ Dump data about SDK variables
"""
- dir_to_walk = d.getVar("SDK_DEPLOY", True)
- artifact_info_data = {}
-
- # collect all artifacts in the sdk directory
- for dirpath, dirnames, filenames in os.walk(dir_to_walk):
- for filename in filenames:
- full_path = os.path.join(dirpath, filename)
- try:
- if not os.path.islink(full_path):
- artifact_info_data[full_path] = os.stat(full_path).st_size
- except OSError as e:
- bb.event.fire(bb.event.MetadataEvent("OSErrorException", e), d)
+ event_data = {
+ "TOOLCHAIN_OUTPUTNAME": d.getVar("TOOLCHAIN_OUTPUTNAME", True)
+ }
- bb.event.fire(bb.event.MetadataEvent("ArtifactFileSize", artifact_info_data), d)
+ bb.event.fire(bb.event.MetadataEvent("SDKArtifactInfo", event_data), d)
}
# collect list of buildstats files based on fired events; when the build completes, collect all stats and fire an event with collected data
@@ -361,17 +317,6 @@ python toaster_buildhistory_dump() {
}
-# dump information related to license manifest path
-
-python toaster_licensemanifest_dump() {
- deploy_dir = d.getVar('DEPLOY_DIR', True);
- image_name = d.getVar('IMAGE_NAME', True);
-
- data = { 'deploy_dir' : deploy_dir, 'image_name' : image_name }
-
- bb.event.fire(bb.event.MetadataEvent("LicenseManifestPath", data), d)
-}
-
# set event handlers
addhandler toaster_layerinfo_dumpdata
toaster_layerinfo_dumpdata[eventmask] = "bb.event.TreeDataPreparationCompleted"
@@ -388,11 +333,8 @@ do_packagedata_setscene[vardepsexclude] += "toaster_package_dumpdata "
do_package[postfuncs] += "toaster_package_dumpdata "
do_package[vardepsexclude] += "toaster_package_dumpdata "
-do_image_complete[postfuncs] += "toaster_image_dumpdata "
-do_image_complete[vardepsexclude] += "toaster_image_dumpdata "
-
-do_rootfs[postfuncs] += "toaster_licensemanifest_dump "
-do_rootfs[vardepsexclude] += "toaster_licensemanifest_dump "
-
do_populate_sdk[postfuncs] += "toaster_artifact_dumpdata "
do_populate_sdk[vardepsexclude] += "toaster_artifact_dumpdata "
+
+do_populate_sdk_ext[postfuncs] += "toaster_artifact_dumpdata "
+do_populate_sdk_ext[vardepsexclude] += "toaster_artifact_dumpdata "
\ No newline at end of file
--
1.9.1
^ permalink raw reply related [flat|nested] 4+ messages in thread* [PATCH 2/2] toaster.bbclass: only scan files-in-image.txt if it exists
2016-07-12 22:54 [PATCH 0/2] toaster: Better SDK artifacts and fstypes V2 bavery
2016-07-12 22:54 ` [PATCH 1/2] toaster.bbclass: remove directory scan logic for detecting artifacts bavery
@ 2016-07-12 22:54 ` bavery
2016-07-19 8:04 ` [PATCH 0/2] toaster: Better SDK artifacts and fstypes V2 Richard Purdie
2 siblings, 0 replies; 4+ messages in thread
From: bavery @ 2016-07-12 22:54 UTC (permalink / raw)
To: openembedded-core
From: Elliot Smith <elliot.smith@intel.com>
We can reach the method in toaster.bbclass which tries to read from
the files-in-image.txt file via a build which doesn't create that
file (e.g. "bitbake core-image-minimal -c rootfs"). This causes
the build to fail with an exception.
Check that this file exists before trying to read from it.
[YOCTO #9784]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
---
classes/toaster.bbclass | 25 ++++++++++++++++---------
1 file changed, 16 insertions(+), 9 deletions(-)
diff --git a/classes/toaster.bbclass b/classes/toaster.bbclass
index 90ea563..972efb9 100644
--- a/classes/toaster.bbclass
+++ b/classes/toaster.bbclass
@@ -288,15 +288,22 @@ python toaster_buildhistory_dump() {
images[target][dependsname] = {'size': 0, 'depends' : []}
images[target][pname]['depends'].append((dependsname, deptype))
- with open("%s/files-in-image.txt" % installed_img_path, "r") as fin:
- for line in fin:
- lc = [ x for x in line.strip().split(" ") if len(x) > 0 ]
- if lc[0].startswith("l"):
- files[target]['syms'].append(lc)
- elif lc[0].startswith("d"):
- files[target]['dirs'].append(lc)
- else:
- files[target]['files'].append(lc)
+ # files-in-image.txt is only generated if an image file is created,
+ # so the file entries ('syms', 'dirs', 'files') for a target will be
+ # empty for rootfs builds and other "image" tasks which don't
+ # produce image files
+ # (e.g. "bitbake core-image-minimal -c populate_sdk")
+ files_in_image_path = "%s/files-in-image.txt" % installed_img_path
+ if os.path.exists(files_in_image_path):
+ with open(files_in_image_path, "r") as fin:
+ for line in fin:
+ lc = [ x for x in line.strip().split(" ") if len(x) > 0 ]
+ if lc[0].startswith("l"):
+ files[target]['syms'].append(lc)
+ elif lc[0].startswith("d"):
+ files[target]['dirs'].append(lc)
+ else:
+ files[target]['files'].append(lc)
for pname in images[target]:
if not pname in allpkgs:
--
1.9.1
^ permalink raw reply related [flat|nested] 4+ messages in thread