[PATCH 0/2] more flexible image conversion

Openembedded Core Discussions
 help / color / mirror / Atom feed

* [PATCH 0/2] more flexible image conversion
@ 2016-03-07 14:51 Patrick Ohly
  2016-03-07 14:51 ` [PATCH 1/2] image creation: allow overriding .rootfs suffix Patrick Ohly
  2016-03-07 14:51 ` [PATCH 2/2] image.bbclass: support chaining compression (aka conversion) commands Patrick Ohly
  0 siblings, 2 replies; 4+ messages in thread
From: Patrick Ohly @ 2016-03-07 14:51 UTC (permalink / raw)
  To: openembedded-core

In Ostro OS (https://ostroproject.org) we have a custom image class
which creates a whole-disk image
(https://github.com/ostroproject/meta-ostro/blob/master/meta-ostro/classes/image-dsk.bbclass).

Currently that class is more complex than it should be because it
re-implements image compression and conversion to VirtualBox, due to
limitations in OE-core's image.bbclass and because we wanted to get
rid of the redundant and misleading ".rootfs" suffix in image file
names.

These two patches make it possible to simplify the custom code to just
the actual dsk image and leverage OE-core for the rest.

The removal of the ".rootfs" suffix would also make sense for
OE-core. The first commit can be seen as a first step towards that. It
does not change the default, so nothing changes yet, but interested
distros or users can already get rid of the suffix by overriding the
IMAGE_NAME_SUFFIX variable.

https://bugzilla.yoctoproject.org/show_bug.cgi?id=9076 describes the
problem and probably can be considered resolved by these changes.

The following changes since commit 331ad5c5f80044a5e82abca0e87a85b162126411:

  mtools: Drop GCONV_PATH manipulation (2016-03-07 12:41:09 +0000)

are available in the git repository at:

  git://github.com/pohly/openembedded-core image-conversion
  https://github.com/pohly/openembedded-core/tree/image-conversion

Patrick Ohly (2):
  image creation: allow overriding .rootfs suffix
  image.bbclass: support chaining compression (aka conversion) commands

 meta/classes/image.bbclass       | 29 +++++++++++++++---
 meta/classes/image_types.bbclass | 63 ++++++++++++++++++++++------------------
 2 files changed, 59 insertions(+), 33 deletions(-)

-- 
2.1.4

^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 1/2] image creation: allow overriding .rootfs suffix
  2016-03-07 14:51 [PATCH 0/2] more flexible image conversion Patrick Ohly
@ 2016-03-07 14:51 ` Patrick Ohly
  2016-03-07 21:13   ` [PATCH] image.bbclass: fix incomplete .rootfs customization Patrick Ohly
  2016-03-07 14:51 ` [PATCH 2/2] image.bbclass: support chaining compression (aka conversion) commands Patrick Ohly
  1 sibling, 1 reply; 4+ messages in thread
From: Patrick Ohly @ 2016-03-07 14:51 UTC (permalink / raw)
  To: openembedded-core

By default, the image file name contains ".rootfs" to distinguish the
image file from other files created during image building. However,
for certain image types (for example, .hddimg) the ".rootfs" suffix is
redundant because the type suffix alone already uniquely identifies
the main image file (core-image-minimal-intel-corei7-64.hddimg instead
of core-image-minimal-intel-corei7-64.rootfs.hddimg).

With this change, distros that prefer the shorter image name can
override the .rootfs suffix like this:
   IMAGE_NAME_ROOTFS = "${@ '' if <using special image type> else '${IMAGE_NAME_ROOTFS_DEFAULT}'}"

The exact logic when to remove the extra suffix depends on the distro
and how it enables its own image type.

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
---
 meta/classes/image.bbclass       |  4 +--
 meta/classes/image_types.bbclass | 63 ++++++++++++++++++++++------------------
 2 files changed, 36 insertions(+), 31 deletions(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 02612ad..081a0b3 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -387,7 +387,7 @@ python () {
                     subimages.append(realt + "." + ctype)
 
         if realt not in alltypes:
-            cmds.append(localdata.expand("\trm ${IMAGE_NAME}.rootfs.${type}"))
+            cmds.append(localdata.expand("\trm ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}"))
         else:
             subimages.append(realt)
 
@@ -474,7 +474,7 @@ python create_symlinks() {
     manifest_name = d.getVar('IMAGE_MANIFEST', True)
     taskname = d.getVar("BB_CURRENTTASK", True)
     subimages = (d.getVarFlag("do_" + taskname, 'subimages', False) or "").split()
-    imgsuffix = d.getVarFlag("do_" + taskname, 'imgsuffix', True) or ".rootfs."
+    imgsuffix = d.getVarFlag("do_" + taskname, 'imgsuffix', True) or "${IMAGE_NAME_SUFFIX}."
     os.chdir(deploy_dir)
 
     if not link_name:
diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index db8cb37..22719ad 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -1,3 +1,8 @@
+# IMAGE_NAME is the base name for everything produced when building images.
+# The actual image that contains the rootfs has an additional suffix (.rootfs
+# by default) followed by additional suffices which describe the format (.ext4,
+# .ext4.xz, etc.).
+IMAGE_NAME_SUFFIX ??= ".rootfs"
 
 # The default aligment of the size of the rootfs is set to 1KiB. In case
 # you're using the SD card emulation of a QEMU system simulator you may
@@ -36,9 +41,9 @@ XZ_INTEGRITY_CHECK ?= "crc32"
 XZ_THREADS ?= "-T 0"
 
 JFFS2_SUM_EXTRA_ARGS ?= ""
-IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
+IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.jffs2 ${EXTRA_IMAGECMD}"
 
-IMAGE_CMD_cramfs = "mkfs.cramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
+IMAGE_CMD_cramfs = "mkfs.cramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cramfs ${EXTRA_IMAGECMD}"
 
 oe_mkext234fs () {
 	fstype=$1
@@ -58,8 +63,8 @@ oe_mkext234fs () {
 		eval COUNT=\"$MIN_COUNT\"
 	fi
 	# Create a sparse image block
-	dd if=/dev/zero of=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype seek=$ROOTFS_SIZE count=$COUNT bs=1024
-	mkfs.$fstype -F $extra_imagecmd ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype -d ${IMAGE_ROOTFS}
+	dd if=/dev/zero of=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.$fstype seek=$ROOTFS_SIZE count=$COUNT bs=1024
+	mkfs.$fstype -F $extra_imagecmd ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.$fstype -d ${IMAGE_ROOTFS}
 }
 
 IMAGE_CMD_ext2 = "oe_mkext234fs ext2 ${EXTRA_IMAGECMD}"
@@ -69,16 +74,16 @@ IMAGE_CMD_ext4 = "oe_mkext234fs ext4 ${EXTRA_IMAGECMD}"
 MIN_BTRFS_SIZE ?= "16384"
 IMAGE_CMD_btrfs () {
 	if [ ${ROOTFS_SIZE} -gt ${MIN_BTRFS_SIZE} ]; then
-		dd if=/dev/zero of=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.btrfs count=${ROOTFS_SIZE} bs=1024
-		mkfs.btrfs ${EXTRA_IMAGECMD} -r ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.btrfs
+		dd if=/dev/zero of=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.btrfs count=${ROOTFS_SIZE} bs=1024
+		mkfs.btrfs ${EXTRA_IMAGECMD} -r ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.btrfs
 	else
 		bbfatal "Rootfs is too small for BTRFS (Rootfs Actual Size: ${ROOTFS_SIZE}, BTRFS Minimum Size: ${MIN_BTRFS_SIZE})"
 	fi
 }
 
-IMAGE_CMD_squashfs = "mksquashfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.squashfs ${EXTRA_IMAGECMD} -noappend"
-IMAGE_CMD_squashfs-xz = "mksquashfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.squashfs-xz ${EXTRA_IMAGECMD} -noappend -comp xz"
-IMAGE_CMD_squashfs-lzo = "mksquashfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.squashfs-lzo ${EXTRA_IMAGECMD} -noappend -comp lzo"
+IMAGE_CMD_squashfs = "mksquashfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.squashfs ${EXTRA_IMAGECMD} -noappend"
+IMAGE_CMD_squashfs-xz = "mksquashfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.squashfs-xz ${EXTRA_IMAGECMD} -noappend -comp xz"
+IMAGE_CMD_squashfs-lzo = "mksquashfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.squashfs-lzo ${EXTRA_IMAGECMD} -noappend -comp lzo"
 
 # By default, tar from the host is used, which can be quite old. If
 # you need special parameters (like --xattrs) which are only supported
@@ -91,18 +96,18 @@ IMAGE_CMD_squashfs-lzo = "mksquashfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE
 # In practice, it turned out to be not needed when creating archives and
 # required when extracting, but it seems prudent to use it in both cases.
 IMAGE_CMD_TAR ?= "tar"
-IMAGE_CMD_tar = "${IMAGE_CMD_TAR} -cvf ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.tar -C ${IMAGE_ROOTFS} ."
+IMAGE_CMD_tar = "${IMAGE_CMD_TAR} -cvf ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.tar -C ${IMAGE_ROOTFS} ."
 
 do_image_cpio[cleandirs] += "${WORKDIR}/cpio_append"
 IMAGE_CMD_cpio () {
-	(cd ${IMAGE_ROOTFS} && find . | cpio -o -H newc >${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cpio)
+	(cd ${IMAGE_ROOTFS} && find . | cpio -o -H newc >${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cpio)
 	if [ ! -L ${IMAGE_ROOTFS}/init -a ! -e ${IMAGE_ROOTFS}/init ]; then
 		if [ -L ${IMAGE_ROOTFS}/sbin/init -o -e ${IMAGE_ROOTFS}/sbin/init ]; then
 			ln -sf /sbin/init ${WORKDIR}/cpio_append/init
 		else
 			touch ${WORKDIR}/cpio_append/init
 		fi
-		(cd  ${WORKDIR}/cpio_append && echo ./init | cpio -oA -H newc -F ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cpio)
+		(cd  ${WORKDIR}/cpio_append && echo ./init | cpio -oA -H newc -F ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cpio)
 	fi
 }
 
@@ -110,8 +115,8 @@ ELF_KERNEL ?= "${STAGING_DIR_HOST}/usr/src/kernel/${KERNEL_IMAGETYPE}"
 ELF_APPEND ?= "ramdisk_size=32768 root=/dev/ram0 rw console="
 
 IMAGE_CMD_elf () {
-	test -f ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.elf && rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.elf
-	mkelfImage --kernel=${ELF_KERNEL} --initrd=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cpio.gz --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.elf --append='${ELF_APPEND}' ${EXTRA_IMAGECMD}
+	test -f ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.elf && rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.elf
+	mkelfImage --kernel=${ELF_KERNEL} --initrd=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cpio.gz --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.elf --append='${ELF_APPEND}' ${EXTRA_IMAGECMD}
 }
 IMAGE_TYPEDEP_elf = "cpio.gz"
 
@@ -128,13 +133,13 @@ multiubi_mkfs() {
 
 	echo \[ubifs\] > ubinize${vname}-${IMAGE_NAME}.cfg
 	echo mode=ubi >> ubinize${vname}-${IMAGE_NAME}.cfg
-	echo image=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${vname}.rootfs.ubifs >> ubinize${vname}-${IMAGE_NAME}.cfg
+	echo image=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${vname}${IMAGE_NAME_SUFFIX}.ubifs >> ubinize${vname}-${IMAGE_NAME}.cfg
 	echo vol_id=0 >> ubinize${vname}-${IMAGE_NAME}.cfg
 	echo vol_type=dynamic >> ubinize${vname}-${IMAGE_NAME}.cfg
 	echo vol_name=${UBI_VOLNAME} >> ubinize${vname}-${IMAGE_NAME}.cfg
 	echo vol_flags=autoresize >> ubinize${vname}-${IMAGE_NAME}.cfg
-	mkfs.ubifs -r ${IMAGE_ROOTFS} -o ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${vname}.rootfs.ubifs ${mkubifs_args}
-	ubinize -o ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${vname}.rootfs.ubi ${ubinize_args} ubinize${vname}-${IMAGE_NAME}.cfg
+	mkfs.ubifs -r ${IMAGE_ROOTFS} -o ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${vname}${IMAGE_NAME_SUFFIX}.ubifs ${mkubifs_args}
+	ubinize -o ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${vname}${IMAGE_NAME_SUFFIX}.ubi ${ubinize_args} ubinize${vname}-${IMAGE_NAME}.cfg
 
 	# Cleanup cfg file
 	mv ubinize${vname}-${IMAGE_NAME}.cfg ${DEPLOY_DIR_IMAGE}/
@@ -142,12 +147,12 @@ multiubi_mkfs() {
 	# Create own symlinks for 'named' volumes
 	if [ -n "$vname" ]; then
 		cd ${DEPLOY_DIR_IMAGE}
-		if [ -e ${IMAGE_NAME}${vname}.rootfs.ubifs ]; then
-			ln -sf ${IMAGE_NAME}${vname}.rootfs.ubifs \
+		if [ -e ${IMAGE_NAME}${vname}${IMAGE_NAME_SUFFIX}.ubifs ]; then
+			ln -sf ${IMAGE_NAME}${vname}${IMAGE_NAME_SUFFIX}.ubifs \
 			${IMAGE_LINK_NAME}${vname}.ubifs
 		fi
-		if [ -e ${IMAGE_NAME}${vname}.rootfs.ubi ]; then
-			ln -sf ${IMAGE_NAME}${vname}.rootfs.ubi \
+		if [ -e ${IMAGE_NAME}${vname}${IMAGE_NAME_SUFFIX}.ubi ]; then
+			ln -sf ${IMAGE_NAME}${vname}${IMAGE_NAME_SUFFIX}.ubi \
 			${IMAGE_LINK_NAME}${vname}.ubi
 		fi
 		cd -
@@ -168,7 +173,7 @@ IMAGE_CMD_ubi () {
 	multiubi_mkfs "${MKUBIFS_ARGS}" "${UBINIZE_ARGS}"
 }
 
-IMAGE_CMD_ubifs = "mkfs.ubifs -r ${IMAGE_ROOTFS} -o ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ubifs ${MKUBIFS_ARGS}"
+IMAGE_CMD_ubifs = "mkfs.ubifs -r ${IMAGE_ROOTFS} -o ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.ubifs ${MKUBIFS_ARGS}"
 
 WKS_FILE ?= "${IMAGE_BASENAME}.${MACHINE}.wks"
 WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
@@ -193,7 +198,7 @@ IMAGE_CMD_wic () {
 	fi
 
 	BUILDDIR="${TOPDIR}" wic create "$wks" --vars "${STAGING_DIR_TARGET}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/"
-	mv "$out/build/$(basename "${wks%.wks}")"*.direct "$out.rootfs.wic"
+	mv "$out/build/$(basename "${wks%.wks}")"*.direct "$out${IMAGE_NAME_SUFFIX}.wic"
 	rm -rf "$out/"
 }
 IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
@@ -255,12 +260,12 @@ IMAGE_TYPES = " \
 "
 
 COMPRESSIONTYPES = "gz bz2 lzma xz lz4 sum"
-COMPRESS_CMD_lzma = "lzma -k -f -7 ${IMAGE_NAME}.rootfs.${type}"
-COMPRESS_CMD_gz = "gzip -f -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.gz"
-COMPRESS_CMD_bz2 = "pbzip2 -f -k ${IMAGE_NAME}.rootfs.${type}"
-COMPRESS_CMD_xz = "xz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.xz"
-COMPRESS_CMD_lz4 = "lz4c -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.lz4"
-COMPRESS_CMD_sum = "sumtool -i ${IMAGE_NAME}.rootfs.${type} -o ${IMAGE_NAME}.rootfs.${type}.sum ${JFFS2_SUM_EXTRA_ARGS}"
+COMPRESS_CMD_lzma = "lzma -k -f -7 ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}"
+COMPRESS_CMD_gz = "gzip -f -9 -c ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.gz"
+COMPRESS_CMD_bz2 = "pbzip2 -f -k ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}"
+COMPRESS_CMD_xz = "xz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.xz"
+COMPRESS_CMD_lz4 = "lz4c -9 -c ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.lz4"
+COMPRESS_CMD_sum = "sumtool -i ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} -o ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.sum ${JFFS2_SUM_EXTRA_ARGS}"
 COMPRESS_DEPENDS_lzma = "xz-native"
 COMPRESS_DEPENDS_gz = ""
 COMPRESS_DEPENDS_bz2 = "pbzip2-native"
-- 
2.1.4



^ permalink raw reply related	[flat|nested] 4+ messages in thread

* [PATCH] image.bbclass: fix incomplete .rootfs customization
  2016-03-07 14:51 ` [PATCH 1/2] image creation: allow overriding .rootfs suffix Patrick Ohly
@ 2016-03-07 21:13   ` Patrick Ohly
  0 siblings, 0 replies; 4+ messages in thread
From: Patrick Ohly @ 2016-03-07 21:13 UTC (permalink / raw)
  To: openembedded-core

The patch for making the .rootfs configurable was incomplete: in the
python create_symlinks() method the new variable must be expanded
explicitly.

Not doing so broke the symlink creation and that led to hard build
failures in image types depending on the boot-directdisk.bbclass (like
qcow2) because the build_boot_dd() method relied on the symlink.

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
---
 meta/classes/image.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 081a0b3..8b6c30b 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -474,7 +474,7 @@ python create_symlinks() {
     manifest_name = d.getVar('IMAGE_MANIFEST', True)
     taskname = d.getVar("BB_CURRENTTASK", True)
     subimages = (d.getVarFlag("do_" + taskname, 'subimages', False) or "").split()
-    imgsuffix = d.getVarFlag("do_" + taskname, 'imgsuffix', True) or "${IMAGE_NAME_SUFFIX}."
+    imgsuffix = d.getVarFlag("do_" + taskname, 'imgsuffix', True) or d.expand("${IMAGE_NAME_SUFFIX}.")
     os.chdir(deploy_dir)
 
     if not link_name:
-- 
2.1.4



^ permalink raw reply related	[flat|nested] 4+ messages in thread

* [PATCH 2/2] image.bbclass: support chaining compression (aka conversion) commands
  2016-03-07 14:51 [PATCH 0/2] more flexible image conversion Patrick Ohly
  2016-03-07 14:51 ` [PATCH 1/2] image creation: allow overriding .rootfs suffix Patrick Ohly
@ 2016-03-07 14:51 ` Patrick Ohly
  1 sibling, 0 replies; 4+ messages in thread
From: Patrick Ohly @ 2016-03-07 14:51 UTC (permalink / raw)
  To: openembedded-core

It makes sense to use the compression mechanism also for conversion,
for example of a whole-disk image into .vdi (VirtualBox). That part
already works, like this:

   COMPRESSIONTYPES_append = " vdi"
   COMPRESS_CMD_vdi = "qemu-img convert -O vdi ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.vdi"
   IMAGE_DEPENDS_vdi = "qemu-native"

But then it also makes sense to allow compressing the resulting image,
which only works after enhancing the image.bbclass.

For example, suppose a custom image command produces "dsk" images. Then
it becomes possible to set
   IMAGE_FSTYPES = " dsk.xz dsk.vdi.xz"
and do_image_dsk will automatically produce the intermediate images,
convert to dsk.xz resp. dsk.vdi -> dsk.vdi.xz and delete all
intermediate images. Symlinks are also set correctly.

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
---
 meta/classes/image.bbclass | 27 ++++++++++++++++++++++++---
 1 file changed, 24 insertions(+), 3 deletions(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 081a0b3..01b49d4 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -303,6 +303,10 @@ python () {
                 basetype = type[:-len("." + ctype)]
                 break
 
+        if basetype != type:
+            # New base type itself might be generated by a conversion command.
+            basetype = _image_base_type(basetype)
+
         return basetype
 
     basetypes = {}
@@ -379,18 +383,35 @@ python () {
             bb.fatal("No IMAGE_CMD defined for IMAGE_FSTYPES entry '%s' - possibly invalid type name or missing support class" % t)
         cmds.append(localdata.expand("\tcd ${DEPLOY_DIR_IMAGE}"))
 
-        for bt in basetypes[t]:
+        rm_tmp_images = set()
+        def gen_conversion_cmds(bt):
             for ctype in ctypes:
                 if bt.endswith("." + ctype):
+                    type = bt[0:-len(ctype) - 1]
+                    # Create input image first.
+                    gen_conversion_cmds(type)
+                    localdata.setVar('type', type)
                     cmds.append("\t" + localdata.getVar("COMPRESS_CMD_" + ctype, True))
                     vardeps.add('COMPRESS_CMD_' + ctype)
-                    subimages.append(realt + "." + ctype)
+                    subimages.append(type + "." + ctype)
+                    if type not in alltypes:
+                        rm_tmp_images.add(localdata.expand("${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}"))
+
+        for bt in basetypes[t]:
+            gen_conversion_cmds(bt)
 
+        localdata.setVar('type', realt)
         if realt not in alltypes:
-            cmds.append(localdata.expand("\trm ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}"))
+            rm_tmp_images.add(localdata.expand("${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}"))
         else:
             subimages.append(realt)
 
+        # Clean up after applying all conversion commands. Some of them might
+        # use the same input, therefore we cannot delete sooner without applying
+        # some complex dependency analysis.
+        for image in rm_tmp_images:
+            cmds.append("\trm " + image)
+
         after = 'do_image'
         for dep in typedeps[t]:
             after += ' do_image_%s' % dep.replace("-", "_").replace(".", "_")
-- 
2.1.4



^ permalink raw reply related	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2016-03-07 21:13 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-03-07 14:51 [PATCH 0/2] more flexible image conversion Patrick Ohly
2016-03-07 14:51 ` [PATCH 1/2] image creation: allow overriding .rootfs suffix Patrick Ohly
2016-03-07 21:13   ` [PATCH] image.bbclass: fix incomplete .rootfs customization Patrick Ohly
2016-03-07 14:51 ` [PATCH 2/2] image.bbclass: support chaining compression (aka conversion) commands Patrick Ohly

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox