* [PATCH 01/12] image.py: write bitbake variables to .env file
2015-08-30 17:46 [PATCH 00/12] V3: Build wic images with bitbake Ed Bartosh
@ 2015-08-30 17:46 ` Ed Bartosh
2015-08-30 17:46 ` [PATCH 02/12] oe-selftest: test generation of <image>.env Ed Bartosh
` (10 subsequent siblings)
11 siblings, 0 replies; 14+ messages in thread
From: Ed Bartosh @ 2015-08-30 17:46 UTC (permalink / raw)
To: openembedded-core
Write set of bitbake variables used by wic into
build/tmp/sysroots/<machine>/imagedata/<image>.env
List of variables is defined in WICVARS variable in
meta/classes/image_types.bbclass.
This is needed for wic to be able to get bitbake variables without
running 'bitbake -e'.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
meta/classes/image_types.bbclass | 4 ++++
meta/lib/oe/image.py | 18 ++++++++++++++++++
2 files changed, 22 insertions(+)
diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index 2fd4c37..acee227 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -236,3 +236,7 @@ IMAGE_EXTENSION_live = "hddimg iso"
# The IMAGE_TYPES_MASKED variable will be used to mask out from the IMAGE_FSTYPES,
# images that will not be built at do_rootfs time: vmdk, vdi, qcow2, hddimg, iso, etc.
IMAGE_TYPES_MASKED ?= ""
+
+# The WICVARS variable is used to define list of bitbake variables used in wic code
+# variables from this list is written to <image>.env file
+WICVARS ?= "BBLAYERS DEPLOY_DIR_IMAGE HDDDIR IMAGE_BASENAME IMAGE_BOOT_FILES IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD ISODIR MACHINE_ARCH ROOTFS_SIZE STAGING_DATADIR STAGING_DIR_NATIVE STAGING_LIBDIR TARGET_SYS"
diff --git a/meta/lib/oe/image.py b/meta/lib/oe/image.py
index a2f94a1..95c62dc 100644
--- a/meta/lib/oe/image.py
+++ b/meta/lib/oe/image.py
@@ -326,6 +326,22 @@ class Image(ImageDepGraph):
return image_cmd_groups
+ def _write_env(self):
+ """
+ Write environment variables used by wic
+ to tmp/sysroots/<machine>/imgdata/<image>.env
+ """
+ stdir = self.d.getVar('STAGING_DIR_TARGET', True)
+ outdir = os.path.join(stdir, 'imgdata')
+ if not os.path.exists(outdir):
+ os.makedirs(outdir)
+ basename = self.d.getVar('IMAGE_BASENAME', True)
+ with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
+ for var in self.d.getVar('WICVARS', True).split():
+ value = self.d.getVar(var, True)
+ if value:
+ envf.write('%s="%s"\n' % (var, value.strip()))
+
def create(self):
bb.note("###### Generate images #######")
pre_process_cmds = self.d.getVar("IMAGE_PREPROCESS_COMMAND", True)
@@ -337,6 +353,8 @@ class Image(ImageDepGraph):
image_cmd_groups = self._get_imagecmds()
+ self._write_env()
+
for image_cmds in image_cmd_groups:
# create the images in parallel
nproc = multiprocessing.cpu_count()
--
2.1.4
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH 02/12] oe-selftest: test generation of <image>.env
2015-08-30 17:46 [PATCH 00/12] V3: Build wic images with bitbake Ed Bartosh
2015-08-30 17:46 ` [PATCH 01/12] image.py: write bitbake variables to .env file Ed Bartosh
@ 2015-08-30 17:46 ` Ed Bartosh
2015-08-30 17:47 ` [PATCH 03/12] wic: add BitbakeVars class Ed Bartosh
` (9 subsequent siblings)
11 siblings, 0 replies; 14+ messages in thread
From: Ed Bartosh @ 2015-08-30 17:46 UTC (permalink / raw)
To: openembedded-core
Added test case to check if <image>.env file is generated
and contains bitbake variables used in wic code.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
meta/lib/oeqa/selftest/wic.py | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/meta/lib/oeqa/selftest/wic.py b/meta/lib/oeqa/selftest/wic.py
index 117bd9d..fe8a2d0 100644
--- a/meta/lib/oeqa/selftest/wic.py
+++ b/meta/lib/oeqa/selftest/wic.py
@@ -23,6 +23,7 @@
"""Test cases for wic."""
+import os
import sys
from glob import glob
@@ -168,3 +169,22 @@ class Wic(oeSelfTest):
self.assertEqual(1, len(glob(self.resultdir + "HYBRID_ISO_IMG-*.direct")))
self.assertEqual(1, len(glob(self.resultdir + "HYBRID_ISO_IMG-*.iso")))
+ def test19_image_env(self):
+ """Test generation of <image>.env files."""
+ image = 'core-image-minimal'
+ stdir = get_bb_var('STAGING_DIR_TARGET', image)
+ imgdatadir = os.path.join(stdir, 'imgdata')
+
+ basename = get_bb_var('IMAGE_BASENAME', image)
+ self.assertEqual(basename, image)
+ path = os.path.join(imgdatadir, basename) + '.env'
+ self.assertTrue(os.path.isfile(path))
+
+ wicvars = get_bb_var('WICVARS', image).split()
+ wicvars.remove('IMAGE_BOOT_FILES') # this variable is optional
+ with open(path) as envfile:
+ content = dict(line.split("=", 1) for line in envfile)
+ # test if variables used by wic present in the .env file
+ for var in wicvars:
+ self.assertTrue(var in content, "%s is not in .env file" % var)
+ self.assertTrue(content[var])
--
2.1.4
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH 03/12] wic: add BitbakeVars class
2015-08-30 17:46 [PATCH 00/12] V3: Build wic images with bitbake Ed Bartosh
2015-08-30 17:46 ` [PATCH 01/12] image.py: write bitbake variables to .env file Ed Bartosh
2015-08-30 17:46 ` [PATCH 02/12] oe-selftest: test generation of <image>.env Ed Bartosh
@ 2015-08-30 17:47 ` Ed Bartosh
2015-08-30 17:47 ` [PATCH 04/12] wic: create new method _parse_line Ed Bartosh
` (8 subsequent siblings)
11 siblings, 0 replies; 14+ messages in thread
From: Ed Bartosh @ 2015-08-30 17:47 UTC (permalink / raw)
To: openembedded-core
Moved code of getting bitbake variables into separate class.
Created singleton object of this class in the module namespace.
Preserved existing API get_bitbake_var.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/utils/oe/misc.py | 91 +++++++++++++++++++++++-----------------
1 file changed, 53 insertions(+), 38 deletions(-)
diff --git a/scripts/lib/wic/utils/oe/misc.py b/scripts/lib/wic/utils/oe/misc.py
index 9c8f52d..3537a2e 100644
--- a/scripts/lib/wic/utils/oe/misc.py
+++ b/scripts/lib/wic/utils/oe/misc.py
@@ -124,48 +124,63 @@ def exec_native_cmd(cmd_and_args, native_sysroot, catch=3):
BOOTDD_EXTRA_SPACE = 16384
-_BITBAKE_VARS = defaultdict(dict)
+class BitbakeVars(defaultdict):
+ """
+ Container for Bitbake variables.
+ """
+ def __init__(self):
+ defaultdict.__init__(self, dict)
+
+ def get_var(self, var, image=None):
+ """
+ Get bitbake variable value lazy way, i.e. run
+ 'bitbake -e' only when variable is requested.
+ """
+ if image not in self:
+ # Get bitbake -e output
+ cmd = "bitbake -e"
+ if image:
+ cmd += " %s" % image
+
+ log_level = msger.get_loglevel()
+ msger.set_loglevel('normal')
+ ret, lines = _exec_cmd(cmd)
+ msger.set_loglevel(log_level)
+
+ if ret:
+ print "Couldn't get '%s' output." % cmd
+ print "Bitbake failed with error:\n%s\n" % lines
+ return
+
+ # Parse bitbake -e output
+ for line in lines.split('\n'):
+ if "=" not in line:
+ continue
+ try:
+ key, val = line.split("=")
+ except ValueError:
+ continue
+ key = key.strip()
+ val = val.strip()
+ if key.replace('_', '').isalnum():
+ self[image][key] = val.strip('"')
+
+ # Make first image a default set of variables
+ images = [key for key in self if key]
+ if len(images) == 1:
+ self[None] = self[image]
+
+ return self[image].get(var)
+
+# Create BB_VARS singleton
+BB_VARS = BitbakeVars()
def get_bitbake_var(var, image=None):
"""
- Get bitbake variable value lazy way, i.e. run
- 'bitbake -e' only when variable is requested.
+ Provide old get_bitbake_var API by wrapping
+ get_var method of BB_VARS singleton.
"""
- if image not in _BITBAKE_VARS:
- # Get bitbake -e output
- cmd = "bitbake -e"
- if image:
- cmd += " %s" % image
-
- log_level = msger.get_loglevel()
- msger.set_loglevel('normal')
- ret, lines = _exec_cmd(cmd)
- msger.set_loglevel(log_level)
-
- if ret:
- print "Couldn't get '%s' output." % cmd
- print "Bitbake failed with error:\n%s\n" % lines
- return
-
- # Parse bitbake -e output
- for line in lines.split('\n'):
- if "=" not in line:
- continue
- try:
- key, val = line.split("=")
- except ValueError:
- continue
- key = key.strip()
- val = val.strip()
- if key.replace('_', '').isalnum():
- _BITBAKE_VARS[image][key] = val.strip('"')
-
- # Make first image a default set of variables
- images = [key for key in _BITBAKE_VARS if key]
- if len(images) == 1:
- _BITBAKE_VARS[None] = _BITBAKE_VARS[image]
-
- return _BITBAKE_VARS[image].get(var)
+ return BB_VARS.get_var(var, image)
def parse_sourceparams(sourceparams):
"""
--
2.1.4
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH 04/12] wic: create new method _parse_line
2015-08-30 17:46 [PATCH 00/12] V3: Build wic images with bitbake Ed Bartosh
` (2 preceding siblings ...)
2015-08-30 17:47 ` [PATCH 03/12] wic: add BitbakeVars class Ed Bartosh
@ 2015-08-30 17:47 ` Ed Bartosh
2015-08-30 17:47 ` [PATCH 05/12] wic: add default_image attribute to BitbakeVars Ed Bartosh
` (7 subsequent siblings)
11 siblings, 0 replies; 14+ messages in thread
From: Ed Bartosh @ 2015-08-30 17:47 UTC (permalink / raw)
To: openembedded-core
Moved code that parses one line of 'bitbake -e' output
to separate method _parse_line.
This method will be also used later to parse lines of .env files.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/utils/oe/misc.py | 27 +++++++++++++++++----------
1 file changed, 17 insertions(+), 10 deletions(-)
diff --git a/scripts/lib/wic/utils/oe/misc.py b/scripts/lib/wic/utils/oe/misc.py
index 3537a2e..41e435f 100644
--- a/scripts/lib/wic/utils/oe/misc.py
+++ b/scripts/lib/wic/utils/oe/misc.py
@@ -131,6 +131,22 @@ class BitbakeVars(defaultdict):
def __init__(self):
defaultdict.__init__(self, dict)
+ def _parse_line(self, line, image):
+ """
+ Parse one line from bitbake -e output.
+ Put result key-value pair into the storage.
+ """
+ if "=" not in line:
+ return
+ try:
+ key, val = line.split("=")
+ except ValueError:
+ return
+ key = key.strip()
+ val = val.strip()
+ if key.replace('_', '').isalnum():
+ self[image][key] = val.strip('"')
+
def get_var(self, var, image=None):
"""
Get bitbake variable value lazy way, i.e. run
@@ -154,16 +170,7 @@ class BitbakeVars(defaultdict):
# Parse bitbake -e output
for line in lines.split('\n'):
- if "=" not in line:
- continue
- try:
- key, val = line.split("=")
- except ValueError:
- continue
- key = key.strip()
- val = val.strip()
- if key.replace('_', '').isalnum():
- self[image][key] = val.strip('"')
+ self._parse_line(line, image)
# Make first image a default set of variables
images = [key for key in self if key]
--
2.1.4
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH 05/12] wic: add default_image attribute to BitbakeVars
2015-08-30 17:46 [PATCH 00/12] V3: Build wic images with bitbake Ed Bartosh
` (3 preceding siblings ...)
2015-08-30 17:47 ` [PATCH 04/12] wic: create new method _parse_line Ed Bartosh
@ 2015-08-30 17:47 ` Ed Bartosh
2015-08-30 17:47 ` [PATCH 06/12] wic: set default image Ed Bartosh
` (6 subsequent siblings)
11 siblings, 0 replies; 14+ messages in thread
From: Ed Bartosh @ 2015-08-30 17:47 UTC (permalink / raw)
To: openembedded-core
New attribute is used when bitbake variable is requested without
specifying image name. The attribute should be set from outside,
for example when wic is called with '-e <image>' option.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/utils/oe/misc.py | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/scripts/lib/wic/utils/oe/misc.py b/scripts/lib/wic/utils/oe/misc.py
index 41e435f..040176d 100644
--- a/scripts/lib/wic/utils/oe/misc.py
+++ b/scripts/lib/wic/utils/oe/misc.py
@@ -131,6 +131,9 @@ class BitbakeVars(defaultdict):
def __init__(self):
defaultdict.__init__(self, dict)
+ # default_image attribute should be set from outside
+ self.default_image = None
+
def _parse_line(self, line, image):
"""
Parse one line from bitbake -e output.
@@ -152,6 +155,9 @@ class BitbakeVars(defaultdict):
Get bitbake variable value lazy way, i.e. run
'bitbake -e' only when variable is requested.
"""
+ if not image:
+ image = self.default_image
+
if image not in self:
# Get bitbake -e output
cmd = "bitbake -e"
--
2.1.4
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH 06/12] wic: set default image
2015-08-30 17:46 [PATCH 00/12] V3: Build wic images with bitbake Ed Bartosh
` (4 preceding siblings ...)
2015-08-30 17:47 ` [PATCH 05/12] wic: add default_image attribute to BitbakeVars Ed Bartosh
@ 2015-08-30 17:47 ` Ed Bartosh
2015-08-30 17:47 ` [PATCH 07/12] wic: implement getting variables from .env files Ed Bartosh
` (5 subsequent siblings)
11 siblings, 0 replies; 14+ messages in thread
From: Ed Bartosh @ 2015-08-30 17:47 UTC (permalink / raw)
To: openembedded-core
Set BitbakeVars.default_image when wic is called with -e option.
This makes get_bitbake_var API to use provided image as a default
source of variables.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/wic | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/scripts/wic b/scripts/wic
index 5aa2393..5818e2d 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -52,7 +52,7 @@ if bitbake_exe:
else:
bitbake_main = None
-from wic.utils.oe.misc import get_bitbake_var
+from wic.utils.oe.misc import get_bitbake_var, BB_VARS
from wic.utils.errors import WicError
from image import engine
from image import help as hlp
@@ -141,7 +141,9 @@ def wic_create_subcommand(args, usage_str):
print " " + ", ".join(missed)
sys.exit(1)
- if not options.image_name:
+ if options.image_name:
+ BB_VARS.default_image = options.image_name
+ else:
options.build_check = False
if options.build_check:
--
2.1.4
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH 07/12] wic: implement getting variables from .env files
2015-08-30 17:46 [PATCH 00/12] V3: Build wic images with bitbake Ed Bartosh
` (5 preceding siblings ...)
2015-08-30 17:47 ` [PATCH 06/12] wic: set default image Ed Bartosh
@ 2015-08-30 17:47 ` Ed Bartosh
2015-08-30 17:47 ` [PATCH 08/12] wic: implement --vars option Ed Bartosh
` (4 subsequent siblings)
11 siblings, 0 replies; 14+ messages in thread
From: Ed Bartosh @ 2015-08-30 17:47 UTC (permalink / raw)
To: openembedded-core
Added functionality of getting variables from <image>.env files to
BitbakeVars class. env files will be parsed if the directory with
env files is known, i.e. when vars_dir attribute is set.
Otherwise 'bitbake -e' output will be parsed.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/utils/oe/misc.py | 59 +++++++++++++++++++++++++---------------
1 file changed, 37 insertions(+), 22 deletions(-)
diff --git a/scripts/lib/wic/utils/oe/misc.py b/scripts/lib/wic/utils/oe/misc.py
index 040176d..0a8f3f1 100644
--- a/scripts/lib/wic/utils/oe/misc.py
+++ b/scripts/lib/wic/utils/oe/misc.py
@@ -26,6 +26,7 @@
#
"""Miscellaneous functions."""
+import os
from collections import defaultdict
from wic import msger
@@ -131,12 +132,13 @@ class BitbakeVars(defaultdict):
def __init__(self):
defaultdict.__init__(self, dict)
- # default_image attribute should be set from outside
+ # default_image and vars_dir attributes should be set from outside
self.default_image = None
+ self.vars_dir = None
def _parse_line(self, line, image):
"""
- Parse one line from bitbake -e output.
+ Parse one line from bitbake -e output or from .env file.
Put result key-value pair into the storage.
"""
if "=" not in line:
@@ -152,31 +154,44 @@ class BitbakeVars(defaultdict):
def get_var(self, var, image=None):
"""
- Get bitbake variable value lazy way, i.e. run
- 'bitbake -e' only when variable is requested.
+ Get bitbake variable from 'bitbake -e' output or from .env file.
+ This is a lazy method, i.e. it runs bitbake or parses file only when
+ only when variable is requested. It also caches results.
"""
if not image:
image = self.default_image
if image not in self:
- # Get bitbake -e output
- cmd = "bitbake -e"
- if image:
- cmd += " %s" % image
-
- log_level = msger.get_loglevel()
- msger.set_loglevel('normal')
- ret, lines = _exec_cmd(cmd)
- msger.set_loglevel(log_level)
-
- if ret:
- print "Couldn't get '%s' output." % cmd
- print "Bitbake failed with error:\n%s\n" % lines
- return
-
- # Parse bitbake -e output
- for line in lines.split('\n'):
- self._parse_line(line, image)
+ if image and self.vars_dir:
+ fname = os.path.join(self.vars_dir, image + '.env')
+ if os.path.isfile(fname):
+ # parse .env file
+ with open(fname) as varsfile:
+ for line in varsfile:
+ self._parse_line(line, image)
+ else:
+ print "Couldn't get bitbake variable from %s." % fname
+ print "File %s doesn't exist." % fname
+ return
+ else:
+ # Get bitbake -e output
+ cmd = "bitbake -e"
+ if image:
+ cmd += " %s" % image
+
+ log_level = msger.get_loglevel()
+ msger.set_loglevel('normal')
+ ret, lines = _exec_cmd(cmd)
+ msger.set_loglevel(log_level)
+
+ if ret:
+ print "Couldn't get '%s' output." % cmd
+ print "Bitbake failed with error:\n%s\n" % lines
+ return
+
+ # Parse bitbake -e output
+ for line in lines.split('\n'):
+ self._parse_line(line, image)
# Make first image a default set of variables
images = [key for key in self if key]
--
2.1.4
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH 08/12] wic: implement --vars option
2015-08-30 17:46 [PATCH 00/12] V3: Build wic images with bitbake Ed Bartosh
` (6 preceding siblings ...)
2015-08-30 17:47 ` [PATCH 07/12] wic: implement getting variables from .env files Ed Bartosh
@ 2015-08-30 17:47 ` Ed Bartosh
2015-08-30 17:47 ` [PATCH 09/12] image_types.bbclass: add wic image type Ed Bartosh
` (3 subsequent siblings)
11 siblings, 0 replies; 14+ messages in thread
From: Ed Bartosh @ 2015-08-30 17:47 UTC (permalink / raw)
To: openembedded-core
This option is used to point wic to the directory with .env
files containing list of bitbake variables and their values.
If this option is used wic will get bitbake variables from
files instead of parsing 'bitbake -e' output.
The main reason for this is to support new mode, when bitbake
runs wic to produce wic images. In this case wic can't run bitbake
again as it's locked, so it will get variables from .env files.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/wic | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/scripts/wic b/scripts/wic
index 5818e2d..25b0d67 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -114,6 +114,9 @@ def wic_create_subcommand(args, usage_str):
parser.add_option("-c", "--compress-with", choices=("gzip", "bzip2", "xz"),
dest='compressor',
help="compress image with specified compressor")
+ parser.add_option("-v", "--vars", dest='vars_dir',
+ help="directory with <image>.env files that store "
+ "bitbake variables")
parser.add_option("-D", "--debug", dest="debug", action="store_true",
default=False, help="output debug information")
@@ -146,6 +149,9 @@ def wic_create_subcommand(args, usage_str):
else:
options.build_check = False
+ if options.vars_dir:
+ BB_VARS.vars_dir = options.vars_dir
+
if options.build_check:
print "Checking basic build environment..."
if not engine.verify_build_env():
--
2.1.4
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH 09/12] image_types.bbclass: add wic image type
2015-08-30 17:46 [PATCH 00/12] V3: Build wic images with bitbake Ed Bartosh
` (7 preceding siblings ...)
2015-08-30 17:47 ` [PATCH 08/12] wic: implement --vars option Ed Bartosh
@ 2015-08-30 17:47 ` Ed Bartosh
2015-08-30 17:47 ` [PATCH 10/12] wic-image-minimal: add wic image recipe and .wks Ed Bartosh
` (2 subsequent siblings)
11 siblings, 0 replies; 14+ messages in thread
From: Ed Bartosh @ 2015-08-30 17:47 UTC (permalink / raw)
To: openembedded-core
wic image type is used to produce partitioned images.
Image configuration should be stored in either <recipe>.<machine>.wks
or <recipe>.wks file.
.wks file should be put to the same location as image recipe
and have the same name.
[YOCTO #7672]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
meta/classes/image_types.bbclass | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index acee227..ecb066b 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -162,6 +162,16 @@ IMAGE_CMD_ubi = "multiubi_mkfs "${MKUBIFS_ARGS}" "${UBINIZE_ARGS}" "${UBI_VOLNAM
IMAGE_CMD_ubifs = "mkfs.ubifs -r ${IMAGE_ROOTFS} -o ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ubifs ${MKUBIFS_ARGS}"
+IMAGE_CMD_wic () {
+ out=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}
+ wks=${FILE_DIRNAME}/${IMAGE_BASENAME}.${MACHINE}.wks
+ [ -e $wks ] || wks=${FILE_DIRNAME}/${IMAGE_BASENAME}.wks
+ [ -e $wks ] || bbfatal "Kiskstart file $wks doesn't exist"
+ BUILDDIR=${TOPDIR} wic create $wks --vars ${STAGING_DIR_TARGET}/imgdata/ -e ${IMAGE_BASENAME} -o $out/
+ mv $out/build/${IMAGE_BASENAME}*.direct $out.rootfs.wic
+ rm -rf $out/
+}
+
EXTRA_IMAGECMD = ""
inherit siteinfo
@@ -190,6 +200,7 @@ IMAGE_DEPENDS_elf = "virtual/kernel mkelfimage-native"
IMAGE_DEPENDS_ubi = "mtd-utils-native"
IMAGE_DEPENDS_ubifs = "mtd-utils-native"
IMAGE_DEPENDS_multiubi = "mtd-utils-native"
+IMAGE_DEPENDS_wic = "parted-native"
# This variable is available to request which values are suitable for IMAGE_FSTYPES
IMAGE_TYPES = " \
@@ -209,6 +220,7 @@ IMAGE_TYPES = " \
vdi \
qcow2 \
elf \
+ wic wic.gz wic.bz2 wic.lzma \
"
COMPRESSIONTYPES = "gz bz2 lzma xz lz4 sum"
--
2.1.4
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH 10/12] wic-image-minimal: add wic image recipe and .wks
2015-08-30 17:46 [PATCH 00/12] V3: Build wic images with bitbake Ed Bartosh
` (8 preceding siblings ...)
2015-08-30 17:47 ` [PATCH 09/12] image_types.bbclass: add wic image type Ed Bartosh
@ 2015-08-30 17:47 ` Ed Bartosh
2015-08-30 20:37 ` Richard Purdie
2015-08-30 17:47 ` [PATCH 11/12] oe-selftest: test building wic image by bitbake Ed Bartosh
2015-08-30 17:47 ` [PATCH 12/12] wic: use bitbake variable ROOTFS_SIZE Ed Bartosh
11 siblings, 1 reply; 14+ messages in thread
From: Ed Bartosh @ 2015-08-30 17:47 UTC (permalink / raw)
To: openembedded-core
Added example of recipe and .wks file to create partitioned image.
This image is using quite complex partitioning scheme.
It uses its own rootfs to populate two partitions in two different ways.
It also uses core-image-minimal rootfs to populate another partition.
This is how wic reports about artifacts used to create this image:
ROOTFS_DIR: tmp/work/qemux86_64-poky-linux/wic-image-minimal/1.0-r0/rootfs
ROOTFS_DIR["/core"]: tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs
ROOTFS_DIR["/backup"]: tmp/work/qemux86_64-poky-linux/wic-image-minimal/1.0-r0/rootfs
BOOTIMG_DIR: tmp/sysroots/qemux86-64/usr/share
KERNEL_DIR: tmp/deploy/images/qemux86-64
NATIVE_SYSROOT: tmp/sysroots/x86_64-linux
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
meta/recipes-extended/images/wic-image-minimal.bb | 14 ++++++++++++++
meta/recipes-extended/images/wic-image-minimal.wks | 10 ++++++++++
2 files changed, 24 insertions(+)
create mode 100644 meta/recipes-extended/images/wic-image-minimal.bb
create mode 100644 meta/recipes-extended/images/wic-image-minimal.wks
diff --git a/meta/recipes-extended/images/wic-image-minimal.bb b/meta/recipes-extended/images/wic-image-minimal.bb
new file mode 100644
index 0000000..073c569
--- /dev/null
+++ b/meta/recipes-extended/images/wic-image-minimal.bb
@@ -0,0 +1,14 @@
+SUMMARY = "An example of partitioned image."
+
+IMAGE_INSTALL = "packagegroup-core-boot ${ROOTFS_PKGMANAGE_BOOTSTRAP}"
+
+IMAGE_FSTYPES = "wic.bz2"
+RM_OLD_IMAGE = "1"
+
+# core-image-minimal is referenced in .wks, so we need its rootfs
+# to be ready before our rootfs
+do_rootfs[depends] += "core-image-minimal:do_rootfs"
+
+IMAGE_ROOTFS_EXTRA_SPACE = "2000"
+
+inherit image
diff --git a/meta/recipes-extended/images/wic-image-minimal.wks b/meta/recipes-extended/images/wic-image-minimal.wks
new file mode 100644
index 0000000..29cd8f2
--- /dev/null
+++ b/meta/recipes-extended/images/wic-image-minimal.wks
@@ -0,0 +1,10 @@
+# short-description: Example of partitioned image with complex layout
+# long-description: This image contains boot partition and 3 rootfs partitions
+# created from core-image-minimal and wic-image-minimal image recipes.
+
+part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024
+part / --source rootfs --ondisk sda --fstype=ext2 --label platform --align 1024
+part /core --source rootfs --rootfs-dir=core-image-minimal --ondisk sda --fstype=ext2 --label core --align 1024
+part /backup --source rootfs --rootfs-dir=wic-image-minimal --ondisk sda --fstype=ext2 --label backup --align 1024
+
+bootloader --timeout=0 --append="rootwait console=tty0"
--
2.1.4
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [PATCH 10/12] wic-image-minimal: add wic image recipe and .wks
2015-08-30 17:47 ` [PATCH 10/12] wic-image-minimal: add wic image recipe and .wks Ed Bartosh
@ 2015-08-30 20:37 ` Richard Purdie
0 siblings, 0 replies; 14+ messages in thread
From: Richard Purdie @ 2015-08-30 20:37 UTC (permalink / raw)
To: Ed Bartosh; +Cc: openembedded-core
On Sun, 2015-08-30 at 20:47 +0300, Ed Bartosh wrote:
> Added example of recipe and .wks file to create partitioned image.
>
> This image is using quite complex partitioning scheme.
> It uses its own rootfs to populate two partitions in two different ways.
> It also uses core-image-minimal rootfs to populate another partition.
>
> This is how wic reports about artifacts used to create this image:
> ROOTFS_DIR: tmp/work/qemux86_64-poky-linux/wic-image-minimal/1.0-r0/rootfs
> ROOTFS_DIR["/core"]: tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs
> ROOTFS_DIR["/backup"]: tmp/work/qemux86_64-poky-linux/wic-image-minimal/1.0-r0/rootfs
> BOOTIMG_DIR: tmp/sysroots/qemux86-64/usr/share
> KERNEL_DIR: tmp/deploy/images/qemux86-64
> NATIVE_SYSROOT: tmp/sysroots/x86_64-linux
>
> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Since we use this as the selftest, I think this may be better positioned
in meta-selftest?
Otherwise the patchset looks good thanks. I'll merge this up to this
patch.
Cheers,
Richard
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH 11/12] oe-selftest: test building wic image by bitbake
2015-08-30 17:46 [PATCH 00/12] V3: Build wic images with bitbake Ed Bartosh
` (9 preceding siblings ...)
2015-08-30 17:47 ` [PATCH 10/12] wic-image-minimal: add wic image recipe and .wks Ed Bartosh
@ 2015-08-30 17:47 ` Ed Bartosh
2015-08-30 17:47 ` [PATCH 12/12] wic: use bitbake variable ROOTFS_SIZE Ed Bartosh
11 siblings, 0 replies; 14+ messages in thread
From: Ed Bartosh @ 2015-08-30 17:47 UTC (permalink / raw)
To: openembedded-core
Added test case to verify building of wic-image-minimal recipe
and produced artifacts: manifest and bzipped partitioned image.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
meta/lib/oeqa/selftest/wic.py | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/meta/lib/oeqa/selftest/wic.py b/meta/lib/oeqa/selftest/wic.py
index fe8a2d0..deb2333 100644
--- a/meta/lib/oeqa/selftest/wic.py
+++ b/meta/lib/oeqa/selftest/wic.py
@@ -188,3 +188,17 @@ class Wic(oeSelfTest):
for var in wicvars:
self.assertTrue(var in content, "%s is not in .env file" % var)
self.assertTrue(content[var])
+
+ def test20_wic_image_type(self):
+ """Test building wic images by bitbake"""
+ self.assertEqual(0, bitbake('wic-image-minimal').status)
+
+ deploy_dir = get_bb_var('DEPLOY_DIR_IMAGE')
+ machine = get_bb_var('MACHINE')
+ prefix = os.path.join(deploy_dir, 'wic-image-minimal-%s.' % machine)
+ # check if we have result image and manifests symlinks
+ # pointing to existing files
+ for suffix in ('wic.bz2', 'manifest'):
+ path = prefix + suffix
+ self.assertTrue(os.path.islink(path))
+ self.assertTrue(os.path.isfile(os.path.realpath(path)))
--
2.1.4
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH 12/12] wic: use bitbake variable ROOTFS_SIZE
2015-08-30 17:46 [PATCH 00/12] V3: Build wic images with bitbake Ed Bartosh
` (10 preceding siblings ...)
2015-08-30 17:47 ` [PATCH 11/12] oe-selftest: test building wic image by bitbake Ed Bartosh
@ 2015-08-30 17:47 ` Ed Bartosh
11 siblings, 0 replies; 14+ messages in thread
From: Ed Bartosh @ 2015-08-30 17:47 UTC (permalink / raw)
To: openembedded-core
If bitbake image is referenced in .ks file and --size is not used
there wic uses ROOTFS_SIZE variable to set minimum partition size.
ROOTFS_SIZE is calculated in meta/lib/oe/image.py when rootfs is
created. The calculation is done using other image parameters:
IMAGE_ROOTFS_SIZE, IMAGE_ROOTFS_ALIGNMENT, IMAGE_OVERHEAD_FACTOR
and IMAGE_ROOTFS_EXTRA_SPACE.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/imager/direct.py | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/scripts/lib/wic/imager/direct.py b/scripts/lib/wic/imager/direct.py
index 57b1335..d68fd2a 100644
--- a/scripts/lib/wic/imager/direct.py
+++ b/scripts/lib/wic/imager/direct.py
@@ -29,6 +29,7 @@ import shutil
from wic import kickstart, msger
from wic.utils import fs_related
+from wic.utils.oe.misc import get_bitbake_var
from wic.utils.partitionedfs import Image
from wic.utils.errors import CreatorError, ImageError
from wic.imager.baseimager import BaseImageCreator
@@ -229,6 +230,19 @@ class DirectImageCreator(BaseImageCreator):
fstab_path = self._write_fstab(self.rootfs_dir.get("ROOTFS_DIR"))
for p in parts:
+ # get rootfs size from bitbake variable if it's not set in .ks file
+ if not p.size:
+ # and if rootfs name is specified for the partition
+ image_name = p.get_rootfs()
+ if image_name:
+ # Bitbake variable ROOTFS_SIZE is calculated in
+ # Image._get_rootfs_size method from meta/lib/oe/image.py
+ # using IMAGE_ROOTFS_SIZE, IMAGE_ROOTFS_ALIGNMENT,
+ # IMAGE_OVERHEAD_FACTOR and IMAGE_ROOTFS_EXTRA_SPACE
+ rsize_bb = get_bitbake_var('ROOTFS_SIZE', image_name)
+ if rsize_bb:
+ # convert from Kb to Mb
+ p.size = int(rsize_bb) / 1024
# need to create the filesystems in order to get their
# sizes before we can add them and do the layout.
# Image.create() actually calls __format_disks() to create
--
2.1.4
^ permalink raw reply related [flat|nested] 14+ messages in thread