* [PATCH 01/17] image.py: write bitbake variables to .env file
2015-08-20 11:56 [PATCH 00/17] Build wic images with bitbake Ed Bartosh
@ 2015-08-20 11:56 ` Ed Bartosh
2015-08-30 11:24 ` Richard Purdie
2015-08-20 11:56 ` [PATCH 02/17] oe-selftest: test generation of <image>.env Ed Bartosh
` (15 subsequent siblings)
16 siblings, 1 reply; 22+ messages in thread
From: Ed Bartosh @ 2015-08-20 11:56 UTC (permalink / raw)
To: openembedded-core
Write set of bitbake variables associated with the image into
build/tmp/sysroots/<machine>/imagedata/<image>.env
This is needed for wic to be able to get bitbake variables without
running 'bitbake -e'.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
meta/lib/oe/image.py | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/meta/lib/oe/image.py b/meta/lib/oe/image.py
index 699c30f..a7fdefa 100644
--- a/meta/lib/oe/image.py
+++ b/meta/lib/oe/image.py
@@ -321,6 +321,27 @@ class Image(ImageDepGraph):
return image_cmd_groups
+ def _write_env(self):
+ """
+ Write environment variables
+ to tmp/sysroots/<machine>/imgdata/<image>.env
+ """
+ stdir = self.d.getVar('STAGING_DIR_TARGET', True)
+ outdir = os.path.join(stdir, 'imgdata')
+ if not os.path.exists(outdir):
+ os.makedirs(outdir)
+ basename = self.d.getVar('IMAGE_BASENAME', True)
+ with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
+ for var in sorted(self.d.keys()):
+ # filter out as much as we can to reduce file size
+ if var.startswith('_') or var.startswith('BB_') \
+ or not var.isupper() or self.d.getVarFlag(var, "func") \
+ or var in ('BBINCLUDED', 'SRCPV', 'MIRRORS'):
+ continue
+ value = self.d.getVar(var, True)
+ if value:
+ envf.write('%s="%s"\n' % (var, value.strip()))
+
def create(self):
bb.note("###### Generate images #######")
pre_process_cmds = self.d.getVar("IMAGE_PREPROCESS_COMMAND", True)
@@ -332,6 +353,8 @@ class Image(ImageDepGraph):
image_cmd_groups = self._get_imagecmds()
+ self._write_env()
+
for image_cmds in image_cmd_groups:
# create the images in parallel
nproc = multiprocessing.cpu_count()
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* Re: [PATCH 01/17] image.py: write bitbake variables to .env file
2015-08-20 11:56 ` [PATCH 01/17] image.py: write bitbake variables to .env file Ed Bartosh
@ 2015-08-30 11:24 ` Richard Purdie
2015-08-30 13:17 ` Ed Bartosh
0 siblings, 1 reply; 22+ messages in thread
From: Richard Purdie @ 2015-08-30 11:24 UTC (permalink / raw)
To: Ed Bartosh; +Cc: openembedded-core
On Thu, 2015-08-20 at 14:56 +0300, Ed Bartosh wrote:
> Write set of bitbake variables associated with the image into
> build/tmp/sysroots/<machine>/imagedata/<image>.env
>
> This is needed for wic to be able to get bitbake variables without
> running 'bitbake -e'.
I've just realised what this code is doing, its writing out nearly the
whole data store :(
Is there no way we can know which variables wic may later need? I'm not
keen at all to see an iteration of d.keys() and the list of things not
to write out looks arbitrary at best :(
We need to find a better way of doing this...
I'd also prefer we only do this if we know we're going to be writing a
wic image.
Cheers,
Richard
> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> ---
> meta/lib/oe/image.py | 23 +++++++++++++++++++++++
> 1 file changed, 23 insertions(+)
>
> diff --git a/meta/lib/oe/image.py b/meta/lib/oe/image.py
> index 699c30f..a7fdefa 100644
> --- a/meta/lib/oe/image.py
> +++ b/meta/lib/oe/image.py
> @@ -321,6 +321,27 @@ class Image(ImageDepGraph):
>
> return image_cmd_groups
>
> + def _write_env(self):
> + """
> + Write environment variables
> + to tmp/sysroots/<machine>/imgdata/<image>.env
> + """
> + stdir = self.d.getVar('STAGING_DIR_TARGET', True)
> + outdir = os.path.join(stdir, 'imgdata')
> + if not os.path.exists(outdir):
> + os.makedirs(outdir)
> + basename = self.d.getVar('IMAGE_BASENAME', True)
> + with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
> + for var in sorted(self.d.keys()):
> + # filter out as much as we can to reduce file size
> + if var.startswith('_') or var.startswith('BB_') \
> + or not var.isupper() or self.d.getVarFlag(var, "func") \
> + or var in ('BBINCLUDED', 'SRCPV', 'MIRRORS'):
> + continue
> + value = self.d.getVar(var, True)
> + if value:
> + envf.write('%s="%s"\n' % (var, value.strip()))
> +
> def create(self):
> bb.note("###### Generate images #######")
> pre_process_cmds = self.d.getVar("IMAGE_PREPROCESS_COMMAND", True)
> @@ -332,6 +353,8 @@ class Image(ImageDepGraph):
>
> image_cmd_groups = self._get_imagecmds()
>
> + self._write_env()
> +
> for image_cmds in image_cmd_groups:
> # create the images in parallel
> nproc = multiprocessing.cpu_count()
> --
> 2.1.4
>
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [PATCH 01/17] image.py: write bitbake variables to .env file
2015-08-30 11:24 ` Richard Purdie
@ 2015-08-30 13:17 ` Ed Bartosh
0 siblings, 0 replies; 22+ messages in thread
From: Ed Bartosh @ 2015-08-30 13:17 UTC (permalink / raw)
To: Richard Purdie; +Cc: openembedded-core
On Sun, Aug 30, 2015 at 12:24:48PM +0100, Richard Purdie wrote:
> On Thu, 2015-08-20 at 14:56 +0300, Ed Bartosh wrote:
> > Write set of bitbake variables associated with the image into
> > build/tmp/sysroots/<machine>/imagedata/<image>.env
> >
> > This is needed for wic to be able to get bitbake variables without
> > running 'bitbake -e'.
>
> I've just realised what this code is doing, its writing out nearly the
> whole data store :(
I'd not say so. The code filters out a lot of content and that filtering can
be improved if needed.
Here is a real example of the file sizes:
$ ls -lh tmp/sysroots/nuc/imgdata/
total 124K
-rw-r--r-- 1 ed users 41K Aug 27 11:34 core-image-minimal.env
-rw-r--r-- 1 ed users 40K Aug 27 11:29 core-image-minimal-initramfs.env
-rw-r--r-- 1 ed users 39K Aug 27 11:35 wic-image-minimal.env
Are they still too big from your point of view?
> Is there no way we can know which variables wic may later need?
Theoretically there is no way to know it. It depends on what wic
plugin writers would want to use.
However, I can make .env files contain only variables that wic
already uses. This would require explicit addition of every new variable
to the code when wic starts to use it, but it's not a big deal as
most probably it will not happen to often.
Frankly, I thought that 40K in size .env file per image is not a big
deal either, but looks like it is.
> I'm not
> keen at all to see an iteration of d.keys() and the list of things not
> to write out looks arbitrary at best :(
It's not arbitrary. It's a result of filtering out variables which wic
will most probably not require and variables with the biggest
content size.
> We need to find a better way of doing this...
See my proposal above.
> I'd also prefer we only do this if we know we're going to be writing a
> wic image.
Any suggestion how to do this?
The problem is that any previously built image can be potentially
used by wic if it's mentioned in .ks file. We can find this out when
we build wic image, but we can't generate .env files on this step.
Regards,
Ed
> > Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> > ---
> > meta/lib/oe/image.py | 23 +++++++++++++++++++++++
> > 1 file changed, 23 insertions(+)
> >
> > diff --git a/meta/lib/oe/image.py b/meta/lib/oe/image.py
> > index 699c30f..a7fdefa 100644
> > --- a/meta/lib/oe/image.py
> > +++ b/meta/lib/oe/image.py
> > @@ -321,6 +321,27 @@ class Image(ImageDepGraph):
> >
> > return image_cmd_groups
> >
> > + def _write_env(self):
> > + """
> > + Write environment variables
> > + to tmp/sysroots/<machine>/imgdata/<image>.env
> > + """
> > + stdir = self.d.getVar('STAGING_DIR_TARGET', True)
> > + outdir = os.path.join(stdir, 'imgdata')
> > + if not os.path.exists(outdir):
> > + os.makedirs(outdir)
> > + basename = self.d.getVar('IMAGE_BASENAME', True)
> > + with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
> > + for var in sorted(self.d.keys()):
> > + # filter out as much as we can to reduce file size
> > + if var.startswith('_') or var.startswith('BB_') \
> > + or not var.isupper() or self.d.getVarFlag(var, "func") \
> > + or var in ('BBINCLUDED', 'SRCPV', 'MIRRORS'):
> > + continue
> > + value = self.d.getVar(var, True)
> > + if value:
> > + envf.write('%s="%s"\n' % (var, value.strip()))
> > +
> > def create(self):
> > bb.note("###### Generate images #######")
> > pre_process_cmds = self.d.getVar("IMAGE_PREPROCESS_COMMAND", True)
> > @@ -332,6 +353,8 @@ class Image(ImageDepGraph):
> >
> > image_cmd_groups = self._get_imagecmds()
> >
> > + self._write_env()
> > +
> > for image_cmds in image_cmd_groups:
> > # create the images in parallel
> > nproc = multiprocessing.cpu_count()
> > --
> > 2.1.4
> >
>
>
--
--
Regards,
Ed
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 02/17] oe-selftest: test generation of <image>.env
2015-08-20 11:56 [PATCH 00/17] Build wic images with bitbake Ed Bartosh
2015-08-20 11:56 ` [PATCH 01/17] image.py: write bitbake variables to .env file Ed Bartosh
@ 2015-08-20 11:56 ` Ed Bartosh
2015-08-20 11:56 ` [PATCH 03/17] wic: remove undescore from function name Ed Bartosh
` (14 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Ed Bartosh @ 2015-08-20 11:56 UTC (permalink / raw)
To: openembedded-core
Added test case to check if <image>.env file is generated
and contains bitbake variables used in wic code.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
meta/lib/oeqa/selftest/wic.py | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/meta/lib/oeqa/selftest/wic.py b/meta/lib/oeqa/selftest/wic.py
index 0c503ef..15de621 100644
--- a/meta/lib/oeqa/selftest/wic.py
+++ b/meta/lib/oeqa/selftest/wic.py
@@ -23,6 +23,7 @@
"""Test cases for wic."""
+import os
import sys
from glob import glob
@@ -160,3 +161,22 @@ class Wic(oeSelfTest):
self.assertEqual(0, status)
self.assertEqual(1, len(glob(self.resultdir + \
"%(wks)s-*.direct" % vars)))
+
+ def test18_image_env(self):
+ """Test generation of <image>.env files."""
+ image = 'core-image-minimal'
+ stdir = get_bb_var('STAGING_DIR_TARGET', image)
+ imgdatadir = os.path.join(stdir, 'imgdata')
+ basename = get_bb_var('IMAGE_BASENAME', image)
+ self.assertEqual(basename, image)
+ path = os.path.join(imgdatadir, basename) + '.env'
+ self.assertTrue(os.path.isfile(path))
+ with open(path) as envfile:
+ content = dict(line.split("=", 1) for line in envfile)
+ # test if variables used by wic present in the .env file
+ for var in ('IMAGE_ROOTFS', 'DEPLOY_DIR_IMAGE', 'BBLAYERS',
+ 'STAGING_DIR_NATIVE', 'STAGING_DATADIR', 'TARGET_SYS',
+ 'IMAGE_BASENAME', 'INITRAMFS_FSTYPES', 'MACHINE_ARCH',
+ 'IMAGE_LINK_NAME', 'STAGING_LIBDIR', 'ROOTFS_SIZE'):
+ self.assertTrue(var in content)
+ self.assertTrue(content[var])
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 03/17] wic: remove undescore from function name
2015-08-20 11:56 [PATCH 00/17] Build wic images with bitbake Ed Bartosh
2015-08-20 11:56 ` [PATCH 01/17] image.py: write bitbake variables to .env file Ed Bartosh
2015-08-20 11:56 ` [PATCH 02/17] oe-selftest: test generation of <image>.env Ed Bartosh
@ 2015-08-20 11:56 ` Ed Bartosh
2015-08-20 11:56 ` [PATCH 04/17] wic: add BitbakeVars class Ed Bartosh
` (13 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Ed Bartosh @ 2015-08-20 11:56 UTC (permalink / raw)
To: openembedded-core
Renamed __exec_cmd -> _exec_cmd as double underscores cause
strange behaviour when function is called in class method.
Python complains that __exec_cmd method(!!!) of the same class
doesn't exist.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/utils/oe/misc.py | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/scripts/lib/wic/utils/oe/misc.py b/scripts/lib/wic/utils/oe/misc.py
index d399f2a..9c8f52d 100644
--- a/scripts/lib/wic/utils/oe/misc.py
+++ b/scripts/lib/wic/utils/oe/misc.py
@@ -46,13 +46,13 @@ NATIVE_RECIPES = {"mcopy": "mtools",
"syslinux": "syslinux"
}
-def __exec_cmd(cmd_and_args, as_shell=False, catch=3):
+def _exec_cmd(cmd_and_args, as_shell=False, catch=3):
"""
Execute command, catching stderr, stdout
Need to execute as_shell if the command uses wildcards
"""
- msger.debug("__exec_cmd: %s" % cmd_and_args)
+ msger.debug("_exec_cmd: %s" % cmd_and_args)
args = cmd_and_args.split()
msger.debug(args)
@@ -61,7 +61,7 @@ def __exec_cmd(cmd_and_args, as_shell=False, catch=3):
else:
ret, out = runner.runtool(args, catch)
out = out.strip()
- msger.debug("__exec_cmd: output for %s (rc = %d): %s" % \
+ msger.debug("_exec_cmd: output for %s (rc = %d): %s" % \
(cmd_and_args, ret, out))
return (ret, out)
@@ -73,7 +73,7 @@ def exec_cmd(cmd_and_args, as_shell=False, catch=3):
Exits if rc non-zero
"""
- ret, out = __exec_cmd(cmd_and_args, as_shell, catch)
+ ret, out = _exec_cmd(cmd_and_args, as_shell, catch)
if ret != 0:
msger.error("exec_cmd: %s returned '%s' instead of 0" % \
@@ -99,7 +99,7 @@ def exec_native_cmd(cmd_and_args, native_sysroot, catch=3):
args = cmd_and_args.split()
msger.debug(args)
- ret, out = __exec_cmd(native_cmd_and_args, True, catch)
+ ret, out = _exec_cmd(native_cmd_and_args, True, catch)
if ret == 127: # shell command-not-found
prog = args[0]
@@ -139,7 +139,7 @@ def get_bitbake_var(var, image=None):
log_level = msger.get_loglevel()
msger.set_loglevel('normal')
- ret, lines = __exec_cmd(cmd)
+ ret, lines = _exec_cmd(cmd)
msger.set_loglevel(log_level)
if ret:
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 04/17] wic: add BitbakeVars class
2015-08-20 11:56 [PATCH 00/17] Build wic images with bitbake Ed Bartosh
` (2 preceding siblings ...)
2015-08-20 11:56 ` [PATCH 03/17] wic: remove undescore from function name Ed Bartosh
@ 2015-08-20 11:56 ` Ed Bartosh
2015-08-20 11:56 ` [PATCH 05/17] wic: create new method _parse_line Ed Bartosh
` (12 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Ed Bartosh @ 2015-08-20 11:56 UTC (permalink / raw)
To: openembedded-core
Moved code of getting bitbake variables into separate class.
Created singleton object of this class in the module namespace.
Preserved existing API get_bitbake_var.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/utils/oe/misc.py | 91 +++++++++++++++++++++++-----------------
1 file changed, 53 insertions(+), 38 deletions(-)
diff --git a/scripts/lib/wic/utils/oe/misc.py b/scripts/lib/wic/utils/oe/misc.py
index 9c8f52d..3537a2e 100644
--- a/scripts/lib/wic/utils/oe/misc.py
+++ b/scripts/lib/wic/utils/oe/misc.py
@@ -124,48 +124,63 @@ def exec_native_cmd(cmd_and_args, native_sysroot, catch=3):
BOOTDD_EXTRA_SPACE = 16384
-_BITBAKE_VARS = defaultdict(dict)
+class BitbakeVars(defaultdict):
+ """
+ Container for Bitbake variables.
+ """
+ def __init__(self):
+ defaultdict.__init__(self, dict)
+
+ def get_var(self, var, image=None):
+ """
+ Get bitbake variable value lazy way, i.e. run
+ 'bitbake -e' only when variable is requested.
+ """
+ if image not in self:
+ # Get bitbake -e output
+ cmd = "bitbake -e"
+ if image:
+ cmd += " %s" % image
+
+ log_level = msger.get_loglevel()
+ msger.set_loglevel('normal')
+ ret, lines = _exec_cmd(cmd)
+ msger.set_loglevel(log_level)
+
+ if ret:
+ print "Couldn't get '%s' output." % cmd
+ print "Bitbake failed with error:\n%s\n" % lines
+ return
+
+ # Parse bitbake -e output
+ for line in lines.split('\n'):
+ if "=" not in line:
+ continue
+ try:
+ key, val = line.split("=")
+ except ValueError:
+ continue
+ key = key.strip()
+ val = val.strip()
+ if key.replace('_', '').isalnum():
+ self[image][key] = val.strip('"')
+
+ # Make first image a default set of variables
+ images = [key for key in self if key]
+ if len(images) == 1:
+ self[None] = self[image]
+
+ return self[image].get(var)
+
+# Create BB_VARS singleton
+BB_VARS = BitbakeVars()
def get_bitbake_var(var, image=None):
"""
- Get bitbake variable value lazy way, i.e. run
- 'bitbake -e' only when variable is requested.
+ Provide old get_bitbake_var API by wrapping
+ get_var method of BB_VARS singleton.
"""
- if image not in _BITBAKE_VARS:
- # Get bitbake -e output
- cmd = "bitbake -e"
- if image:
- cmd += " %s" % image
-
- log_level = msger.get_loglevel()
- msger.set_loglevel('normal')
- ret, lines = _exec_cmd(cmd)
- msger.set_loglevel(log_level)
-
- if ret:
- print "Couldn't get '%s' output." % cmd
- print "Bitbake failed with error:\n%s\n" % lines
- return
-
- # Parse bitbake -e output
- for line in lines.split('\n'):
- if "=" not in line:
- continue
- try:
- key, val = line.split("=")
- except ValueError:
- continue
- key = key.strip()
- val = val.strip()
- if key.replace('_', '').isalnum():
- _BITBAKE_VARS[image][key] = val.strip('"')
-
- # Make first image a default set of variables
- images = [key for key in _BITBAKE_VARS if key]
- if len(images) == 1:
- _BITBAKE_VARS[None] = _BITBAKE_VARS[image]
-
- return _BITBAKE_VARS[image].get(var)
+ return BB_VARS.get_var(var, image)
def parse_sourceparams(sourceparams):
"""
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 05/17] wic: create new method _parse_line
2015-08-20 11:56 [PATCH 00/17] Build wic images with bitbake Ed Bartosh
` (3 preceding siblings ...)
2015-08-20 11:56 ` [PATCH 04/17] wic: add BitbakeVars class Ed Bartosh
@ 2015-08-20 11:56 ` Ed Bartosh
2015-08-20 11:56 ` [PATCH 06/17] wic: add default_image attribute to BitbakeVars Ed Bartosh
` (11 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Ed Bartosh @ 2015-08-20 11:56 UTC (permalink / raw)
To: openembedded-core
Moved code that parses one line of 'bitbake -e' output
to separate method _parse_line.
This method will be also used later to parse lines of .env files.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/utils/oe/misc.py | 27 +++++++++++++++++----------
1 file changed, 17 insertions(+), 10 deletions(-)
diff --git a/scripts/lib/wic/utils/oe/misc.py b/scripts/lib/wic/utils/oe/misc.py
index 3537a2e..41e435f 100644
--- a/scripts/lib/wic/utils/oe/misc.py
+++ b/scripts/lib/wic/utils/oe/misc.py
@@ -131,6 +131,22 @@ class BitbakeVars(defaultdict):
def __init__(self):
defaultdict.__init__(self, dict)
+ def _parse_line(self, line, image):
+ """
+ Parse one line from bitbake -e output.
+ Put result key-value pair into the storage.
+ """
+ if "=" not in line:
+ return
+ try:
+ key, val = line.split("=")
+ except ValueError:
+ return
+ key = key.strip()
+ val = val.strip()
+ if key.replace('_', '').isalnum():
+ self[image][key] = val.strip('"')
+
def get_var(self, var, image=None):
"""
Get bitbake variable value lazy way, i.e. run
@@ -154,16 +170,7 @@ class BitbakeVars(defaultdict):
# Parse bitbake -e output
for line in lines.split('\n'):
- if "=" not in line:
- continue
- try:
- key, val = line.split("=")
- except ValueError:
- continue
- key = key.strip()
- val = val.strip()
- if key.replace('_', '').isalnum():
- self[image][key] = val.strip('"')
+ self._parse_line(line, image)
# Make first image a default set of variables
images = [key for key in self if key]
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 06/17] wic: add default_image attribute to BitbakeVars
2015-08-20 11:56 [PATCH 00/17] Build wic images with bitbake Ed Bartosh
` (4 preceding siblings ...)
2015-08-20 11:56 ` [PATCH 05/17] wic: create new method _parse_line Ed Bartosh
@ 2015-08-20 11:56 ` Ed Bartosh
2015-08-20 11:56 ` [PATCH 07/17] wic: set default image Ed Bartosh
` (10 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Ed Bartosh @ 2015-08-20 11:56 UTC (permalink / raw)
To: openembedded-core
New attribute is used when bitbake variable is requested without
specifying image name. The attribute should be set from outside,
for example when wic is called with '-e <image>' option.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/utils/oe/misc.py | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/scripts/lib/wic/utils/oe/misc.py b/scripts/lib/wic/utils/oe/misc.py
index 41e435f..040176d 100644
--- a/scripts/lib/wic/utils/oe/misc.py
+++ b/scripts/lib/wic/utils/oe/misc.py
@@ -131,6 +131,9 @@ class BitbakeVars(defaultdict):
def __init__(self):
defaultdict.__init__(self, dict)
+ # default_image attribute should be set from outside
+ self.default_image = None
+
def _parse_line(self, line, image):
"""
Parse one line from bitbake -e output.
@@ -152,6 +155,9 @@ class BitbakeVars(defaultdict):
Get bitbake variable value lazy way, i.e. run
'bitbake -e' only when variable is requested.
"""
+ if not image:
+ image = self.default_image
+
if image not in self:
# Get bitbake -e output
cmd = "bitbake -e"
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 07/17] wic: set default image
2015-08-20 11:56 [PATCH 00/17] Build wic images with bitbake Ed Bartosh
` (5 preceding siblings ...)
2015-08-20 11:56 ` [PATCH 06/17] wic: add default_image attribute to BitbakeVars Ed Bartosh
@ 2015-08-20 11:56 ` Ed Bartosh
2015-08-20 11:56 ` [PATCH 08/17] wic: implement getting variables from .env files Ed Bartosh
` (9 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Ed Bartosh @ 2015-08-20 11:56 UTC (permalink / raw)
To: openembedded-core
Set BitbakeVars.default_image when wic is called with -e option.
This makes get_bitbake_var API to use provided image as a default
source of variables.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/wic | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/scripts/wic b/scripts/wic
index c1d3003..87ada36 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -52,7 +52,7 @@ if bitbake_exe:
else:
bitbake_main = None
-from wic.utils.oe.misc import get_bitbake_var
+from wic.utils.oe.misc import get_bitbake_var, BB_VARS
from wic.utils.errors import WicError
from image import engine
from image import help as hlp
@@ -141,7 +141,9 @@ def wic_create_subcommand(args, usage_str):
print " " + ", ".join(missed)
sys.exit(1)
- if not options.image_name:
+ if options.image_name:
+ BB_VARS.default_image = options.image_name
+ else:
options.build_check = False
if options.build_check:
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 08/17] wic: implement getting variables from .env files
2015-08-20 11:56 [PATCH 00/17] Build wic images with bitbake Ed Bartosh
` (6 preceding siblings ...)
2015-08-20 11:56 ` [PATCH 07/17] wic: set default image Ed Bartosh
@ 2015-08-20 11:56 ` Ed Bartosh
2015-08-20 11:56 ` [PATCH 09/17] wic: implement --vars option Ed Bartosh
` (8 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Ed Bartosh @ 2015-08-20 11:56 UTC (permalink / raw)
To: openembedded-core
Added functionality of getting variables from <image>.env files to
BitbakeVars class. env files will be parsed if the directory with
env files is known, i.e. when vars_dir attribute is set.
Otherwise 'bitbake -e' output will be parsed.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/utils/oe/misc.py | 59 +++++++++++++++++++++++++---------------
1 file changed, 37 insertions(+), 22 deletions(-)
diff --git a/scripts/lib/wic/utils/oe/misc.py b/scripts/lib/wic/utils/oe/misc.py
index 040176d..0a8f3f1 100644
--- a/scripts/lib/wic/utils/oe/misc.py
+++ b/scripts/lib/wic/utils/oe/misc.py
@@ -26,6 +26,7 @@
#
"""Miscellaneous functions."""
+import os
from collections import defaultdict
from wic import msger
@@ -131,12 +132,13 @@ class BitbakeVars(defaultdict):
def __init__(self):
defaultdict.__init__(self, dict)
- # default_image attribute should be set from outside
+ # default_image and vars_dir attributes should be set from outside
self.default_image = None
+ self.vars_dir = None
def _parse_line(self, line, image):
"""
- Parse one line from bitbake -e output.
+ Parse one line from bitbake -e output or from .env file.
Put result key-value pair into the storage.
"""
if "=" not in line:
@@ -152,31 +154,44 @@ class BitbakeVars(defaultdict):
def get_var(self, var, image=None):
"""
- Get bitbake variable value lazy way, i.e. run
- 'bitbake -e' only when variable is requested.
+ Get bitbake variable from 'bitbake -e' output or from .env file.
+ This is a lazy method, i.e. it runs bitbake or parses file only when
+ only when variable is requested. It also caches results.
"""
if not image:
image = self.default_image
if image not in self:
- # Get bitbake -e output
- cmd = "bitbake -e"
- if image:
- cmd += " %s" % image
-
- log_level = msger.get_loglevel()
- msger.set_loglevel('normal')
- ret, lines = _exec_cmd(cmd)
- msger.set_loglevel(log_level)
-
- if ret:
- print "Couldn't get '%s' output." % cmd
- print "Bitbake failed with error:\n%s\n" % lines
- return
-
- # Parse bitbake -e output
- for line in lines.split('\n'):
- self._parse_line(line, image)
+ if image and self.vars_dir:
+ fname = os.path.join(self.vars_dir, image + '.env')
+ if os.path.isfile(fname):
+ # parse .env file
+ with open(fname) as varsfile:
+ for line in varsfile:
+ self._parse_line(line, image)
+ else:
+ print "Couldn't get bitbake variable from %s." % fname
+ print "File %s doesn't exist." % fname
+ return
+ else:
+ # Get bitbake -e output
+ cmd = "bitbake -e"
+ if image:
+ cmd += " %s" % image
+
+ log_level = msger.get_loglevel()
+ msger.set_loglevel('normal')
+ ret, lines = _exec_cmd(cmd)
+ msger.set_loglevel(log_level)
+
+ if ret:
+ print "Couldn't get '%s' output." % cmd
+ print "Bitbake failed with error:\n%s\n" % lines
+ return
+
+ # Parse bitbake -e output
+ for line in lines.split('\n'):
+ self._parse_line(line, image)
# Make first image a default set of variables
images = [key for key in self if key]
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 09/17] wic: implement --vars option
2015-08-20 11:56 [PATCH 00/17] Build wic images with bitbake Ed Bartosh
` (7 preceding siblings ...)
2015-08-20 11:56 ` [PATCH 08/17] wic: implement getting variables from .env files Ed Bartosh
@ 2015-08-20 11:56 ` Ed Bartosh
2015-08-20 11:56 ` [PATCH 10/17] wic: rename variable Ed Bartosh
` (7 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Ed Bartosh @ 2015-08-20 11:56 UTC (permalink / raw)
To: openembedded-core
This option is used to point wic to the directory with .env
files containing list of bitbake variables and their values.
If this option is used wic will get bitbake variables from
files instead of parsing 'bitbake -e' output.
The main reason for this is to support new mode, when bitbake
runs wic to produce wic images. In this case wic can't run bitbake
again as it's locked, so it will get variables from .env files.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/wic | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/scripts/wic b/scripts/wic
index 87ada36..815b84e 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -114,6 +114,9 @@ def wic_create_subcommand(args, usage_str):
parser.add_option("-c", "--compress-with", choices=("gzip", "bzip2", "xz"),
dest='compressor',
help="compress image with specified compressor")
+ parser.add_option("-v", "--vars", dest='vars_dir',
+ help="directory with <image>.env files that store "
+ "bitbake variables")
parser.add_option("-D", "--debug", dest="debug", action="store_true",
default=False, help="output debug information")
@@ -146,6 +149,9 @@ def wic_create_subcommand(args, usage_str):
else:
options.build_check = False
+ if options.vars_dir:
+ BB_VARS.vars_dir = options.vars_dir
+
if options.build_check:
print "Checking basic build environment..."
if not engine.verify_build_env():
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 10/17] wic: rename variable
2015-08-20 11:56 [PATCH 00/17] Build wic images with bitbake Ed Bartosh
` (8 preceding siblings ...)
2015-08-20 11:56 ` [PATCH 09/17] wic: implement --vars option Ed Bartosh
@ 2015-08-20 11:56 ` Ed Bartosh
2015-08-20 11:56 ` [PATCH 11/17] wic: deferred call of hlp.get_wic_plugins_help() Ed Bartosh
` (6 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Ed Bartosh @ 2015-08-20 11:56 UTC (permalink / raw)
To: openembedded-core
Renamed variable help -> hlp as 'help' is a name of Python
built-in function.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/image/help.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/scripts/lib/image/help.py b/scripts/lib/image/help.py
index 5fa5836..dc6ff36 100644
--- a/scripts/lib/image/help.py
+++ b/scripts/lib/image/help.py
@@ -41,9 +41,9 @@ def display_help(subcommand, subcommands):
if subcommand not in subcommands:
return False
- help = subcommands.get(subcommand, subcommand_error)[2]
+ hlp = subcommands.get(subcommand, subcommand_error)[2]
pager = subprocess.Popen('less', stdin=subprocess.PIPE)
- pager.communicate(help)
+ pager.communicate(hlp)
return True
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 11/17] wic: deferred call of hlp.get_wic_plugins_help()
2015-08-20 11:56 [PATCH 00/17] Build wic images with bitbake Ed Bartosh
` (9 preceding siblings ...)
2015-08-20 11:56 ` [PATCH 10/17] wic: rename variable Ed Bartosh
@ 2015-08-20 11:56 ` Ed Bartosh
2015-08-20 11:56 ` [PATCH 12/17] image_types.bbclass: add wic image type Ed Bartosh
` (5 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Ed Bartosh @ 2015-08-20 11:56 UTC (permalink / raw)
To: openembedded-core
If get_wic_plugins_help is called from wic main module
it calls git_bitbake_var at some point. This fails when
wic is called from bitbake as 'bitbake -e' can't be
run.
Moved call of this method to help.py in order to call it
later, when BitbakeVariables singleton is properly initialized
to get variables from .env files.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/image/help.py | 2 ++
scripts/wic | 2 +-
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/scripts/lib/image/help.py b/scripts/lib/image/help.py
index dc6ff36..717d847 100644
--- a/scripts/lib/image/help.py
+++ b/scripts/lib/image/help.py
@@ -42,6 +42,8 @@ def display_help(subcommand, subcommands):
return False
hlp = subcommands.get(subcommand, subcommand_error)[2]
+ if callable(hlp):
+ hlp = hlp()
pager = subprocess.Popen('less', stdin=subprocess.PIPE)
pager.communicate(hlp)
diff --git a/scripts/wic b/scripts/wic
index 815b84e..25b0d67 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -288,7 +288,7 @@ subcommands = {
hlp.wic_list_help],
"plugins": [wic_help_topic_subcommand,
wic_help_topic_usage,
- hlp.get_wic_plugins_help()],
+ hlp.get_wic_plugins_help],
"overview": [wic_help_topic_subcommand,
wic_help_topic_usage,
hlp.wic_overview_help],
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 12/17] image_types.bbclass: add wic image type
2015-08-20 11:56 [PATCH 00/17] Build wic images with bitbake Ed Bartosh
` (10 preceding siblings ...)
2015-08-20 11:56 ` [PATCH 11/17] wic: deferred call of hlp.get_wic_plugins_help() Ed Bartosh
@ 2015-08-20 11:56 ` Ed Bartosh
2015-08-20 11:56 ` [PATCH 13/17] wic-image-minimal: add wic image recipe and .wks Ed Bartosh
` (4 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Ed Bartosh @ 2015-08-20 11:56 UTC (permalink / raw)
To: openembedded-core
wic image type is used to produce partitioned images.
Image configuration should be stored in either <recipe>.<machine>.wks
or <recipe>.wks file.
.wks file should be put to the same location as image recipe
and have the same name.
[YOCTO #7672]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
meta/classes/image_types.bbclass | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index 8547574..cdd66ff 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -154,6 +154,16 @@ IMAGE_CMD_ubi = "multiubi_mkfs "${MKUBIFS_ARGS}" "${UBINIZE_ARGS}" "${UBI_VOLNAM
IMAGE_CMD_ubifs = "mkfs.ubifs -r ${IMAGE_ROOTFS} -o ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ubifs ${MKUBIFS_ARGS}"
+IMAGE_CMD_wic () {
+ out=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}
+ wks=${FILE_DIRNAME}/${IMAGE_BASENAME}.${MACHINE}.wks
+ [ -e $wks ] || wks=${FILE_DIRNAME}/${IMAGE_BASENAME}.wks
+ [ -e $wks ] || bbfatal "Kiskstart file $wks doesn't exist"
+ BUILDDIR=${TOPDIR} wic create $wks --vars ${STAGING_DIR_TARGET}/imgdata/ -e ${IMAGE_BASENAME} -o $out/
+ mv $out/build/${IMAGE_BASENAME}*.direct $out.rootfs.wic
+ rm -rf $out/
+}
+
EXTRA_IMAGECMD = ""
inherit siteinfo
@@ -182,6 +192,7 @@ IMAGE_DEPENDS_elf = "virtual/kernel mkelfimage-native"
IMAGE_DEPENDS_ubi = "mtd-utils-native"
IMAGE_DEPENDS_ubifs = "mtd-utils-native"
IMAGE_DEPENDS_multiubi = "mtd-utils-native"
+IMAGE_DEPENDS_wic = "parted-native"
# This variable is available to request which values are suitable for IMAGE_FSTYPES
IMAGE_TYPES = " \
@@ -201,6 +212,7 @@ IMAGE_TYPES = " \
vdi \
qcow2 \
elf \
+ wic wic.gz wic.bz2 wic.lzma \
"
COMPRESSIONTYPES = "gz bz2 lzma xz lz4 sum"
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 13/17] wic-image-minimal: add wic image recipe and .wks
2015-08-20 11:56 [PATCH 00/17] Build wic images with bitbake Ed Bartosh
` (11 preceding siblings ...)
2015-08-20 11:56 ` [PATCH 12/17] image_types.bbclass: add wic image type Ed Bartosh
@ 2015-08-20 11:56 ` Ed Bartosh
2015-08-20 11:56 ` [PATCH 14/17] oe-selftest: test building wic image by bitbake Ed Bartosh
` (3 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Ed Bartosh @ 2015-08-20 11:56 UTC (permalink / raw)
To: openembedded-core
Added example of recipe and .wks file to create partitioned image.
This image is using quite complex partitioning scheme.
It uses its own rootfs to populate two partitions in two different ways.
It also uses core-image-minimal rootfs to populate another partition.
This is how wic reports about artifacts used to create this image:
ROOTFS_DIR: tmp/work/qemux86_64-poky-linux/wic-image-minimal/1.0-r0/rootfs
ROOTFS_DIR["/core"]: tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs
ROOTFS_DIR["/backup"]: tmp/work/qemux86_64-poky-linux/wic-image-minimal/1.0-r0/rootfs
BOOTIMG_DIR: tmp/sysroots/qemux86-64/usr/share
KERNEL_DIR: tmp/deploy/images/qemux86-64
NATIVE_SYSROOT: tmp/sysroots/x86_64-linux
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
meta/recipes-extended/images/wic-image-minimal.bb | 14 ++++++++++++++
meta/recipes-extended/images/wic-image-minimal.wks | 10 ++++++++++
2 files changed, 24 insertions(+)
create mode 100644 meta/recipes-extended/images/wic-image-minimal.bb
create mode 100644 meta/recipes-extended/images/wic-image-minimal.wks
diff --git a/meta/recipes-extended/images/wic-image-minimal.bb b/meta/recipes-extended/images/wic-image-minimal.bb
new file mode 100644
index 0000000..073c569
--- /dev/null
+++ b/meta/recipes-extended/images/wic-image-minimal.bb
@@ -0,0 +1,14 @@
+SUMMARY = "An example of partitioned image."
+
+IMAGE_INSTALL = "packagegroup-core-boot ${ROOTFS_PKGMANAGE_BOOTSTRAP}"
+
+IMAGE_FSTYPES = "wic.bz2"
+RM_OLD_IMAGE = "1"
+
+# core-image-minimal is referenced in .wks, so we need its rootfs
+# to be ready before our rootfs
+do_rootfs[depends] += "core-image-minimal:do_rootfs"
+
+IMAGE_ROOTFS_EXTRA_SPACE = "2000"
+
+inherit image
diff --git a/meta/recipes-extended/images/wic-image-minimal.wks b/meta/recipes-extended/images/wic-image-minimal.wks
new file mode 100644
index 0000000..29cd8f2
--- /dev/null
+++ b/meta/recipes-extended/images/wic-image-minimal.wks
@@ -0,0 +1,10 @@
+# short-description: Example of partitioned image with complex layout
+# long-description: This image contains boot partition and 3 rootfs partitions
+# created from core-image-minimal and wic-image-minimal image recipes.
+
+part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024
+part / --source rootfs --ondisk sda --fstype=ext2 --label platform --align 1024
+part /core --source rootfs --rootfs-dir=core-image-minimal --ondisk sda --fstype=ext2 --label core --align 1024
+part /backup --source rootfs --rootfs-dir=wic-image-minimal --ondisk sda --fstype=ext2 --label backup --align 1024
+
+bootloader --timeout=0 --append="rootwait console=tty0"
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 14/17] oe-selftest: test building wic image by bitbake
2015-08-20 11:56 [PATCH 00/17] Build wic images with bitbake Ed Bartosh
` (12 preceding siblings ...)
2015-08-20 11:56 ` [PATCH 13/17] wic-image-minimal: add wic image recipe and .wks Ed Bartosh
@ 2015-08-20 11:56 ` Ed Bartosh
2015-08-20 12:55 ` Mario Domenech Goulart
2015-08-20 11:56 ` [PATCH 15/17] image.py: set bitbake variable ROOTFS_SIZE Ed Bartosh
` (2 subsequent siblings)
16 siblings, 1 reply; 22+ messages in thread
From: Ed Bartosh @ 2015-08-20 11:56 UTC (permalink / raw)
To: openembedded-core
Added test case to verify building of wic-image-minimal recipe
and produced artifacts: manifest and bzipped partitioned image.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
meta/lib/oeqa/selftest/wic.py | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/meta/lib/oeqa/selftest/wic.py b/meta/lib/oeqa/selftest/wic.py
index 15de621..dc0745a 100644
--- a/meta/lib/oeqa/selftest/wic.py
+++ b/meta/lib/oeqa/selftest/wic.py
@@ -180,3 +180,17 @@ class Wic(oeSelfTest):
'IMAGE_LINK_NAME', 'STAGING_LIBDIR', 'ROOTFS_SIZE'):
self.assertTrue(var in content)
self.assertTrue(content[var])
+
+ def test19_wic_image_type(self):
+ """Test building wic images by bitbake"""
+ self.assertEqual(0, bitbake('wic-image-minimal').status)
+
+ deploy_dir = get_bb_var('DEPLOY_DIR_IMAGE')
+ machine = get_bb_var('MACHINE')
+ prefix = os.path.join(deploy_dir, 'wic-image-minimal-%s.' % machine)
+ # check if we have result image and manifests synmlinks
+ # pointing to existing files
+ for suffix in ('wic.bz2', 'manifest'):
+ path = prefix + suffix
+ self.assertTrue(os.path.islink(path))
+ self.assertTrue(os.path.isfile(os.path.realpath(path)))
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* Re: [PATCH 14/17] oe-selftest: test building wic image by bitbake
2015-08-20 11:56 ` [PATCH 14/17] oe-selftest: test building wic image by bitbake Ed Bartosh
@ 2015-08-20 12:55 ` Mario Domenech Goulart
0 siblings, 0 replies; 22+ messages in thread
From: Mario Domenech Goulart @ 2015-08-20 12:55 UTC (permalink / raw)
To: Ed Bartosh; +Cc: openembedded-core
Hi Ed,
On Thu, 20 Aug 2015 14:56:27 +0300 Ed Bartosh <ed.bartosh@linux.intel.com> wrote:
> Added test case to verify building of wic-image-minimal recipe
> and produced artifacts: manifest and bzipped partitioned image.
>
> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> ---
> meta/lib/oeqa/selftest/wic.py | 14 ++++++++++++++
> 1 file changed, 14 insertions(+)
>
> diff --git a/meta/lib/oeqa/selftest/wic.py b/meta/lib/oeqa/selftest/wic.py
> index 15de621..dc0745a 100644
> --- a/meta/lib/oeqa/selftest/wic.py
> +++ b/meta/lib/oeqa/selftest/wic.py
> @@ -180,3 +180,17 @@ class Wic(oeSelfTest):
> 'IMAGE_LINK_NAME', 'STAGING_LIBDIR', 'ROOTFS_SIZE'):
> self.assertTrue(var in content)
> self.assertTrue(content[var])
> +
> + def test19_wic_image_type(self):
> + """Test building wic images by bitbake"""
> + self.assertEqual(0, bitbake('wic-image-minimal').status)
> +
> + deploy_dir = get_bb_var('DEPLOY_DIR_IMAGE')
> + machine = get_bb_var('MACHINE')
> + prefix = os.path.join(deploy_dir, 'wic-image-minimal-%s.' % machine)
> + # check if we have result image and manifests synmlinks
Small typo: s/synmlinks/symlinks/.
> + # pointing to existing files
> + for suffix in ('wic.bz2', 'manifest'):
> + path = prefix + suffix
> + self.assertTrue(os.path.islink(path))
> + self.assertTrue(os.path.isfile(os.path.realpath(path)))
> --
> 2.1.4
Best wishes.
Mario
--
http://www.ossystems.com.br
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 15/17] image.py: set bitbake variable ROOTFS_SIZE
2015-08-20 11:56 [PATCH 00/17] Build wic images with bitbake Ed Bartosh
` (13 preceding siblings ...)
2015-08-20 11:56 ` [PATCH 14/17] oe-selftest: test building wic image by bitbake Ed Bartosh
@ 2015-08-20 11:56 ` Ed Bartosh
2015-08-20 11:56 ` [PATCH 16/17] wic: use " Ed Bartosh
2015-08-20 11:56 ` [PATCH 17/17] image.py: add script output to the rootfs log Ed Bartosh
16 siblings, 0 replies; 22+ messages in thread
From: Ed Bartosh @ 2015-08-20 11:56 UTC (permalink / raw)
To: openembedded-core
This variable is going to be used by wic to set partition
size. Setting it in image.py makes it possible for wic to
use it without calculating it again.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
meta/lib/oe/image.py | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/meta/lib/oe/image.py b/meta/lib/oe/image.py
index a7fdefa..7f82748 100644
--- a/meta/lib/oe/image.py
+++ b/meta/lib/oe/image.py
@@ -262,14 +262,16 @@ class Image(ImageDepGraph):
def _write_script(self, type, cmds):
tempdir = self.d.getVar('T', True)
script_name = os.path.join(tempdir, "create_image." + type)
+ rootfs_size = self._get_rootfs_size()
self.d.setVar('img_creation_func', '\n'.join(cmds))
self.d.setVarFlag('img_creation_func', 'func', 1)
self.d.setVarFlag('img_creation_func', 'fakeroot', 1)
+ self.d.setVar('ROOTFS_SIZE', str(rootfs_size))
with open(script_name, "w+") as script:
script.write("%s" % bb.build.shell_trap_code())
- script.write("export ROOTFS_SIZE=%d\n" % self._get_rootfs_size())
+ script.write("export ROOTFS_SIZE=%d\n" % rootfs_size)
bb.data.emit_func('img_creation_func', script, self.d)
script.write("img_creation_func\n")
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 16/17] wic: use bitbake variable ROOTFS_SIZE
2015-08-20 11:56 [PATCH 00/17] Build wic images with bitbake Ed Bartosh
` (14 preceding siblings ...)
2015-08-20 11:56 ` [PATCH 15/17] image.py: set bitbake variable ROOTFS_SIZE Ed Bartosh
@ 2015-08-20 11:56 ` Ed Bartosh
2015-08-20 11:56 ` [PATCH 17/17] image.py: add script output to the rootfs log Ed Bartosh
16 siblings, 0 replies; 22+ messages in thread
From: Ed Bartosh @ 2015-08-20 11:56 UTC (permalink / raw)
To: openembedded-core
If bitbake image is referenced in .ks file and --size is not used
there wic uses ROOTFS_SIZE variable to set minimum partition size.
ROOTFS_SIZE is calculated in meta/lib/oe/image.py when rootfs is
created. The calculation is done using other image parameters:
IMAGE_ROOTFS_SIZE, IMAGE_ROOTFS_ALIGNMENT, IMAGE_OVERHEAD_FACTOR
and IMAGE_ROOTFS_EXTRA_SPACE.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/imager/direct.py | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/scripts/lib/wic/imager/direct.py b/scripts/lib/wic/imager/direct.py
index 57b1335..d68fd2a 100644
--- a/scripts/lib/wic/imager/direct.py
+++ b/scripts/lib/wic/imager/direct.py
@@ -29,6 +29,7 @@ import shutil
from wic import kickstart, msger
from wic.utils import fs_related
+from wic.utils.oe.misc import get_bitbake_var
from wic.utils.partitionedfs import Image
from wic.utils.errors import CreatorError, ImageError
from wic.imager.baseimager import BaseImageCreator
@@ -229,6 +230,19 @@ class DirectImageCreator(BaseImageCreator):
fstab_path = self._write_fstab(self.rootfs_dir.get("ROOTFS_DIR"))
for p in parts:
+ # get rootfs size from bitbake variable if it's not set in .ks file
+ if not p.size:
+ # and if rootfs name is specified for the partition
+ image_name = p.get_rootfs()
+ if image_name:
+ # Bitbake variable ROOTFS_SIZE is calculated in
+ # Image._get_rootfs_size method from meta/lib/oe/image.py
+ # using IMAGE_ROOTFS_SIZE, IMAGE_ROOTFS_ALIGNMENT,
+ # IMAGE_OVERHEAD_FACTOR and IMAGE_ROOTFS_EXTRA_SPACE
+ rsize_bb = get_bitbake_var('ROOTFS_SIZE', image_name)
+ if rsize_bb:
+ # convert from Kb to Mb
+ p.size = int(rsize_bb) / 1024
# need to create the filesystems in order to get their
# sizes before we can add them and do the layout.
# Image.create() actually calls __format_disks() to create
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [PATCH 17/17] image.py: add script output to the rootfs log
2015-08-20 11:56 [PATCH 00/17] Build wic images with bitbake Ed Bartosh
` (15 preceding siblings ...)
2015-08-20 11:56 ` [PATCH 16/17] wic: use " Ed Bartosh
@ 2015-08-20 11:56 ` Ed Bartosh
16 siblings, 0 replies; 22+ messages in thread
From: Ed Bartosh @ 2015-08-20 11:56 UTC (permalink / raw)
To: openembedded-core
Let's add output of image creation script to the bitbake log
as it can contain useful information.
One good example of such an information is wic report about
artifacts and .wks file used for image creation.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
meta/lib/oe/image.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/meta/lib/oe/image.py b/meta/lib/oe/image.py
index 7f82748..6f57df2 100644
--- a/meta/lib/oe/image.py
+++ b/meta/lib/oe/image.py
@@ -11,11 +11,14 @@ def generate_image(arg):
(type, create_img_cmd))
try:
- subprocess.check_output(create_img_cmd, stderr=subprocess.STDOUT)
+ output = subprocess.check_output(create_img_cmd,
+ stderr=subprocess.STDOUT)
except subprocess.CalledProcessError as e:
return("Error: The image creation script '%s' returned %d:\n%s" %
(e.cmd, e.returncode, e.output))
+ bb.note("Script output:\n%s" % output)
+
return None
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread