[wic][PATCH v2 00/20] miscellaneous fixes. poky-conrib:ed/wic/misc

[wic][PATCH v2 00/20] miscellaneous fixes. poky-conrib:ed/wic/misc

[Ed Bartosh]

Hi,

This patchset includes usability fixes, new functionality,
code cleanup, refactoring and 2 new test cases for wic.

The work was done during verification of multi-rootfs EFI images.

Fixes YOCTO #7854 and #7912

Changes in v2:
 - removed global statement in get_bitbake_var. Thanks to Christopher Larson to point that out to me
 - added one indentation fix

Ed Bartosh (20):
  wic: Fix misleading message
  wic: Test rootfs plugin using image recipes
  wic: Test rootfs plugin using rootfs paths
  wic: Refactor getting bitbake variables
  wic: Include mount point into image report
  wic: Remove annoing debug message
  wic: Turn off debug output for 'bitbake -e'
  wic: Refactor prepare_rootfs API
  wic: Rename partition images
  wic: Get rid of useless variable 'image_rootfs'
  wic: Call methods better way
  wic: Refactor prepare_empty_partition API
  wic: Remove duplicated code
  wic: Fix naming conflict
  wic: Add --uuid partition option
  wic: Refactor fstab update code
  wic: Remove __write_partition method
  wic: Fix confusing error message
  wic: Code cleanup: long lines, identation and whitespaces
  wic: Code cleanup: unused imports

 meta/lib/oeqa/selftest/wic.py                      |  27 +++
 .../image/canned-wks/directdisk-multi-rootfs.wks   |  23 +++
 scripts/lib/image/engine.py                        |  38 +---
 scripts/lib/image/help.py                          |   5 +
 scripts/lib/wic/conf.py                            |  28 ++-
 scripts/lib/wic/imager/baseimager.py               |   3 +-
 scripts/lib/wic/imager/direct.py                   |  91 ++++-----
 .../lib/wic/kickstart/custom_commands/micboot.py   |   2 +-
 .../lib/wic/kickstart/custom_commands/partition.py | 209 +++++++--------------
 scripts/lib/wic/msger.py                           |  23 +--
 scripts/lib/wic/plugin.py                          |   2 -
 scripts/lib/wic/pluginbase.py                      |   1 -
 scripts/lib/wic/plugins/imager/direct_plugin.py    |   4 +-
 scripts/lib/wic/plugins/source/bootimg-efi.py      |   1 -
 scripts/lib/wic/plugins/source/bootimg-pcbios.py   |   4 +-
 scripts/lib/wic/plugins/source/rootfs.py           |   9 +-
 .../lib/wic/plugins/source/rootfs_pcbios_ext.py    |   6 +-
 scripts/lib/wic/utils/cmdln.py                     |   1 -
 scripts/lib/wic/utils/oe/misc.py                   |  87 ++++-----
 scripts/lib/wic/utils/partitionedfs.py             |  41 ++--
 scripts/lib/wic/utils/runner.py                    |   2 +-
 scripts/wic                                        |  66 +++----
 22 files changed, 282 insertions(+), 391 deletions(-)
 create mode 100644 scripts/lib/image/canned-wks/directdisk-multi-rootfs.wks

--
Ed

[wic][PATCH v2 01/20] wic: Fix misleading message

[Ed Bartosh]

Due to usage of incorrect variable wic produces strange message
"No image named None found, exiting." when specified canned .wks
doesn't exist.

Fixed by replacing wks_file -> argv[0]

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/wic b/scripts/wic
index f9be972..a39ec95 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -173,7 +173,7 @@ def wic_create_subcommand(args, usage_str):
     if not wks_file.endswith(".wks"):
         wks_file = engine.find_canned_image(scripts_path, wks_file)
         if not wks_file:
-            print "No image named %s found, exiting.  (Use 'wic list images' to list available images, or specify a fully-qualified OE kickstart (.wks) filename)\n" % wks_file
+            print "No image named %s found, exiting.  (Use 'wic list images' to list available images, or specify a fully-qualified OE kickstart (.wks) filename)\n" % args[0]
             sys.exit(1)
 
     image_output_dir = ""
-- 
2.1.4

[wic][PATCH v2 02/20] wic: Test rootfs plugin using image recipes

[Ed Bartosh]

Added canned wks and testcase to create multi-rootfs images referring
bitbake image recipes.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

 create mode 100644 scripts/lib/image/canned-wks/directdisk-multi-rootfs.wks

diff --git a/meta/lib/oeqa/selftest/wic.py b/meta/lib/oeqa/selftest/wic.py
index 5385562..3cfc2ff 100644
--- a/meta/lib/oeqa/selftest/wic.py
+++ b/meta/lib/oeqa/selftest/wic.py
@@ -133,3 +133,13 @@ class Wic(oeSelfTest):
         self.assertEqual(2, runCmd("wic create directdisk "
                                    "--image-name core-image-minimal "
                                    "-c wrong", ignore_status=True).status)
+
+    def test16_rootfs_indirect_recipes(self):
+        """Test usage of rootfs plugin with rootfs recipes"""
+        wks = "directdisk-multi-rootfs"
+        self.assertEqual(0, runCmd("wic create %s "
+                                   "--image-name core-image-minimal "
+                                   "--rootfs rootfs1=core-image-minimal "
+                                   "--rootfs rootfs2=core-image-minimal" \
+                                   % wks).status)
+        self.assertEqual(1, len(glob(self.resultdir + "%s*.direct" % wks)))
diff --git a/scripts/lib/image/canned-wks/directdisk-multi-rootfs.wks b/scripts/lib/image/canned-wks/directdisk-multi-rootfs.wks
new file mode 100644
index 0000000..f3c6df5
--- /dev/null
+++ b/scripts/lib/image/canned-wks/directdisk-multi-rootfs.wks
@@ -0,0 +1,23 @@
+# short-description: Create multi rootfs image using rootfs plugin
+# long-description: Creates a partitioned disk image with two rootfs partitions
+# using rootfs plugin.
+#
+# Partitions can use either
+#   - indirect rootfs references to image recipe(s):
+#     wic create directdisk-multi-indirect-recipes -e core-image-minimal \
+#         --rootfs-dir rootfs1=core-image-minimal
+#         --rootfs-dir rootfs2=core-image-minimal-dev
+#
+#   - or paths to rootfs directories:
+#     wic create directdisk-multi-rootfs \
+#         --rootfs-dir rootfs1=tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/
+#         --rootfs-dir rootfs2=tmp/work/qemux86_64-poky-linux/core-image-minimal-dev/1.0-r0/rootfs/
+#
+#   - or any combinations of -r and --rootfs command line options
+
+part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024
+part / --source rootfs --rootfs-dir=rootfs1 --ondisk sda --fstype=ext3 --label platform --align 1024
+part /rescue --source rootfs --rootfs-dir=rootfs2 --ondisk sda --fstype=ext3 --label secondary --align 1024
+
+bootloader  --timeout=0  --append="rootwait rootfstype=ext3 video=vesafb vga=0x318 console=tty0"
+
-- 
2.1.4

[wic][PATCH v2 03/20] wic: Test rootfs plugin using rootfs paths

[Ed Bartosh]

Added testcase to create multi-rootfs images using rootfs plugin
with paths to rootfs directories in wic command line.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/meta/lib/oeqa/selftest/wic.py b/meta/lib/oeqa/selftest/wic.py
index 3cfc2ff..0c503ef 100644
--- a/meta/lib/oeqa/selftest/wic.py
+++ b/meta/lib/oeqa/selftest/wic.py
@@ -143,3 +143,20 @@ class Wic(oeSelfTest):
                                    "--rootfs rootfs2=core-image-minimal" \
                                    % wks).status)
         self.assertEqual(1, len(glob(self.resultdir + "%s*.direct" % wks)))
+
+    def test17_rootfs_artifacts(self):
+        """Test usage of rootfs plugin with rootfs paths"""
+        vars = dict((var.lower(), get_bb_var(var, 'core-image-minimal')) \
+                        for var in ('STAGING_DATADIR', 'DEPLOY_DIR_IMAGE',
+                                    'STAGING_DIR_NATIVE', 'IMAGE_ROOTFS'))
+        vars['wks'] = "directdisk-multi-rootfs"
+        status = runCmd("wic create %(wks)s "
+                        "-b %(staging_datadir)s "
+                        "-k %(deploy_dir_image)s "
+                        "-n %(staging_dir_native)s "
+                        "--rootfs-dir rootfs1=%(image_rootfs)s "
+                        "--rootfs-dir rootfs2=%(image_rootfs)s" \
+                        % vars).status
+        self.assertEqual(0, status)
+        self.assertEqual(1, len(glob(self.resultdir + \
+                                     "%(wks)s-*.direct" % vars)))
-- 
2.1.4

[wic][PATCH v2 04/20] wic: Refactor getting bitbake variables

[Ed Bartosh]

Wic gets bitbake variables by parsing output of 'bitbake -e' command.

This implementation improves this procedure as it runs 'bitbake -e' only
when API is called and does it only once, i.e. in a "lazy" way. As parsing
results are cached 'bitbake -e' is run only once and results are parsed
only once per requested set of variables.

get_bitbake_var became the only API call. It replaces find_artifacts,
find_artifact, find_bitbake_env_lines, get_bitbake_env_lines,
set_bitbake_env_lines and get_line_val calls making API much more clear.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/image/engine.py b/scripts/lib/image/engine.py
index 47950f8..92dcc5a 100644
--- a/scripts/lib/image/engine.py
+++ b/scripts/lib/image/engine.py
@@ -60,29 +60,6 @@ def verify_build_env():
     return True
 
 
-def find_artifacts(image_name):
-    """
-    Gather the build artifacts for the current image (the image_name
-    e.g. core-image-minimal) for the current MACHINE set in local.conf
-    """
-    bitbake_env_lines = misc.get_bitbake_env_lines()
-
-    rootfs_dir = kernel_dir = bootimg_dir = native_sysroot = ""
-
-    for line in bitbake_env_lines.split('\n'):
-        if misc.get_line_val(line, "IMAGE_ROOTFS"):
-            rootfs_dir = misc.get_line_val(line, "IMAGE_ROOTFS")
-            continue
-        if misc.get_line_val(line, "DEPLOY_DIR_IMAGE"):
-            kernel_dir = misc.get_line_val(line, "DEPLOY_DIR_IMAGE")
-            continue
-        if misc.get_line_val(line, "STAGING_DIR_NATIVE"):
-            native_sysroot = misc.get_line_val(line, "STAGING_DIR_NATIVE")
-            continue
-
-    return (rootfs_dir, kernel_dir, bootimg_dir, native_sysroot)
-
-
 CANNED_IMAGE_DIR = "lib/image/canned-wks" # relative to scripts
 SCRIPTS_CANNED_IMAGE_DIR = "scripts/" + CANNED_IMAGE_DIR
 
diff --git a/scripts/lib/wic/plugins/source/rootfs.py b/scripts/lib/wic/plugins/source/rootfs.py
index 12fbf67..a90712b 100644
--- a/scripts/lib/wic/plugins/source/rootfs.py
+++ b/scripts/lib/wic/plugins/source/rootfs.py
@@ -29,7 +29,7 @@ import os
 
 from wic import msger
 from wic.pluginbase import SourcePlugin
-from wic.utils.oe.misc import find_bitbake_env_lines, find_artifact
+from wic.utils.oe.misc import get_bitbake_var
 
 class RootfsPlugin(SourcePlugin):
     """
@@ -43,12 +43,7 @@ class RootfsPlugin(SourcePlugin):
         if os.path.isdir(rootfs_dir):
             return rootfs_dir
 
-        bitbake_env_lines = find_bitbake_env_lines(rootfs_dir)
-        if not bitbake_env_lines:
-            msg = "Couldn't get bitbake environment, exiting."
-            msger.error(msg)
-
-        image_rootfs_dir = find_artifact(bitbake_env_lines, "IMAGE_ROOTFS")
+        image_rootfs_dir = get_bitbake_var("IMAGE_ROOTFS", rootfs_dir)
         if not os.path.isdir(image_rootfs_dir):
             msg = "No valid artifact IMAGE_ROOTFS from image named"
             msg += " %s has been found at %s, exiting.\n" % \
diff --git a/scripts/lib/wic/plugins/source/rootfs_pcbios_ext.py b/scripts/lib/wic/plugins/source/rootfs_pcbios_ext.py
index 533eaa7..76e7b03 100644
--- a/scripts/lib/wic/plugins/source/rootfs_pcbios_ext.py
+++ b/scripts/lib/wic/plugins/source/rootfs_pcbios_ext.py
@@ -59,11 +59,7 @@ class RootfsPlugin(SourcePlugin):
         if os.path.isdir(rootfs_dir):
             return rootfs_dir
 
-        bitbake_env_lines = misc.find_bitbake_env_lines(rootfs_dir)
-        if not bitbake_env_lines:
-            msger.error("Couldn't get bitbake environment, exiting.")
-
-        image_rootfs_dir = misc.find_artifact(bitbake_env_lines, "IMAGE_ROOTFS")
+        image_rootfs_dir = misc.get_bitbake_var("IMAGE_ROOTFS", rootfs_dir)
         if not os.path.isdir(image_rootfs_dir):
             msg = "No valid artifact IMAGE_ROOTFS from image named"
             msg += " %s has been found at %s, exiting.\n" % \
diff --git a/scripts/lib/wic/utils/oe/misc.py b/scripts/lib/wic/utils/oe/misc.py
index 2f916dd..1de6f46 100644
--- a/scripts/lib/wic/utils/oe/misc.py
+++ b/scripts/lib/wic/utils/oe/misc.py
@@ -25,6 +25,8 @@
 # Tom Zanussi <tom.zanussi (at] linux.intel.com>
 #
 
+from collections import defaultdict
+
 from wic import msger
 from wic.utils import runner
 
@@ -108,62 +110,38 @@ def add_wks_var(key, val):
 
 BOOTDD_EXTRA_SPACE = 16384
 
-__bitbake_env_lines = ""
-
-def set_bitbake_env_lines(bitbake_env_lines):
-    global __bitbake_env_lines
-    __bitbake_env_lines = bitbake_env_lines
+_BITBAKE_VARS = defaultdict(dict)
 
-def get_bitbake_env_lines():
-    return __bitbake_env_lines
-
-def find_bitbake_env_lines(image_name):
-    """
-    If image_name is empty, plugins might still be able to use the
-    environment, so set it regardless.
+def get_bitbake_var(var, image=None):
     """
-    if image_name:
-        bitbake_env_cmd = "bitbake -e %s" % image_name
-    else:
-        bitbake_env_cmd = "bitbake -e"
-    rc, bitbake_env_lines = __exec_cmd(bitbake_env_cmd)
-    if rc != 0:
-        print "Couldn't get '%s' output." % bitbake_env_cmd
-        print "Bitbake failed with error:\n%s\n" % bitbake_env_lines
-        return None
-
-    return bitbake_env_lines
-
-def find_artifact(bitbake_env_lines, variable):
+    Get bitbake variable value lazy way, i.e. run
+    'bitbake -e' only when variable is requested.
     """
-    Gather the build artifact for the current image (the image_name
-    e.g. core-image-minimal) for the current MACHINE set in local.conf
-    """
-    retval = ""
-
-    for line in bitbake_env_lines.split('\n'):
-        if get_line_val(line, variable):
-            retval = get_line_val(line, variable)
-            break
-
-    return retval
+    if image not in _BITBAKE_VARS:
+        # Get bitbake -e output
+        cmd = "bitbake -e"
+        if image:
+            cmd += " %s" % image
+        rc, lines = __exec_cmd(cmd)
+        if rc:
+            print "Couldn't get '%s' output." % cmd
+            print "Bitbake failed with error:\n%s\n" % lines
+            return
+
+        # Parse bitbake -e output
+        for line in lines.split('\n'):
+            if "=" not in line:
+                continue
+            try:
+                key, val = line.split("=")
+            except ValueError:
+                continue
+            key = key.strip()
+            val = val.strip()
+            if key.replace('_', '').isalnum():
+                _BITBAKE_VARS[image][key] = val.strip('"')
 
-def get_line_val(line, key):
-    """
-    Extract the value from the VAR="val" string
-    """
-    if line.startswith(key + "="):
-        stripped_line = line.split('=')[1]
-        stripped_line = stripped_line.replace('\"', '')
-        return stripped_line
-    return None
-
-def get_bitbake_var(key):
-    for line in __bitbake_env_lines.split('\n'):
-        if get_line_val(line, key):
-            val = get_line_val(line, key)
-            return val
-    return None
+    return _BITBAKE_VARS[image].get(var)
 
 def parse_sourceparams(sourceparams):
     """
diff --git a/scripts/wic b/scripts/wic
index a39ec95..b75d122 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -52,7 +52,7 @@ if bitbake_exe:
 else:
     bitbake_main = None
 
-from wic.utils.oe.misc import find_bitbake_env_lines, set_bitbake_env_lines
+from wic.utils.oe.misc import get_bitbake_var
 from wic.utils.errors import WicError
 from image import engine
 from image import help as hlp
@@ -141,12 +141,6 @@ def wic_create_subcommand(args, usage_str):
         else:
             print "Done.\n"
 
-    bitbake_env_lines = find_bitbake_env_lines(options.image_name)
-    if not bitbake_env_lines:
-        print "Couldn't get bitbake environment, exiting."
-        sys.exit(1)
-    set_bitbake_env_lines(bitbake_env_lines)
-
     bootimg_dir = ""
 
     if options.image_name:
@@ -160,9 +154,10 @@ def wic_create_subcommand(args, usage_str):
                             cookerdata.CookerConfiguration()):
                 sys.exit(1)
 
-        (rootfs_dir, kernel_dir, bootimg_dir, native_sysroot) \
-            = engine.find_artifacts(options.image_name)
-
+        rootfs_dir = get_bitbake_var("IMAGE_ROOTFS", options.image_name)
+        kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE", options.image_name)
+        native_sysroot = get_bitbake_var("STAGING_DIR_NATIVE",
+                                         options.image_name)
     else:
         if options.build_rootfs:
             print "Image name is not specified, exiting. (Use -e/--image-name to specify it)\n"
@@ -244,12 +239,6 @@ def wic_list_subcommand(args, usage_str):
 
     (options, args) = parser.parse_args(args)
 
-    bitbake_env_lines = find_bitbake_env_lines(None)
-    if not bitbake_env_lines:
-        print "Couldn't get bitbake environment, exiting."
-        sys.exit(1)
-    set_bitbake_env_lines(bitbake_env_lines)
-
     if not engine.wic_list(args, scripts_path, options.properties_file):
         logging.error("Bad list arguments, exiting\n")
         parser.print_help()
-- 
2.1.4

[wic][PATCH v2 05/20] wic: Include mount point into image report

[Ed Bartosh]

Wic doesn't show any information for the partition if label is not set.
Fixed this by adding mount point to the report.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/wic/imager/direct.py b/scripts/lib/wic/imager/direct.py
index 2ea7e4e..58a9e9d 100644
--- a/scripts/lib/wic/imager/direct.py
+++ b/scripts/lib/wic/imager/direct.py
@@ -351,7 +351,7 @@ class DirectImageCreator(BaseImageCreator):
             if p.mountpoint == '/':
                 str = ':'
             else:
-                str = '["%s"]:' % p.label
+                str = '["%s"]:' % (p.mountpoint or p.label)
             msg += '  ROOTFS_DIR%s%s\n' % (str.ljust(20), p.get_rootfs())
 
         msg += '  BOOTIMG_DIR:                  %s\n' % self.bootimg_dir
-- 
2.1.4

[wic][PATCH v2 06/20] wic: Remove annoing debug message

[Ed Bartosh]

Wic tries to find plugins in every layer and prints a message
'Plugin dir is not a directory or does not exist' if layer
doesn't have plugin dir. It causes a lot of duplicated
messages in the debug output, which makes it hard to find
useful info there.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/wic/plugin.py b/scripts/lib/wic/plugin.py
index fff02c0..d714b6d 100644
--- a/scripts/lib/wic/plugin.py
+++ b/scripts/lib/wic/plugin.py
@@ -74,8 +74,6 @@ class PluginMgr(object):
         path = os.path.abspath(os.path.expanduser(path))
 
         if not os.path.isdir(path):
-            msger.debug("Plugin dir is not a directory or does not exist: %s"\
-                          % path)
             return
 
         if path not in self.plugin_dirs:
-- 
2.1.4

[wic][PATCH v2 07/20] wic: Turn off debug output for 'bitbake -e'

[Ed Bartosh]

Switched debug level to 'normal' to prevent huge 'bitbake -e'
output to go into wic debug output. This should help to make
wic debug info much more clean and easier to read.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/wic/utils/oe/misc.py b/scripts/lib/wic/utils/oe/misc.py
index 1de6f46..2f9f515 100644
--- a/scripts/lib/wic/utils/oe/misc.py
+++ b/scripts/lib/wic/utils/oe/misc.py
@@ -122,7 +122,12 @@ def get_bitbake_var(var, image=None):
         cmd = "bitbake -e"
         if image:
             cmd += " %s" % image
+
+        log_level = msger.get_loglevel()
+        msger.set_loglevel('normal')
         rc, lines = __exec_cmd(cmd)
+        msger.set_loglevel(log_level)
+
         if rc:
             print "Couldn't get '%s' output." % cmd
             print "Bitbake failed with error:\n%s\n" % lines
-- 
2.1.4

[wic][PATCH v2 08/20] wic: Refactor prepare_rootfs API

[Ed Bartosh]

Moved code out of prepare_roots* methods to avoid code duplication.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/wic/kickstart/custom_commands/partition.py b/scripts/lib/wic/kickstart/custom_commands/partition.py
index 324ea69..489ebe3 100644
--- a/scripts/lib/wic/kickstart/custom_commands/partition.py
+++ b/scripts/lib/wic/kickstart/custom_commands/partition.py
@@ -219,34 +219,36 @@ class Wic_PartData(Mic_PartData):
         pseudo += "export PSEUDO_NOSYMLINKEXP=%s;" % p_nosymlinkexp
         pseudo += "%s/usr/bin/pseudo " % native_sysroot
 
+        rootfs = "%s/rootfs_%s.%s" % (cr_workdir, self.label, self.fstype)
+        if os.path.isfile(rootfs):
+            os.remove(rootfs)
+
         if self.fstype.startswith("ext"):
-            return self.prepare_rootfs_ext(cr_workdir, oe_builddir,
+            return self.prepare_rootfs_ext(rootfs, oe_builddir,
                                            rootfs_dir, native_sysroot,
                                            pseudo)
         elif self.fstype.startswith("btrfs"):
-            return self.prepare_rootfs_btrfs(cr_workdir, oe_builddir,
+            return self.prepare_rootfs_btrfs(rootfs, oe_builddir,
                                              rootfs_dir, native_sysroot,
                                              pseudo)
 
         elif self.fstype.startswith("vfat"):
-            return self.prepare_rootfs_vfat(cr_workdir, oe_builddir,
+            return self.prepare_rootfs_vfat(rootfs, oe_builddir,
                                             rootfs_dir, native_sysroot,
                                             pseudo)
         elif self.fstype.startswith("squashfs"):
-            return self.prepare_rootfs_squashfs(cr_workdir, oe_builddir,
+            return self.prepare_rootfs_squashfs(rootfs, oe_builddir,
                                                 rootfs_dir, native_sysroot,
                                                 pseudo)
 
-    def prepare_rootfs_ext(self, cr_workdir, oe_builddir, rootfs_dir,
+    def prepare_rootfs_ext(self, rootfs, oe_builddir, rootfs_dir,
                            native_sysroot, pseudo):
         """
         Prepare content for an ext2/3/4 rootfs partition.
         """
 
         image_rootfs = rootfs_dir
-        rootfs = "%s/rootfs_%s.%s" % (cr_workdir, self.label ,self.fstype)
 
-        os.path.isfile(rootfs) and os.remove(rootfs)
         du_cmd = "du -ks %s" % image_rootfs
         out = exec_cmd(du_cmd)
         actual_rootfs_size = int(out.split()[0])
@@ -285,7 +287,7 @@ class Wic_PartData(Mic_PartData):
 
         return 0
 
-    def prepare_rootfs_btrfs(self, cr_workdir, oe_builddir, rootfs_dir,
+    def prepare_rootfs_btrfs(self, rootfs, oe_builddir, rootfs_dir,
                              native_sysroot, pseudo):
         """
         Prepare content for a btrfs rootfs partition.
@@ -293,9 +295,7 @@ class Wic_PartData(Mic_PartData):
         Currently handles ext2/3/4 and btrfs.
         """
         image_rootfs = rootfs_dir
-        rootfs = "%s/rootfs_%s.%s" % (cr_workdir, self.label, self.fstype)
 
-        os.path.isfile(rootfs) and os.remove(rootfs)
         du_cmd = "du -ks %s" % image_rootfs
         out = exec_cmd(du_cmd)
         actual_rootfs_size = int(out.split()[0])
@@ -330,15 +330,13 @@ class Wic_PartData(Mic_PartData):
         self.size = rootfs_size
         self.source_file = rootfs
 
-    def prepare_rootfs_vfat(self, cr_workdir, oe_builddir, rootfs_dir,
+    def prepare_rootfs_vfat(self, rootfs, oe_builddir, rootfs_dir,
                             native_sysroot, pseudo):
         """
         Prepare content for a vfat rootfs partition.
         """
         image_rootfs = rootfs_dir
-        rootfs = "%s/rootfs_%s.%s" % (cr_workdir, self.label, self.fstype)
 
-        os.path.isfile(rootfs) and os.remove(rootfs)
         du_cmd = "du -bks %s" % image_rootfs
         out = exec_cmd(du_cmd)
         blocks = int(out.split()[0])
@@ -381,15 +379,13 @@ class Wic_PartData(Mic_PartData):
         self.set_size(rootfs_size)
         self.set_source_file(rootfs)
 
-    def prepare_rootfs_squashfs(self, cr_workdir, oe_builddir, rootfs_dir,
+    def prepare_rootfs_squashfs(self, rootfs, oe_builddir, rootfs_dir,
                                 native_sysroot, pseudo):
         """
         Prepare content for a squashfs rootfs partition.
         """
         image_rootfs = rootfs_dir
-        rootfs = "%s/rootfs_%s.%s" % (cr_workdir, self.label ,self.fstype)
 
-        os.path.isfile(rootfs) and os.remove(rootfs)
         squashfs_cmd = "mksquashfs %s %s -noappend" % \
                        (image_rootfs, rootfs)
         exec_native_cmd(pseudo + squashfs_cmd, native_sysroot)
-- 
2.1.4

[wic][PATCH v2 09/20] wic: Rename partition images

[Ed Bartosh]

Renamed partition images into <final image>.p<partition number>
This should make output directory look more organized and easier
to understand.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/wic/utils/partitionedfs.py b/scripts/lib/wic/utils/partitionedfs.py
index e093ec5..d61087a 100644
--- a/scripts/lib/wic/utils/partitionedfs.py
+++ b/scripts/lib/wic/utils/partitionedfs.py
@@ -18,6 +18,7 @@
 # with this program; if not, write to the Free Software Foundation, Inc., 59
 # Temple Place - Suite 330, Boston, MA 02111-1307, USA.
 
+import os
 from wic import msger
 from wic.utils.errors import ImageError
 from wic.utils.oe.misc import exec_cmd, exec_native_cmd
@@ -356,6 +357,7 @@ class Image(object):
         for p in self.partitions:
             self.__write_partition(p['num'], p['source_file'],
                                    p['start'], p['size'], image_file)
+            os.rename(p['source_file'], image_file + '.p%d' % p['num'])
 
     def create(self):
         for dev in self.disks.keys():
-- 
2.1.4

[wic][PATCH v2 10/20] wic: Get rid of useless variable 'image_rootfs'

[Ed Bartosh]

Removed useless variable 'image_rootfs' from 4 prepare_rootfs_* methods.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/wic/kickstart/custom_commands/partition.py b/scripts/lib/wic/kickstart/custom_commands/partition.py
index 489ebe3..e3ae3ce 100644
--- a/scripts/lib/wic/kickstart/custom_commands/partition.py
+++ b/scripts/lib/wic/kickstart/custom_commands/partition.py
@@ -246,10 +246,7 @@ class Wic_PartData(Mic_PartData):
         """
         Prepare content for an ext2/3/4 rootfs partition.
         """
-
-        image_rootfs = rootfs_dir
-
-        du_cmd = "du -ks %s" % image_rootfs
+        du_cmd = "du -ks %s" % rootfs_dir
         out = exec_cmd(du_cmd)
         actual_rootfs_size = int(out.split()[0])
 
@@ -274,7 +271,7 @@ class Wic_PartData(Mic_PartData):
             label_str = "-L %s" % self.label
 
         mkfs_cmd = "mkfs.%s -F %s %s %s -d %s" % \
-            (self.fstype, extra_imagecmd, rootfs, label_str, image_rootfs)
+            (self.fstype, extra_imagecmd, rootfs, label_str, rootfs_dir)
         exec_native_cmd(pseudo + mkfs_cmd, native_sysroot)
 
         # get the rootfs size in the right units for kickstart (kB)
@@ -294,9 +291,7 @@ class Wic_PartData(Mic_PartData):
 
         Currently handles ext2/3/4 and btrfs.
         """
-        image_rootfs = rootfs_dir
-
-        du_cmd = "du -ks %s" % image_rootfs
+        du_cmd = "du -ks %s" % rootfs_dir
         out = exec_cmd(du_cmd)
         actual_rootfs_size = int(out.split()[0])
 
@@ -319,7 +314,7 @@ class Wic_PartData(Mic_PartData):
             label_str = "-L %s" % self.label
 
         mkfs_cmd = "mkfs.%s -b %d -r %s %s %s" % \
-            (self.fstype, rootfs_size * 1024, image_rootfs, label_str, rootfs)
+            (self.fstype, rootfs_size * 1024, rootfs_dir, label_str, rootfs)
         exec_native_cmd(pseudo + mkfs_cmd, native_sysroot)
 
         # get the rootfs size in the right units for kickstart (kB)
@@ -335,9 +330,7 @@ class Wic_PartData(Mic_PartData):
         """
         Prepare content for a vfat rootfs partition.
         """
-        image_rootfs = rootfs_dir
-
-        du_cmd = "du -bks %s" % image_rootfs
+        du_cmd = "du -bks %s" % rootfs_dir
         out = exec_cmd(du_cmd)
         blocks = int(out.split()[0])
 
@@ -365,7 +358,7 @@ class Wic_PartData(Mic_PartData):
         dosfs_cmd = "mkdosfs %s -S 512 -C %s %d" % (label_str, rootfs, blocks)
         exec_native_cmd(dosfs_cmd, native_sysroot)
 
-        mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, image_rootfs)
+        mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
         exec_native_cmd(mcopy_cmd, native_sysroot)
 
         chmod_cmd = "chmod 644 %s" % rootfs
@@ -384,10 +377,8 @@ class Wic_PartData(Mic_PartData):
         """
         Prepare content for a squashfs rootfs partition.
         """
-        image_rootfs = rootfs_dir
-
         squashfs_cmd = "mksquashfs %s %s -noappend" % \
-                       (image_rootfs, rootfs)
+                       (rootfs_dir, rootfs)
         exec_native_cmd(pseudo + squashfs_cmd, native_sysroot)
 
         # get the rootfs size in the right units for kickstart (kB)
-- 
2.1.4

[wic][PATCH v2 11/20] wic: Call methods better way

[Ed Bartosh]

Shortened code by using getattr to obtain a method to call for
prepare_empty_partition_* and prepare_rootfs_* methods.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/wic/kickstart/custom_commands/partition.py b/scripts/lib/wic/kickstart/custom_commands/partition.py
index e3ae3ce..0741bb2 100644
--- a/scripts/lib/wic/kickstart/custom_commands/partition.py
+++ b/scripts/lib/wic/kickstart/custom_commands/partition.py
@@ -160,8 +160,12 @@ class Wic_PartData(Mic_PartData):
                 self.prepare_swap_partition(cr_workdir, oe_builddir,
                                             native_sysroot)
             elif self.fstype:
-                self.prepare_empty_partition(cr_workdir, oe_builddir,
-                                             native_sysroot)
+                for prefix in ("ext", "btrfs", "vfat", "squashfs"):
+                    if self.fstype.startswith(prefix):
+                        method = getattr(self,
+                                         "prepare_empty_partition_" + prefix)
+                        method(cr_workdir, oe_builddir, native_sysroot)
+                        break
             return
 
         plugins = pluginmgr.get_source_plugins()
@@ -223,23 +227,11 @@ class Wic_PartData(Mic_PartData):
         if os.path.isfile(rootfs):
             os.remove(rootfs)
 
-        if self.fstype.startswith("ext"):
-            return self.prepare_rootfs_ext(rootfs, oe_builddir,
-                                           rootfs_dir, native_sysroot,
-                                           pseudo)
-        elif self.fstype.startswith("btrfs"):
-            return self.prepare_rootfs_btrfs(rootfs, oe_builddir,
-                                             rootfs_dir, native_sysroot,
-                                             pseudo)
-
-        elif self.fstype.startswith("vfat"):
-            return self.prepare_rootfs_vfat(rootfs, oe_builddir,
-                                            rootfs_dir, native_sysroot,
-                                            pseudo)
-        elif self.fstype.startswith("squashfs"):
-            return self.prepare_rootfs_squashfs(rootfs, oe_builddir,
-                                                rootfs_dir, native_sysroot,
-                                                pseudo)
+        for prefix in ("ext", "btrfs", "vfat", "squashfs"):
+            if self.fstype.startswith(prefix):
+                method = getattr(self, "prepare_rootfs_" + prefix)
+                return method(rootfs, oe_builddir, rootfs_dir,
+                              native_sysroot, pseudo)
 
     def prepare_rootfs_ext(self, rootfs, oe_builddir, rootfs_dir,
                            native_sysroot, pseudo):
@@ -391,23 +383,6 @@ class Wic_PartData(Mic_PartData):
 
         return 0
 
-    def prepare_empty_partition(self, cr_workdir, oe_builddir, native_sysroot):
-        """
-        Prepare an empty partition.
-        """
-        if self.fstype.startswith("ext"):
-            return self.prepare_empty_partition_ext(cr_workdir, oe_builddir,
-                                                    native_sysroot)
-        elif self.fstype.startswith("btrfs"):
-            return self.prepare_empty_partition_btrfs(cr_workdir, oe_builddir,
-                                                      native_sysroot)
-        elif self.fstype.startswith("vfat"):
-            return self.prepare_empty_partition_vfat(cr_workdir, oe_builddir,
-                                                     native_sysroot)
-        elif self.fstype.startswith("squashfs"):
-            return self.prepare_empty_partition_squashfs(cr_workdir, oe_builddir,
-                                                         native_sysroot)
-
     def prepare_empty_partition_ext(self, cr_workdir, oe_builddir,
                                     native_sysroot):
         """
-- 
2.1.4

[wic][PATCH v2 12/20] wic: Refactor prepare_empty_partition API

[Ed Bartosh]

Moved code out of prepare_empty_partition* methods
to avoid code duplication.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/wic/kickstart/custom_commands/partition.py b/scripts/lib/wic/kickstart/custom_commands/partition.py
index 0741bb2..4e8a6a8 100644
--- a/scripts/lib/wic/kickstart/custom_commands/partition.py
+++ b/scripts/lib/wic/kickstart/custom_commands/partition.py
@@ -160,11 +160,15 @@ class Wic_PartData(Mic_PartData):
                 self.prepare_swap_partition(cr_workdir, oe_builddir,
                                             native_sysroot)
             elif self.fstype:
+                rootfs = "%s/fs_%s.%s" % (cr_workdir, self.label, self.fstype)
+                if os.path.isfile(rootfs):
+                    os.remove(rootfs)
                 for prefix in ("ext", "btrfs", "vfat", "squashfs"):
                     if self.fstype.startswith(prefix):
                         method = getattr(self,
                                          "prepare_empty_partition_" + prefix)
-                        method(cr_workdir, oe_builddir, native_sysroot)
+                        method(rootfs, oe_builddir, native_sysroot)
+                        self.source_file = rootfs
                         break
             return
 
@@ -383,16 +387,13 @@ class Wic_PartData(Mic_PartData):
 
         return 0
 
-    def prepare_empty_partition_ext(self, cr_workdir, oe_builddir,
+    def prepare_empty_partition_ext(self, rootfs, oe_builddir,
                                     native_sysroot):
         """
         Prepare an empty ext2/3/4 partition.
         """
-        fs = "%s/fs_%s.%s" % (cr_workdir, self.label, self.fstype)
-
-        os.path.isfile(fs) and os.remove(fs)
         dd_cmd = "dd if=/dev/zero of=%s bs=1k seek=%d count=0" % \
-            (fs, self.size)
+            (rootfs, self.size)
         exec_cmd(dd_cmd)
 
         extra_imagecmd = "-i 8192"
@@ -402,23 +403,16 @@ class Wic_PartData(Mic_PartData):
             label_str = "-L %s" % self.label
 
         mkfs_cmd = "mkfs.%s -F %s %s %s" % \
-            (self.fstype, extra_imagecmd, label_str, fs)
+            (self.fstype, extra_imagecmd, label_str, rootfs)
         exec_native_cmd(mkfs_cmd, native_sysroot)
 
-        self.source_file = fs
-
-        return 0
-
-    def prepare_empty_partition_btrfs(self, cr_workdir, oe_builddir,
+    def prepare_empty_partition_btrfs(self, rootfs, oe_builddir,
                                       native_sysroot):
         """
         Prepare an empty btrfs partition.
         """
-        fs = "%s/fs_%s.%s" % (cr_workdir, self.label, self.fstype)
-
-        os.path.isfile(fs) and os.remove(fs)
         dd_cmd = "dd if=/dev/zero of=%s bs=1k seek=%d count=0" % \
-            (fs, self.size)
+            (rootfs, self.size)
         exec_cmd(dd_cmd)
 
         label_str = ""
@@ -426,21 +420,14 @@ class Wic_PartData(Mic_PartData):
             label_str = "-L %s" % self.label
 
         mkfs_cmd = "mkfs.%s -b %d %s %s" % \
-            (self.fstype, self.size * 1024, label_str, fs)
+            (self.fstype, self.size * 1024, label_str, rootfs)
         exec_native_cmd(mkfs_cmd, native_sysroot)
 
-        self.source_file = fs
-
-        return 0
-
-    def prepare_empty_partition_vfat(self, cr_workdir, oe_builddir,
+    def prepare_empty_partition_vfat(self, rootfs, oe_builddir,
                                      native_sysroot):
         """
         Prepare an empty vfat partition.
         """
-        fs = "%s/fs_%s.%s" % (cr_workdir, self.label, self.fstype)
-        os.path.isfile(fs) and os.remove(fs)
-
         blocks = self.size
 
         label_str = "-n boot"
@@ -453,10 +440,6 @@ class Wic_PartData(Mic_PartData):
         chmod_cmd = "chmod 644 %s" % fs
         exec_cmd(chmod_cmd)
 
-        self.source_file = fs
-
-        return 0
-
     def prepare_empty_partition_squashfs(self, cr_workdir, oe_builddir,
                                          native_sysroot):
         """
@@ -484,9 +467,6 @@ class Wic_PartData(Mic_PartData):
         fs_size = out.split()[0]
 
         self.size = fs_size
-        self.source_file = fs
-
-        return 0
 
     def prepare_swap_partition(self, cr_workdir, oe_builddir, native_sysroot):
         """
@@ -505,10 +485,6 @@ class Wic_PartData(Mic_PartData):
         mkswap_cmd = "mkswap %s -U %s %s" % (label_str, str(uuid.uuid1()), fs)
         exec_native_cmd(mkswap_cmd, native_sysroot)
 
-        self.source_file = fs
-
-        return 0
-
 class Wic_Partition(Mic_Partition):
     removedKeywords = Mic_Partition.removedKeywords
     removedAttrs = Mic_Partition.removedAttrs
-- 
2.1.4

[wic][PATCH v2 13/20] wic: Remove duplicated code

[Ed Bartosh]

Moved duplicated code of geting rootfs size
out of prepare_rootfs* methods.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/wic/kickstart/custom_commands/partition.py b/scripts/lib/wic/kickstart/custom_commands/partition.py
index 4e8a6a8..72f617a 100644
--- a/scripts/lib/wic/kickstart/custom_commands/partition.py
+++ b/scripts/lib/wic/kickstart/custom_commands/partition.py
@@ -234,8 +234,16 @@ class Wic_PartData(Mic_PartData):
         for prefix in ("ext", "btrfs", "vfat", "squashfs"):
             if self.fstype.startswith(prefix):
                 method = getattr(self, "prepare_rootfs_" + prefix)
-                return method(rootfs, oe_builddir, rootfs_dir,
-                              native_sysroot, pseudo)
+                method(rootfs, oe_builddir, rootfs_dir, native_sysroot, pseudo)
+
+                self.source_file = rootfs
+
+                # get the rootfs size in the right units for kickstart (kB)
+                du_cmd = "du -Lbks %s" % rootfs
+                out = exec_cmd(du_cmd)
+                self.size = out.split()[0]
+
+                break
 
     def prepare_rootfs_ext(self, rootfs, oe_builddir, rootfs_dir,
                            native_sysroot, pseudo):
@@ -270,16 +278,6 @@ class Wic_PartData(Mic_PartData):
             (self.fstype, extra_imagecmd, rootfs, label_str, rootfs_dir)
         exec_native_cmd(pseudo + mkfs_cmd, native_sysroot)
 
-        # get the rootfs size in the right units for kickstart (kB)
-        du_cmd = "du -Lbks %s" % rootfs
-        out = exec_cmd(du_cmd)
-        rootfs_size = out.split()[0]
-
-        self.size = rootfs_size
-        self.source_file = rootfs
-
-        return 0
-
     def prepare_rootfs_btrfs(self, rootfs, oe_builddir, rootfs_dir,
                              native_sysroot, pseudo):
         """
@@ -313,14 +311,6 @@ class Wic_PartData(Mic_PartData):
             (self.fstype, rootfs_size * 1024, rootfs_dir, label_str, rootfs)
         exec_native_cmd(pseudo + mkfs_cmd, native_sysroot)
 
-        # get the rootfs size in the right units for kickstart (kB)
-        du_cmd = "du -Lbks %s" % rootfs
-        out = exec_cmd(du_cmd)
-        rootfs_size = out.split()[0]
-
-        self.size = rootfs_size
-        self.source_file = rootfs
-
     def prepare_rootfs_vfat(self, rootfs, oe_builddir, rootfs_dir,
                             native_sysroot, pseudo):
         """
@@ -360,14 +350,6 @@ class Wic_PartData(Mic_PartData):
         chmod_cmd = "chmod 644 %s" % rootfs
         exec_cmd(chmod_cmd)
 
-        # get the rootfs size in the right units for kickstart (kB)
-        du_cmd = "du -Lbks %s" % rootfs
-        out = exec_cmd(du_cmd)
-        rootfs_size = out.split()[0]
-
-        self.set_size(rootfs_size)
-        self.set_source_file(rootfs)
-
     def prepare_rootfs_squashfs(self, rootfs, oe_builddir, rootfs_dir,
                                 native_sysroot, pseudo):
         """
@@ -377,16 +359,6 @@ class Wic_PartData(Mic_PartData):
                        (rootfs_dir, rootfs)
         exec_native_cmd(pseudo + squashfs_cmd, native_sysroot)
 
-        # get the rootfs size in the right units for kickstart (kB)
-        du_cmd = "du -Lbks %s" % rootfs
-        out = exec_cmd(du_cmd)
-        rootfs_size = out.split()[0]
-
-        self.size = rootfs_size
-        self.source_file = rootfs
-
-        return 0
-
     def prepare_empty_partition_ext(self, rootfs, oe_builddir,
                                     native_sysroot):
         """
-- 
2.1.4

[wic][PATCH v2 14/20] wic: Fix naming conflict

[Ed Bartosh]

Image file name is not unique for the partitions without label.
This causes image being rewritten and used as a source for all
partitions without label. Wic produces broken or incorrect result
images because of that.

Added wks line number to the image name to make it unique.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/wic/kickstart/custom_commands/partition.py b/scripts/lib/wic/kickstart/custom_commands/partition.py
index 72f617a..874b207 100644
--- a/scripts/lib/wic/kickstart/custom_commands/partition.py
+++ b/scripts/lib/wic/kickstart/custom_commands/partition.py
@@ -160,7 +160,8 @@ class Wic_PartData(Mic_PartData):
                 self.prepare_swap_partition(cr_workdir, oe_builddir,
                                             native_sysroot)
             elif self.fstype:
-                rootfs = "%s/fs_%s.%s" % (cr_workdir, self.label, self.fstype)
+                rootfs = "%s/fs_%s.%s.%s" % (cr_workdir, self.label,
+                                             self.lineno, self.fstype)
                 if os.path.isfile(rootfs):
                     os.remove(rootfs)
                 for prefix in ("ext", "btrfs", "vfat", "squashfs"):
@@ -227,7 +228,8 @@ class Wic_PartData(Mic_PartData):
         pseudo += "export PSEUDO_NOSYMLINKEXP=%s;" % p_nosymlinkexp
         pseudo += "%s/usr/bin/pseudo " % native_sysroot
 
-        rootfs = "%s/rootfs_%s.%s" % (cr_workdir, self.label, self.fstype)
+        rootfs = "%s/rootfs_%s.%s.%s" % (cr_workdir, self.label,
+                                         self.lineno, self.fstype)
         if os.path.isfile(rootfs):
             os.remove(rootfs)
 
-- 
2.1.4

[wic][PATCH v2 15/20] wic: Add --uuid partition option

[Ed Bartosh]

Added --uuid option to the configuration of wks parser.
This option specifies partition UUID. The code to process
it is already in place. It was implemented for --use-uuid
option.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/image/help.py b/scripts/lib/image/help.py
index c8b2fab..0666d80 100644
--- a/scripts/lib/image/help.py
+++ b/scripts/lib/image/help.py
@@ -787,6 +787,11 @@ DESCRIPTION
                      random globally unique identifier (GUID) for the partition
                      and use it in bootloader configuration to specify root partition.
 
+         --uuid: This option is specific to wic. It specifies partition UUID.
+                 It's useful if preconfigured partition UUID is added to kernel command line
+                 in bootloader configuration before running wic. In this case .wks file can
+                 be generated or modified to set preconfigured parition UUID using this option.
+
     * bootloader
 
       This command allows the user to specify various bootloader
diff --git a/scripts/lib/wic/kickstart/custom_commands/partition.py b/scripts/lib/wic/kickstart/custom_commands/partition.py
index 874b207..fea4d6d 100644
--- a/scripts/lib/wic/kickstart/custom_commands/partition.py
+++ b/scripts/lib/wic/kickstart/custom_commands/partition.py
@@ -53,7 +53,7 @@ class Wic_PartData(Mic_PartData):
         self.extra_space = kwargs.get("extra-space", "10M")
         self.overhead_factor = kwargs.get("overhead-factor", 1.3)
         self._use_uuid = False
-        self.uuid = None
+        self.uuid = kwargs.get("uuid", None)
         self.use_uuid = kwargs.get("use-uuid", False)
         self.source_file = ""
         self.size = 0
@@ -71,6 +71,8 @@ class Wic_PartData(Mic_PartData):
             retval += " --no-table"
         if self.use_uuid:
             retval += " --use-uuid"
+        if self.uuid:
+            retval += " --uuid=%s" % self.uuid
         retval += " --extra-space=%s" % self.extra_space
         retval += " --overhead-factor=%f" % self.overhead_factor
 
@@ -491,5 +493,6 @@ class Wic_Partition(Mic_Partition):
                       nargs=1, default=1.3)
         op.add_option("--use-uuid", dest="use_uuid", action="store_true",
                       default=False)
+        op.add_option("--uuid")
 
         return op
-- 
2.1.4

[wic][PATCH v2 16/20] wic: Refactor fstab update code

[Ed Bartosh]

Made the code to backup and restore fstab only if it's modified.

Cleaned up the code. Made it more pythonic.

Improved code readability by moving code from several tiny
methods into one place.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/wic/imager/direct.py b/scripts/lib/wic/imager/direct.py
index 58a9e9d..561c396 100644
--- a/scripts/lib/wic/imager/direct.py
+++ b/scripts/lib/wic/imager/direct.py
@@ -90,74 +90,50 @@ class DirectImageCreator(BaseImageCreator):
                     return realnum + 1
                 return realnum
 
-    def __write_fstab(self, image_rootfs):
+    def _write_fstab(self, image_rootfs):
         """overriden to generate fstab (temporarily) in rootfs. This is called
         from _create, make sure it doesn't get called from
         BaseImage.create()
         """
-        if image_rootfs is None:
-            return None
+        if not image_rootfs:
+            return
+
+        fstab_path = image_rootfs + "/etc/fstab"
+        if not os.path.isfile(fstab_path):
+            return
 
-        fstab = image_rootfs + "/etc/fstab"
-        if not os.path.isfile(fstab):
-            return None
+        with open(fstab_path) as fstab:
+            fstab_lines = fstab.readlines()
 
-        parts = self._get_parts()
+        if self._update_fstab(fstab_lines, self._get_parts()):
+            shutil.copyfile(fstab_path, fstab_path + ".orig")
 
-        self._save_fstab(fstab)
-        fstab_lines = self._get_fstab(fstab, parts)
-        self._update_fstab(fstab_lines, parts)
-        self._write_fstab(fstab, fstab_lines)
+            with open(fstab_path, "w") as fstab:
+                fstab.writelines(fstab_lines)
 
-        return fstab
+            return fstab_path
 
     def _update_fstab(self, fstab_lines, parts):
         """Assume partition order same as in wks"""
-        for num, p in enumerate(parts, 1):
+        updated = False
+        for num, part in enumerate(parts, 1):
             pnum = self.__get_part_num(num, parts)
-            if not p.mountpoint or p.mountpoint == "/" or p.mountpoint == "/boot" or pnum == 0:
+            if not pnum or not part.mountpoint \
+               or part.mountpoint in ("/", "/boot"):
                 continue
 
-            part = ''
             # mmc device partitions are named mmcblk0p1, mmcblk0p2..
-            if p.disk.startswith('mmcblk'):
-                part = 'p'
-
-            device_name = "/dev/" + p.disk + part + str(pnum)
-
-            opts = "defaults"
-            if p.fsopts:
-                opts = p.fsopts
-
-            fstab_entry = device_name + "\t" + \
-                          p.mountpoint + "\t" + \
-                          p.fstype + "\t" + \
-                          opts + "\t0\t0\n"
-            fstab_lines.append(fstab_entry)
-
-    def _write_fstab(self, fstab, fstab_lines):
-        fstab = open(fstab, "w")
-        for line in fstab_lines:
-            fstab.write(line)
-        fstab.close()
-
-    def _save_fstab(self, fstab):
-        """Save the current fstab in rootfs"""
-        shutil.copyfile(fstab, fstab + ".orig")
-
-    def _restore_fstab(self, fstab):
-        """Restore the saved fstab in rootfs"""
-        if fstab is None:
-            return
-        shutil.move(fstab + ".orig", fstab)
+            prefix = 'p' if  part.disk.startswith('mmcblk') else ''
+            device_name = "/dev/%s%s%d" % (part.disk, prefix, pnum)
+
+            opts = part.fsopts if part.fsopts else "defaults"
+            line = "\t".join([device_name, part.mountpoint, part.fstype,
+                              opts, "0", "0"]) + "\n"
 
-    def _get_fstab(self, fstab, parts):
-        """Return the desired contents of /etc/fstab."""
-        f = open(fstab, "r")
-        fstab_contents = f.readlines()
-        f.close()
+            fstab_lines.append(line)
+            updated = True
 
-        return fstab_contents
+        return updated
 
     def set_bootimg_dir(self, bootimg_dir):
         """
@@ -250,7 +226,7 @@ class DirectImageCreator(BaseImageCreator):
             if not self.ks.handler.bootloader.source and p.mountpoint == "/boot":
                 self.ks.handler.bootloader.source = p.source
 
-        fstab = self.__write_fstab(self.rootfs_dir.get("ROOTFS_DIR"))
+        fstab_path = self._write_fstab(self.rootfs_dir.get("ROOTFS_DIR"))
 
         for p in parts:
             # need to create the filesystems in order to get their
@@ -277,7 +253,8 @@ class DirectImageCreator(BaseImageCreator):
                                        part_type=p.part_type,
                                        uuid=p.uuid)
 
-        self._restore_fstab(fstab)
+        if fstab_path:
+            shutil.move(fstab_path + ".orig", fstab_path)
 
         self.__image.layout_partitions(self.ptable_format)
 
-- 
2.1.4

[wic][PATCH v2 17/20] wic: Remove __write_partition method

[Ed Bartosh]

Moved code of __write_partition to 'assemble' method.
This way it should be more readable.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/wic/utils/partitionedfs.py b/scripts/lib/wic/utils/partitionedfs.py
index d61087a..ca4b1f0 100644
--- a/scripts/lib/wic/utils/partitionedfs.py
+++ b/scripts/lib/wic/utils/partitionedfs.py
@@ -334,30 +334,24 @@ class Image(object):
                 except:
                     pass
 
-    def __write_partition(self, num, source_file, start, size, image_file):
-        """
-        Install source_file contents into a partition.
-        """
-        if not source_file: # nothing to write
-            return
-
-        # Start is included in the size so need to substract one from the end.
-        end = start + size - 1
-        msger.debug("Installed %s in partition %d, sectors %d-%d, "
-                    "size %d sectors" % (source_file, num, start, end, size))
-
-        dd_cmd = "dd if=%s of=%s bs=%d seek=%d count=%d conv=notrunc" % \
-            (source_file, image_file, self.sector_size, start, size)
-        exec_cmd(dd_cmd)
-
-
     def assemble(self, image_file):
         msger.debug("Installing partitions")
 
-        for p in self.partitions:
-            self.__write_partition(p['num'], p['source_file'],
-                                   p['start'], p['size'], image_file)
-            os.rename(p['source_file'], image_file + '.p%d' % p['num'])
+        for part in self.partitions:
+            source = part['source_file']
+            if source:
+                # install source_file contents into a partition
+                cmd = "dd if=%s of=%s bs=%d seek=%d count=%d conv=notrunc" % \
+                      (source, image_file, self.sector_size,
+                       part['start'], part['size'])
+                exec_cmd(cmd)
+
+                msger.debug("Installed %s in partition %d, sectors %d-%d, "
+                            "size %d sectors" % \
+                            (source, part['num'], part['start'],
+                             part['start'] + part['size'] - 1, part['size']))
+
+                os.rename(source, image_file + '.p%d' % part['num'])
 
     def create(self):
         for dev in self.disks.keys():
-- 
2.1.4

[wic][PATCH v2 18/20] wic: Fix confusing error message

[Ed Bartosh]

Wic throws this message when any of the build artifacts are
not provided:
  Build artifacts not completely specified, exiting.
    (Use 'wic -e' or 'wic -r -b -k -n' to specify artifacts)
It was not clear which artifact was not specified.

Reworked the code to specify list of missed artifacts.
Now the message looks like this:
  The following build artifacts are not specified:
    bootimg-dir, kernel-dir, native-sysroot

[YOCTO #7912]

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/wic b/scripts/wic
index b75d122..ac272c6 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -122,13 +122,18 @@ def wic_create_subcommand(args, usage_str):
         logging.error("Can't build roofs as bitbake is not in the $PATH")
         sys.exit(1)
 
-    if not options.image_name and not (options.rootfs_dir and
-                                       options.bootimg_dir and
-                                       options.kernel_dir and
-                                       options.native_sysroot):
-        print "Build artifacts not completely specified, exiting."
-        print "  (Use 'wic -e' or 'wic -r -b -k -n' to specify artifacts)"
-        sys.exit(1)
+    if not options.image_name:
+        missed = []
+        for val, opt in [(options.rootfs_dir, 'rootfs-dir'),
+                         (options.bootimg_dir, 'bootimg-dir'),
+                         (options.kernel_dir, 'kernel-dir'),
+                         (options.native_sysroot, 'native-sysroot')]:
+            if not val:
+                missed.append(opt)
+        if missed:
+            print "The following build artifacts are not specified:"
+            print "  " + ", ".join(missed)
+            sys.exit(1)
 
     if not options.image_name:
         options.build_check = False
-- 
2.1.4

[wic][PATCH v2 19/20] wic: Code cleanup: long lines, identation and whitespaces

[Ed Bartosh]

Fixed pylint warnings bad-continuation, bad-continuation and
line-too-long.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/image/engine.py b/scripts/lib/image/engine.py
index 92dcc5a..0dc7920 100644
--- a/scripts/lib/image/engine.py
+++ b/scripts/lib/image/engine.py
@@ -236,7 +236,10 @@ def wic_list(args, scripts_path, properties_file):
             wks_file = args[0]
             fullpath = find_canned_image(scripts_path, wks_file)
             if not fullpath:
-                print "No image named %s found, exiting.  (Use 'wic list images' to list available images, or specify a fully-qualified OE kickstart (.wks) filename)\n" % wks_file
+                print "No image named %s found, exiting. "\
+                      "(Use 'wic list images' to list available images, or "\
+                      "specify a fully-qualified OE kickstart (.wks) "\
+                      "filename)\n" % wks_file
                 sys.exit(1)
             list_canned_image_help(scripts_path, fullpath)
             return True
diff --git a/scripts/lib/wic/conf.py b/scripts/lib/wic/conf.py
index ff6e2ca..cecf396 100644
--- a/scripts/lib/wic/conf.py
+++ b/scripts/lib/wic/conf.py
@@ -30,20 +30,18 @@ def get_siteconf():
     return scripts_path + "/lib/image/config/wic.conf"
 
 class ConfigMgr(object):
-    DEFAULTS = {'common': {
-                    "distro_name": "Default Distribution",
-                    "plugin_dir": "/usr/lib/wic/plugins", # TODO use prefix also?
-                },
-                'create': {
-                    "tmpdir": '/var/tmp/wic',
-                    "outdir": './wic-output',
-
-                    "release": None,
-                    "logfile": None,
-                    "name_prefix": None,
-                    "name_suffix": None,
-                },
-               }
+    DEFAULTS = {
+        'common': {
+            "distro_name": "Default Distribution",
+            "plugin_dir": "/usr/lib/wic/plugins"}, # TODO use prefix also?
+        'create': {
+            "tmpdir": '/var/tmp/wic',
+            "outdir": './wic-output',
+            "release": None,
+            "logfile": None,
+            "name_prefix": None,
+            "name_suffix": None}
+        }
 
     # make the manager class as singleton
     _instance = None
diff --git a/scripts/lib/wic/imager/direct.py b/scripts/lib/wic/imager/direct.py
index 561c396..e2f8058 100644
--- a/scripts/lib/wic/imager/direct.py
+++ b/scripts/lib/wic/imager/direct.py
@@ -179,8 +179,8 @@ class DirectImageCreator(BaseImageCreator):
 
             if parts[i].mountpoint and not parts[i].fstype:
                 raise CreatorError("Failed to create disks, no --fstype "
-                                    "specified for partition with mountpoint "
-                                    "'%s' in the ks file")
+                                   "specified for partition with mountpoint "
+                                   "'%s' in the ks file")
 
             self._disk_names.append(disk_name)
 
diff --git a/scripts/lib/wic/kickstart/custom_commands/micboot.py b/scripts/lib/wic/kickstart/custom_commands/micboot.py
index 358b0ea..1fa282b 100644
--- a/scripts/lib/wic/kickstart/custom_commands/micboot.py
+++ b/scripts/lib/wic/kickstart/custom_commands/micboot.py
@@ -27,7 +27,7 @@ class Mic_Bootloader(F8_Bootloader):
                  forceLBA=False, location="", md5pass="", password="",
                  upgrade=False, menus=""):
         F8_Bootloader.__init__(self, writePriority, appendLine, driveorder,
-                                forceLBA, location, md5pass, password, upgrade)
+                               forceLBA, location, md5pass, password, upgrade)
 
         self.menus = ""
         self.ptable = "msdos"
diff --git a/scripts/lib/wic/kickstart/custom_commands/partition.py b/scripts/lib/wic/kickstart/custom_commands/partition.py
index fea4d6d..045b290 100644
--- a/scripts/lib/wic/kickstart/custom_commands/partition.py
+++ b/scripts/lib/wic/kickstart/custom_commands/partition.py
@@ -157,7 +157,9 @@ class Wic_PartData(Mic_PartData):
 
         if not self.source:
             if not self.size:
-                msger.error("The %s partition has a size of zero.  Please specify a non-zero --size for that partition." % self.mountpoint)
+                msger.error("The %s partition has a size of zero.  Please "
+                            "specify a non-zero --size for that partition." % \
+                            self.mountpoint)
             if self.fstype and self.fstype == "swap":
                 self.prepare_swap_partition(cr_workdir, oe_builddir,
                                             native_sysroot)
@@ -178,9 +180,14 @@ class Wic_PartData(Mic_PartData):
         plugins = pluginmgr.get_source_plugins()
 
         if self.source not in plugins:
-            msger.error("The '%s' --source specified for %s doesn't exist.\n\tSee 'wic list source-plugins' for a list of available --sources.\n\tSee 'wic help source-plugins' for details on adding a new source plugin." % (self.source, self.mountpoint))
-
-        self._source_methods = pluginmgr.get_source_plugin_methods(self.source, partition_methods)
+            msger.error("The '%s' --source specified for %s doesn't exist.\n\t"
+                        "See 'wic list source-plugins' for a list of available"
+                        " --sources.\n\tSee 'wic help source-plugins' for "
+                        "details on adding a new source plugin." % \
+                        (self.source, self.mountpoint))
+
+        self._source_methods = pluginmgr.get_source_plugin_methods(\
+                                   self.source, partition_methods)
         self._source_methods["do_configure_partition"](self, self.sourceparams_dict,
                                                        cr, cr_workdir,
                                                        oe_builddir,
@@ -466,9 +473,10 @@ class Wic_Partition(Mic_Partition):
     removedAttrs = Mic_Partition.removedAttrs
 
     def _getParser(self):
-        def overhead_cb (option, opt_str, value, parser):
+        def overhead_cb(option, opt_str, value, parser):
             if value < 1:
-                raise OptionValueError("Option %s: invalid value: %r" % (option, value))
+                raise OptionValueError("Option %s: invalid value: %r" % \
+                                       (option, value))
             setattr(parser.values, option.dest, value)
 
         op = Mic_Partition._getParser(self)
diff --git a/scripts/lib/wic/msger.py b/scripts/lib/wic/msger.py
index 35b705b..1b60980 100644
--- a/scripts/lib/wic/msger.py
+++ b/scripts/lib/wic/msger.py
@@ -16,7 +16,8 @@
 # with this program; if not, write to the Free Software Foundation, Inc., 59
 # Temple Place - Suite 330, Boston, MA 02111-1307, USA.
 
-import os,sys
+import os
+import sys
 import re
 import time
 
@@ -37,8 +38,8 @@ __ALL__ = ['set_mode',
 # COLORs in ANSI
 INFO_COLOR = 32 # green
 WARN_COLOR = 33 # yellow
-ERR_COLOR  = 31 # red
-ASK_COLOR  = 34 # blue
+ERR_COLOR = 31 # red
+ASK_COLOR = 34 # blue
 NO_COLOR = 0
 
 PREFIX_RE = re.compile('^<(.*?)>\s*(.*)', re.S)
@@ -47,12 +48,12 @@ INTERACTIVE = True
 
 LOG_LEVEL = 1
 LOG_LEVELS = {
-                'quiet': 0,
-                'normal': 1,
-                'verbose': 2,
-                'debug': 3,
-                'never': 4,
-             }
+    'quiet': 0,
+    'normal': 1,
+    'verbose': 2,
+    'debug': 3,
+    'never': 4,
+}
 
 LOG_FILE_FP = None
 LOG_CONTENT = ''
@@ -75,7 +76,7 @@ def _general_print(head, color, msg=None, stream=None, level='normal'):
 
     errormsg = ''
     if CATCHERR_BUFFILE_FD > 0:
-        size = os.lseek(CATCHERR_BUFFILE_FD , 0, os.SEEK_END)
+        size = os.lseek(CATCHERR_BUFFILE_FD, 0, os.SEEK_END)
         os.lseek(CATCHERR_BUFFILE_FD, 0, os.SEEK_SET)
         errormsg = os.read(CATCHERR_BUFFILE_FD, size)
         os.ftruncate(CATCHERR_BUFFILE_FD, 0)
@@ -158,7 +159,7 @@ def _split_msg(head, msg):
     return head, msg
 
 def get_loglevel():
-    return (k for k,v in LOG_LEVELS.items() if v==LOG_LEVEL).next()
+    return (k for k, v in LOG_LEVELS.items() if v == LOG_LEVEL).next()
 
 def set_loglevel(level):
     global LOG_LEVEL
diff --git a/scripts/lib/wic/plugins/source/bootimg-pcbios.py b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
index 6ca68f6..721c524 100644
--- a/scripts/lib/wic/plugins/source/bootimg-pcbios.py
+++ b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
@@ -56,7 +56,9 @@ class BootimgPcbiosPlugin(SourcePlugin):
             msger.error("Unsupported partition table: %s" % cr.ptable_format)
 
         if not os.path.exists(mbrfile):
-            msger.error("Couldn't find %s.  If using the -e option, do you have the right MACHINE set in local.conf?  If not, is the bootimg_dir path correct?" % mbrfile)
+            msger.error("Couldn't find %s.  If using the -e option, do you "
+                        "have the right MACHINE set in local.conf?  If not, "
+                        "is the bootimg_dir path correct?" % mbrfile)
 
         full_path = cr._full_path(workdir, disk_name, "direct")
         msger.debug("Installing MBR on disk %s as %s with size %s bytes" \
diff --git a/scripts/lib/wic/utils/partitionedfs.py b/scripts/lib/wic/utils/partitionedfs.py
index ca4b1f0..2f884a3 100644
--- a/scripts/lib/wic/utils/partitionedfs.py
+++ b/scripts/lib/wic/utils/partitionedfs.py
@@ -302,9 +302,8 @@ class Image(object):
                 msger.debug("partition %d: set UUID to %s" % \
                             (p['num'], p['uuid']))
                 exec_native_cmd("sgdisk --partition-guid=%d:%s %s" % \
-                                         (p['num'], p['uuid'],
-                                          d['disk'].device),
-                                        self.native_sysroot)
+                                (p['num'], p['uuid'], d['disk'].device),
+                                self.native_sysroot)
 
             if p['boot']:
                 flag_name = "legacy_boot" if d['ptable_format'] == 'gpt' else "boot"
diff --git a/scripts/lib/wic/utils/runner.py b/scripts/lib/wic/utils/runner.py
index 2ae9f41..8d48f68 100644
--- a/scripts/lib/wic/utils/runner.py
+++ b/scripts/lib/wic/utils/runner.py
@@ -89,7 +89,7 @@ def show(cmdln_or_args):
     else:
         cmd = cmdln_or_args
 
-    msg =  'running command: "%s"' % cmd
+    msg = 'running command: "%s"' % cmd
     if out: out = out.strip()
     if out:
         msg += ', with output::'
diff --git a/scripts/wic b/scripts/wic
index ac272c6..9435ce0 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -90,18 +90,24 @@ def wic_create_subcommand(args, usage_str):
     parser = optparse.OptionParser(usage=usage_str)
 
     parser.add_option("-o", "--outdir", dest="outdir",
-                      action="store", help="name of directory to create image in")
+                      help="name of directory to create image in")
     parser.add_option("-e", "--image-name", dest="image_name",
-                      action="store", help="name of the image to use the artifacts from e.g. core-image-sato")
-    parser.add_option("-r", "--rootfs-dir", dest="rootfs_dir",
-                      action="callback", callback=callback_rootfs_dir, type="string",
-                      help="path to the /rootfs dir to use as the .wks rootfs source")
+                      help="name of the image to use the artifacts from "
+                           "e.g. core-image-sato")
+    parser.add_option("-r", "--rootfs-dir", dest="rootfs_dir", type="string",
+                      action="callback", callback=callback_rootfs_dir,
+                      help="path to the /rootfs dir to use as the "
+                           ".wks rootfs source")
     parser.add_option("-b", "--bootimg-dir", dest="bootimg_dir",
-                      action="store", help="path to the dir containing the boot artifacts (e.g. /EFI or /syslinux dirs) to use as the .wks bootimg source")
+                      help="path to the dir containing the boot artifacts "
+                           "(e.g. /EFI or /syslinux dirs) to use as the "
+                           ".wks bootimg source")
     parser.add_option("-k", "--kernel-dir", dest="kernel_dir",
-                      action="store", help="path to the dir containing the kernel to use in the .wks bootimg")
+                      help="path to the dir containing the kernel to use "
+                           "in the .wks bootimg")
     parser.add_option("-n", "--native-sysroot", dest="native_sysroot",
-                      action="store", help="path to the native sysroot containing the tools to use to build the image")
+                      help="path to the native sysroot containing the tools "
+                           "to use to build the image")
     parser.add_option("-p", "--skip-build-check", dest="build_check",
                       action="store_false", default=True, help="skip the build check")
     parser.add_option("-f", "--build-rootfs", action="store_true", help="build rootfs")
@@ -173,7 +179,9 @@ def wic_create_subcommand(args, usage_str):
     if not wks_file.endswith(".wks"):
         wks_file = engine.find_canned_image(scripts_path, wks_file)
         if not wks_file:
-            print "No image named %s found, exiting.  (Use 'wic list images' to list available images, or specify a fully-qualified OE kickstart (.wks) filename)\n" % args[0]
+            print "No image named %s found, exiting.  (Use 'wic list images' "\
+                  "to list available images, or specify a fully-qualified OE "\
+                  "kickstart (.wks) filename)\n" % args[0]
             sys.exit(1)
 
     image_output_dir = ""
-- 
2.1.4

[wic][PATCH v2 20/20] wic: Code cleanup: unused imports

[Ed Bartosh]

Fixed pylint warning unused-import

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/image/engine.py b/scripts/lib/image/engine.py
index 0dc7920..73e8f8b 100644
--- a/scripts/lib/image/engine.py
+++ b/scripts/lib/image/engine.py
@@ -30,18 +30,10 @@
 
 import os
 import sys
-from abc import ABCMeta, abstractmethod
-import shlex
-import json
-import subprocess
-import shutil
 
-import os, sys, errno
 from wic import msger, creator
-from wic.utils import cmdln, misc, errors
-from wic.conf import configmgr
+from wic.utils import misc
 from wic.plugin import pluginmgr
-from wic.__version__ import VERSION
 from wic.utils.oe import misc
 
 
diff --git a/scripts/lib/wic/conf.py b/scripts/lib/wic/conf.py
index cecf396..402ed62 100644
--- a/scripts/lib/wic/conf.py
+++ b/scripts/lib/wic/conf.py
@@ -19,7 +19,7 @@ import os
 
 from wic import msger
 from wic import kickstart
-from wic.utils import misc, runner, errors
+from wic.utils import misc
 
 
 def get_siteconf():
diff --git a/scripts/lib/wic/imager/baseimager.py b/scripts/lib/wic/imager/baseimager.py
index 0afbc2c..67a2d01 100644
--- a/scripts/lib/wic/imager/baseimager.py
+++ b/scripts/lib/wic/imager/baseimager.py
@@ -21,10 +21,9 @@ import os
 import tempfile
 import shutil
 
-from wic import kickstart
 from wic import msger
 from wic.utils.errors import CreatorError
-from wic.utils import misc, runner, fs_related as fs
+from wic.utils import runner
 
 class BaseImageCreator(object):
     """Base class for image creation.
diff --git a/scripts/lib/wic/imager/direct.py b/scripts/lib/wic/imager/direct.py
index e2f8058..57b1335 100644
--- a/scripts/lib/wic/imager/direct.py
+++ b/scripts/lib/wic/imager/direct.py
@@ -28,7 +28,7 @@ import os
 import shutil
 
 from wic import kickstart, msger
-from wic.utils import fs_related, runner, misc
+from wic.utils import fs_related
 from wic.utils.partitionedfs import Image
 from wic.utils.errors import CreatorError, ImageError
 from wic.imager.baseimager import BaseImageCreator
diff --git a/scripts/lib/wic/pluginbase.py b/scripts/lib/wic/pluginbase.py
index 50e442d..6d96de2 100644
--- a/scripts/lib/wic/pluginbase.py
+++ b/scripts/lib/wic/pluginbase.py
@@ -16,7 +16,6 @@
 # Temple Place - Suite 330, Boston, MA 02111-1307, USA.
 
 from wic import msger
-from wic.utils import errors
 
 class _Plugin(object):
     class __metaclass__(type):
diff --git a/scripts/lib/wic/plugins/imager/direct_plugin.py b/scripts/lib/wic/plugins/imager/direct_plugin.py
index db01c41..eb17e8d 100644
--- a/scripts/lib/wic/plugins/imager/direct_plugin.py
+++ b/scripts/lib/wic/plugins/imager/direct_plugin.py
@@ -24,10 +24,8 @@
 # Tom Zanussi <tom.zanussi (at] linux.intel.com>
 #
 
-from wic import msger
-from wic.utils import misc, fs_related, errors, runner, cmdln
+from wic.utils import errors
 from wic.conf import configmgr
-from wic.plugin import pluginmgr
 
 import wic.imager.direct as direct
 from wic.pluginbase import ImagerPlugin
diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py b/scripts/lib/wic/plugins/source/bootimg-efi.py
index 903b478..49d5c8a 100644
--- a/scripts/lib/wic/plugins/source/bootimg-efi.py
+++ b/scripts/lib/wic/plugins/source/bootimg-efi.py
@@ -27,7 +27,6 @@
 import os
 import shutil
 
-from wic.utils.errors import ImageError
 from wic import kickstart, msger
 from wic.pluginbase import SourcePlugin
 from wic.utils.oe.misc import exec_cmd, exec_native_cmd, get_bitbake_var, \
diff --git a/scripts/lib/wic/utils/cmdln.py b/scripts/lib/wic/utils/cmdln.py
index b099473..d8f1c05 100644
--- a/scripts/lib/wic/utils/cmdln.py
+++ b/scripts/lib/wic/utils/cmdln.py
@@ -43,7 +43,6 @@ import sys
 import re
 import cmd
 import optparse
-from pprint import pprint
 import sys
 
 
-- 
2.1.4