* [PATCH 00/18] #10619: refactor wic codebase (start)
@ 2017-02-01 13:48 Ed Bartosh
2017-02-01 13:48 ` [PATCH 01/18] wic: creator: stop using config manager Ed Bartosh
` (17 more replies)
0 siblings, 18 replies; 19+ messages in thread
From: Ed Bartosh @ 2017-02-01 13:48 UTC (permalink / raw)
To: openembedded-core
Hi,
This patchset consolidates wic APIs in a more maintainable way,
removes unused APIs and cleans up wic code.
This is a first series of a refactoring work. The changes in this patchset are
relatively simple. They're a preparation for upcoming heavy work on making wic
codebase less complex and more maintainable.
The following changes since commit ec3d83f9a90288403b96be25da855fa280aadd8d:
xmlto: Don't hardcode the path to tail (2017-01-31 23:47:33 +0000)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib ed/wic/refactoring-10619
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/wic/refactoring-10619
Ed Bartosh (18):
wic: creator: stop using config manager
wic: direct_plugin: stop using config manager
wic: removed conf.py and empty config file.
wic: moved content of direct.py to direct_plugin
wic: get rid of __rootfs_dir_to_dict method
wic: improve naming in direct_plugin classes
wic: pylinted direct_plugin
wic: simplified code of direct_plugin
wic: renamd direct_plugin.py -> direct.py
wic: removed test file
wic: partition: simlify calling plugin methods
wci: misc: removed build_name API
wic: move 2 APIs to wic.engine
wic: move oe/misc.py one level up
wic: removed code from __init__.py
wic: msger.py: remove unused APIs
wic: code cleanup
wic: remove syslinux.py
scripts/lib/wic/__init__.py | 4 -
scripts/lib/wic/__version__.py | 1 -
scripts/lib/wic/conf.py | 103 --------
scripts/lib/wic/config/wic.conf | 6 -
scripts/lib/wic/creator.py | 19 --
scripts/lib/wic/engine.py | 39 ++-
scripts/lib/wic/help.py | 4 +-
scripts/lib/wic/imager/__init__.py | 0
scripts/lib/wic/ksparser.py | 2 +-
scripts/lib/wic/msger.py | 26 --
scripts/lib/wic/partition.py | 66 ++---
scripts/lib/wic/plugin.py | 5 +-
scripts/lib/wic/pluginbase.py | 1 -
scripts/lib/wic/{ => plugins}/imager/direct.py | 139 ++++++-----
scripts/lib/wic/plugins/imager/direct_plugin.py | 103 --------
scripts/lib/wic/plugins/source/bootimg-efi.py | 20 +-
.../lib/wic/plugins/source/bootimg-partition.py | 6 +-
scripts/lib/wic/plugins/source/bootimg-pcbios.py | 12 +-
scripts/lib/wic/plugins/source/fsimage.py | 2 +-
.../lib/wic/plugins/source/isoimage-isohybrid.py | 15 +-
scripts/lib/wic/plugins/source/rawcopy.py | 3 +-
scripts/lib/wic/plugins/source/rootfs.py | 6 +-
.../lib/wic/plugins/source/rootfs_pcbios_ext.py | 46 +++-
scripts/lib/wic/test | 1 -
scripts/lib/wic/utils/misc.py | 274 +++++++++++++++------
scripts/lib/wic/utils/oe/misc.py | 247 -------------------
scripts/lib/wic/utils/partitionedfs.py | 7 +-
scripts/lib/wic/utils/syslinux.py | 58 -----
scripts/wic | 2 +-
29 files changed, 426 insertions(+), 791 deletions(-)
delete mode 100644 scripts/lib/wic/__version__.py
delete mode 100644 scripts/lib/wic/conf.py
delete mode 100644 scripts/lib/wic/config/wic.conf
delete mode 100644 scripts/lib/wic/imager/__init__.py
rename scripts/lib/wic/{ => plugins}/imager/direct.py (80%)
delete mode 100644 scripts/lib/wic/plugins/imager/direct_plugin.py
delete mode 100644 scripts/lib/wic/test
delete mode 100644 scripts/lib/wic/utils/oe/misc.py
delete mode 100644 scripts/lib/wic/utils/syslinux.py
--
2.1.4
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH 01/18] wic: creator: stop using config manager
2017-02-01 13:48 [PATCH 00/18] #10619: refactor wic codebase (start) Ed Bartosh
@ 2017-02-01 13:48 ` Ed Bartosh
2017-02-01 13:48 ` [PATCH 02/18] wic: direct_plugin: " Ed Bartosh
` (16 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Ed Bartosh @ 2017-02-01 13:48 UTC (permalink / raw)
To: openembedded-core
This is a preparation to removing conf.py and config/wic.conf
from the codebase.
confmgr object is complicated for no reason and almost
useless as all configuration info comes from command line and
bitbake variables. It's used it creator.py to store information
about output directory, logs and some never used functionality
like tmpfs for future use, which doesn't actually happen.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/creator.py | 19 -------------------
1 file changed, 19 deletions(-)
diff --git a/scripts/lib/wic/creator.py b/scripts/lib/wic/creator.py
index 8f7d150..74db83c 100644
--- a/scripts/lib/wic/creator.py
+++ b/scripts/lib/wic/creator.py
@@ -20,10 +20,8 @@ from optparse import OptionParser, SUPPRESS_HELP
from wic import msger
from wic.utils import errors
-from wic.conf import configmgr
from wic.plugin import pluginmgr
-
class Creator():
"""${name}: create an image
@@ -89,23 +87,6 @@ class Creator():
os.makedirs(os.path.dirname(logfile_abs_path))
msger.set_interactive(False)
msger.set_logfile(logfile_abs_path)
- configmgr.create['logfile'] = options.logfile
-
- if options.config:
- configmgr.reset()
- configmgr._siteconf = options.config
-
- if options.outdir is not None:
- configmgr.create['outdir'] = abspath(options.outdir)
-
- cdir = 'outdir'
- if os.path.exists(configmgr.create[cdir]) \
- and not os.path.isdir(configmgr.create[cdir]):
- msger.error('Invalid directory specified: %s' \
- % configmgr.create[cdir])
-
- if options.enabletmpfs:
- configmgr.create['enabletmpfs'] = options.enabletmpfs
def main(self, argv=None):
if argv is None:
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 02/18] wic: direct_plugin: stop using config manager
2017-02-01 13:48 [PATCH 00/18] #10619: refactor wic codebase (start) Ed Bartosh
2017-02-01 13:48 ` [PATCH 01/18] wic: creator: stop using config manager Ed Bartosh
@ 2017-02-01 13:48 ` Ed Bartosh
2017-02-01 13:48 ` [PATCH 03/18] wic: removed conf.py and empty config file Ed Bartosh
` (15 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Ed Bartosh @ 2017-02-01 13:48 UTC (permalink / raw)
To: openembedded-core
This is a preparation to removing conf.py and config/wic.conf
from the codebase.
Got rid of using configmgr global object in direct_plugin and direct
modules. It was used to implicitly parse kickstart file and set
couple of variables.
Replaced usage of configmgr by passing parameters directly to the
DirectImageCreator.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/imager/direct.py | 10 +++++-----
scripts/lib/wic/plugins/imager/direct_plugin.py | 22 +++++++++++++++-------
2 files changed, 20 insertions(+), 12 deletions(-)
diff --git a/scripts/lib/wic/imager/direct.py b/scripts/lib/wic/imager/direct.py
index 575fd95..ff06b50 100644
--- a/scripts/lib/wic/imager/direct.py
+++ b/scripts/lib/wic/imager/direct.py
@@ -72,18 +72,18 @@ class DirectImageCreator:
media and used on actual hardware.
"""
- def __init__(self, oe_builddir, image_output_dir, rootfs_dir,
- bootimg_dir, kernel_dir, native_sysroot, compressor,
- creatoropts, bmap=False):
+ def __init__(self, image_name, ksobj, oe_builddir, image_output_dir,
+ rootfs_dir, bootimg_dir, kernel_dir, native_sysroot,
+ compressor, bmap=False):
"""
Initialize a DirectImageCreator instance.
This method takes the same arguments as ImageCreator.__init__()
"""
- self.name = creatoropts['name']
+ self.name = image_name
self.outdir = image_output_dir
self.workdir = tempfile.mktemp(prefix='wic')
- self.ks = creatoropts['ks']
+ self.ks = ksobj
self.__image = None
self.__disks = {}
diff --git a/scripts/lib/wic/plugins/imager/direct_plugin.py b/scripts/lib/wic/plugins/imager/direct_plugin.py
index 8fe3930..e839d2f 100644
--- a/scripts/lib/wic/plugins/imager/direct_plugin.py
+++ b/scripts/lib/wic/plugins/imager/direct_plugin.py
@@ -24,8 +24,12 @@
# Tom Zanussi <tom.zanussi (at] linux.intel.com>
#
+from time import strftime
+
+from os.path import basename, splitext
from wic.utils import errors
-from wic.conf import configmgr
+from wic.ksparser import KickStart, KickStartError
+from wic import msger
import wic.imager.direct as direct
from wic.pluginbase import ImagerPlugin
@@ -68,27 +72,31 @@ class DirectPlugin(ImagerPlugin):
bootimg_dir = args[2]
rootfs_dir = args[3]
- creatoropts = configmgr.create
ksconf = args[4]
image_output_dir = args[5]
oe_builddir = args[6]
compressor = args[7]
- krootfs_dir = cls.__rootfs_dir_to_dict(rootfs_dir)
+ try:
+ ksobj = KickStart(ksconf)
+ except KickStartError as err:
+ msger.error(str(err))
- configmgr._ksconf = ksconf
+ image_name = "%s-%s" % (splitext(basename(ksconf))[0],
+ strftime("%Y%m%d%H%M"))
+ krootfs_dir = cls.__rootfs_dir_to_dict(rootfs_dir)
- creator = direct.DirectImageCreator(oe_builddir,
+ creator = direct.DirectImageCreator(image_name,
+ ksobj,
+ oe_builddir,
image_output_dir,
krootfs_dir,
bootimg_dir,
kernel_dir,
native_sysroot,
compressor,
- creatoropts,
opts.bmap)
-
try:
creator.create()
creator.assemble()
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 03/18] wic: removed conf.py and empty config file.
2017-02-01 13:48 [PATCH 00/18] #10619: refactor wic codebase (start) Ed Bartosh
2017-02-01 13:48 ` [PATCH 01/18] wic: creator: stop using config manager Ed Bartosh
2017-02-01 13:48 ` [PATCH 02/18] wic: direct_plugin: " Ed Bartosh
@ 2017-02-01 13:48 ` Ed Bartosh
2017-02-01 13:48 ` [PATCH 04/18] wic: moved content of direct.py to direct_plugin Ed Bartosh
` (14 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Ed Bartosh @ 2017-02-01 13:48 UTC (permalink / raw)
To: openembedded-core
Removed as they're not used anymore in wic code.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/conf.py | 103 ----------------------------------------
scripts/lib/wic/config/wic.conf | 6 ---
2 files changed, 109 deletions(-)
delete mode 100644 scripts/lib/wic/conf.py
delete mode 100644 scripts/lib/wic/config/wic.conf
diff --git a/scripts/lib/wic/conf.py b/scripts/lib/wic/conf.py
deleted file mode 100644
index 070ec30..0000000
--- a/scripts/lib/wic/conf.py
+++ /dev/null
@@ -1,103 +0,0 @@
-#!/usr/bin/env python -tt
-#
-# Copyright (c) 2011 Intel, Inc.
-#
-# This program is free software; you can redistribute it and/or modify it
-# under the terms of the GNU General Public License as published by the Free
-# Software Foundation; version 2 of the License
-#
-# This program is distributed in the hope that it will be useful, but
-# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
-# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
-# for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc., 59
-# Temple Place - Suite 330, Boston, MA 02111-1307, USA.
-
-import os
-
-from wic.ksparser import KickStart, KickStartError
-from wic import msger
-from wic.utils import misc
-
-
-def get_siteconf():
- wic_path = os.path.dirname(__file__)
- eos = wic_path.find('scripts') + len('scripts')
- scripts_path = wic_path[:eos]
-
- return scripts_path + "/lib/image/config/wic.conf"
-
-class ConfigMgr(object):
- DEFAULTS = {
- 'common': {
- "distro_name": "Default Distribution",
- "plugin_dir": "/usr/lib/wic/plugins"}, # TODO use prefix also?
- 'create': {
- "tmpdir": '/var/tmp/wic',
- "outdir": './wic-output',
- "release": None,
- "logfile": None,
- "name_prefix": None,
- "name_suffix": None}
- }
-
- # make the manager class as singleton
- _instance = None
- def __new__(cls, *args, **kwargs):
- if not cls._instance:
- cls._instance = super(ConfigMgr, cls).__new__(cls, *args, **kwargs)
-
- return cls._instance
-
- def __init__(self, ksconf=None, siteconf=None):
- # reset config options
- self.reset()
-
- if not siteconf:
- siteconf = get_siteconf()
-
- # initial options from siteconf
- self._siteconf = siteconf
-
- if ksconf:
- self._ksconf = ksconf
-
- def reset(self):
- self.__ksconf = None
- self.__siteconf = None
- self.create = {}
-
- # initialize the values with defaults
- for sec, vals in self.DEFAULTS.items():
- setattr(self, sec, vals)
-
- def __set_ksconf(self, ksconf):
- if not os.path.isfile(ksconf):
- msger.error('Cannot find ks file: %s' % ksconf)
-
- self.__ksconf = ksconf
- self._parse_kickstart(ksconf)
- def __get_ksconf(self):
- return self.__ksconf
- _ksconf = property(__get_ksconf, __set_ksconf)
-
- def _parse_kickstart(self, ksconf=None):
- if not ksconf:
- return
-
- try:
- ksobj = KickStart(ksconf)
- except KickStartError as err:
- msger.error(str(err))
-
- self.create['ks'] = ksobj
- self.create['name'] = os.path.splitext(os.path.basename(ksconf))[0]
-
- self.create['name'] = misc.build_name(ksconf,
- self.create['release'],
- self.create['name_prefix'],
- self.create['name_suffix'])
-
-configmgr = ConfigMgr()
diff --git a/scripts/lib/wic/config/wic.conf b/scripts/lib/wic/config/wic.conf
deleted file mode 100644
index a51bcb5..0000000
--- a/scripts/lib/wic/config/wic.conf
+++ /dev/null
@@ -1,6 +0,0 @@
-[common]
-; general settings
-distro_name = OpenEmbedded
-
-[create]
-; settings for create subcommand
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 04/18] wic: moved content of direct.py to direct_plugin
2017-02-01 13:48 [PATCH 00/18] #10619: refactor wic codebase (start) Ed Bartosh
` (2 preceding siblings ...)
2017-02-01 13:48 ` [PATCH 03/18] wic: removed conf.py and empty config file Ed Bartosh
@ 2017-02-01 13:48 ` Ed Bartosh
2017-02-01 13:48 ` [PATCH 05/18] wic: get rid of __rootfs_dir_to_dict method Ed Bartosh
` (13 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Ed Bartosh @ 2017-02-01 13:48 UTC (permalink / raw)
To: openembedded-core
This move simplifies directory structure and makes
further refactoring easier. The code from direct.py was used
only in direct_plugin, so it's safe to move it there.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/imager/__init__.py | 0
scripts/lib/wic/imager/direct.py | 402 ------------------------
scripts/lib/wic/plugins/imager/direct_plugin.py | 396 ++++++++++++++++++++++-
3 files changed, 380 insertions(+), 418 deletions(-)
delete mode 100644 scripts/lib/wic/imager/__init__.py
delete mode 100644 scripts/lib/wic/imager/direct.py
diff --git a/scripts/lib/wic/imager/__init__.py b/scripts/lib/wic/imager/__init__.py
deleted file mode 100644
index e69de29..0000000
diff --git a/scripts/lib/wic/imager/direct.py b/scripts/lib/wic/imager/direct.py
deleted file mode 100644
index ff06b50..0000000
--- a/scripts/lib/wic/imager/direct.py
+++ /dev/null
@@ -1,402 +0,0 @@
-# ex:ts=4:sw=4:sts=4:et
-# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
-#
-# Copyright (c) 2013, Intel Corporation.
-# All rights reserved.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License version 2 as
-# published by the Free Software Foundation.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-#
-# DESCRIPTION
-# This implements the 'direct' image creator class for 'wic'
-#
-# AUTHORS
-# Tom Zanussi <tom.zanussi (at] linux.intel.com>
-#
-
-import os
-import shutil
-import uuid
-import tempfile
-
-from wic import msger
-from wic.utils.oe.misc import get_bitbake_var
-from wic.utils.partitionedfs import Image
-from wic.utils.errors import CreatorError, ImageError
-from wic.plugin import pluginmgr
-from wic.utils.oe.misc import exec_cmd, exec_native_cmd
-
-disk_methods = {
- "do_install_disk":None,
-}
-
-class DiskImage():
- """
- A Disk backed by a file.
- """
- def __init__(self, device, size):
- self.size = size
- self.device = device
- self.created = False
-
- def exists(self):
- return os.path.exists(self.device)
-
- def create(self):
- if self.created:
- return
- # create sparse disk image
- with open(self.device, 'w') as sparse:
- os.ftruncate(sparse.fileno(), self.size)
-
- self.created = True
-
-class DirectImageCreator:
- """
- Installs a system into a file containing a partitioned disk image.
-
- DirectImageCreator is an advanced ImageCreator subclass; an image
- file is formatted with a partition table, each partition created
- from a rootfs or other OpenEmbedded build artifact and dd'ed into
- the virtual disk. The disk image can subsequently be dd'ed onto
- media and used on actual hardware.
- """
-
- def __init__(self, image_name, ksobj, oe_builddir, image_output_dir,
- rootfs_dir, bootimg_dir, kernel_dir, native_sysroot,
- compressor, bmap=False):
- """
- Initialize a DirectImageCreator instance.
-
- This method takes the same arguments as ImageCreator.__init__()
- """
- self.name = image_name
- self.outdir = image_output_dir
- self.workdir = tempfile.mktemp(prefix='wic')
- self.ks = ksobj
-
- self.__image = None
- self.__disks = {}
- self.__disk_format = "direct"
- self._disk_names = []
- self.ptable_format = self.ks.bootloader.ptable
-
- self.oe_builddir = oe_builddir
- self.rootfs_dir = rootfs_dir
- self.bootimg_dir = bootimg_dir
- self.kernel_dir = kernel_dir
- self.native_sysroot = native_sysroot
- self.compressor = compressor
- self.bmap = bmap
-
- def _get_part_num(self, num, parts):
- """calculate the real partition number, accounting for partitions not
- in the partition table and logical partitions
- """
- realnum = 0
- for pnum, part in enumerate(parts, 1):
- if not part.no_table:
- realnum += 1
- if pnum == num:
- if part.no_table:
- return 0
- if self.ptable_format == 'msdos' and realnum > 3:
- # account for logical partition numbering, ex. sda5..
- return realnum + 1
- return realnum
-
- def _write_fstab(self, image_rootfs):
- """overriden to generate fstab (temporarily) in rootfs. This is called
- from _create, make sure it doesn't get called from
- BaseImage.create()
- """
- if not image_rootfs:
- return
-
- fstab_path = image_rootfs + "/etc/fstab"
- if not os.path.isfile(fstab_path):
- return
-
- with open(fstab_path) as fstab:
- fstab_lines = fstab.readlines()
-
- if self._update_fstab(fstab_lines, self._get_parts()):
- shutil.copyfile(fstab_path, fstab_path + ".orig")
-
- with open(fstab_path, "w") as fstab:
- fstab.writelines(fstab_lines)
-
- return fstab_path
-
- def _update_fstab(self, fstab_lines, parts):
- """Assume partition order same as in wks"""
- updated = False
- for num, part in enumerate(parts, 1):
- pnum = self._get_part_num(num, parts)
- if not pnum or not part.mountpoint \
- or part.mountpoint in ("/", "/boot"):
- continue
-
- # mmc device partitions are named mmcblk0p1, mmcblk0p2..
- prefix = 'p' if part.disk.startswith('mmcblk') else ''
- device_name = "/dev/%s%s%d" % (part.disk, prefix, pnum)
-
- opts = part.fsopts if part.fsopts else "defaults"
- line = "\t".join([device_name, part.mountpoint, part.fstype,
- opts, "0", "0"]) + "\n"
-
- fstab_lines.append(line)
- updated = True
-
- return updated
-
- def set_bootimg_dir(self, bootimg_dir):
- """
- Accessor for bootimg_dir, the actual location used for the source
- of the bootimg. Should be set by source plugins (only if they
- change the default bootimg source) so the correct info gets
- displayed for print_outimage_info().
- """
- self.bootimg_dir = bootimg_dir
-
- def _get_parts(self):
- if not self.ks:
- raise CreatorError("Failed to get partition info, "
- "please check your kickstart setting.")
-
- # Set a default partition if no partition is given out
- if not self.ks.partitions:
- partstr = "part / --size 1900 --ondisk sda --fstype=ext3"
- args = partstr.split()
- part = self.ks.parse(args[1:])
- if part not in self.ks.partitions:
- self.ks.partitions.append(part)
-
- # partitions list from kickstart file
- return self.ks.partitions
-
- def _full_name(self, name, extention):
- """ Construct full file name for a file we generate. """
- return "%s-%s.%s" % (self.name, name, extention)
-
- def _full_path(self, path, name, extention):
- """ Construct full file path to a file we generate. """
- return os.path.join(path, self._full_name(name, extention))
-
- def get_default_source_plugin(self):
- """
- The default source plugin i.e. the plugin that's consulted for
- overall image generation tasks outside of any particular
- partition. For convenience, we just hang it off the
- bootloader handler since it's the one non-partition object in
- any setup. By default the default plugin is set to the same
- plugin as the /boot partition; since we hang it off the
- bootloader object, the default can be explicitly set using the
- --source bootloader param.
- """
- return self.ks.bootloader.source
-
- #
- # Actual implemention
- #
- def create(self):
- """
- For 'wic', we already have our build artifacts - we just create
- filesystems from the artifacts directly and combine them into
- a partitioned image.
- """
- parts = self._get_parts()
-
- self._image = Image(self.native_sysroot)
-
- disk_ids = {}
- for num, part in enumerate(parts, 1):
- # as a convenience, set source to the boot partition source
- # instead of forcing it to be set via bootloader --source
- if not self.ks.bootloader.source and part.mountpoint == "/boot":
- self.ks.bootloader.source = part.source
-
- # generate parition UUIDs
- if not part.uuid and part.use_uuid:
- if self.ptable_format == 'gpt':
- part.uuid = str(uuid.uuid4())
- else: # msdos partition table
- if part.disk not in disk_ids:
- disk_ids[part.disk] = int.from_bytes(os.urandom(4), 'little')
- disk_id = disk_ids[part.disk]
- part.uuid = '%0x-%02d' % (disk_id, self._get_part_num(num, parts))
-
- fstab_path = self._write_fstab(self.rootfs_dir.get("ROOTFS_DIR"))
-
- for part in parts:
- # get rootfs size from bitbake variable if it's not set in .ks file
- if not part.size:
- # and if rootfs name is specified for the partition
- image_name = self.rootfs_dir.get(part.rootfs_dir)
- if image_name and os.path.sep not in image_name:
- # Bitbake variable ROOTFS_SIZE is calculated in
- # Image._get_rootfs_size method from meta/lib/oe/image.py
- # using IMAGE_ROOTFS_SIZE, IMAGE_ROOTFS_ALIGNMENT,
- # IMAGE_OVERHEAD_FACTOR and IMAGE_ROOTFS_EXTRA_SPACE
- rsize_bb = get_bitbake_var('ROOTFS_SIZE', image_name)
- if rsize_bb:
- part.size = int(round(float(rsize_bb)))
- # need to create the filesystems in order to get their
- # sizes before we can add them and do the layout.
- # Image.create() actually calls __format_disks() to create
- # the disk images and carve out the partitions, then
- # self.assemble() calls Image.assemble() which calls
- # __write_partitition() for each partition to dd the fs
- # into the partitions.
- part.prepare(self, self.workdir, self.oe_builddir, self.rootfs_dir,
- self.bootimg_dir, self.kernel_dir, self.native_sysroot)
-
-
- self._image.add_partition(part.disk_size,
- part.disk,
- part.mountpoint,
- part.source_file,
- part.fstype,
- part.label,
- fsopts=part.fsopts,
- boot=part.active,
- align=part.align,
- no_table=part.no_table,
- part_type=part.part_type,
- uuid=part.uuid,
- system_id=part.system_id)
-
- if fstab_path:
- shutil.move(fstab_path + ".orig", fstab_path)
-
- self._image.layout_partitions(self.ptable_format)
-
- for disk_name, disk in self._image.disks.items():
- full_path = self._full_path(self.workdir, disk_name, "direct")
- msger.debug("Adding disk %s as %s with size %s bytes" \
- % (disk_name, full_path, disk['min_size']))
- disk_obj = DiskImage(full_path, disk['min_size'])
- #self._disks[disk_name] = disk_obj
- self._image.add_disk(disk_name, disk_obj, disk_ids.get(disk_name))
-
- self._image.create()
-
- def assemble(self):
- """
- Assemble partitions into disk image(s)
- """
- for disk_name, disk in self._image.disks.items():
- full_path = self._full_path(self.workdir, disk_name, "direct")
- msger.debug("Assembling disk %s as %s with size %s bytes" \
- % (disk_name, full_path, disk['min_size']))
- self._image.assemble(full_path)
-
- def finalize(self):
- """
- Finalize the disk image.
-
- For example, prepare the image to be bootable by e.g.
- creating and installing a bootloader configuration.
- """
- source_plugin = self.get_default_source_plugin()
- if source_plugin:
- self._source_methods = pluginmgr.get_source_plugin_methods(source_plugin, disk_methods)
- for disk_name, disk in self._image.disks.items():
- self._source_methods["do_install_disk"](disk, disk_name, self,
- self.workdir,
- self.oe_builddir,
- self.bootimg_dir,
- self.kernel_dir,
- self.native_sysroot)
-
- for disk_name, disk in self._image.disks.items():
- full_path = self._full_path(self.workdir, disk_name, "direct")
- # Generate .bmap
- if self.bmap:
- msger.debug("Generating bmap file for %s" % disk_name)
- exec_native_cmd("bmaptool create %s -o %s.bmap" % (full_path, full_path),
- self.native_sysroot)
- # Compress the image
- if self.compressor:
- msger.debug("Compressing disk %s with %s" % (disk_name, self.compressor))
- exec_cmd("%s %s" % (self.compressor, full_path))
-
- def print_outimage_info(self):
- """
- Print the image(s) and artifacts used, for the user.
- """
- msg = "The new image(s) can be found here:\n"
-
- parts = self._get_parts()
-
- for disk_name in self._image.disks:
- extension = "direct" + {"gzip": ".gz",
- "bzip2": ".bz2",
- "xz": ".xz",
- "": ""}.get(self.compressor)
- full_path = self._full_path(self.outdir, disk_name, extension)
- msg += ' %s\n\n' % full_path
-
- msg += 'The following build artifacts were used to create the image(s):\n'
- for part in parts:
- if part.rootfs_dir is None:
- continue
- if part.mountpoint == '/':
- suffix = ':'
- else:
- suffix = '["%s"]:' % (part.mountpoint or part.label)
- msg += ' ROOTFS_DIR%s%s\n' % (suffix.ljust(20), part.rootfs_dir)
-
- msg += ' BOOTIMG_DIR: %s\n' % self.bootimg_dir
- msg += ' KERNEL_DIR: %s\n' % self.kernel_dir
- msg += ' NATIVE_SYSROOT: %s\n' % self.native_sysroot
-
- msger.info(msg)
-
- @property
- def rootdev(self):
- """
- Get root device name to use as a 'root' parameter
- in kernel command line.
-
- Assume partition order same as in wks
- """
- parts = self._get_parts()
- for num, part in enumerate(parts, 1):
- if part.mountpoint == "/":
- if part.uuid:
- return "PARTUUID=%s" % part.uuid
- else:
- suffix = 'p' if part.disk.startswith('mmcblk') else ''
- pnum = self._get_part_num(num, parts)
- return "/dev/%s%s%-d" % (part.disk, suffix, pnum)
-
- def cleanup(self):
- if self._image:
- try:
- self._image.cleanup()
- except ImageError as err:
- msger.warning("%s" % err)
-
- # Move results to the output dir
- if not os.path.exists(self.outdir):
- os.makedirs(self.outdir)
-
- for fname in os.listdir(self.workdir):
- path = os.path.join(self.workdir, fname)
- if os.path.isfile(path):
- shutil.move(path, os.path.join(self.outdir, fname))
-
- # remove work directory
- shutil.rmtree(self.workdir, ignore_errors=True)
-
diff --git a/scripts/lib/wic/plugins/imager/direct_plugin.py b/scripts/lib/wic/plugins/imager/direct_plugin.py
index e839d2f..91a9792 100644
--- a/scripts/lib/wic/plugins/imager/direct_plugin.py
+++ b/scripts/lib/wic/plugins/imager/direct_plugin.py
@@ -23,16 +23,23 @@
# AUTHORS
# Tom Zanussi <tom.zanussi (at] linux.intel.com>
#
+import os
+import shutil
+import uuid
+import tempfile
+import time
from time import strftime
-
from os.path import basename, splitext
-from wic.utils import errors
-from wic.ksparser import KickStart, KickStartError
-from wic import msger
-import wic.imager.direct as direct
+from wic import msger
+from wic.ksparser import KickStart, KickStartError
+from wic.plugin import pluginmgr
from wic.pluginbase import ImagerPlugin
+from wic.utils import errors
+from wic.utils.errors import CreatorError, ImageError
+from wic.utils.oe.misc import get_bitbake_var, exec_cmd, exec_native_cmd
+from wic.utils.partitionedfs import Image
class DirectPlugin(ImagerPlugin):
"""
@@ -87,16 +94,10 @@ class DirectPlugin(ImagerPlugin):
strftime("%Y%m%d%H%M"))
krootfs_dir = cls.__rootfs_dir_to_dict(rootfs_dir)
- creator = direct.DirectImageCreator(image_name,
- ksobj,
- oe_builddir,
- image_output_dir,
- krootfs_dir,
- bootimg_dir,
- kernel_dir,
- native_sysroot,
- compressor,
- opts.bmap)
+ creator = DirectImageCreator(image_name, ksobj, oe_builddir,
+ image_output_dir, krootfs_dir,
+ bootimg_dir, kernel_dir, native_sysroot,
+ compressor, opts.bmap)
try:
creator.create()
creator.assemble()
@@ -108,4 +109,367 @@ class DirectPlugin(ImagerPlugin):
finally:
creator.cleanup()
- return 0
+disk_methods = {
+ "do_install_disk":None,
+}
+
+class DiskImage():
+ """
+ A Disk backed by a file.
+ """
+ def __init__(self, device, size):
+ self.size = size
+ self.device = device
+ self.created = False
+
+ def exists(self):
+ return os.path.exists(self.device)
+
+ def create(self):
+ if self.created:
+ return
+ # create sparse disk image
+ with open(self.device, 'w') as sparse:
+ os.ftruncate(sparse.fileno(), self.size)
+
+ self.created = True
+
+class DirectImageCreator:
+ """
+ Installs a system into a file containing a partitioned disk image.
+
+ DirectImageCreator is an advanced ImageCreator subclass; an image
+ file is formatted with a partition table, each partition created
+ from a rootfs or other OpenEmbedded build artifact and dd'ed into
+ the virtual disk. The disk image can subsequently be dd'ed onto
+ media and used on actual hardware.
+ """
+
+ def __init__(self, image_name, ksobj, oe_builddir, image_output_dir,
+ rootfs_dir, bootimg_dir, kernel_dir, native_sysroot,
+ compressor, bmap=False):
+ """
+ Initialize a DirectImageCreator instance.
+
+ This method takes the same arguments as ImageCreator.__init__()
+ """
+ self.name = image_name
+ self.outdir = image_output_dir
+ self.workdir = tempfile.mktemp(prefix='wic')
+ self.ks = ksobj
+
+ self.__image = None
+ self.__disks = {}
+ self.__disk_format = "direct"
+ self._disk_names = []
+ self.ptable_format = self.ks.bootloader.ptable
+
+ self.oe_builddir = oe_builddir
+ self.rootfs_dir = rootfs_dir
+ self.bootimg_dir = bootimg_dir
+ self.kernel_dir = kernel_dir
+ self.native_sysroot = native_sysroot
+ self.compressor = compressor
+ self.bmap = bmap
+
+ def _get_part_num(self, num, parts):
+ """calculate the real partition number, accounting for partitions not
+ in the partition table and logical partitions
+ """
+ realnum = 0
+ for pnum, part in enumerate(parts, 1):
+ if not part.no_table:
+ realnum += 1
+ if pnum == num:
+ if part.no_table:
+ return 0
+ if self.ptable_format == 'msdos' and realnum > 3:
+ # account for logical partition numbering, ex. sda5..
+ return realnum + 1
+ return realnum
+
+ def _write_fstab(self, image_rootfs):
+ """overriden to generate fstab (temporarily) in rootfs. This is called
+ from _create, make sure it doesn't get called from
+ BaseImage.create()
+ """
+ if not image_rootfs:
+ return
+
+ fstab_path = image_rootfs + "/etc/fstab"
+ if not os.path.isfile(fstab_path):
+ return
+
+ with open(fstab_path) as fstab:
+ fstab_lines = fstab.readlines()
+
+ if self._update_fstab(fstab_lines, self._get_parts()):
+ shutil.copyfile(fstab_path, fstab_path + ".orig")
+
+ with open(fstab_path, "w") as fstab:
+ fstab.writelines(fstab_lines)
+
+ return fstab_path
+
+ def _update_fstab(self, fstab_lines, parts):
+ """Assume partition order same as in wks"""
+ updated = False
+ for num, part in enumerate(parts, 1):
+ pnum = self._get_part_num(num, parts)
+ if not pnum or not part.mountpoint \
+ or part.mountpoint in ("/", "/boot"):
+ continue
+
+ # mmc device partitions are named mmcblk0p1, mmcblk0p2..
+ prefix = 'p' if part.disk.startswith('mmcblk') else ''
+ device_name = "/dev/%s%s%d" % (part.disk, prefix, pnum)
+
+ opts = part.fsopts if part.fsopts else "defaults"
+ line = "\t".join([device_name, part.mountpoint, part.fstype,
+ opts, "0", "0"]) + "\n"
+
+ fstab_lines.append(line)
+ updated = True
+
+ return updated
+
+ def set_bootimg_dir(self, bootimg_dir):
+ """
+ Accessor for bootimg_dir, the actual location used for the source
+ of the bootimg. Should be set by source plugins (only if they
+ change the default bootimg source) so the correct info gets
+ displayed for print_outimage_info().
+ """
+ self.bootimg_dir = bootimg_dir
+
+ def _get_parts(self):
+ if not self.ks:
+ raise CreatorError("Failed to get partition info, "
+ "please check your kickstart setting.")
+
+ # Set a default partition if no partition is given out
+ if not self.ks.partitions:
+ partstr = "part / --size 1900 --ondisk sda --fstype=ext3"
+ args = partstr.split()
+ part = self.ks.parse(args[1:])
+ if part not in self.ks.partitions:
+ self.ks.partitions.append(part)
+
+ # partitions list from kickstart file
+ return self.ks.partitions
+
+ def _full_name(self, name, extention):
+ """ Construct full file name for a file we generate. """
+ return "%s-%s.%s" % (self.name, name, extention)
+
+ def _full_path(self, path, name, extention):
+ """ Construct full file path to a file we generate. """
+ return os.path.join(path, self._full_name(name, extention))
+
+ def get_default_source_plugin(self):
+ """
+ The default source plugin i.e. the plugin that's consulted for
+ overall image generation tasks outside of any particular
+ partition. For convenience, we just hang it off the
+ bootloader handler since it's the one non-partition object in
+ any setup. By default the default plugin is set to the same
+ plugin as the /boot partition; since we hang it off the
+ bootloader object, the default can be explicitly set using the
+ --source bootloader param.
+ """
+ return self.ks.bootloader.source
+
+ #
+ # Actual implemention
+ #
+ def create(self):
+ """
+ For 'wic', we already have our build artifacts - we just create
+ filesystems from the artifacts directly and combine them into
+ a partitioned image.
+ """
+ parts = self._get_parts()
+
+ self._image = Image(self.native_sysroot)
+
+ disk_ids = {}
+ for num, part in enumerate(parts, 1):
+ # as a convenience, set source to the boot partition source
+ # instead of forcing it to be set via bootloader --source
+ if not self.ks.bootloader.source and part.mountpoint == "/boot":
+ self.ks.bootloader.source = part.source
+
+ # generate parition UUIDs
+ if not part.uuid and part.use_uuid:
+ if self.ptable_format == 'gpt':
+ part.uuid = str(uuid.uuid4())
+ else: # msdos partition table
+ if part.disk not in disk_ids:
+ disk_ids[part.disk] = int.from_bytes(os.urandom(4), 'little')
+ disk_id = disk_ids[part.disk]
+ part.uuid = '%0x-%02d' % (disk_id, self._get_part_num(num, parts))
+
+ fstab_path = self._write_fstab(self.rootfs_dir.get("ROOTFS_DIR"))
+
+ for part in parts:
+ # get rootfs size from bitbake variable if it's not set in .ks file
+ if not part.size:
+ # and if rootfs name is specified for the partition
+ image_name = self.rootfs_dir.get(part.rootfs_dir)
+ if image_name and os.path.sep not in image_name:
+ # Bitbake variable ROOTFS_SIZE is calculated in
+ # Image._get_rootfs_size method from meta/lib/oe/image.py
+ # using IMAGE_ROOTFS_SIZE, IMAGE_ROOTFS_ALIGNMENT,
+ # IMAGE_OVERHEAD_FACTOR and IMAGE_ROOTFS_EXTRA_SPACE
+ rsize_bb = get_bitbake_var('ROOTFS_SIZE', image_name)
+ if rsize_bb:
+ part.size = int(round(float(rsize_bb)))
+ # need to create the filesystems in order to get their
+ # sizes before we can add them and do the layout.
+ # Image.create() actually calls __format_disks() to create
+ # the disk images and carve out the partitions, then
+ # self.assemble() calls Image.assemble() which calls
+ # __write_partitition() for each partition to dd the fs
+ # into the partitions.
+ part.prepare(self, self.workdir, self.oe_builddir, self.rootfs_dir,
+ self.bootimg_dir, self.kernel_dir, self.native_sysroot)
+
+
+ self._image.add_partition(part.disk_size,
+ part.disk,
+ part.mountpoint,
+ part.source_file,
+ part.fstype,
+ part.label,
+ fsopts=part.fsopts,
+ boot=part.active,
+ align=part.align,
+ no_table=part.no_table,
+ part_type=part.part_type,
+ uuid=part.uuid,
+ system_id=part.system_id)
+
+ if fstab_path:
+ shutil.move(fstab_path + ".orig", fstab_path)
+
+ self._image.layout_partitions(self.ptable_format)
+
+ for disk_name, disk in self._image.disks.items():
+ full_path = self._full_path(self.workdir, disk_name, "direct")
+ msger.debug("Adding disk %s as %s with size %s bytes" \
+ % (disk_name, full_path, disk['min_size']))
+ disk_obj = DiskImage(full_path, disk['min_size'])
+ #self._disks[disk_name] = disk_obj
+ self._image.add_disk(disk_name, disk_obj, disk_ids.get(disk_name))
+
+ self._image.create()
+
+ def assemble(self):
+ """
+ Assemble partitions into disk image(s)
+ """
+ for disk_name, disk in self._image.disks.items():
+ full_path = self._full_path(self.workdir, disk_name, "direct")
+ msger.debug("Assembling disk %s as %s with size %s bytes" \
+ % (disk_name, full_path, disk['min_size']))
+ self._image.assemble(full_path)
+
+ def finalize(self):
+ """
+ Finalize the disk image.
+
+ For example, prepare the image to be bootable by e.g.
+ creating and installing a bootloader configuration.
+ """
+ source_plugin = self.get_default_source_plugin()
+ if source_plugin:
+ self._source_methods = pluginmgr.get_source_plugin_methods(source_plugin, disk_methods)
+ for disk_name, disk in self._image.disks.items():
+ self._source_methods["do_install_disk"](disk, disk_name, self,
+ self.workdir,
+ self.oe_builddir,
+ self.bootimg_dir,
+ self.kernel_dir,
+ self.native_sysroot)
+
+ for disk_name, disk in self._image.disks.items():
+ full_path = self._full_path(self.workdir, disk_name, "direct")
+ # Generate .bmap
+ if self.bmap:
+ msger.debug("Generating bmap file for %s" % disk_name)
+ exec_native_cmd("bmaptool create %s -o %s.bmap" % (full_path, full_path),
+ self.native_sysroot)
+ # Compress the image
+ if self.compressor:
+ msger.debug("Compressing disk %s with %s" % (disk_name, self.compressor))
+ exec_cmd("%s %s" % (self.compressor, full_path))
+
+ def print_outimage_info(self):
+ """
+ Print the image(s) and artifacts used, for the user.
+ """
+ msg = "The new image(s) can be found here:\n"
+
+ parts = self._get_parts()
+
+ for disk_name in self._image.disks:
+ extension = "direct" + {"gzip": ".gz",
+ "bzip2": ".bz2",
+ "xz": ".xz",
+ "": ""}.get(self.compressor)
+ full_path = self._full_path(self.outdir, disk_name, extension)
+ msg += ' %s\n\n' % full_path
+
+ msg += 'The following build artifacts were used to create the image(s):\n'
+ for part in parts:
+ if part.rootfs_dir is None:
+ continue
+ if part.mountpoint == '/':
+ suffix = ':'
+ else:
+ suffix = '["%s"]:' % (part.mountpoint or part.label)
+ msg += ' ROOTFS_DIR%s%s\n' % (suffix.ljust(20), part.rootfs_dir)
+
+ msg += ' BOOTIMG_DIR: %s\n' % self.bootimg_dir
+ msg += ' KERNEL_DIR: %s\n' % self.kernel_dir
+ msg += ' NATIVE_SYSROOT: %s\n' % self.native_sysroot
+
+ msger.info(msg)
+
+ @property
+ def rootdev(self):
+ """
+ Get root device name to use as a 'root' parameter
+ in kernel command line.
+
+ Assume partition order same as in wks
+ """
+ parts = self._get_parts()
+ for num, part in enumerate(parts, 1):
+ if part.mountpoint == "/":
+ if part.uuid:
+ return "PARTUUID=%s" % part.uuid
+ else:
+ suffix = 'p' if part.disk.startswith('mmcblk') else ''
+ pnum = self._get_part_num(num, parts)
+ return "/dev/%s%s%-d" % (part.disk, suffix, pnum)
+
+ def cleanup(self):
+ if self._image:
+ try:
+ self._image.cleanup()
+ except ImageError as err:
+ msger.warning("%s" % err)
+
+ # Move results to the output dir
+ if not os.path.exists(self.outdir):
+ os.makedirs(self.outdir)
+
+ for fname in os.listdir(self.workdir):
+ path = os.path.join(self.workdir, fname)
+ if os.path.isfile(path):
+ shutil.move(path, os.path.join(self.outdir, fname))
+
+ # remove work directory
+ shutil.rmtree(self.workdir, ignore_errors=True)
+
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 05/18] wic: get rid of __rootfs_dir_to_dict method
2017-02-01 13:48 [PATCH 00/18] #10619: refactor wic codebase (start) Ed Bartosh
` (3 preceding siblings ...)
2017-02-01 13:48 ` [PATCH 04/18] wic: moved content of direct.py to direct_plugin Ed Bartosh
@ 2017-02-01 13:48 ` Ed Bartosh
2017-02-01 13:48 ` [PATCH 06/18] wic: improve naming in direct_plugin classes Ed Bartosh
` (12 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Ed Bartosh @ 2017-02-01 13:48 UTC (permalink / raw)
To: openembedded-core
Replaced class method __rootfs_dir_to_dict with a list
comprehension.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/plugins/imager/direct_plugin.py | 21 +++++----------------
1 file changed, 5 insertions(+), 16 deletions(-)
diff --git a/scripts/lib/wic/plugins/imager/direct_plugin.py b/scripts/lib/wic/plugins/imager/direct_plugin.py
index 91a9792..c6df5fb 100644
--- a/scripts/lib/wic/plugins/imager/direct_plugin.py
+++ b/scripts/lib/wic/plugins/imager/direct_plugin.py
@@ -53,21 +53,8 @@ class DirectPlugin(ImagerPlugin):
name = 'direct'
- @classmethod
- def __rootfs_dir_to_dict(cls, rootfs_dirs):
- """
- Gets a string that contain 'connection=dir' splitted by
- space and return a dict
- """
- krootfs_dir = {}
- for rootfs_dir in rootfs_dirs.split(' '):
- key, val = rootfs_dir.split('=')
- krootfs_dir[key] = val
-
- return krootfs_dir
-
- @classmethod
- def do_create(cls, opts, *args):
+ @staticmethod
+ def do_create(opts, *args):
"""
Create direct image, called from creator as 'direct' cmd
"""
@@ -92,7 +79,9 @@ class DirectPlugin(ImagerPlugin):
image_name = "%s-%s" % (splitext(basename(ksconf))[0],
strftime("%Y%m%d%H%M"))
- krootfs_dir = cls.__rootfs_dir_to_dict(rootfs_dir)
+
+ # parse possible 'rootfs=name' items
+ krootfs_dir = dict(rdir.split('=') for rdir in rootfs_dir.split(' '))
creator = DirectImageCreator(image_name, ksobj, oe_builddir,
image_output_dir, krootfs_dir,
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 06/18] wic: improve naming in direct_plugin classes
2017-02-01 13:48 [PATCH 00/18] #10619: refactor wic codebase (start) Ed Bartosh
` (4 preceding siblings ...)
2017-02-01 13:48 ` [PATCH 05/18] wic: get rid of __rootfs_dir_to_dict method Ed Bartosh
@ 2017-02-01 13:48 ` Ed Bartosh
2017-02-01 13:48 ` [PATCH 07/18] wic: pylinted direct_plugin Ed Bartosh
` (11 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Ed Bartosh @ 2017-02-01 13:48 UTC (permalink / raw)
To: openembedded-core
Synchronized attribure names in DirectImageCreator and
DirectPlugin for better readability. Simplified code,
removed unneeded global variable disk_methods.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/plugins/imager/direct_plugin.py | 62 +++++++++----------------
1 file changed, 22 insertions(+), 40 deletions(-)
diff --git a/scripts/lib/wic/plugins/imager/direct_plugin.py b/scripts/lib/wic/plugins/imager/direct_plugin.py
index c6df5fb..48588db 100644
--- a/scripts/lib/wic/plugins/imager/direct_plugin.py
+++ b/scripts/lib/wic/plugins/imager/direct_plugin.py
@@ -58,50 +58,33 @@ class DirectPlugin(ImagerPlugin):
"""
Create direct image, called from creator as 'direct' cmd
"""
- if len(args) != 8:
- raise errors.Usage("Extra arguments given")
-
- native_sysroot = args[0]
- kernel_dir = args[1]
- bootimg_dir = args[2]
- rootfs_dir = args[3]
-
- ksconf = args[4]
-
- image_output_dir = args[5]
- oe_builddir = args[6]
- compressor = args[7]
+ native_sysroot, kernel_dir, bootimg_dir, rootfs_dir, ksconf, \
+ outdir, oe_builddir, compressor = args
try:
ksobj = KickStart(ksconf)
except KickStartError as err:
msger.error(str(err))
- image_name = "%s-%s" % (splitext(basename(ksconf))[0],
+ name = "%s-%s" % (splitext(basename(ksconf))[0],
strftime("%Y%m%d%H%M"))
# parse possible 'rootfs=name' items
krootfs_dir = dict(rdir.split('=') for rdir in rootfs_dir.split(' '))
- creator = DirectImageCreator(image_name, ksobj, oe_builddir,
- image_output_dir, krootfs_dir,
- bootimg_dir, kernel_dir, native_sysroot,
- compressor, opts.bmap)
+ creator = DirectImageCreator(name, ksobj, oe_builddir, outdir,
+ krootfs_dir, bootimg_dir, kernel_dir,
+ native_sysroot, compressor, opts.bmap)
try:
creator.create()
creator.assemble()
creator.finalize()
- creator.print_outimage_info()
-
+ creator.print_info()
except errors.CreatorError:
raise
finally:
creator.cleanup()
-disk_methods = {
- "do_install_disk":None,
-}
-
class DiskImage():
"""
A Disk backed by a file.
@@ -134,22 +117,22 @@ class DirectImageCreator:
media and used on actual hardware.
"""
- def __init__(self, image_name, ksobj, oe_builddir, image_output_dir,
- rootfs_dir, bootimg_dir, kernel_dir, native_sysroot,
- compressor, bmap=False):
+ def __init__(self, name, ksobj, oe_builddir, outdir,
+ rootfs_dir, bootimg_dir, kernel_dir,
+ native_sysroot, compressor, bmap=False):
"""
Initialize a DirectImageCreator instance.
This method takes the same arguments as ImageCreator.__init__()
"""
- self.name = image_name
- self.outdir = image_output_dir
+ self.name = name
+ self.outdir = outdir
self.workdir = tempfile.mktemp(prefix='wic')
self.ks = ksobj
- self.__image = None
- self.__disks = {}
- self.__disk_format = "direct"
+ self._image = None
+ self._disks = {}
+ self._disk_format = "direct"
self._disk_names = []
self.ptable_format = self.ks.bootloader.ptable
@@ -372,14 +355,13 @@ class DirectImageCreator:
"""
source_plugin = self.get_default_source_plugin()
if source_plugin:
- self._source_methods = pluginmgr.get_source_plugin_methods(source_plugin, disk_methods)
+ name = "do_install_disk"
+ methods = pluginmgr.get_source_plugin_methods(source_plugin,
+ {name: None})
for disk_name, disk in self._image.disks.items():
- self._source_methods["do_install_disk"](disk, disk_name, self,
- self.workdir,
- self.oe_builddir,
- self.bootimg_dir,
- self.kernel_dir,
- self.native_sysroot)
+ methods["do_install_disk"](disk, disk_name, self, self.workdir,
+ self.oe_builddir, self.bootimg_dir,
+ self.kernel_dir, self.native_sysroot)
for disk_name, disk in self._image.disks.items():
full_path = self._full_path(self.workdir, disk_name, "direct")
@@ -393,7 +375,7 @@ class DirectImageCreator:
msger.debug("Compressing disk %s with %s" % (disk_name, self.compressor))
exec_cmd("%s %s" % (self.compressor, full_path))
- def print_outimage_info(self):
+ def print_info(self):
"""
Print the image(s) and artifacts used, for the user.
"""
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 07/18] wic: pylinted direct_plugin
2017-02-01 13:48 [PATCH 00/18] #10619: refactor wic codebase (start) Ed Bartosh
` (5 preceding siblings ...)
2017-02-01 13:48 ` [PATCH 06/18] wic: improve naming in direct_plugin classes Ed Bartosh
@ 2017-02-01 13:48 ` Ed Bartosh
2017-02-01 13:48 ` [PATCH 08/18] wic: simplified code of direct_plugin Ed Bartosh
` (10 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Ed Bartosh @ 2017-02-01 13:48 UTC (permalink / raw)
To: openembedded-core
Fixed wrong continued indentation, unused import and
trailing new line pyling warnings.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/plugins/imager/direct_plugin.py | 22 +++++++---------------
1 file changed, 7 insertions(+), 15 deletions(-)
diff --git a/scripts/lib/wic/plugins/imager/direct_plugin.py b/scripts/lib/wic/plugins/imager/direct_plugin.py
index 48588db..1d0d3ff 100644
--- a/scripts/lib/wic/plugins/imager/direct_plugin.py
+++ b/scripts/lib/wic/plugins/imager/direct_plugin.py
@@ -27,7 +27,6 @@ import os
import shutil
import uuid
import tempfile
-import time
from time import strftime
from os.path import basename, splitext
@@ -307,19 +306,13 @@ class DirectImageCreator:
self.bootimg_dir, self.kernel_dir, self.native_sysroot)
- self._image.add_partition(part.disk_size,
- part.disk,
- part.mountpoint,
- part.source_file,
- part.fstype,
- part.label,
- fsopts=part.fsopts,
- boot=part.active,
- align=part.align,
- no_table=part.no_table,
- part_type=part.part_type,
- uuid=part.uuid,
- system_id=part.system_id)
+ self._image.add_partition(part.disk_size, part.disk,
+ part.mountpoint, part.source_file,
+ part.fstype, part.label,
+ fsopts=part.fsopts, boot=part.active,
+ align=part.align, no_table=part.no_table,
+ part_type=part.part_type, uuid=part.uuid,
+ system_id=part.system_id)
if fstab_path:
shutil.move(fstab_path + ".orig", fstab_path)
@@ -443,4 +436,3 @@ class DirectImageCreator:
# remove work directory
shutil.rmtree(self.workdir, ignore_errors=True)
-
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 08/18] wic: simplified code of direct_plugin
2017-02-01 13:48 [PATCH 00/18] #10619: refactor wic codebase (start) Ed Bartosh
` (6 preceding siblings ...)
2017-02-01 13:48 ` [PATCH 07/18] wic: pylinted direct_plugin Ed Bartosh
@ 2017-02-01 13:48 ` Ed Bartosh
2017-02-01 13:48 ` [PATCH 09/18] wic: renamd direct_plugin.py -> direct.py Ed Bartosh
` (9 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Ed Bartosh @ 2017-02-01 13:48 UTC (permalink / raw)
To: openembedded-core
Removed unused methods.
Got rid of get_default_source_plugin and _full_name methods.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/plugins/imager/direct_plugin.py | 24 ++----------------------
1 file changed, 2 insertions(+), 22 deletions(-)
diff --git a/scripts/lib/wic/plugins/imager/direct_plugin.py b/scripts/lib/wic/plugins/imager/direct_plugin.py
index 1d0d3ff..9cd7068 100644
--- a/scripts/lib/wic/plugins/imager/direct_plugin.py
+++ b/scripts/lib/wic/plugins/imager/direct_plugin.py
@@ -93,9 +93,6 @@ class DiskImage():
self.device = device
self.created = False
- def exists(self):
- return os.path.exists(self.device)
-
def create(self):
if self.created:
return
@@ -229,26 +226,9 @@ class DirectImageCreator:
# partitions list from kickstart file
return self.ks.partitions
- def _full_name(self, name, extention):
- """ Construct full file name for a file we generate. """
- return "%s-%s.%s" % (self.name, name, extention)
-
def _full_path(self, path, name, extention):
""" Construct full file path to a file we generate. """
- return os.path.join(path, self._full_name(name, extention))
-
- def get_default_source_plugin(self):
- """
- The default source plugin i.e. the plugin that's consulted for
- overall image generation tasks outside of any particular
- partition. For convenience, we just hang it off the
- bootloader handler since it's the one non-partition object in
- any setup. By default the default plugin is set to the same
- plugin as the /boot partition; since we hang it off the
- bootloader object, the default can be explicitly set using the
- --source bootloader param.
- """
- return self.ks.bootloader.source
+ return os.path.join(path, "%s-%s.%s" % (self.name, name, extention))
#
# Actual implemention
@@ -346,7 +326,7 @@ class DirectImageCreator:
For example, prepare the image to be bootable by e.g.
creating and installing a bootloader configuration.
"""
- source_plugin = self.get_default_source_plugin()
+ source_plugin = self.ks.bootloader.source
if source_plugin:
name = "do_install_disk"
methods = pluginmgr.get_source_plugin_methods(source_plugin,
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 09/18] wic: renamd direct_plugin.py -> direct.py
2017-02-01 13:48 [PATCH 00/18] #10619: refactor wic codebase (start) Ed Bartosh
` (7 preceding siblings ...)
2017-02-01 13:48 ` [PATCH 08/18] wic: simplified code of direct_plugin Ed Bartosh
@ 2017-02-01 13:48 ` Ed Bartosh
2017-02-01 13:48 ` [PATCH 10/18] wic: removed test file Ed Bartosh
` (8 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Ed Bartosh @ 2017-02-01 13:48 UTC (permalink / raw)
To: openembedded-core
As this files is located in plugins/imager subdirectory it's
obvious that it's an imager plugin. Renamed to direct.py to
be consistent with plugin naming scheme.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/plugins/imager/{direct_plugin.py => direct.py} | 0
1 file changed, 0 insertions(+), 0 deletions(-)
rename scripts/lib/wic/plugins/imager/{direct_plugin.py => direct.py} (100%)
diff --git a/scripts/lib/wic/plugins/imager/direct_plugin.py b/scripts/lib/wic/plugins/imager/direct.py
similarity index 100%
rename from scripts/lib/wic/plugins/imager/direct_plugin.py
rename to scripts/lib/wic/plugins/imager/direct.py
--
2.1.4
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH 10/18] wic: removed test file
2017-02-01 13:48 [PATCH 00/18] #10619: refactor wic codebase (start) Ed Bartosh
` (8 preceding siblings ...)
2017-02-01 13:48 ` [PATCH 09/18] wic: renamd direct_plugin.py -> direct.py Ed Bartosh
@ 2017-02-01 13:48 ` Ed Bartosh
2017-02-01 13:48 ` [PATCH 11/18] wic: partition: simlify calling plugin methods Ed Bartosh
` (7 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Ed Bartosh @ 2017-02-01 13:48 UTC (permalink / raw)
To: openembedded-core
This file is not used anywhere in the wic code.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/test | 1 -
1 file changed, 1 deletion(-)
delete mode 100644 scripts/lib/wic/test
diff --git a/scripts/lib/wic/test b/scripts/lib/wic/test
deleted file mode 100644
index 9daeafb..0000000
--- a/scripts/lib/wic/test
+++ /dev/null
@@ -1 +0,0 @@
-test
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 11/18] wic: partition: simlify calling plugin methods
2017-02-01 13:48 [PATCH 00/18] #10619: refactor wic codebase (start) Ed Bartosh
` (9 preceding siblings ...)
2017-02-01 13:48 ` [PATCH 10/18] wic: removed test file Ed Bartosh
@ 2017-02-01 13:48 ` Ed Bartosh
2017-02-01 13:48 ` [PATCH 12/18] wci: misc: removed build_name API Ed Bartosh
` (6 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Ed Bartosh @ 2017-02-01 13:48 UTC (permalink / raw)
To: openembedded-core
Replaced parse_sourceparams function with list comprehension.
Used local variables instead of attributes.
Moved global variable to the local scope.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/partition.py | 53 +++++++++++++++++++---------------------
scripts/lib/wic/utils/oe/misc.py | 23 -----------------
2 files changed, 25 insertions(+), 51 deletions(-)
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 49d1327..094a8c3 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -27,16 +27,10 @@
import os
import tempfile
-from wic.utils.oe.misc import msger, parse_sourceparams
+from wic.utils.oe.misc import msger
from wic.utils.oe.misc import exec_cmd, exec_native_cmd, get_bitbake_var
from wic.plugin import pluginmgr
-partition_methods = {
- "do_stage_partition":None,
- "do_prepare_partition":None,
- "do_configure_partition":None,
-}
-
class Partition():
def __init__(self, args, lineno):
@@ -129,9 +123,6 @@ class Partition():
Prepare content for individual partitions, depending on
partition command parameters.
"""
- if self.sourceparams:
- self.sourceparams_dict = parse_sourceparams(self.sourceparams)
-
if not self.source:
if not self.size and not self.fixed_size:
msger.error("The %s partition has a size of zero. Please "
@@ -164,24 +155,30 @@ class Partition():
"details on adding a new source plugin." % \
(self.source, self.mountpoint))
- self._source_methods = pluginmgr.get_source_plugin_methods(\
- self.source, partition_methods)
- self._source_methods["do_configure_partition"](self, self.sourceparams_dict,
- creator, cr_workdir,
- oe_builddir,
- bootimg_dir,
- kernel_dir,
- native_sysroot)
- self._source_methods["do_stage_partition"](self, self.sourceparams_dict,
- creator, cr_workdir,
- oe_builddir,
- bootimg_dir, kernel_dir,
- native_sysroot)
- self._source_methods["do_prepare_partition"](self, self.sourceparams_dict,
- creator, cr_workdir,
- oe_builddir,
- bootimg_dir, kernel_dir, rootfs_dir,
- native_sysroot)
+ srcparams_dict = {}
+ if self.sourceparams:
+ # Split sourceparams string of the form key1=val1[,key2=val2,...]
+ # into a dict. Also accepts valueless keys i.e. without =
+ splitted = self.sourceparams.split(',')
+ srcparams_dict = dict(par.split('=') for par in splitted if par)
+
+ partition_methods = {
+ "do_stage_partition": None,
+ "do_prepare_partition": None,
+ "do_configure_partition": None
+ }
+
+ methods = pluginmgr.get_source_plugin_methods(self.source,
+ partition_methods)
+ methods["do_configure_partition"](self, srcparams_dict, creator,
+ cr_workdir, oe_builddir, bootimg_dir,
+ kernel_dir, native_sysroot)
+ methods["do_stage_partition"](self, srcparams_dict, creator,
+ cr_workdir, oe_builddir, bootimg_dir,
+ kernel_dir, native_sysroot)
+ methods["do_prepare_partition"](self, srcparams_dict, creator,
+ cr_workdir, oe_builddir, bootimg_dir,
+ kernel_dir, rootfs_dir, native_sysroot)
# further processing required Partition.size to be an integer, make
# sure that it is one
diff --git a/scripts/lib/wic/utils/oe/misc.py b/scripts/lib/wic/utils/oe/misc.py
index 3737c4b..6781d83 100644
--- a/scripts/lib/wic/utils/oe/misc.py
+++ b/scripts/lib/wic/utils/oe/misc.py
@@ -222,26 +222,3 @@ def get_bitbake_var(var, image=None, cache=True):
get_var method of BB_VARS singleton.
"""
return BB_VARS.get_var(var, image, cache)
-
-def parse_sourceparams(sourceparams):
- """
- Split sourceparams string of the form key1=val1[,key2=val2,...]
- into a dict. Also accepts valueless keys i.e. without =.
-
- Returns dict of param key/val pairs (note that val may be None).
- """
- params_dict = {}
-
- params = sourceparams.split(',')
- if params:
- for par in params:
- if not par:
- continue
- if not '=' in par:
- key = par
- val = None
- else:
- key, val = par.split('=')
- params_dict[key] = val
-
- return params_dict
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 12/18] wci: misc: removed build_name API
2017-02-01 13:48 [PATCH 00/18] #10619: refactor wic codebase (start) Ed Bartosh
` (10 preceding siblings ...)
2017-02-01 13:48 ` [PATCH 11/18] wic: partition: simlify calling plugin methods Ed Bartosh
@ 2017-02-01 13:48 ` Ed Bartosh
2017-02-01 13:48 ` [PATCH 13/18] wic: move 2 APIs to wic.engine Ed Bartosh
` (5 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Ed Bartosh @ 2017-02-01 13:48 UTC (permalink / raw)
To: openembedded-core
This API is not used in wic code.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/utils/misc.py | 39 ---------------------------------------
1 file changed, 39 deletions(-)
diff --git a/scripts/lib/wic/utils/misc.py b/scripts/lib/wic/utils/misc.py
index 1415ae9..7d09f6f 100644
--- a/scripts/lib/wic/utils/misc.py
+++ b/scripts/lib/wic/utils/misc.py
@@ -19,45 +19,6 @@ import os
import time
import wic.engine
-def build_name(kscfg, release=None, prefix=None, suffix=None):
- """Construct and return an image name string.
-
- This is a utility function to help create sensible name and fslabel
- strings. The name is constructed using the sans-prefix-and-extension
- kickstart filename and the supplied prefix and suffix.
-
- kscfg -- a path to a kickstart file
- release -- a replacement to suffix for image release
- prefix -- a prefix to prepend to the name; defaults to None, which causes
- no prefix to be used
- suffix -- a suffix to append to the name; defaults to None, which causes
- a YYYYMMDDHHMM suffix to be used
-
- Note, if maxlen is less then the len(suffix), you get to keep both pieces.
-
- """
- name = os.path.basename(kscfg)
- idx = name.rfind('.')
- if idx >= 0:
- name = name[:idx]
-
- if release is not None:
- suffix = ""
- if prefix is None:
- prefix = ""
- if suffix is None:
- suffix = time.strftime("%Y%m%d%H%M")
-
- if name.startswith(prefix):
- name = name[len(prefix):]
-
- prefix = "%s-" % prefix if prefix else ""
- suffix = "-%s" % suffix if suffix else ""
-
- ret = prefix + name + suffix
-
- return ret
-
def find_canned(scripts_path, file_name):
"""
Find a file either by its path or by name in the canned files dir.
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 13/18] wic: move 2 APIs to wic.engine
2017-02-01 13:48 [PATCH 00/18] #10619: refactor wic codebase (start) Ed Bartosh
` (11 preceding siblings ...)
2017-02-01 13:48 ` [PATCH 12/18] wci: misc: removed build_name API Ed Bartosh
@ 2017-02-01 13:48 ` Ed Bartosh
2017-02-01 13:48 ` [PATCH 14/18] wic: move oe/misc.py one level up Ed Bartosh
` (4 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Ed Bartosh @ 2017-02-01 13:48 UTC (permalink / raw)
To: openembedded-core
Moved find_canned and get_custom_config APIs to engine module.
Removed empty wic.utils.misc module.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/engine.py | 33 ++++++++++++-
scripts/lib/wic/ksparser.py | 2 +-
scripts/lib/wic/plugins/source/bootimg-efi.py | 2 +-
scripts/lib/wic/plugins/source/bootimg-pcbios.py | 4 +-
.../lib/wic/plugins/source/isoimage-isohybrid.py | 2 +-
scripts/lib/wic/utils/misc.py | 56 ----------------------
6 files changed, 37 insertions(+), 62 deletions(-)
delete mode 100644 scripts/lib/wic/utils/misc.py
diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py
index 2adef2f..4abea87 100644
--- a/scripts/lib/wic/engine.py
+++ b/scripts/lib/wic/engine.py
@@ -32,7 +32,6 @@ import os
import sys
from wic import msger, creator
-from wic.utils import misc
from wic.plugin import pluginmgr
from wic.utils.oe import misc
@@ -226,3 +225,35 @@ def wic_list(args, scripts_path):
return True
return False
+
+def find_canned(scripts_path, file_name):
+ """
+ Find a file either by its path or by name in the canned files dir.
+
+ Return None if not found
+ """
+ if os.path.exists(file_name):
+ return file_name
+
+ layers_canned_wks_dir = build_canned_image_list(scripts_path)
+ for canned_wks_dir in layers_canned_wks_dir:
+ for root, dirs, files in os.walk(canned_wks_dir):
+ for fname in files:
+ if fname == file_name:
+ fullpath = os.path.join(canned_wks_dir, fname)
+ return fullpath
+
+def get_custom_config(boot_file):
+ """
+ Get the custom configuration to be used for the bootloader.
+
+ Return None if the file can't be found.
+ """
+ # Get the scripts path of poky
+ scripts_path = os.path.abspath("%s/../.." % os.path.dirname(__file__))
+
+ cfg_file = find_canned(scripts_path, boot_file)
+ if cfg_file:
+ with open(cfg_file, "r") as f:
+ config = f.read()
+ return config
diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
index 62c4902..41d3cc6 100644
--- a/scripts/lib/wic/ksparser.py
+++ b/scripts/lib/wic/ksparser.py
@@ -30,8 +30,8 @@ import shlex
from argparse import ArgumentParser, ArgumentError, ArgumentTypeError
from wic import msger
+from wic.engine import find_canned
from wic.partition import Partition
-from wic.utils.misc import find_canned
class KickStartError(Exception):
"""Custom exception."""
diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py b/scripts/lib/wic/plugins/source/bootimg-efi.py
index 74a1557..28b941e 100644
--- a/scripts/lib/wic/plugins/source/bootimg-efi.py
+++ b/scripts/lib/wic/plugins/source/bootimg-efi.py
@@ -28,8 +28,8 @@ import os
import shutil
from wic import msger
+from wic.engine import get_custom_config
from wic.pluginbase import SourcePlugin
-from wic.utils.misc import get_custom_config
from wic.utils.oe.misc import exec_cmd, exec_native_cmd, get_bitbake_var, \
BOOTDD_EXTRA_SPACE
diff --git a/scripts/lib/wic/plugins/source/bootimg-pcbios.py b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
index cff8aec..283b834 100644
--- a/scripts/lib/wic/plugins/source/bootimg-pcbios.py
+++ b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
@@ -26,10 +26,10 @@
import os
-from wic.utils.errors import ImageError
from wic import msger
+from wic.engine import get_custom_config
from wic.utils import runner
-from wic.utils.misc import get_custom_config
+from wic.utils.errors import ImageError
from wic.pluginbase import SourcePlugin
from wic.utils.oe.misc import exec_cmd, exec_native_cmd, \
get_bitbake_var, BOOTDD_EXTRA_SPACE
diff --git a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
index bceaa84..4979d8e 100644
--- a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
+++ b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
@@ -26,8 +26,8 @@ import shutil
import glob
from wic import msger
+from wic.engine import get_custom_config
from wic.pluginbase import SourcePlugin
-from wic.utils.misc import get_custom_config
from wic.utils.oe.misc import exec_cmd, exec_native_cmd, get_bitbake_var
class IsoImagePlugin(SourcePlugin):
diff --git a/scripts/lib/wic/utils/misc.py b/scripts/lib/wic/utils/misc.py
deleted file mode 100644
index 7d09f6f..0000000
--- a/scripts/lib/wic/utils/misc.py
+++ /dev/null
@@ -1,56 +0,0 @@
-#!/usr/bin/env python -tt
-#
-# Copyright (c) 2010, 2011 Intel Inc.
-#
-# This program is free software; you can redistribute it and/or modify it
-# under the terms of the GNU General Public License as published by the Free
-# Software Foundation; version 2 of the License
-#
-# This program is distributed in the hope that it will be useful, but
-# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
-# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
-# for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc., 59
-# Temple Place - Suite 330, Boston, MA 02111-1307, USA.
-
-import os
-import time
-import wic.engine
-
-def find_canned(scripts_path, file_name):
- """
- Find a file either by its path or by name in the canned files dir.
-
- Return None if not found
- """
- if os.path.exists(file_name):
- return file_name
-
- layers_canned_wks_dir = wic.engine.build_canned_image_list(scripts_path)
- for canned_wks_dir in layers_canned_wks_dir:
- for root, dirs, files in os.walk(canned_wks_dir):
- for fname in files:
- if fname == file_name:
- fullpath = os.path.join(canned_wks_dir, fname)
- return fullpath
-
-def get_custom_config(boot_file):
- """
- Get the custom configuration to be used for the bootloader.
-
- Return None if the file can't be found.
- """
- scripts_path = os.path.abspath(os.path.dirname(__file__))
- # Get the scripts path of poky
- for x in range(0, 3):
- scripts_path = os.path.dirname(scripts_path)
-
- cfg_file = find_canned(scripts_path, boot_file)
- if cfg_file:
- with open(cfg_file, "r") as f:
- config = f.read()
- return config
-
- return None
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 14/18] wic: move oe/misc.py one level up
2017-02-01 13:48 [PATCH 00/18] #10619: refactor wic codebase (start) Ed Bartosh
` (12 preceding siblings ...)
2017-02-01 13:48 ` [PATCH 13/18] wic: move 2 APIs to wic.engine Ed Bartosh
@ 2017-02-01 13:48 ` Ed Bartosh
2017-02-01 13:48 ` [PATCH 15/18] wic: removed code from __init__.py Ed Bartosh
` (3 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Ed Bartosh @ 2017-02-01 13:48 UTC (permalink / raw)
To: openembedded-core
Flattened directory structure:
moved wic/utils/oe/misc.py -> wic/utils/misc.py
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/engine.py | 4 ++--
scripts/lib/wic/partition.py | 4 ++--
scripts/lib/wic/plugin.py | 2 +-
scripts/lib/wic/plugins/imager/direct.py | 2 +-
scripts/lib/wic/plugins/source/bootimg-efi.py | 4 ++--
scripts/lib/wic/plugins/source/bootimg-partition.py | 2 +-
scripts/lib/wic/plugins/source/bootimg-pcbios.py | 4 ++--
scripts/lib/wic/plugins/source/fsimage.py | 2 +-
scripts/lib/wic/plugins/source/isoimage-isohybrid.py | 2 +-
scripts/lib/wic/plugins/source/rawcopy.py | 2 +-
scripts/lib/wic/plugins/source/rootfs.py | 2 +-
scripts/lib/wic/plugins/source/rootfs_pcbios_ext.py | 8 ++++----
scripts/lib/wic/utils/{oe => }/misc.py | 0
scripts/lib/wic/utils/partitionedfs.py | 2 +-
scripts/wic | 2 +-
15 files changed, 21 insertions(+), 21 deletions(-)
rename scripts/lib/wic/utils/{oe => }/misc.py (100%)
diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py
index 4abea87..685bc88 100644
--- a/scripts/lib/wic/engine.py
+++ b/scripts/lib/wic/engine.py
@@ -33,7 +33,7 @@ import sys
from wic import msger, creator
from wic.plugin import pluginmgr
-from wic.utils.oe import misc
+from wic.utils.misc import get_bitbake_var
def verify_build_env():
@@ -54,7 +54,7 @@ SCRIPTS_CANNED_IMAGE_DIR = "scripts/" + CANNED_IMAGE_DIR
WIC_DIR = "wic"
def build_canned_image_list(path):
- layers_path = misc.get_bitbake_var("BBLAYERS")
+ layers_path = get_bitbake_var("BBLAYERS")
canned_wks_layer_dirs = []
if layers_path is not None:
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 094a8c3..31a0350 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -27,8 +27,8 @@
import os
import tempfile
-from wic.utils.oe.misc import msger
-from wic.utils.oe.misc import exec_cmd, exec_native_cmd, get_bitbake_var
+from wic import msger
+from wic.utils.misc import exec_cmd, exec_native_cmd, get_bitbake_var
from wic.plugin import pluginmgr
class Partition():
diff --git a/scripts/lib/wic/plugin.py b/scripts/lib/wic/plugin.py
index 306b324..6b06ed6 100644
--- a/scripts/lib/wic/plugin.py
+++ b/scripts/lib/wic/plugin.py
@@ -20,7 +20,7 @@ import os, sys
from wic import msger
from wic import pluginbase
from wic.utils import errors
-from wic.utils.oe.misc import get_bitbake_var
+from wic.utils.misc import get_bitbake_var
__ALL__ = ['PluginMgr', 'pluginmgr']
diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py
index 9cd7068..592412a 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -37,7 +37,7 @@ from wic.plugin import pluginmgr
from wic.pluginbase import ImagerPlugin
from wic.utils import errors
from wic.utils.errors import CreatorError, ImageError
-from wic.utils.oe.misc import get_bitbake_var, exec_cmd, exec_native_cmd
+from wic.utils.misc import get_bitbake_var, exec_cmd, exec_native_cmd
from wic.utils.partitionedfs import Image
class DirectPlugin(ImagerPlugin):
diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py b/scripts/lib/wic/plugins/source/bootimg-efi.py
index 28b941e..2b66a58 100644
--- a/scripts/lib/wic/plugins/source/bootimg-efi.py
+++ b/scripts/lib/wic/plugins/source/bootimg-efi.py
@@ -30,8 +30,8 @@ import shutil
from wic import msger
from wic.engine import get_custom_config
from wic.pluginbase import SourcePlugin
-from wic.utils.oe.misc import exec_cmd, exec_native_cmd, get_bitbake_var, \
- BOOTDD_EXTRA_SPACE
+from wic.utils.misc import (exec_cmd, exec_native_cmd, get_bitbake_var,
+ BOOTDD_EXTRA_SPACE)
class BootimgEFIPlugin(SourcePlugin):
"""
diff --git a/scripts/lib/wic/plugins/source/bootimg-partition.py b/scripts/lib/wic/plugins/source/bootimg-partition.py
index b76c121..f94dfab 100644
--- a/scripts/lib/wic/plugins/source/bootimg-partition.py
+++ b/scripts/lib/wic/plugins/source/bootimg-partition.py
@@ -28,7 +28,7 @@ import re
from wic import msger
from wic.pluginbase import SourcePlugin
-from wic.utils.oe.misc import exec_cmd, get_bitbake_var
+from wic.utils.misc import exec_cmd, get_bitbake_var
from glob import glob
class BootimgPartitionPlugin(SourcePlugin):
diff --git a/scripts/lib/wic/plugins/source/bootimg-pcbios.py b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
index 283b834..d796433 100644
--- a/scripts/lib/wic/plugins/source/bootimg-pcbios.py
+++ b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
@@ -31,8 +31,8 @@ from wic.engine import get_custom_config
from wic.utils import runner
from wic.utils.errors import ImageError
from wic.pluginbase import SourcePlugin
-from wic.utils.oe.misc import exec_cmd, exec_native_cmd, \
- get_bitbake_var, BOOTDD_EXTRA_SPACE
+from wic.utils.misc import (exec_cmd, exec_native_cmd,
+ get_bitbake_var, BOOTDD_EXTRA_SPACE)
class BootimgPcbiosPlugin(SourcePlugin):
"""
diff --git a/scripts/lib/wic/plugins/source/fsimage.py b/scripts/lib/wic/plugins/source/fsimage.py
index f894e89..9193892 100644
--- a/scripts/lib/wic/plugins/source/fsimage.py
+++ b/scripts/lib/wic/plugins/source/fsimage.py
@@ -19,7 +19,7 @@ import os
from wic import msger
from wic.pluginbase import SourcePlugin
-from wic.utils.oe.misc import get_bitbake_var
+from wic.utils.misc import get_bitbake_var
class FSImagePlugin(SourcePlugin):
"""
diff --git a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
index 4979d8e..6df3756 100644
--- a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
+++ b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
@@ -28,7 +28,7 @@ import glob
from wic import msger
from wic.engine import get_custom_config
from wic.pluginbase import SourcePlugin
-from wic.utils.oe.misc import exec_cmd, exec_native_cmd, get_bitbake_var
+from wic.utils.misc import exec_cmd, exec_native_cmd, get_bitbake_var
class IsoImagePlugin(SourcePlugin):
"""
diff --git a/scripts/lib/wic/plugins/source/rawcopy.py b/scripts/lib/wic/plugins/source/rawcopy.py
index 5bd22fd..79e41ca 100644
--- a/scripts/lib/wic/plugins/source/rawcopy.py
+++ b/scripts/lib/wic/plugins/source/rawcopy.py
@@ -19,7 +19,7 @@ import os
from wic import msger
from wic.pluginbase import SourcePlugin
-from wic.utils.oe.misc import exec_cmd, get_bitbake_var
+from wic.utils.misc import exec_cmd, get_bitbake_var
from wic.filemap import sparse_copy
class RawCopyPlugin(SourcePlugin):
diff --git a/scripts/lib/wic/plugins/source/rootfs.py b/scripts/lib/wic/plugins/source/rootfs.py
index 425da8b..891f095 100644
--- a/scripts/lib/wic/plugins/source/rootfs.py
+++ b/scripts/lib/wic/plugins/source/rootfs.py
@@ -29,7 +29,7 @@ import os
from wic import msger
from wic.pluginbase import SourcePlugin
-from wic.utils.oe.misc import get_bitbake_var
+from wic.utils.misc import get_bitbake_var
class RootfsPlugin(SourcePlugin):
"""
diff --git a/scripts/lib/wic/plugins/source/rootfs_pcbios_ext.py b/scripts/lib/wic/plugins/source/rootfs_pcbios_ext.py
index 3d60e6f..1032019 100644
--- a/scripts/lib/wic/plugins/source/rootfs_pcbios_ext.py
+++ b/scripts/lib/wic/plugins/source/rootfs_pcbios_ext.py
@@ -22,7 +22,7 @@ import os
from wic import msger
from wic.utils import syslinux
from wic.utils import runner
-from wic.utils.oe import misc
+from wic.utils.misc import get_bitbake_var, exec_cmd, exec_native_cmd
from wic.utils.errors import ImageError
from wic.pluginbase import SourcePlugin
@@ -58,7 +58,7 @@ class RootfsPlugin(SourcePlugin):
if os.path.isdir(rootfs_dir):
return rootfs_dir
- image_rootfs_dir = misc.get_bitbake_var("IMAGE_ROOTFS", rootfs_dir)
+ image_rootfs_dir = get_bitbake_var("IMAGE_ROOTFS", rootfs_dir)
if not os.path.isdir(image_rootfs_dir):
msg = "No valid artifact IMAGE_ROOTFS from image named"
msg += " %s has been found at %s, exiting.\n" % \
@@ -119,7 +119,7 @@ class RootfsPlugin(SourcePlugin):
native_syslinux_nomtools = os.path.join(native_sysroot, "usr/bin/syslinux-nomtools")
if not is_exe(native_syslinux_nomtools):
msger.info("building syslinux-native...")
- misc.exec_cmd("bitbake syslinux-native")
+ exec_cmd("bitbake syslinux-native")
if not is_exe(native_syslinux_nomtools):
msger.error("Couldn't find syslinux-nomtools (%s), exiting\n" %
native_syslinux_nomtools)
@@ -145,7 +145,7 @@ class RootfsPlugin(SourcePlugin):
# install syslinux into rootfs partition
syslinux_cmd = "syslinux-nomtools -d /boot -i %s" % part.source_file
- misc.exec_native_cmd(syslinux_cmd, native_sysroot)
+ exec_native_cmd(syslinux_cmd, native_sysroot)
@classmethod
def do_install_disk(cls, disk, disk_name, image_creator, workdir, oe_builddir,
diff --git a/scripts/lib/wic/utils/oe/misc.py b/scripts/lib/wic/utils/misc.py
similarity index 100%
rename from scripts/lib/wic/utils/oe/misc.py
rename to scripts/lib/wic/utils/misc.py
diff --git a/scripts/lib/wic/utils/partitionedfs.py b/scripts/lib/wic/utils/partitionedfs.py
index ed67b9a..eb4d7e6 100644
--- a/scripts/lib/wic/utils/partitionedfs.py
+++ b/scripts/lib/wic/utils/partitionedfs.py
@@ -21,7 +21,7 @@
import os
from wic import msger
from wic.utils.errors import ImageError
-from wic.utils.oe.misc import exec_cmd, exec_native_cmd
+from wic.utils.misc import exec_cmd, exec_native_cmd
from wic.filemap import sparse_copy
# Overhead of the MBR partitioning scheme (just one sector)
diff --git a/scripts/wic b/scripts/wic
index 54cbe96..33355ee 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -51,7 +51,7 @@ if bitbake_exe:
else:
bitbake_main = None
-from wic.utils.oe.misc import get_bitbake_var, BB_VARS
+from wic.utils.misc import get_bitbake_var, BB_VARS
from wic.utils.errors import WicError
from wic import engine
from wic import help as hlp
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 15/18] wic: removed code from __init__.py
2017-02-01 13:48 [PATCH 00/18] #10619: refactor wic codebase (start) Ed Bartosh
` (13 preceding siblings ...)
2017-02-01 13:48 ` [PATCH 14/18] wic: move oe/misc.py one level up Ed Bartosh
@ 2017-02-01 13:48 ` Ed Bartosh
2017-02-01 13:48 ` [PATCH 16/18] wic: msger.py: remove unused APIs Ed Bartosh
` (2 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Ed Bartosh @ 2017-02-01 13:48 UTC (permalink / raw)
To: openembedded-core
The code deals with non-existing directory
and can be removed.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/__init__.py | 4 ----
1 file changed, 4 deletions(-)
diff --git a/scripts/lib/wic/__init__.py b/scripts/lib/wic/__init__.py
index 63c1d9c..e69de29 100644
--- a/scripts/lib/wic/__init__.py
+++ b/scripts/lib/wic/__init__.py
@@ -1,4 +0,0 @@
-import os, sys
-
-cur_path = os.path.dirname(__file__) or '.'
-sys.path.insert(0, cur_path + '/3rdparty')
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 16/18] wic: msger.py: remove unused APIs
2017-02-01 13:48 [PATCH 00/18] #10619: refactor wic codebase (start) Ed Bartosh
` (14 preceding siblings ...)
2017-02-01 13:48 ` [PATCH 15/18] wic: removed code from __init__.py Ed Bartosh
@ 2017-02-01 13:48 ` Ed Bartosh
2017-02-01 13:48 ` [PATCH 17/18] wic: code cleanup Ed Bartosh
2017-02-01 13:48 ` [PATCH 18/18] wic: remove syslinux.py Ed Bartosh
17 siblings, 0 replies; 19+ messages in thread
From: Ed Bartosh @ 2017-02-01 13:48 UTC (permalink / raw)
To: openembedded-core
Removed unused enable_logstderr and disable_logstderr APIs.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/msger.py | 26 --------------------------
1 file changed, 26 deletions(-)
diff --git a/scripts/lib/wic/msger.py b/scripts/lib/wic/msger.py
index fb8336d..dc9b734 100644
--- a/scripts/lib/wic/msger.py
+++ b/scripts/lib/wic/msger.py
@@ -207,29 +207,3 @@ def set_logfile(fpath):
import atexit
atexit.register(_savelogf)
-
-def enable_logstderr(fpath):
- global CATCHERR_BUFFILE_FD
- global CATCHERR_BUFFILE_PATH
- global CATCHERR_SAVED_2
-
- if os.path.exists(fpath):
- os.remove(fpath)
- CATCHERR_BUFFILE_PATH = fpath
- CATCHERR_BUFFILE_FD = os.open(CATCHERR_BUFFILE_PATH, os.O_RDWR|os.O_CREAT)
- CATCHERR_SAVED_2 = os.dup(2)
- os.dup2(CATCHERR_BUFFILE_FD, 2)
-
-def disable_logstderr():
- global CATCHERR_BUFFILE_FD
- global CATCHERR_BUFFILE_PATH
- global CATCHERR_SAVED_2
-
- raw(msg=None) # flush message buffer and print it.
- os.dup2(CATCHERR_SAVED_2, 2)
- os.close(CATCHERR_SAVED_2)
- os.close(CATCHERR_BUFFILE_FD)
- os.unlink(CATCHERR_BUFFILE_PATH)
- CATCHERR_BUFFILE_FD = -1
- CATCHERR_BUFFILE_PATH = None
- CATCHERR_SAVED_2 = -1
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 17/18] wic: code cleanup
2017-02-01 13:48 [PATCH 00/18] #10619: refactor wic codebase (start) Ed Bartosh
` (15 preceding siblings ...)
2017-02-01 13:48 ` [PATCH 16/18] wic: msger.py: remove unused APIs Ed Bartosh
@ 2017-02-01 13:48 ` Ed Bartosh
2017-02-01 13:48 ` [PATCH 18/18] wic: remove syslinux.py Ed Bartosh
17 siblings, 0 replies; 19+ messages in thread
From: Ed Bartosh @ 2017-02-01 13:48 UTC (permalink / raw)
To: openembedded-core
Fixed indentation, unused imports, trailing lines etc.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
scripts/lib/wic/__version__.py | 1 -
scripts/lib/wic/engine.py | 2 +-
scripts/lib/wic/help.py | 4 ++--
scripts/lib/wic/partition.py | 11 +++++++----
scripts/lib/wic/plugin.py | 3 ++-
scripts/lib/wic/pluginbase.py | 1 -
scripts/lib/wic/plugins/imager/direct.py | 3 +--
scripts/lib/wic/plugins/source/bootimg-efi.py | 14 +++++++-------
scripts/lib/wic/plugins/source/bootimg-partition.py | 4 ++--
scripts/lib/wic/plugins/source/bootimg-pcbios.py | 4 +---
scripts/lib/wic/plugins/source/isoimage-isohybrid.py | 11 +++++------
scripts/lib/wic/plugins/source/rawcopy.py | 1 -
scripts/lib/wic/plugins/source/rootfs.py | 4 ++--
scripts/lib/wic/utils/misc.py | 9 +++------
scripts/lib/wic/utils/partitionedfs.py | 7 +++----
15 files changed, 36 insertions(+), 43 deletions(-)
delete mode 100644 scripts/lib/wic/__version__.py
diff --git a/scripts/lib/wic/__version__.py b/scripts/lib/wic/__version__.py
deleted file mode 100644
index 5452a46..0000000
--- a/scripts/lib/wic/__version__.py
+++ /dev/null
@@ -1 +0,0 @@
-VERSION = "2.00"
diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py
index 685bc88..592ef77 100644
--- a/scripts/lib/wic/engine.py
+++ b/scripts/lib/wic/engine.py
@@ -190,7 +190,7 @@ def wic_create(wks_file, rootfs_dir, bootimg_dir, kernel_dir,
crobj = creator.Creator()
cmdline = ["direct", native_sysroot, kernel_dir, bootimg_dir, rootfs_dir,
- wks_file, image_output_dir, oe_builddir, compressor or ""]
+ wks_file, image_output_dir, oe_builddir, compressor or ""]
if bmap:
cmdline.append('--bmap')
diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
index daa11bf..1bd411d 100644
--- a/scripts/lib/wic/help.py
+++ b/scripts/lib/wic/help.py
@@ -31,7 +31,7 @@ import logging
from wic.plugin import pluginmgr, PLUGIN_TYPES
def subcommand_error(args):
- logging.info("invalid subcommand %s" % args[0])
+ logging.info("invalid subcommand %s", args[0])
def display_help(subcommand, subcommands):
@@ -87,7 +87,7 @@ def invoke_subcommand(args, parser, main_command_usage, subcommands):
elif args[0] == "help":
wic_help(args, main_command_usage, subcommands)
elif args[0] not in subcommands:
- logging.error("Unsupported subcommand %s, exiting\n" % (args[0]))
+ logging.error("Unsupported subcommand %s, exiting\n", args[0])
parser.print_help()
return 1
else:
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 31a0350..69b369c 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -182,7 +182,7 @@ class Partition():
# further processing required Partition.size to be an integer, make
# sure that it is one
- if type(self.size) is not int:
+ if not isinstance(self.size, int):
msger.error("Partition %s internal size is not an integer. " \
"This a bug in source plugin %s and needs to be fixed." \
% (self.mountpoint, self.source))
@@ -242,7 +242,10 @@ class Partition():
# IMAGE_OVERHEAD_FACTOR and IMAGE_ROOTFS_EXTRA_SPACE
rsize_bb = get_bitbake_var('ROOTFS_SIZE')
if rsize_bb:
- msger.warning('overhead-factor was specified, but size was not, so bitbake variables will be used for the size. In this case both IMAGE_OVERHEAD_FACTOR and --overhead-factor will be applied')
+ msger.warning('overhead-factor was specified, but size was not,'
+ ' so bitbake variables will be used for the size.'
+ ' In this case both IMAGE_OVERHEAD_FACTOR and '
+ '--overhead-factor will be applied')
self.size = int(round(float(rsize_bb)))
for prefix in ("ext", "btrfs", "vfat", "squashfs"):
@@ -402,7 +405,8 @@ class Partition():
"Proceeding as requested." % self.mountpoint)
path = "%s/fs_%s.%s" % (cr_workdir, self.label, self.fstype)
- os.path.isfile(path) and os.remove(path)
+ if os.path.isfile(path):
+ os.remove(path)
# it is not possible to create a squashfs without source data,
# thus prepare an empty temp dir that is used as source
@@ -436,4 +440,3 @@ class Partition():
label_str = "-L %s" % self.label
mkswap_cmd = "mkswap %s -U %s %s" % (label_str, str(uuid.uuid1()), path)
exec_native_cmd(mkswap_cmd, native_sysroot)
-
diff --git a/scripts/lib/wic/plugin.py b/scripts/lib/wic/plugin.py
index 6b06ed6..f04a034 100644
--- a/scripts/lib/wic/plugin.py
+++ b/scripts/lib/wic/plugin.py
@@ -15,7 +15,8 @@
# with this program; if not, write to the Free Software Foundation, Inc., 59
# Temple Place - Suite 330, Boston, MA 02111-1307, USA.
-import os, sys
+import os
+import sys
from wic import msger
from wic import pluginbase
diff --git a/scripts/lib/wic/pluginbase.py b/scripts/lib/wic/pluginbase.py
index e737dee..2f747a9 100644
--- a/scripts/lib/wic/pluginbase.py
+++ b/scripts/lib/wic/pluginbase.py
@@ -17,7 +17,6 @@
__all__ = ['ImagerPlugin', 'SourcePlugin', 'get_plugins']
-import sys
from collections import defaultdict
from wic import msger
diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py
index 592412a..4637fbf3 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -29,7 +29,6 @@ import uuid
import tempfile
from time import strftime
-from os.path import basename, splitext
from wic import msger
from wic.ksparser import KickStart, KickStartError
@@ -65,7 +64,7 @@ class DirectPlugin(ImagerPlugin):
except KickStartError as err:
msger.error(str(err))
- name = "%s-%s" % (splitext(basename(ksconf))[0],
+ name = "%s-%s" % (os.path.splitext(os.path.basename(ksconf))[0],
strftime("%Y%m%d%H%M"))
# parse possible 'rootfs=name' items
diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py b/scripts/lib/wic/plugins/source/bootimg-efi.py
index 2b66a58..dd6c920 100644
--- a/scripts/lib/wic/plugins/source/bootimg-efi.py
+++ b/scripts/lib/wic/plugins/source/bootimg-efi.py
@@ -42,7 +42,7 @@ class BootimgEFIPlugin(SourcePlugin):
name = 'bootimg-efi'
@classmethod
- def do_configure_grubefi(cls, hdddir, creator, cr_workdir, source_params):
+ def do_configure_grubefi(cls, creator, cr_workdir):
"""
Create loader-specific (grub-efi) config
"""
@@ -54,10 +54,10 @@ class BootimgEFIPlugin(SourcePlugin):
# Use a custom configuration for grub
grubefi_conf = custom_cfg
msger.debug("Using custom configuration file "
- "%s for grub.cfg" % configfile)
+ "%s for grub.cfg" % configfile)
else:
msger.error("configfile is specified but failed to "
- "get it from %s." % configfile)
+ "get it from %s." % configfile)
if not custom_cfg:
# Create grub configuration using parameters from wks file
@@ -125,10 +125,10 @@ class BootimgEFIPlugin(SourcePlugin):
# Use a custom configuration for systemd-boot
boot_conf = custom_cfg
msger.debug("Using custom configuration file "
- "%s for systemd-boots's boot.conf" % configfile)
+ "%s for systemd-boots's boot.conf" % configfile)
else:
msger.error("configfile is specified but failed to "
- "get it from %s." % configfile)
+ "get it from %s." % configfile)
if not custom_cfg:
# Create systemd-boot configuration using parameters from wks file
@@ -164,7 +164,7 @@ class BootimgEFIPlugin(SourcePlugin):
try:
if source_params['loader'] == 'grub-efi':
- cls.do_configure_grubefi(hdddir, creator, cr_workdir, source_params)
+ cls.do_configure_grubefi(creator, cr_workdir)
elif source_params['loader'] == 'systemd-boot':
cls.do_configure_systemdboot(hdddir, creator, cr_workdir, source_params)
else:
@@ -185,7 +185,7 @@ class BootimgEFIPlugin(SourcePlugin):
if not bootimg_dir:
bootimg_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
if not bootimg_dir:
- msger.error("Couldn't find DEPLOY_DIR_IMAGE, exiting\n")
+ msger.error("Couldn't find DEPLOY_DIR_IMAGE, exiting\n")
# just so the result notes display it
creator.set_bootimg_dir(bootimg_dir)
diff --git a/scripts/lib/wic/plugins/source/bootimg-partition.py b/scripts/lib/wic/plugins/source/bootimg-partition.py
index f94dfab..e0d9a50 100644
--- a/scripts/lib/wic/plugins/source/bootimg-partition.py
+++ b/scripts/lib/wic/plugins/source/bootimg-partition.py
@@ -26,10 +26,11 @@
import os
import re
+from glob import glob
+
from wic import msger
from wic.pluginbase import SourcePlugin
from wic.utils.misc import exec_cmd, get_bitbake_var
-from glob import glob
class BootimgPartitionPlugin(SourcePlugin):
"""
@@ -137,4 +138,3 @@ class BootimgPartitionPlugin(SourcePlugin):
msger.debug('Prepare boot partition using rootfs in %s' % (hdddir))
part.prepare_rootfs(cr_workdir, oe_builddir, hdddir,
native_sysroot)
-
diff --git a/scripts/lib/wic/plugins/source/bootimg-pcbios.py b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
index d796433..4b9b265 100644
--- a/scripts/lib/wic/plugins/source/bootimg-pcbios.py
+++ b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
@@ -198,7 +198,5 @@ class BootimgPcbiosPlugin(SourcePlugin):
out = exec_cmd(du_cmd)
bootimg_size = out.split()[0]
- part.size = int(out.split()[0])
+ part.size = int(bootimg_size)
part.source_file = bootimg
-
-
diff --git a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
index 6df3756..56bb62d 100644
--- a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
+++ b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
@@ -100,10 +100,10 @@ class IsoImagePlugin(SourcePlugin):
grubefi_conf = get_custom_config(configfile)
if grubefi_conf:
msger.debug("Using custom configuration file "
- "%s for grub.cfg" % configfile)
+ "%s for grub.cfg" % configfile)
else:
msger.error("configfile is specified but failed to "
- "get it from %s." % configfile)
+ "get it from %s." % configfile)
else:
splash = os.path.join(cr_workdir, "EFI/boot/splash.jpg")
if os.path.exists(splash):
@@ -285,6 +285,7 @@ class IsoImagePlugin(SourcePlugin):
if not deploy_dir:
msger.error("Couldn't find DEPLOY_DIR_IMAGE, exiting\n")
cp_cmd = "cp %s/%s %s" % (deploy_dir, initrd, cr_workdir)
+ exec_cmd(cp_cmd)
else:
# Prepare initial ramdisk
initrd = "%s/initrd" % deploy_dir
@@ -293,8 +294,7 @@ class IsoImagePlugin(SourcePlugin):
if not os.path.isfile(initrd):
initrd = cls._build_initramfs_path(rootfs_dir, cr_workdir)
- install_cmd = "install -m 0644 %s %s/initrd" \
- % (initrd, isodir)
+ install_cmd = "install -m 0644 %s %s/initrd" % (initrd, isodir)
exec_cmd(install_cmd)
# Remove the temporary file created by _build_initramfs_path function
@@ -303,7 +303,7 @@ class IsoImagePlugin(SourcePlugin):
# Install bzImage
install_cmd = "install -m 0644 %s/bzImage %s/bzImage" % \
- (kernel_dir, isodir)
+ (kernel_dir, isodir)
exec_cmd(install_cmd)
#Create bootloader for efi boot
@@ -362,7 +362,6 @@ class IsoImagePlugin(SourcePlugin):
exec_native_cmd(grub_cmd, native_sysroot)
else:
- # TODO: insert gummiboot stuff
msger.error("unrecognized bootimg-efi loader: %s" \
% source_params['loader'])
except KeyError:
diff --git a/scripts/lib/wic/plugins/source/rawcopy.py b/scripts/lib/wic/plugins/source/rawcopy.py
index 79e41ca..4e42e3e 100644
--- a/scripts/lib/wic/plugins/source/rawcopy.py
+++ b/scripts/lib/wic/plugins/source/rawcopy.py
@@ -84,4 +84,3 @@ class RawCopyPlugin(SourcePlugin):
part.size = filesize
part.source_file = dst
-
diff --git a/scripts/lib/wic/plugins/source/rootfs.py b/scripts/lib/wic/plugins/source/rootfs.py
index 891f095..9d959fa 100644
--- a/scripts/lib/wic/plugins/source/rootfs.py
+++ b/scripts/lib/wic/plugins/source/rootfs.py
@@ -79,5 +79,5 @@ class RootfsPlugin(SourcePlugin):
real_rootfs_dir = cls.__get_rootfs_dir(rootfs_dir)
part.rootfs_dir = real_rootfs_dir
- part.prepare_rootfs(cr_workdir, oe_builddir, real_rootfs_dir, native_sysroot)
-
+ part.prepare_rootfs(cr_workdir, oe_builddir,
+ real_rootfs_dir, native_sysroot)
diff --git a/scripts/lib/wic/utils/misc.py b/scripts/lib/wic/utils/misc.py
index 6781d83..edb9c5b 100644
--- a/scripts/lib/wic/utils/misc.py
+++ b/scripts/lib/wic/utils/misc.py
@@ -77,18 +77,15 @@ def _exec_cmd(cmd_and_args, as_shell=False, catch=3):
msger.debug("_exec_cmd: output for %s (rc = %d): %s" % \
(cmd_and_args, ret, out))
- return (ret, out)
+ return ret, out
def exec_cmd(cmd_and_args, as_shell=False, catch=3):
"""
- Execute command, catching stderr, stdout
-
- Exits if rc non-zero
+ Execute command, return output
"""
- ret, out = _exec_cmd(cmd_and_args, as_shell, catch)
+ return _exec_cmd(cmd_and_args, as_shell, catch)[1]
- return out
def exec_native_cmd(cmd_and_args, native_sysroot, catch=3, pseudo=""):
"""
diff --git a/scripts/lib/wic/utils/partitionedfs.py b/scripts/lib/wic/utils/partitionedfs.py
index eb4d7e6..5fc5765 100644
--- a/scripts/lib/wic/utils/partitionedfs.py
+++ b/scripts/lib/wic/utils/partitionedfs.py
@@ -21,7 +21,7 @@
import os
from wic import msger
from wic.utils.errors import ImageError
-from wic.utils.misc import exec_cmd, exec_native_cmd
+from wic.utils.misc import exec_native_cmd
from wic.filemap import sparse_copy
# Overhead of the MBR partitioning scheme (just one sector)
@@ -350,10 +350,9 @@ class Image():
if self.disks:
for dev in self.disks:
disk = self.disks[dev]
- try:
+ if hasattr(disk['disk'], 'cleanup'):
disk['disk'].cleanup()
- except:
- pass
+
# remove partition images
for image in self.partimages:
if os.path.isfile(image):
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 18/18] wic: remove syslinux.py
2017-02-01 13:48 [PATCH 00/18] #10619: refactor wic codebase (start) Ed Bartosh
` (16 preceding siblings ...)
2017-02-01 13:48 ` [PATCH 17/18] wic: code cleanup Ed Bartosh
@ 2017-02-01 13:48 ` Ed Bartosh
17 siblings, 0 replies; 19+ messages in thread
From: Ed Bartosh @ 2017-02-01 13:48 UTC (permalink / raw)
To: openembedded-core
This module contains singe function serial_console_form_kargs, which
is used only by rootfs_pcbios_ext plugin. Moved it there and removed
syslinux module to make it easy to find and mainain plugin code.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
.../lib/wic/plugins/source/rootfs_pcbios_ext.py | 38 +++++++++++++-
scripts/lib/wic/utils/syslinux.py | 58 ----------------------
2 files changed, 37 insertions(+), 59 deletions(-)
delete mode 100644 scripts/lib/wic/utils/syslinux.py
diff --git a/scripts/lib/wic/plugins/source/rootfs_pcbios_ext.py b/scripts/lib/wic/plugins/source/rootfs_pcbios_ext.py
index 1032019..bd6fd6c 100644
--- a/scripts/lib/wic/plugins/source/rootfs_pcbios_ext.py
+++ b/scripts/lib/wic/plugins/source/rootfs_pcbios_ext.py
@@ -19,13 +19,49 @@
#
import os
+import re
+
from wic import msger
-from wic.utils import syslinux
from wic.utils import runner
from wic.utils.misc import get_bitbake_var, exec_cmd, exec_native_cmd
from wic.utils.errors import ImageError
from wic.pluginbase import SourcePlugin
+def serial_console_form_kargs(kernel_args):
+ """
+ Create SERIAL... line from kernel parameters
+
+ syslinux needs a line SERIAL port [baudrate [flowcontrol]]
+ in the syslinux.cfg file. The config line is generated based
+ on kernel boot parameters. The the parameters of the first
+ ttyS console are considered for syslinux config.
+ @param kernel_args kernel command line
+ @return line for syslinux config file e.g. "SERIAL 0 115200"
+ """
+ syslinux_conf = ""
+ for param in kernel_args.split():
+ param_match = re.match("console=ttyS([0-9]+),?([0-9]*)([noe]?)([0-9]?)(r?)", param)
+ if param_match:
+ syslinux_conf += "SERIAL " + param_match.group(1)
+ # baudrate
+ if param_match.group(2):
+ syslinux_conf += " " + param_match.group(2)
+ # parity
+ if param_match.group(3) and param_match.group(3) != 'n':
+ msger.warning("syslinux does not support parity for console. {} is ignored."
+ .format(param_match.group(3)))
+ # number of bits
+ if param_match.group(4) and param_match.group(4) != '8':
+ msger.warning("syslinux supports 8 bit console configuration only. {} is ignored."
+ .format(param_match.group(4)))
+ # flow control
+ if param_match.group(5) and param_match.group(5) != '':
+ msger.warning("syslinux console flowcontrol configuration. {} is ignored."
+ .format(param_match.group(5)))
+ break
+
+ return syslinux_conf
+
# pylint: disable=no-init
class RootfsPlugin(SourcePlugin):
diff --git a/scripts/lib/wic/utils/syslinux.py b/scripts/lib/wic/utils/syslinux.py
deleted file mode 100644
index aace286..0000000
--- a/scripts/lib/wic/utils/syslinux.py
+++ /dev/null
@@ -1,58 +0,0 @@
-# ex:ts=4:sw=4:sts=4:et
-# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
-#
-# This program is free software; you can redistribute it and/or modify it
-# under the terms of the GNU General Public License as published by the Free
-# Software Foundation; version 2 of the License
-#
-# This program is distributed in the hope that it will be useful, but
-# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
-# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
-# for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc., 59
-# Temple Place - Suite 330, Boston, MA 02111-1307, USA.
-#
-# AUTHOR
-# Adrian Freihofer <adrian.freihofer (at] neratec.com>
-
-
-import re
-from wic import msger
-
-
-def serial_console_form_kargs(kernel_args):
- """
- Create SERIAL... line from kernel parameters
-
- syslinux needs a line SERIAL port [baudrate [flowcontrol]]
- in the syslinux.cfg file. The config line is generated based
- on kernel boot parameters. The the parameters of the first
- ttyS console are considered for syslinux config.
- @param kernel_args kernel command line
- @return line for syslinux config file e.g. "SERIAL 0 115200"
- """
- syslinux_conf = ""
- for param in kernel_args.split():
- param_match = re.match("console=ttyS([0-9]+),?([0-9]*)([noe]?)([0-9]?)(r?)", param)
- if param_match:
- syslinux_conf += "SERIAL " + param_match.group(1)
- # baudrate
- if param_match.group(2):
- syslinux_conf += " " + param_match.group(2)
- # parity
- if param_match.group(3) and param_match.group(3) != 'n':
- msger.warning("syslinux does not support parity for console. {} is ignored."
- .format(param_match.group(3)))
- # number of bits
- if param_match.group(4) and param_match.group(4) != '8':
- msger.warning("syslinux supports 8 bit console configuration only. {} is ignored."
- .format(param_match.group(4)))
- # flow control
- if param_match.group(5) and param_match.group(5) != '':
- msger.warning("syslinux console flowcontrol configuration. {} is ignored."
- .format(param_match.group(5)))
- break
-
- return syslinux_conf
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
end of thread, other threads:[~2017-02-01 14:11 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-02-01 13:48 [PATCH 00/18] #10619: refactor wic codebase (start) Ed Bartosh
2017-02-01 13:48 ` [PATCH 01/18] wic: creator: stop using config manager Ed Bartosh
2017-02-01 13:48 ` [PATCH 02/18] wic: direct_plugin: " Ed Bartosh
2017-02-01 13:48 ` [PATCH 03/18] wic: removed conf.py and empty config file Ed Bartosh
2017-02-01 13:48 ` [PATCH 04/18] wic: moved content of direct.py to direct_plugin Ed Bartosh
2017-02-01 13:48 ` [PATCH 05/18] wic: get rid of __rootfs_dir_to_dict method Ed Bartosh
2017-02-01 13:48 ` [PATCH 06/18] wic: improve naming in direct_plugin classes Ed Bartosh
2017-02-01 13:48 ` [PATCH 07/18] wic: pylinted direct_plugin Ed Bartosh
2017-02-01 13:48 ` [PATCH 08/18] wic: simplified code of direct_plugin Ed Bartosh
2017-02-01 13:48 ` [PATCH 09/18] wic: renamd direct_plugin.py -> direct.py Ed Bartosh
2017-02-01 13:48 ` [PATCH 10/18] wic: removed test file Ed Bartosh
2017-02-01 13:48 ` [PATCH 11/18] wic: partition: simlify calling plugin methods Ed Bartosh
2017-02-01 13:48 ` [PATCH 12/18] wci: misc: removed build_name API Ed Bartosh
2017-02-01 13:48 ` [PATCH 13/18] wic: move 2 APIs to wic.engine Ed Bartosh
2017-02-01 13:48 ` [PATCH 14/18] wic: move oe/misc.py one level up Ed Bartosh
2017-02-01 13:48 ` [PATCH 15/18] wic: removed code from __init__.py Ed Bartosh
2017-02-01 13:48 ` [PATCH 16/18] wic: msger.py: remove unused APIs Ed Bartosh
2017-02-01 13:48 ` [PATCH 17/18] wic: code cleanup Ed Bartosh
2017-02-01 13:48 ` [PATCH 18/18] wic: remove syslinux.py Ed Bartosh
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox