[PATCH 0/3] 'wic'- OpenEmbedded Image Creator

[PATCH 0/3] 'wic'- OpenEmbedded Image Creator

[Tom Zanussi]

This patchset implements a new command named 'wic' (for OpenEmbedded
Image Creator).  Please see [YOCTO #3847] for extensive background on
what's implemented here.

This is a completely independent standalone utility that initially
provides easier-to-use and more flexible replacements for a couple
bits of existing functionality in oe-core: directdisk.bbclass and
mkefidisk.sh.  The more interesting aspect of the current patchset
though is that the replaced (this patchset doesn't actually replace
those scripts, however) scripts are implemented by a general-purpose
partitioning 'language' based on Redhat kickstart syntax (with the
underlying code borrowed from Tizen mic, which in turn was borrowed
from Meego mic, in turn borrowed from Fedora livecd, etc.).

Though this patchset only uses the kickstart syntax related to
partitioning and bootloaders and only for creating images, because the
code is based on the mic/pykickstart code, future deployment efforts
such as those partially described by [YOCTO #4106], but also others
including package selection (from e.g. binary feeds) and deployment
configuration of users/network/services, etc, could be implemented
under this framework, considering that all of those are implemented
in some form by the base system.

The current patchset has successfully been used to generate and boot
images from core-image-minimal build artifacts using the 'mkefidisk'
kickstart file (in scripts/lib/image/canned-wks) on minnow and the
'directdisk' kickstart file using both core-image-minimal and
core-image-sato artifacts for crownbay.

The 'wic' command generates partitioned images from existing
OpenEmbedded build artifacts.  Image generation is driven by
partitioning commands contained in an 'Openembedded kickstart' (.wks)
file specified either directly on the command-line or as one of a
selection of canned .wks files (see 'wic list images').  When applied
to a given set of build artifacts, the result is an image or set of
images that can be directly written onto media and used on a
particular system.

It can be used in 'raw' mode, where artifacts are explicitly specified
via command-line arguments (see example below), or it can be used in a
more easily usable 'cooked' mode which uses the current MACHINE
setting and a specified image name to automatically locate the
artifacts used to create the image.

OE kickstart files (.wks) can of course be specified directly on the
command-line, but the user can also choose from a set of 'canned' .wks
files available via the 'wic list images' command (example below).

In any case, the prerequisite for generating any image is to have the
build artifacts already available.  The below examples assume the user
has already build a 'core-image-minimal' for a specific machine
(future versions won't require this redundant step, but for now that's
typically how build artifacts get generated).

The other prerequisite is to source the build environment:

$ source oe-init-build-env

To start out with, we'll generate an image from one of the canned .wks
files.  The following generates a list of availailable images:

$ wic list images
  mkefidisk             Create an EFI disk image
  directdisk            Create a 'pcbios' direct disk image

You can get more information about any of the available images by typing
'wic list xxx help', where 'xxx' is one of the image names:

$ wic list mkefidisk help

Creates a partitioned EFI disk image that the user
can directly dd to boot media.

At any time, you can get help on the 'wic' command or any subcommand
(currently 'list' and 'create').  For instance, to get the description
of 'wic create' command and its parameters:

$ wic create

Usage: 

 Create a new OpenEmbedded image

 usage: wic create <wks file or image name> [-o <DIRNAME> | --outdir <DIRNAME>]
            [-i <JSON PROPERTY FILE> | --infile <JSON PROPERTY_FILE>]
            [-e | --image-name] [-r, --rootfs-dir] [-b, --bootimg-dir]
            [-k, --kernel-dir] [-n, --native-sysroot] [-s, --skip-build-check]

 This command creates an OpenEmbedded image based on the 'OE kickstart
 commands' found in the <wks file>.

 The -o option can be used to place the image in a directory with a
 different name and location.

 See 'wic help create' for more detailed instructions.

...

As mentioned in the command, you can get even more detailed information
by adding 'help' to the above:

$ wic help create

NAME
    wic create - Create a new OpenEmbedded image

SYNOPSIS
    wic create <wks file or image name> [-o <DIRNAME> | --outdir
<DIRNAME>]
        [-i <JSON PROPERTY FILE> | --infile <JSON PROPERTY_FILE>]
        [-e | --image-name] [-r, --rootfs-dir] [-b, --bootimg-dir]
        [-k, --kernel-dir] [-n, --native-sysroot] [-s,
--skip-build-check]

DESCRIPTION
    This command creates an OpenEmbedded image based on the 'OE
    kickstart commands' found in the <wks file>.

    In order to do this, wic needs to know the locations of the
    various build artifacts required to build the image.

    Users can explicitly specify the build artifact locations using
    the -r, -b, -k, and -n options.  See below for details on where
    the corresponding artifacts are typically found in a normal
    OpenEmbedded build.

    Alternatively, users can use the -e option to have 'mic' determine
    those locations for a given image.  If the -e option is used, the
    user needs to have set the appropriate MACHINE variable in
    local.conf, and have sourced the build environment.

    The -e option is used to specify the name of the image to use the
    artifacts from e.g. core-image-sato.

    The -r option is used to specify the path to the /rootfs dir to
    use as the .wks rootfs source.

    The -b option is used to specify the path to the dir containing
    the boot artifacts (e.g. /EFI or /syslinux dirs) to use as the
    .wks bootimg source.

    The -k option is used to specify the path to the dir containing
    the kernel to use in the .wks bootimg.

    The -n option is used to specify the path to the native sysroot
    containing the tools to use to build the image.

    The -s option is used to skip the build check.  The build check is
    a simple sanity check used to determine whether the user has
    sourced the build environment so that the -e option can operate
    correctly.  If the user has specified the build artifact locations
    explicitly, 'wic' assumes the user knows what he or she is doing
    and skips the build check.

    When 'wic -e' is used, the locations for the build artifacts
    values are determined by 'wic -e' from the output of the 'bitbake
    -e' command given an image name e.g. 'core-image-minimal' and a
    given machine set in local.conf.  In that case, the image is
    created as if the following 'bitbake -e' variables were used:

    -r:        IMAGE_ROOTFS
    -k:        STAGING_KERNEL_DIR
    -n:        STAGING_DIR_NATIVE
    -b:        HDDDIR and STAGING_DATA_DIR (handlers decide which to
use)

    If 'wic -e' is not used, the user needs to select the appropriate
    value for -b (as well as -r, -k, and -n).

    The -o option can be used to place the image in a directory with a
    different name and location.

    As an alternative to the wks file, the image-specific properties
    that define the values that will be used to generate a particular
    image can be specified on the command-line using the -i option and
    supplying a JSON object consisting of the set of name:value pairs
    needed by image creation.

    The set of properties available for a given image type can be
    listed using the 'wic list' command.

So, the easiest way to create an image is to use the -e option with a
canned .wks file.  To use the -e option, you need to specify the image
used to generate the artifacts and you actually need to have the MACHINE
used to build them specified in your local.conf (these requirements
aren't necessary if you aren't using the -e options.  Below, we generate
a directdisk image, pointing the process at the core-image-minimal
artifacts for the current MACHINE:

$ wic create directdisk -e core-image-minimal

Checking basic build environment...
Done.

Creating image(s)...

Info: The new image(s) can be found here:
  /var/tmp/wic/build/directdisk-201309252350-sda.direct

The following build artifacts were used to create the image(s):

ROOTFS_DIR:      /home/trz/yocto/yocto-image/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs
BOOTIMG_DIR:     /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/share
KERNEL_DIR:      /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel
NATIVE_SYSROOT:  /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel

The image(s) were created using OE kickstart file:
  /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/directdisk.wks

The output shows the name and location of the image created, and so that
you know exactly what was used to generate the image, each of the
artifacts and the kickstart file used.

Similarly, I can create a 'mkefidisk' image in the same way (notice that
I'm using a different machine - because I'm using the -e option, I
needed to change the MACHINE in my local.conf):

$ wic create mkefidisk -e core-image-minimal
Checking basic build environment...
Done.

Creating image(s)...

Info: The new image(s) can be found here:
  /var/tmp/wic/build/mkefidisk-201309260027-sda.direct

The following build artifacts were used to create the image(s):

ROOTFS_DIR:      /home/trz/yocto/yocto-image/build/tmp/work/minnow-poky-linux/core-image-minimal/1.0-r0/rootfs
BOOTIMG_DIR:     /home/trz/yocto/yocto-image/build/tmp/work/minnow-poky-linux/core-image-minimal/1.0-r0/core-image-minimal-1.0/hddimg
KERNEL_DIR:      /home/trz/yocto/yocto-image/build/tmp/sysroots/minnow/usr/src/kernel
NATIVE_SYSROOT:  /home/trz/yocto/yocto-image/build/tmp/sysroots/minnow/usr/src/kernel

The image(s) were created using OE kickstart file:
  /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/mkefidisk.wks

Here's an example that doesn't take the easy way out and manually
specifies each build artifact, along with a non-canned .wks file, and
also uses the -o option to have wic create the output somewhere other
than the default /var/tmp/wic:

$ wic create ~/test.wks -o /home/trz/testwic
--rootfs-dir /home/trz/yocto/yocto-image/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs --bootimg-dir /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/share --kernel-dir /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel --native-sysroot /home/trz/yocto/yocto-image/build/tmp/sysroots/x86_64-linux

Creating image(s)...

Info: The new image(s) can be found here:
  /home/trz/testwic/build/test-201309260032-sda.direct

The following build artifacts were used to create the image(s):

ROOTFS_DIR:      /home/trz/yocto/yocto-image/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs
BOOTIMG_DIR:     /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/share
KERNEL_DIR:      /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel
NATIVE_SYSROOT:  /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel

The image(s) were created using OE kickstart file:
  /home/trz/test.wks

Finally, here's an example of the actual partition language commands
used to generate the mkefidisk image i.e. these are the contents of the
mkefidisk.wks OE kickstart file:

# short-description: Create an EFI disk image
# long-description: Creates a partitioned EFI disk image that the user
# can directly dd to boot media.

part /boot --source bootimg --ondisk sda --fstype=efi --label msdos --active --align 1024

part / --source rootfs --ondisk sda --fstype=ext3 --label platform --align 1024

part swap --ondisk sda --size 44 --label swap1 --fstype=swap

bootloader  --timeout=10  --append="rootwait rootfstype=ext3 console=ttyPCH0,115200 console=tty0 vmalloc=256MB snd-hda-intel.enable_msi=0"

The initial 'wic' implementation here supports only the basic
kickstart partitioning commands: 'partition' (or 'part' for short) and
'bootloader'.

Currently, only the options used in the 'canned' scripts (in
scripts/lib/image/canned-wks) are supported.

They're listed below and mostly follow the syntax and meaning of the
standard kickstart options for those commands:

http://fedoraproject.org/wiki/Anaconda/Kickstart#part_or_partition
http://fedoraproject.org/wiki/Anaconda/Kickstart#bootloader

For the 'partition' ('part') command, here are the supported options:

--source
	bootimg
	rootfs
--ondisk sda
--size size (in mB)
--fstype
	msdos
	efi
	ext4
	ext3
	ext2
	btrfs
	swap
--label label
--active
---align (in kB)

The --source option is a wic-specific option that can currently have
one of two values, 'bootimg' or 'rootfs'.

If '--source rootfs' is used, it tells the wic command to create a
partition as large as needed to fill with the contents of /rootfs
(specified by the -r 'wic' option) and to fill it with the contents of
/rootfs.

If '--source bootimg' is used, it tells the wic command to create a
partition as large as needed to fill with the contents of the boot
partition (specified by the -b 'wic' option).  Exactly what those
contents are depend on the value of the --fstype option for that
partition.  If '--fstype=efi' is specified, the boot artifacts
contained in HDDDIR are used, and if '--fstype=msdos' is specified,
the boot artifacts found in STAGING_DATADIR are used.

The '--align' option is a mic-specific option that says to start a
partition on an x kB boundary.

For the 'bootloader' command, these are the supported options:

--timeout
--append

Future updates will implement more options - using anything not
explicitly supported can result in unpredicable results.


The following changes since commit b049d532f6e0ab9e458e486f81b00be47ee69acf:

  testimage: Exclude BB_ORIGENV variable (2013-09-26 17:27:02 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib.git tzanussi/wic-v1
  http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=tzanussi/wic-v1

Tom Zanussi (3):
  wic: Initial code for wic (OpenEmbedded Image Creator)
  wic: Add mic w/pykickstart
  wic: Add OpenEmbedded-specific implementation

 scripts/lib/image/__init__.py                      |   22 +
 scripts/lib/image/canned-wks/directdisk.wks        |   10 +
 scripts/lib/image/canned-wks/mkefidisk.wks         |   11 +
 scripts/lib/image/config/wic.conf                  |    7 +
 scripts/lib/image/engine.py                        |  280 ++++
 scripts/lib/image/help.py                          |  311 ++++
 scripts/lib/mic/3rdparty/pykickstart/base.py       |  466 ++++++
 .../mic/3rdparty/pykickstart/commands/__init__.py  |   26 +
 .../3rdparty/pykickstart/commands/authconfig.py    |   40 +
 .../mic/3rdparty/pykickstart/commands/autopart.py  |  119 ++
 .../mic/3rdparty/pykickstart/commands/autostep.py  |   55 +
 .../3rdparty/pykickstart/commands/bootloader.py    |  265 ++++
 .../mic/3rdparty/pykickstart/commands/clearpart.py |   86 ++
 .../mic/3rdparty/pykickstart/commands/device.py    |  125 ++
 .../3rdparty/pykickstart/commands/deviceprobe.py   |   40 +
 .../3rdparty/pykickstart/commands/displaymode.py   |   68 +
 .../mic/3rdparty/pykickstart/commands/dmraid.py    |   91 ++
 .../3rdparty/pykickstart/commands/driverdisk.py    |  184 +++
 .../lib/mic/3rdparty/pykickstart/commands/fcoe.py  |  114 ++
 .../mic/3rdparty/pykickstart/commands/firewall.py  |  193 +++
 .../mic/3rdparty/pykickstart/commands/firstboot.py |   62 +
 .../lib/mic/3rdparty/pykickstart/commands/group.py |   88 ++
 .../3rdparty/pykickstart/commands/ignoredisk.py    |  139 ++
 .../3rdparty/pykickstart/commands/interactive.py   |   58 +
 .../lib/mic/3rdparty/pykickstart/commands/iscsi.py |  133 ++
 .../mic/3rdparty/pykickstart/commands/iscsiname.py |   54 +
 .../lib/mic/3rdparty/pykickstart/commands/key.py   |   64 +
 .../mic/3rdparty/pykickstart/commands/keyboard.py  |   55 +
 .../lib/mic/3rdparty/pykickstart/commands/lang.py  |   60 +
 .../3rdparty/pykickstart/commands/langsupport.py   |   58 +
 .../mic/3rdparty/pykickstart/commands/lilocheck.py |   54 +
 .../mic/3rdparty/pykickstart/commands/logging.py   |   66 +
 .../mic/3rdparty/pykickstart/commands/logvol.py    |  304 ++++
 .../3rdparty/pykickstart/commands/mediacheck.py    |   53 +
 .../mic/3rdparty/pykickstart/commands/method.py    |  186 +++
 .../mic/3rdparty/pykickstart/commands/monitor.py   |  106 ++
 .../lib/mic/3rdparty/pykickstart/commands/mouse.py |   70 +
 .../mic/3rdparty/pykickstart/commands/multipath.py |  111 ++
 .../mic/3rdparty/pykickstart/commands/network.py   |  363 +++++
 .../mic/3rdparty/pykickstart/commands/partition.py |  353 +++++
 .../lib/mic/3rdparty/pykickstart/commands/raid.py  |  365 +++++
 .../mic/3rdparty/pykickstart/commands/reboot.py    |   79 +
 .../lib/mic/3rdparty/pykickstart/commands/repo.py  |  249 +++
 .../mic/3rdparty/pykickstart/commands/rescue.py    |   68 +
 .../mic/3rdparty/pykickstart/commands/rootpw.py    |   93 ++
 .../mic/3rdparty/pykickstart/commands/selinux.py   |   64 +
 .../mic/3rdparty/pykickstart/commands/services.py  |   71 +
 .../lib/mic/3rdparty/pykickstart/commands/skipx.py |   54 +
 .../lib/mic/3rdparty/pykickstart/commands/sshpw.py |  105 ++
 .../mic/3rdparty/pykickstart/commands/timezone.py  |   86 ++
 .../mic/3rdparty/pykickstart/commands/updates.py   |   60 +
 .../mic/3rdparty/pykickstart/commands/upgrade.py   |  106 ++
 .../lib/mic/3rdparty/pykickstart/commands/user.py  |  173 +++
 .../lib/mic/3rdparty/pykickstart/commands/vnc.py   |  114 ++
 .../mic/3rdparty/pykickstart/commands/volgroup.py  |  102 ++
 .../mic/3rdparty/pykickstart/commands/xconfig.py   |  184 +++
 .../mic/3rdparty/pykickstart/commands/zerombr.py   |   69 +
 .../lib/mic/3rdparty/pykickstart/commands/zfcp.py  |  134 ++
 scripts/lib/mic/3rdparty/pykickstart/constants.py  |   57 +
 scripts/lib/mic/3rdparty/pykickstart/errors.py     |  103 ++
 .../mic/3rdparty/pykickstart/handlers/control.py   | 1307 ++++++++++++++++
 .../lib/mic/3rdparty/pykickstart/handlers/f10.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/f11.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/f12.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/f13.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/f14.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/f15.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/f16.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/f7.py    |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/f8.py    |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/f9.py    |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/fc3.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/fc4.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/fc5.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/fc6.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/rhel3.py |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/rhel4.py |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/rhel5.py |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/rhel6.py |   24 +
 scripts/lib/mic/3rdparty/pykickstart/ko.py         |   37 +
 scripts/lib/mic/3rdparty/pykickstart/options.py    |  204 +++
 scripts/lib/mic/3rdparty/pykickstart/parser.py     |  702 +++++++++
 scripts/lib/mic/3rdparty/pykickstart/sections.py   |  244 +++
 .../3rdparty/pykickstart/urlgrabber/__init__.py    |   53 +
 .../3rdparty/pykickstart/urlgrabber/byterange.py   |  463 ++++++
 .../mic/3rdparty/pykickstart/urlgrabber/grabber.py | 1477 ++++++++++++++++++
 .../3rdparty/pykickstart/urlgrabber/keepalive.py   |  617 ++++++++
 .../mic/3rdparty/pykickstart/urlgrabber/mirror.py  |  458 ++++++
 .../3rdparty/pykickstart/urlgrabber/progress.py    |  530 +++++++
 .../3rdparty/pykickstart/urlgrabber/sslfactory.py  |   90 ++
 scripts/lib/mic/3rdparty/pykickstart/version.py    |  197 +++
 scripts/lib/mic/__init__.py                        |    4 +
 scripts/lib/mic/__version__.py                     |    1 +
 scripts/lib/mic/bootstrap.py                       |  279 ++++
 scripts/lib/mic/chroot.py                          |  343 +++++
 scripts/lib/mic/conf.py                            |  201 +++
 scripts/lib/mic/creator.py                         |  351 +++++
 scripts/lib/mic/imager/baseimager.py               | 1265 ++++++++++++++++
 scripts/lib/mic/imager/direct.py                   |  472 ++++++
 scripts/lib/mic/imager/fs.py                       |   99 ++
 scripts/lib/mic/imager/livecd.py                   |  750 +++++++++
 scripts/lib/mic/imager/liveusb.py                  |  308 ++++
 scripts/lib/mic/imager/loop.py                     |  418 ++++++
 scripts/lib/mic/imager/raw.py                      |  501 +++++++
 scripts/lib/mic/kickstart/__init__.py              |  892 +++++++++++
 .../lib/mic/kickstart/custom_commands/__init__.py  |   17 +
 .../lib/mic/kickstart/custom_commands/desktop.py   |   95 ++
 .../mic/kickstart/custom_commands/installerfw.py   |   63 +
 .../lib/mic/kickstart/custom_commands/micboot.py   |   49 +
 .../mic/kickstart/custom_commands/micpartition.py  |   57 +
 .../lib/mic/kickstart/custom_commands/micrepo.py   |  127 ++
 .../lib/mic/kickstart/custom_commands/partition.py |  370 +++++
 scripts/lib/mic/msger.py                           |  309 ++++
 scripts/lib/mic/plugin.py                          |  102 ++
 scripts/lib/mic/pluginbase.py                      |  101 ++
 scripts/lib/mic/plugins/backend/yumpkgmgr.py       |  490 ++++++
 scripts/lib/mic/plugins/backend/zypppkgmgr.py      |  973 ++++++++++++
 scripts/lib/mic/plugins/hook/empty_hook.py         |    3 +
 scripts/lib/mic/plugins/imager/direct_plugin.py    |   92 ++
 scripts/lib/mic/plugins/imager/fs_plugin.py        |  143 ++
 scripts/lib/mic/plugins/imager/livecd_plugin.py    |  255 ++++
 scripts/lib/mic/plugins/imager/liveusb_plugin.py   |  260 ++++
 scripts/lib/mic/plugins/imager/loop_plugin.py      |  255 ++++
 scripts/lib/mic/plugins/imager/raw_plugin.py       |  275 ++++
 scripts/lib/mic/rt_util.py                         |  223 +++
 scripts/lib/mic/test                               |    1 +
 scripts/lib/mic/utils/BmapCreate.py                |  298 ++++
 scripts/lib/mic/utils/Fiemap.py                    |  252 ++++
 scripts/lib/mic/utils/cmdln.py                     | 1586 ++++++++++++++++++++
 scripts/lib/mic/utils/errors.py                    |   71 +
 scripts/lib/mic/utils/fs_related.py                | 1057 +++++++++++++
 scripts/lib/mic/utils/gpt_parser.py                |  331 ++++
 scripts/lib/mic/utils/grabber.py                   |   97 ++
 scripts/lib/mic/utils/misc.py                      | 1067 +++++++++++++
 scripts/lib/mic/utils/oe/__init__.py               |   22 +
 scripts/lib/mic/utils/oe/misc.py                   |  108 ++
 scripts/lib/mic/utils/partitionedfs.py             |  776 ++++++++++
 scripts/lib/mic/utils/proxy.py                     |  183 +++
 scripts/lib/mic/utils/rpmmisc.py                   |  600 ++++++++
 scripts/lib/mic/utils/runner.py                    |  109 ++
 scripts/wic                                        |  185 +++
 141 files changed, 30983 insertions(+)
 create mode 100644 scripts/lib/image/__init__.py
 create mode 100644 scripts/lib/image/canned-wks/directdisk.wks
 create mode 100644 scripts/lib/image/canned-wks/mkefidisk.wks
 create mode 100644 scripts/lib/image/config/wic.conf
 create mode 100644 scripts/lib/image/engine.py
 create mode 100644 scripts/lib/image/help.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/__init__.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/base.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/__init__.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/authconfig.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/autopart.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/autostep.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/bootloader.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/clearpart.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/device.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/deviceprobe.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/displaymode.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/dmraid.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/driverdisk.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/fcoe.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/firewall.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/firstboot.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/group.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/ignoredisk.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/interactive.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/iscsi.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/iscsiname.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/key.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/keyboard.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/lang.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/langsupport.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/lilocheck.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/logging.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/logvol.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/mediacheck.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/method.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/monitor.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/mouse.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/multipath.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/network.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/partition.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/raid.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/reboot.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/repo.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/rescue.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/rootpw.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/selinux.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/services.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/skipx.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/sshpw.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/timezone.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/updates.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/upgrade.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/user.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/vnc.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/volgroup.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/xconfig.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/zerombr.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/zfcp.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/constants.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/errors.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/__init__.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/control.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f10.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f11.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f12.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f13.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f14.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f15.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f16.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f7.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f8.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f9.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/fc3.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/fc4.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/fc5.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/fc6.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/rhel3.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/rhel4.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/rhel5.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/rhel6.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/ko.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/options.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/parser.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/sections.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/urlgrabber/__init__.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/urlgrabber/byterange.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/urlgrabber/grabber.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/urlgrabber/keepalive.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/urlgrabber/mirror.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/urlgrabber/progress.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/urlgrabber/sslfactory.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/version.py
 create mode 100644 scripts/lib/mic/__init__.py
 create mode 100644 scripts/lib/mic/__version__.py
 create mode 100644 scripts/lib/mic/bootstrap.py
 create mode 100644 scripts/lib/mic/chroot.py
 create mode 100644 scripts/lib/mic/conf.py
 create mode 100644 scripts/lib/mic/creator.py
 create mode 100644 scripts/lib/mic/imager/__init__.py
 create mode 100644 scripts/lib/mic/imager/baseimager.py
 create mode 100644 scripts/lib/mic/imager/direct.py
 create mode 100644 scripts/lib/mic/imager/fs.py
 create mode 100644 scripts/lib/mic/imager/livecd.py
 create mode 100644 scripts/lib/mic/imager/liveusb.py
 create mode 100644 scripts/lib/mic/imager/loop.py
 create mode 100644 scripts/lib/mic/imager/raw.py
 create mode 100644 scripts/lib/mic/kickstart/__init__.py
 create mode 100644 scripts/lib/mic/kickstart/custom_commands/__init__.py
 create mode 100644 scripts/lib/mic/kickstart/custom_commands/desktop.py
 create mode 100644 scripts/lib/mic/kickstart/custom_commands/installerfw.py
 create mode 100644 scripts/lib/mic/kickstart/custom_commands/micboot.py
 create mode 100644 scripts/lib/mic/kickstart/custom_commands/micpartition.py
 create mode 100644 scripts/lib/mic/kickstart/custom_commands/micrepo.py
 create mode 100644 scripts/lib/mic/kickstart/custom_commands/partition.py
 create mode 100644 scripts/lib/mic/msger.py
 create mode 100644 scripts/lib/mic/plugin.py
 create mode 100644 scripts/lib/mic/pluginbase.py
 create mode 100644 scripts/lib/mic/plugins/backend/yumpkgmgr.py
 create mode 100755 scripts/lib/mic/plugins/backend/zypppkgmgr.py
 create mode 100644 scripts/lib/mic/plugins/hook/.py
 create mode 100644 scripts/lib/mic/plugins/hook/empty_hook.py
 create mode 100644 scripts/lib/mic/plugins/imager/direct_plugin.py
 create mode 100644 scripts/lib/mic/plugins/imager/fs_plugin.py
 create mode 100644 scripts/lib/mic/plugins/imager/livecd_plugin.py
 create mode 100644 scripts/lib/mic/plugins/imager/liveusb_plugin.py
 create mode 100644 scripts/lib/mic/plugins/imager/loop_plugin.py
 create mode 100644 scripts/lib/mic/plugins/imager/raw_plugin.py
 create mode 100644 scripts/lib/mic/rt_util.py
 create mode 100644 scripts/lib/mic/test
 create mode 100644 scripts/lib/mic/utils/BmapCreate.py
 create mode 100644 scripts/lib/mic/utils/Fiemap.py
 create mode 100644 scripts/lib/mic/utils/__init__.py
 create mode 100644 scripts/lib/mic/utils/cmdln.py
 create mode 100644 scripts/lib/mic/utils/errors.py
 create mode 100644 scripts/lib/mic/utils/fs_related.py
 create mode 100644 scripts/lib/mic/utils/gpt_parser.py
 create mode 100644 scripts/lib/mic/utils/grabber.py
 create mode 100644 scripts/lib/mic/utils/misc.py
 create mode 100644 scripts/lib/mic/utils/oe/__init__.py
 create mode 100644 scripts/lib/mic/utils/oe/misc.py
 create mode 100644 scripts/lib/mic/utils/partitionedfs.py
 create mode 100644 scripts/lib/mic/utils/proxy.py
 create mode 100644 scripts/lib/mic/utils/rpmmisc.py
 create mode 100644 scripts/lib/mic/utils/runner.py
 create mode 100755 scripts/wic

-- 
1.7.11.4

[PATCH 1/3] wic: Initial code for wic (OpenEmbedded Image Creator)

[Tom Zanussi]

Initial implementation of the 'wic' command.

The 'wic' command generates partitioned images from existing
OpenEmbedded build artifacts.  Image generation is driven by
partitioning commands contained in an 'Openembedded kickstart' (.wks)
file specified either directly on the command-line or as one of a
selection of canned .wks files (see 'wic list images').  When applied
to a given set of build artifacts, the result is an image or set of
images that can be directly written onto media and used on a
particular system.

'wic' is based loosely on the 'mic' (Meego Image Creator) framework,
but heavily modified to make direct use of OpenEmbedded build
artifacts instead of package installation and configuration, things
already incorporated int the OE artifacts.

The name 'wic' comes from 'oeic' with the 'oe' diphthong promoted to
the letter 'w', because 'oeic' is impossible to remember or pronounce.

This covers the mechanics of invoking and providing help for the
command and sub-commands; it contains hooks for future commits to
connect with the actual functionality, once implemented.

Help is integrated into the 'wic' command - see that for details on
usage.

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
---
 scripts/lib/image/__init__.py |  22 +++
 scripts/lib/image/engine.py   | 256 ++++++++++++++++++++++++++++++++++
 scripts/lib/image/help.py     | 311 ++++++++++++++++++++++++++++++++++++++++++
 scripts/wic                   | 185 +++++++++++++++++++++++++
 4 files changed, 774 insertions(+)
 create mode 100644 scripts/lib/image/__init__.py
 create mode 100644 scripts/lib/image/engine.py
 create mode 100644 scripts/lib/image/help.py
 create mode 100755 scripts/wic

diff --git a/scripts/lib/image/__init__.py b/scripts/lib/image/__init__.py
new file mode 100644
index 0000000..1ff814e
--- /dev/null
+++ b/scripts/lib/image/__init__.py
@@ -0,0 +1,22 @@
+#
+# OpenEmbedded Image tools library
+#
+# Copyright (c) 2013, Intel Corporation.
+# All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+# AUTHORS
+# Tom Zanussi <tom.zanussi (at] linux.intel.com>
+#
diff --git a/scripts/lib/image/engine.py b/scripts/lib/image/engine.py
new file mode 100644
index 0000000..a9b530c
--- /dev/null
+++ b/scripts/lib/image/engine.py
@@ -0,0 +1,256 @@
+# ex:ts=4:sw=4:sts=4:et
+# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
+#
+# Copyright (c) 2013, Intel Corporation.
+# All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+# DESCRIPTION
+
+# This module implements the image creation engine used by 'wic' to
+# create images.  The engine parses through the OpenEmbedded kickstart
+# (wks) file specified and generates images that can then be directly
+# written onto media.
+#
+# AUTHORS
+# Tom Zanussi <tom.zanussi (at] linux.intel.com>
+#
+
+import os
+import sys
+from abc import ABCMeta, abstractmethod
+import shlex
+import json
+import subprocess
+import shutil
+
+import os, sys, errno
+
+
+def verify_build_env():
+    """
+    Verify that the build environment is sane.
+
+    Returns True if it is, false otherwise
+    """
+    try:
+        builddir = os.environ["BUILDDIR"]
+    except KeyError:
+        print "BUILDDIR not found, exiting. (Did you forget to source oe-init-build-env?)"
+        sys.exit(1)
+
+    return True
+
+
+def get_line_val(line, key):
+    """
+    Extract the value from the VAR="val" string
+    """
+    if line.startswith(key + "="):
+        stripped_line = line.split('=')[1]
+        stripped_line = stripped_line.replace('\"', '')
+        return stripped_line
+    return None
+
+
+def find_artifacts(image_name):
+    """
+    Gather the build artifacts for the current image (the image_name
+    e.g. core-image-minimal) for the current MACHINE set in local.conf
+    """
+    bitbake_env_cmd = "bitbake -e %s" % image_name
+    rc, bitbake_env_lines = exec_cmd(bitbake_env_cmd)
+    if rc != 0:
+        print "Couldn't get '%s' output, exiting." % bitbake_env_cmd
+        sys.exit(1)
+
+    for line in bitbake_env_lines.split('\n'):
+        if (get_line_val(line, "IMAGE_ROOTFS")):
+            rootfs_dir = get_line_val(line, "IMAGE_ROOTFS")
+            continue
+        if (get_line_val(line, "STAGING_KERNEL_DIR")):
+            kernel_dir = get_line_val(line, "STAGING_KERNEL_DIR")
+            continue
+        if (get_line_val(line, "HDDDIR")):
+            hdddir = get_line_val(line, "HDDDIR")
+            continue
+        if (get_line_val(line, "STAGING_DATADIR")):
+            staging_data_dir = get_line_val(line, "STAGING_DATADIR")
+            continue
+        if (get_line_val(line, "STAGING_DIR_NATIVE")):
+            native_sysroot = get_line_val(line, "STAGING_DIR_NATIVE")
+            continue
+
+    return (rootfs_dir, kernel_dir, hdddir, staging_data_dir, native_sysroot)
+
+
+CANNED_IMAGE_DIR = "lib/image/canned-wks" # relative to scripts
+
+def find_canned_image(scripts_path, wks_file):
+    """
+    Find a .wks file with the given name in the canned files dir.
+
+    Return False if not found
+    """
+    canned_wks_dir = os.path.join(scripts_path, CANNED_IMAGE_DIR)
+
+    for root, dirs, files in os.walk(canned_wks_dir):
+        for file in files:
+            if file.endswith("~") or file.endswith("#"):
+                continue
+            if file.endswith(".wks") and wks_file + ".wks" == file:
+                fullpath = os.path.join(canned_wks_dir, file)
+                return fullpath
+    return None
+
+
+def list_canned_images(scripts_path):
+    """
+    List the .wks files in the canned image dir, minus the extension.
+    """
+    canned_wks_dir = os.path.join(scripts_path, CANNED_IMAGE_DIR)
+
+    for root, dirs, files in os.walk(canned_wks_dir):
+        for file in files:
+            if file.endswith("~") or file.endswith("#"):
+                continue
+            if file.endswith(".wks"):
+                fullpath = os.path.join(canned_wks_dir, file)
+                f = open(fullpath, "r")
+                lines = f.readlines()
+                for line in lines:
+                    desc = ""
+                    idx = line.find("short-description:")
+                    if idx != -1:
+                        desc = line[idx + len("short-description:"):].strip()
+                        break
+                basename = os.path.splitext(file)[0]
+                print "  %s\t\t%s" % (basename, desc)
+
+
+def list_canned_image_help(scripts_path, fullpath):
+    """
+    List the help and params in the specified canned image.
+    """
+    canned_wks_dir = os.path.join(scripts_path, CANNED_IMAGE_DIR)
+
+    f = open(fullpath, "r")
+    lines = f.readlines()
+    found = False
+    for line in lines:
+        if not found:
+            idx = line.find("long-description:")
+            if idx != -1:
+                print
+                print line[idx + len("long-description:"):].strip()
+                found = True
+            continue
+        if not line.strip():
+            break
+        idx = line.find("#")
+        if idx != -1:
+            print line[idx + len("#:"):].rstrip()
+        else:
+            break
+
+
+def wic_create(args, wks_file, rootfs_dir, bootimg_dir, kernel_dir,
+               native_sysroot, hdddir, staging_data_dir, scripts_path,
+               image_output_dir, properties_file, properties=None):
+    """
+    Create image
+
+    wks_file - user-defined OE kickstart file
+    rootfs_dir - absolute path to the build's /rootfs dir
+    bootimg_dir - absolute path to the build's boot artifacts directory
+    kernel_dir - absolute path to the build's kernel directory
+    native_sysroot - absolute path to the build's native sysroots dir
+    hdddir - absolute path to the build's HDDDIR dir
+    staging_data_dir - absolute path to the build's STAGING_DATA_DIR dir
+    scripts_path - absolute path to /scripts dir
+    image_output_dir - dirname to create for image
+    properties_file - use values from this file if nonempty i.e no prompting
+    properties - use values from this string if nonempty i.e no prompting
+
+    Normally, the values for the build artifacts values are determined
+    by 'wic -e' from the output of the 'bitbake -e' command given an
+    image name e.g. 'core-image-minimal' and a given machine set in
+    local.conf.  If that's the case, the variables get the following
+    values from the output of 'bitbake -e':
+
+    rootfs_dir:        IMAGE_ROOTFS
+    kernel_dir:        STAGING_KERNEL_DIR
+    native_sysroot:    STAGING_DIR_NATIVE
+    hdddir:            HDDDIR
+    staging_data_dir:  STAGING_DATA_DIR
+
+    In the above case, bootimg_dir remains unset and the image
+    creation code determines which of the passed-in directories to
+    use.
+
+    In the case where the values are passed in explicitly i.e 'wic -e'
+    is not used but rather the individual 'wic' options are used to
+    explicitly specify these values, hdddir and staging_data_dir will
+    be unset, but bootimg_dir must be explicit i.e. explicitly set to
+    either hdddir or staging_data_dir, depending on the image being
+    generated.  The other values (rootfs_dir, kernel_dir, and
+    native_sysroot) correspond to the same values found above via
+    'bitbake -e').
+
+    """
+    try:
+        oe_builddir = os.environ["BUILDDIR"]
+    except KeyError:
+        print "BUILDDIR not found, exiting. (Did you forget to source oe-init-build-env?)"
+        sys.exit(1)
+
+
+def wic_list(args, scripts_path, properties_file):
+    """
+    Print the complete list of properties defined by the image, or the
+    possible values for a particular image property.
+    """
+    if len(args) < 1:
+        return False
+
+    if len(args) == 1:
+        if args[0] == "images":
+            list_canned_images(scripts_path)
+            return True
+        elif args[0] == "properties":
+            return True
+        else:
+            return False
+
+    if len(args) == 2:
+        if args[0] == "properties":
+            wks_file = args[1]
+            print "print properties contained in wks file: %s" % wks_file
+            return True
+        elif args[0] == "property":
+            print "print property values for property: %s" % args[1]
+            return True
+        elif args[1] == "help":
+            wks_file = args[0]
+            fullpath = find_canned_image(scripts_path, wks_file)
+            if not fullpath:
+                print "No image named %s found, exiting.  (Use 'wic list images' to list available images, or specify a fully-qualified OE kickstart (.wks) filename)\n" % wks_file
+                sys.exit(1)
+            list_canned_image_help(scripts_path, fullpath)
+            return True
+        else:
+            return False
+
+    return False
diff --git a/scripts/lib/image/help.py b/scripts/lib/image/help.py
new file mode 100644
index 0000000..cb3112c
--- /dev/null
+++ b/scripts/lib/image/help.py
@@ -0,0 +1,311 @@
+# ex:ts=4:sw=4:sts=4:et
+# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
+#
+# Copyright (c) 2013, Intel Corporation.
+# All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+# DESCRIPTION
+# This module implements some basic help invocation functions along
+# with the bulk of the help topic text for the OE Core Image Tools.
+#
+# AUTHORS
+# Tom Zanussi <tom.zanussi (at] linux.intel.com>
+#
+
+import subprocess
+import logging
+
+
+def subcommand_error(args):
+    logging.info("invalid subcommand %s" % args[0])
+
+
+def display_help(subcommand, subcommands):
+    """
+    Display help for subcommand.
+    """
+    if subcommand not in subcommands:
+        return False
+
+    help = subcommands.get(subcommand, subcommand_error)[2]
+    pager = subprocess.Popen('less', stdin=subprocess.PIPE)
+    pager.communicate(help)
+
+    return True
+
+
+def wic_help(args, usage_str, subcommands):
+    """
+    Subcommand help dispatcher.
+    """
+    if len(args) == 1 or not display_help(args[1], subcommands):
+        print(usage_str)
+
+
+def invoke_subcommand(args, parser, main_command_usage, subcommands):
+    """
+    Dispatch to subcommand handler borrowed from combo-layer.
+    Should use argparse, but has to work in 2.6.
+    """
+    if not args:
+        logging.error("No subcommand specified, exiting")
+        parser.print_help()
+    elif args[0] == "help":
+        wic_help(args, main_command_usage, subcommands)
+    elif args[0] not in subcommands:
+        logging.error("Unsupported subcommand %s, exiting\n" % (args[0]))
+        parser.print_help()
+    else:
+        usage = subcommands.get(args[0], subcommand_error)[1]
+        subcommands.get(args[0], subcommand_error)[0](args[1:], usage)
+
+
+##
+# wic help and usage strings
+##
+
+wic_usage = """
+
+ Create a customized OpenEmbedded image
+
+ usage: wic [--version] [--help] COMMAND [ARGS]
+
+ Current 'wic' commands are:
+    create            Create a new OpenEmbedded image
+    list              List available values for options and image properties
+
+ See 'wic help COMMAND' for more information on a specific command.
+"""
+
+wic_help_usage = """
+
+ usage: wic help <subcommand>
+
+ This command displays detailed help for the specified subcommand.
+"""
+
+wic_create_usage = """
+
+ Create a new OpenEmbedded image
+
+ usage: wic create <wks file or image name> [-o <DIRNAME> | --outdir <DIRNAME>]
+            [-i <JSON PROPERTY FILE> | --infile <JSON PROPERTY_FILE>]
+            [-e | --image-name] [-r, --rootfs-dir] [-b, --bootimg-dir]
+            [-k, --kernel-dir] [-n, --native-sysroot] [-s, --skip-build-check]
+
+ This command creates an OpenEmbedded image based on the 'OE kickstart
+ commands' found in the <wks file>.
+
+ The -o option can be used to place the image in a directory with a
+ different name and location.
+
+ See 'wic help create' for more detailed instructions.
+"""
+
+wic_create_help = """
+
+NAME
+    wic create - Create a new OpenEmbedded image
+
+SYNOPSIS
+    wic create <wks file or image name> [-o <DIRNAME> | --outdir <DIRNAME>]
+        [-i <JSON PROPERTY FILE> | --infile <JSON PROPERTY_FILE>]
+        [-e | --image-name] [-r, --rootfs-dir] [-b, --bootimg-dir]
+        [-k, --kernel-dir] [-n, --native-sysroot] [-s, --skip-build-check]
+
+DESCRIPTION
+    This command creates an OpenEmbedded image based on the 'OE
+    kickstart commands' found in the <wks file>.
+
+    In order to do this, wic needs to know the locations of the
+    various build artifacts required to build the image.
+
+    Users can explicitly specify the build artifact locations using
+    the -r, -b, -k, and -n options.  See below for details on where
+    the corresponding artifacts are typically found in a normal
+    OpenEmbedded build.
+
+    Alternatively, users can use the -e option to have 'mic' determine
+    those locations for a given image.  If the -e option is used, the
+    user needs to have set the appropriate MACHINE variable in
+    local.conf, and have sourced the build environment.
+
+    The -e option is used to specify the name of the image to use the
+    artifacts from e.g. core-image-sato.
+
+    The -r option is used to specify the path to the /rootfs dir to
+    use as the .wks rootfs source.
+
+    The -b option is used to specify the path to the dir containing
+    the boot artifacts (e.g. /EFI or /syslinux dirs) to use as the
+    .wks bootimg source.
+
+    The -k option is used to specify the path to the dir containing
+    the kernel to use in the .wks bootimg.
+
+    The -n option is used to specify the path to the native sysroot
+    containing the tools to use to build the image.
+
+    The -s option is used to skip the build check.  The build check is
+    a simple sanity check used to determine whether the user has
+    sourced the build environment so that the -e option can operate
+    correctly.  If the user has specified the build artifact locations
+    explicitly, 'wic' assumes the user knows what he or she is doing
+    and skips the build check.
+
+    When 'wic -e' is used, the locations for the build artifacts
+    values are determined by 'wic -e' from the output of the 'bitbake
+    -e' command given an image name e.g. 'core-image-minimal' and a
+    given machine set in local.conf.  In that case, the image is
+    created as if the following 'bitbake -e' variables were used:
+
+    -r:        IMAGE_ROOTFS
+    -k:        STAGING_KERNEL_DIR
+    -n:        STAGING_DIR_NATIVE
+    -b:        HDDDIR and STAGING_DATA_DIR (handlers decide which to use)
+
+    If 'wic -e' is not used, the user needs to select the appropriate
+    value for -b (as well as -r, -k, and -n).
+
+    The -o option can be used to place the image in a directory with a
+    different name and location.
+
+    As an alternative to the wks file, the image-specific properties
+    that define the values that will be used to generate a particular
+    image can be specified on the command-line using the -i option and
+    supplying a JSON object consisting of the set of name:value pairs
+    needed by image creation.
+
+    The set of properties available for a given image type can be
+    listed using the 'wic list' command.
+"""
+
+wic_list_usage = """
+
+ List available OpenEmbedded image properties and values
+
+ usage: wic list images
+        wic list <image> help
+        wic list properties
+        wic list properties <wks file>
+        wic list property <property>
+                [-o <JSON PROPERTY FILE> | --outfile <JSON PROPERTY_FILE>]
+
+ This command enumerates the set of available canned images as well as
+ help for those images.  It also can be used to enumerate the complete
+ set of possible values for a specified option or property needed by
+ the image creation process.
+
+ The first form enumerates all the available 'canned' images.
+
+ The second form lists the detailed help information for a specific
+ 'canned' image.
+
+ The third form enumerates all the possible values that exist and can
+ be specified in an OE kickstart (wks) file.
+
+ The fourth form enumerates all the possible options that exist for
+ the set of properties specified in a given OE kickstart (ks) file.
+
+ The final form enumerates all the possible values that exist and can
+ be specified for any given OE kickstart (wks) property.
+
+ See 'wic help list' for more details.
+"""
+
+wic_list_help = """
+
+NAME
+    wic list - List available OpenEmbedded image properties and values
+
+SYNOPSIS
+    wic list images
+    wic list <image> help
+    wic list properties
+    wic list properties <wks file>
+    wic list property <property>
+            [-o <JSON PROPERTY FILE> | --outfile <JSON PROPERTY_FILE>]
+
+DESCRIPTION
+    This command enumerates the complete set of possible values for a
+    specified option or property needed by the image creation process.
+
+    This command enumerates the set of available canned images as well
+    as help for those images.  It also can be used to enumerate the
+    complete set of possible values for a specified option or property
+    needed by the image creation process.
+
+    The first form enumerates all the available 'canned' images.
+    These are actually just the set of .wks files that have been moved
+    into the /scripts/lib/image/canned-wks directory).
+
+    The second form lists the detailed help information for a specific
+    'canned' image.
+
+    The third form enumerates all the possible values that exist and
+    can be specified in a OE kickstart (wks) file.  The output of this
+    can be used by the third form to print the description and
+    possible values of a specific property.
+
+    The fourth form enumerates all the possible options that exist for
+    the set of properties specified in a given OE kickstart (wks)
+    file.  If the -o option is specified, the list of properties, in
+    addition to being displayed, will be written to the specified file
+    as a JSON object.  In this case, the object will consist of the
+    set of name:value pairs corresponding to the (possibly nested)
+    dictionary of properties defined by the input statements used by
+    the image.  Some example output for the 'list <wks file>' command:
+
+    $ wic list test.ks
+    "part" : {
+        "mountpoint" : "/"
+        "fstype" : "ext3"
+    }
+    "part" : {
+        "mountpoint" : "/home"
+        "fstype" : "ext3"
+        "offset" : "10000"
+    }
+    "bootloader" : {
+        "type" : "efi"
+    }
+    .
+    .
+    .
+
+    Each entry in the output consists of the name of the input element
+    e.g. "part", followed by the properties defined for that
+    element enclosed in braces.  This information should provide
+    sufficient information to create a complete user interface with.
+
+    The final form enumerates all the possible values that exist and
+    can be specified for any given OE kickstart (wks) property.  If
+    the -o option is specified, the list of values for the given
+    property, in addition to being displayed, will be written to the
+    specified file as a JSON object.  In this case, the object will
+    consist of the set of name:value pairs corresponding to the array
+    of property values associated with the property.
+
+    $ wic list property part
+        ["mountpoint", "where the partition should be mounted"]
+        ["fstype", "filesytem type of the partition"]
+            ["ext3"]
+            ["ext4"]
+            ["btrfs"]
+            ["swap"]
+        ["offset", "offset of the partition within the image"]
+
+"""
diff --git a/scripts/wic b/scripts/wic
new file mode 100755
index 0000000..06e72bb
--- /dev/null
+++ b/scripts/wic
@@ -0,0 +1,185 @@
+#!/usr/bin/env python
+# ex:ts=4:sw=4:sts=4:et
+# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
+#
+# Copyright (c) 2013, Intel Corporation.
+# All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+# DESCRIPTION 'wic' is the OpenEmbedded Image Creator that users can
+# use to generate bootable images.  Invoking it without any arguments
+# will display help screens for the 'wic' command and list the
+# available 'wic' subcommands.  Invoking a subcommand without any
+# arguments will likewise display help screens for the specified
+# subcommand.  Please use that interface for detailed help.
+#
+# AUTHORS
+# Tom Zanussi <tom.zanussi (at] linux.intel.com>
+#
+
+__version__ = "0.1.0"
+
+import os
+import sys
+import optparse
+import logging
+
+scripts_path = os.path.abspath(os.path.dirname(os.path.abspath(sys.argv[0])))
+lib_path = scripts_path + '/lib'
+sys.path = sys.path + [lib_path]
+
+from image.help import *
+from image.engine import *
+
+
+def wic_create_subcommand(args, usage_str):
+    """
+    Command-line handling for image creation.  The real work is done
+    by image.engine.wic_create()
+    """
+    parser = optparse.OptionParser(usage = usage_str)
+
+    parser.add_option("-o", "--outdir", dest = "outdir",
+                      action = "store", help = "name of directory to create image in")
+    parser.add_option("-i", "--infile", dest = "properties_file",
+                      action = "store", help = "name of file containing the values for image properties as a JSON file")
+    parser.add_option("-e", "--image-name", dest = "image_name",
+                      action = "store", help = "name of the image to use the artifacts from e.g. core-image-sato")
+    parser.add_option("-r", "--rootfs-dir", dest = "rootfs_dir",
+                      action = "store", help = "path to the /rootfs dir to use as the .wks rootfs source")
+    parser.add_option("-b", "--bootimg-dir", dest = "bootimg_dir",
+                      action = "store", help = "path to the dir containing the boot artifacts (e.g. /EFI or /syslinux dirs) to use as the .wks bootimg source")
+    parser.add_option("-k", "--kernel-dir", dest = "kernel_dir",
+                      action = "store", help = "path to the dir containing the kernel to use in the .wks bootimg")
+    parser.add_option("-n", "--native-sysroot", dest = "native_sysroot",
+                      action = "store", help = "path to the native sysroot containing the tools to use to build the image")
+    parser.add_option("-p", "--skip-build-check", dest = "build_check",
+                      action = "store_false", default = True, help = "skip the build check")
+
+    (options, args) = parser.parse_args(args)
+
+    if len(args) != 1:
+        logging.error("Wrong number of arguments, exiting\n")
+        parser.print_help()
+        sys.exit(1)
+
+    if not options.image_name:
+        options.build_check = False
+
+    if options.build_check and not options.properties_file:
+        print "Checking basic build environment..."
+        if not verify_build_env():
+            print "Couldn't verify build environment, exiting\n"
+            sys.exit(1)
+        else:
+            print "Done.\n"
+
+    print "Creating image(s)...\n"
+
+    bootimg_dir = staging_data_dir = hdddir = ""
+
+    if options.image_name:
+        (rootfs_dir, kernel_dir, hdddir, staging_data_dir, native_sysroot) = \
+            find_artifacts(options.image_name)
+
+    wks_file = args[0]
+
+    if not wks_file.endswith(".wks"):
+        wks_file = find_canned_image(scripts_path, wks_file)
+        if not wks_file:
+            print "No image named %s found, exiting.  (Use 'wic list images' to list available images, or specify a fully-qualified OE kickstart (.wks) filename)\n" % wks_file
+            sys.exit(1)
+
+    image_output_dir = ""
+    if options.outdir:
+        image_output_dir = options.outdir
+
+    if not options.image_name:
+        rootfs_dir = options.rootfs_dir
+        bootimg_dir = options.bootimg_dir
+        kernel_dir = options.kernel_dir
+        native_sysroot = options.native_sysroot
+
+    wic_create(args, wks_file, rootfs_dir, bootimg_dir, kernel_dir,
+               native_sysroot, hdddir, staging_data_dir, scripts_path,
+               image_output_dir, options.properties_file)
+
+
+def wic_list_subcommand(args, usage_str):
+    """
+    Command-line handling for listing available image properties and
+    values.  The real work is done by image.engine.wic_list()
+    """
+    parser = optparse.OptionParser(usage = usage_str)
+
+    parser.add_option("-o", "--outfile", action = "store",
+                      dest = "properties_file",
+                      help = "dump the possible values for image properties to a JSON file")
+
+    (options, args) = parser.parse_args(args)
+
+    if not wic_list(args, scripts_path, options.properties_file):
+        logging.error("Bad list arguments, exiting\n")
+        parser.print_help()
+        sys.exit(1)
+
+
+subcommands = {
+    "create": [wic_create_subcommand,
+               wic_create_usage,
+               wic_create_help],
+    "list":   [wic_list_subcommand,
+               wic_list_usage,
+               wic_list_help],
+}
+
+
+def start_logging(loglevel):
+    logging.basicConfig(filname = 'wic.log', filemode = 'w', level=loglevel)
+
+
+def main():
+    parser = optparse.OptionParser(version = "wic version %s" % __version__,
+                                   usage = wic_usage)
+
+    parser.disable_interspersed_args()
+    parser.add_option("-D", "--debug", dest = "debug", action = "store_true",
+                      default = False, help = "output debug information")
+
+    (options, args) = parser.parse_args()
+
+    loglevel = logging.INFO
+    if options.debug:
+        loglevel = logging.DEBUG
+    start_logging(loglevel)
+
+    if len(args):
+        if args[0] == "help":
+            if len(args) == 1:
+                parser.print_help()
+                sys.exit(1)
+
+    invoke_subcommand(args, parser, wic_help_usage, subcommands)
+
+
+if __name__ == "__main__":
+    try:
+        ret = main()
+    except Exception:
+        ret = 1
+        import traceback
+        traceback.print_exc(5)
+    sys.exit(ret)
+
-- 
1.7.11.4

Re: [PATCH 1/3] wic: Initial code for wic (OpenEmbedded Image Creator)

[Otavio Salvador]

Hello Tom,

On Thu, Sep 26, 2013 at 11:17 PM, Tom Zanussi
<tom.zanussi@linux.intel.com> wrote:
> Initial implementation of the 'wic' command.
>
> The 'wic' command generates partitioned images from existing
> OpenEmbedded build artifacts.  Image generation is driven by
> partitioning commands contained in an 'Openembedded kickstart' (.wks)
> file specified either directly on the command-line or as one of a
> selection of canned .wks files (see 'wic list images').  When applied
> to a given set of build artifacts, the result is an image or set of
> images that can be directly written onto media and used on a
> particular system.
>
> 'wic' is based loosely on the 'mic' (Meego Image Creator) framework,
> but heavily modified to make direct use of OpenEmbedded build
> artifacts instead of package installation and configuration, things
> already incorporated int the OE artifacts.
>
> The name 'wic' comes from 'oeic' with the 'oe' diphthong promoted to
> the letter 'w', because 'oeic' is impossible to remember or pronounce.
>
> This covers the mechanics of invoking and providing help for the
> command and sub-commands; it contains hooks for future commits to
> connect with the actual functionality, once implemented.
>
> Help is integrated into the 'wic' command - see that for details on
> usage.
>
> Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>

Could you please elaborate why to make a new command instead of using
the class system?

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 1/3] wic: Initial code for wic (OpenEmbedded Image Creator)

[Tom Zanussi]

Hi Otavio,

On Fri, 2013-09-27 at 11:01 -0300, Otavio Salvador wrote:
> Hello Tom,
> 
> On Thu, Sep 26, 2013 at 11:17 PM, Tom Zanussi
> <tom.zanussi@linux.intel.com> wrote:
> > Initial implementation of the 'wic' command.
> >
> > The 'wic' command generates partitioned images from existing
> > OpenEmbedded build artifacts.  Image generation is driven by
> > partitioning commands contained in an 'Openembedded kickstart' (.wks)
> > file specified either directly on the command-line or as one of a
> > selection of canned .wks files (see 'wic list images').  When applied
> > to a given set of build artifacts, the result is an image or set of
> > images that can be directly written onto media and used on a
> > particular system.
> >
> > 'wic' is based loosely on the 'mic' (Meego Image Creator) framework,
> > but heavily modified to make direct use of OpenEmbedded build
> > artifacts instead of package installation and configuration, things
> > already incorporated int the OE artifacts.
> >
> > The name 'wic' comes from 'oeic' with the 'oe' diphthong promoted to
> > the letter 'w', because 'oeic' is impossible to remember or pronounce.
> >
> > This covers the mechanics of invoking and providing help for the
> > command and sub-commands; it contains hooks for future commits to
> > connect with the actual functionality, once implemented.
> >
> > Help is integrated into the 'wic' command - see that for details on
> > usage.
> >
> > Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
> 
> Could you please elaborate why to make a new command instead of using
> the class system?
> 

This isn't an either/or thing - the initial requirements were that the
overall deployment effort end up being something that would be usable
both from an external tool as well as from within the class system.

This just happens to be the initial (easier) part of that, the external
tool, and I expect in 1.6 to be doing a lot of the harder part,
integration with the build system.

The most important part, I think, is that this provides a high-level
user-oriented 'language' (the kickstart files) that users can use to
define custom images, rather than having to muck around in class files
or variable settings, or write specialized scripts such as mkefidisk.sh
for example.

Making that available from a standalone tool such as 'wic' is
straightforward, doing the same from within the build system will
require more thought and work, but that's what I'm hoping to do in
1.6...

Tom

Re: [PATCH 1/3] wic: Initial code for wic (OpenEmbedded Image Creator)

[David Nyström]

On 09/27/2013 04:21 PM, Tom Zanussi wrote:
> Hi Otavio,
>
> On Fri, 2013-09-27 at 11:01 -0300, Otavio Salvador wrote:
>> Hello Tom,
>>
>> On Thu, Sep 26, 2013 at 11:17 PM, Tom Zanussi
>> <tom.zanussi@linux.intel.com> wrote:
>>> Initial implementation of the 'wic' command.
>>>
><snip>
>> Could you please elaborate why to make a new command instead of using
>> the class system?
>>
>
> This isn't an either/or thing - the initial requirements were that the
> overall deployment effort end up being something that would be usable
> both from an external tool as well as from within the class system.

What do you mean when you say "within the class system" here ?
* A tool using only (kickstart for image cfg, partitioning et.c.) + 
(tmp/deploy/ipk|rpm|deb) as input data for image creation ?

Just my five cents,

I would like to see reproducible image creation from both the bitbake/OE 
build env and the nativesdk SDK build env.
This would require a common interface for input distribution data, It 
naturally feels like this interface should be the package repository.
i.e. if X is not packaged as class-target, it can't be included on the 
generated image.
Also, if X is required to generate the image, it should be packaged as 
class-nativesdk.

afaict, the standalone wic tool uses a hybrid approach, using 
OpenEmbedded build artifacts + a package repository as input for rootfs 
generation.
What is the long term plan for wic in regards to the above ?

Br,
David

> This just happens to be the initial (easier) part of that, the external
> tool, and I expect in 1.6 to be doing a lot of the harder part,
> integration with the build system.


> The most important part, I think, is that this provides a high-level
> user-oriented 'language' (the kickstart files) that users can use to
> define custom images, rather than having to muck around in class files
> or variable settings, or write specialized scripts such as mkefidisk.sh
> for example.
>
> Making that available from a standalone tool such as 'wic' is
> straightforward, doing the same from within the build system will
> require more thought and work, but that's what I'm hoping to do in
> 1.6...



>
> Tom
>
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>

Re: [PATCH 1/3] wic: Initial code for wic (OpenEmbedded Image Creator)

[Tom Zanussi]

On Sat, 2013-09-28 at 14:17 +0200, David Nyström wrote:
> On 09/27/2013 04:21 PM, Tom Zanussi wrote:
> > Hi Otavio,
> >
> > On Fri, 2013-09-27 at 11:01 -0300, Otavio Salvador wrote:
> >> Hello Tom,
> >>
> >> On Thu, Sep 26, 2013 at 11:17 PM, Tom Zanussi
> >> <tom.zanussi@linux.intel.com> wrote:
> >>> Initial implementation of the 'wic' command.
> >>>
> ><snip>
> >> Could you please elaborate why to make a new command instead of using
> >> the class system?
> >>
> >
> > This isn't an either/or thing - the initial requirements were that the
> > overall deployment effort end up being something that would be usable
> > both from an external tool as well as from within the class system.
> 
> What do you mean when you say "within the class system" here ?
> * A tool using only (kickstart for image cfg, partitioning et.c.) + 
> (tmp/deploy/ipk|rpm|deb) as input data for image creation ?
> 

For me, 'within the class system' just covers what I originally outlined
in the analysis (Bug 3847 - New partitioning description and tooling),
basically providing a more flexible alternative/eventual replacement for
things like boot-directdisk.class/image-vmdk.bbclass and mkefidisk.sh,
and the IMAGE_FSTYPEs mechanism used all over the place for creating
custom images.  At this point, that's the scope of it for me - I'm not
even sure yet where the kickstart interface will hook into things or
exactly how it will presented to users - that's for 1.6.  What I do know
is that the current wic functionality should be sufficient and that it's
pretty well modularized - the 'wic' command itself is just a thin
wrapper that gathers info from the user e.g. paths to the build
artifacts and a kickstart file and invokes the image creator through a
well-defined entry point.

For current work (and presumably also when integrated into the class
system), I'm directly using the target rootfs - my first versions
actually just used the rootfs image e.g. xx.ext3, but because I needed
access to the filesystem for e.g. generating the fstab, and can't do a
loop mount because it requires root, that's what the tool uses.

Using tmp/deploy/ipk|rpm|deb as input for image creation is a step
beyond what I had scoped out for this immediate task - things like image
configuration and package selection from package repositories/feeds are
things I believe other people are interested in; using kickstart/mic as
the underlying infrastructure for those additional capabilities at first
glance seems that it might also make sense in those cases, since those
things are already implemented in some form, but I haven't looked deeply
and that's probably something for those who have the need/interest to
determine...

> Just my five cents,
> 
> I would like to see reproducible image creation from both the bitbake/OE 
> build env and the nativesdk SDK build env.
> This would require a common interface for input distribution data, It 
> naturally feels like this interface should be the package repository.
> i.e. if X is not packaged as class-target, it can't be included on the 
> generated image.
> Also, if X is required to generate the image, it should be packaged as 
> class-nativesdk.
> 
> afaict, the standalone wic tool uses a hybrid approach, using 
> OpenEmbedded build artifacts + a package repository as input for rootfs 
> generation.

Yeah, that's what the tool currently uses, but as you say, some more
thought towards a common interface needs to be done.  I originally had
this for the --source param to the 'part commmand': 

parser.add_option("-s", "--source", action = "append",
                  default = True, help = "define additional wks sources
[sourcename=/path/to/source], referenced in .wks part commands as
--source sourcename")

For the current wic code, I punted on trying to come up with something
more general purpose like this (and as you can see it's not really
general purpose, in my case just allowing multiple filesystem sources
and not very well thought out at that), and simply defined 'rootfs' to
mean the entire /rootfs passed in using the -r option.

I think there needs to be a way to specify arbitrary (user-defined as
well) sources and those sources could really be anything, including
package repositories in whatever form you need them to be.  I don't have
the answer at the moment, just that there needs to be a generic
mechanism for providing and making use of arbitrary sources - any input
you could provide for the use cases it sounds like you've given more
thought to handling would help move that along...

Thanks,

Tom

> What is the long term plan for wic in regards to the above ?
> 
> Br,
> David
> 
> > This just happens to be the initial (easier) part of that, the external
> > tool, and I expect in 1.6 to be doing a lot of the harder part,
> > integration with the build system.
> 
> 
> > The most important part, I think, is that this provides a high-level
> > user-oriented 'language' (the kickstart files) that users can use to
> > define custom images, rather than having to muck around in class files
> > or variable settings, or write specialized scripts such as mkefidisk.sh
> > for example.
> >
> > Making that available from a standalone tool such as 'wic' is
> > straightforward, doing the same from within the build system will
> > require more thought and work, but that's what I'm hoping to do in
> > 1.6...
> 
> 
> 
> >
> > Tom
> >
> >
> >
> >
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> >
> 

Re: [PATCH 1/3] wic: Initial code for wic (OpenEmbedded Image Creator)

[David Nyström]

On 09/30/2013 03:11 AM, Tom Zanussi wrote:
> On Sat, 2013-09-28 at 14:17 +0200, David Nyström wrote:
>> On 09/27/2013 04:21 PM, Tom Zanussi wrote:
>>> Hi Otavio,
>>>
>>> On Fri, 2013-09-27 at 11:01 -0300, Otavio Salvador wrote:
>>>> Hello Tom,
>>>>
>>>> On Thu, Sep 26, 2013 at 11:17 PM, Tom Zanussi
>>>> <tom.zanussi@linux.intel.com> wrote:
>>>>> Initial implementation of the 'wic' command.
>>>>>
>>> <snip>
>>>> Could you please elaborate why to make a new command instead of using
>>>> the class system?
>>>>
>>>
>>> This isn't an either/or thing - the initial requirements were that the
>>> overall deployment effort end up being something that would be usable
>>> both from an external tool as well as from within the class system.
>>
>> What do you mean when you say "within the class system" here ?
>> * A tool using only (kickstart for image cfg, partitioning et.c.) +
>> (tmp/deploy/ipk|rpm|deb) as input data for image creation ?
>>
>
> For me, 'within the class system' just covers what I originally outlined
> in the analysis (Bug 3847 - New partitioning description and tooling),
> basically providing a more flexible alternative/eventual replacement for
> things like boot-directdisk.class/image-vmdk.bbclass and mkefidisk.sh,
> and the IMAGE_FSTYPEs mechanism used all over the place for creating
> custom images.  At this point, that's the scope of it for me - I'm not
> even sure yet where the kickstart interface will hook into things or
> exactly how it will presented to users - that's for 1.6.  What I do know
> is that the current wic functionality should be sufficient and that it's
> pretty well modularized - the 'wic' command itself is just a thin
> wrapper that gathers info from the user e.g. paths to the build
> artifacts and a kickstart file and invokes the image creator through a
> well-defined entry point.
>
> For current work (and presumably also when integrated into the class
> system), I'm directly using the target rootfs - my first versions
> actually just used the rootfs image e.g. xx.ext3, but because I needed
> access to the filesystem for e.g. generating the fstab, and can't do a
> loop mount because it requires root, that's what the tool uses.
>
> Using tmp/deploy/ipk|rpm|deb as input for image creation is a step
> beyond what I had scoped out for this immediate task - things like image
> configuration and package selection from package repositories/feeds are
> things I believe other people are interested in; using kickstart/mic as
> the underlying infrastructure for those additional capabilities at first
> glance seems that it might also make sense in those cases, since those
> things are already implemented in some form, but I haven't looked deeply
> and that's probably something for those who have the need/interest to
> determine...

I started hacking on a simple test to evaluate dependencies et.c. for 
SDK rootfs + image generation from a repo, similiar to "mic-chroot".

https://github.com/nysan/rootfs-sandbox , dep: (oe-core master-next)

some postinst hooks are OE dependent and require special attention, and 
there is also the need to expand the nativesdk with some postinst 
dependencies to avoid host contamination. et.c.

When I saw the wic/kickstart patches, my hope was that we could, long 
term, share the codebase between the nativeSDK 'mic-like-interface', and 
bitbake/OE 'mic-like-interface', since functionality probably will be 
overlapping.

The nativeSDK 'mic-like-interface' would naturally only have the package 
repo + wks as input data.

Br,
David

>> Just my five cents,
>>
>> I would like to see reproducible image creation from both the bitbake/OE
>> build env and the nativesdk SDK build env.
>> This would require a common interface for input distribution data, It
>> naturally feels like this interface should be the package repository.
>> i.e. if X is not packaged as class-target, it can't be included on the
>> generated image.
>> Also, if X is required to generate the image, it should be packaged as
>> class-nativesdk.
>>
>> afaict, the standalone wic tool uses a hybrid approach, using
>> OpenEmbedded build artifacts + a package repository as input for rootfs
>> generation.
>
> Yeah, that's what the tool currently uses, but as you say, some more
> thought towards a common interface needs to be done.  I originally had
> this for the --source param to the 'part commmand':
>
> parser.add_option("-s", "--source", action = "append",
>                    default = True, help = "define additional wks sources
> [sourcename=/path/to/source], referenced in .wks part commands as
> --source sourcename")
>
> For the current wic code, I punted on trying to come up with something
> more general purpose like this (and as you can see it's not really
> general purpose, in my case just allowing multiple filesystem sources
> and not very well thought out at that), and simply defined 'rootfs' to
> mean the entire /rootfs passed in using the -r option.
>
> I think there needs to be a way to specify arbitrary (user-defined as
> well) sources and those sources could really be anything, including
> package repositories in whatever form you need them to be.  I don't have
> the answer at the moment, just that there needs to be a generic
> mechanism for providing and making use of arbitrary sources - any input
> you could provide for the use cases it sounds like you've given more
> thought to handling would help move that along...
>
> Thanks,
>
> Tom
>
>> What is the long term plan for wic in regards to the above ?
>>
>> Br,
>> David
>>
>>> This just happens to be the initial (easier) part of that, the external
>>> tool, and I expect in 1.6 to be doing a lot of the harder part,
>>> integration with the build system.
>>
>>
>>> The most important part, I think, is that this provides a high-level
>>> user-oriented 'language' (the kickstart files) that users can use to
>>> define custom images, rather than having to muck around in class files
>>> or variable settings, or write specialized scripts such as mkefidisk.sh
>>> for example.
>>>
>>> Making that available from a standalone tool such as 'wic' is
>>> straightforward, doing the same from within the build system will
>>> require more thought and work, but that's what I'm hoping to do in
>>> 1.6...
>>
>>
>>
>>>
>>> Tom
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Openembedded-core mailing list
>>> Openembedded-core@lists.openembedded.org
>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>>
>>
>
>

Re: [PATCH 1/3] wic: Initial code for wic (OpenEmbedded Image Creator)

[Tom Zanussi]

On Mon, 2013-09-30 at 14:58 +0200, David Nyström wrote:
> On 09/30/2013 03:11 AM, Tom Zanussi wrote:
> > On Sat, 2013-09-28 at 14:17 +0200, David Nyström wrote:
> >> On 09/27/2013 04:21 PM, Tom Zanussi wrote:
> >>> Hi Otavio,
> >>>
> >>> On Fri, 2013-09-27 at 11:01 -0300, Otavio Salvador wrote:
> >>>> Hello Tom,
> >>>>
> >>>> On Thu, Sep 26, 2013 at 11:17 PM, Tom Zanussi
> >>>> <tom.zanussi@linux.intel.com> wrote:
> >>>>> Initial implementation of the 'wic' command.
> >>>>>
> >>> <snip>
> >>>> Could you please elaborate why to make a new command instead of using
> >>>> the class system?
> >>>>
> >>>
> >>> This isn't an either/or thing - the initial requirements were that the
> >>> overall deployment effort end up being something that would be usable
> >>> both from an external tool as well as from within the class system.
> >>
> >> What do you mean when you say "within the class system" here ?
> >> * A tool using only (kickstart for image cfg, partitioning et.c.) +
> >> (tmp/deploy/ipk|rpm|deb) as input data for image creation ?
> >>
> >
> > For me, 'within the class system' just covers what I originally outlined
> > in the analysis (Bug 3847 - New partitioning description and tooling),
> > basically providing a more flexible alternative/eventual replacement for
> > things like boot-directdisk.class/image-vmdk.bbclass and mkefidisk.sh,
> > and the IMAGE_FSTYPEs mechanism used all over the place for creating
> > custom images.  At this point, that's the scope of it for me - I'm not
> > even sure yet where the kickstart interface will hook into things or
> > exactly how it will presented to users - that's for 1.6.  What I do know
> > is that the current wic functionality should be sufficient and that it's
> > pretty well modularized - the 'wic' command itself is just a thin
> > wrapper that gathers info from the user e.g. paths to the build
> > artifacts and a kickstart file and invokes the image creator through a
> > well-defined entry point.
> >
> > For current work (and presumably also when integrated into the class
> > system), I'm directly using the target rootfs - my first versions
> > actually just used the rootfs image e.g. xx.ext3, but because I needed
> > access to the filesystem for e.g. generating the fstab, and can't do a
> > loop mount because it requires root, that's what the tool uses.
> >
> > Using tmp/deploy/ipk|rpm|deb as input for image creation is a step
> > beyond what I had scoped out for this immediate task - things like image
> > configuration and package selection from package repositories/feeds are
> > things I believe other people are interested in; using kickstart/mic as
> > the underlying infrastructure for those additional capabilities at first
> > glance seems that it might also make sense in those cases, since those
> > things are already implemented in some form, but I haven't looked deeply
> > and that's probably something for those who have the need/interest to
> > determine...
> 
> I started hacking on a simple test to evaluate dependencies et.c. for 
> SDK rootfs + image generation from a repo, similiar to "mic-chroot".
> 
> https://github.com/nysan/rootfs-sandbox , dep: (oe-core master-next)
> 
> some postinst hooks are OE dependent and require special attention, and 
> there is also the need to expand the nativesdk with some postinst 
> dependencies to avoid host contamination. et.c.
> 
> When I saw the wic/kickstart patches, my hope was that we could, long 
> term, share the codebase between the nativeSDK 'mic-like-interface', and 
> bitbake/OE 'mic-like-interface', since functionality probably will be 
> overlapping.
> 
> The nativeSDK 'mic-like-interface' would naturally only have the package 
> repo + wks as input data.
> 

Looking at your code, it definitely seems there could be a good fit, if
I understand it correctly.

As I understand it, you allow users to interactively specify packages to
include in an image and then generate the image.

If I were going to take a first stab at using the wic infrastructure for
this, I'd probably have the interactive tool start off by having the
user specify a kickstart file and, instead of a pointer to the rootfs as
-r does with 'wic', a pointer to the package repo.

The initial kickstart file would have an initial set of packages that
would automatically be added, using the standard kickstart package
selection syntax e.g.

%packages
@packagegroup-core-boot
%end

http://fedoraproject.org/wiki/Anaconda/Kickstart#Chapter_3._Package_Selection

Your interactive wrapper would allow the user to select and add packages
from the package repo, which would be dynamically added to the %packages
section of the kickstart file.

Once done, all the packages selected to go into the image would generate
the rootfs at an internally known temporary location that would just be
passed to the existing image creation code in place of the -r option,
which is a previously solved problem ;-)

Well, there would be a lot more details involved but at a high level,
that's one way I could see it working...

Tom


> Br,
> David
> 
> >> Just my five cents,
> >>
> >> I would like to see reproducible image creation from both the bitbake/OE
> >> build env and the nativesdk SDK build env.
> >> This would require a common interface for input distribution data, It
> >> naturally feels like this interface should be the package repository.
> >> i.e. if X is not packaged as class-target, it can't be included on the
> >> generated image.
> >> Also, if X is required to generate the image, it should be packaged as
> >> class-nativesdk.
> >>
> >> afaict, the standalone wic tool uses a hybrid approach, using
> >> OpenEmbedded build artifacts + a package repository as input for rootfs
> >> generation.
> >
> > Yeah, that's what the tool currently uses, but as you say, some more
> > thought towards a common interface needs to be done.  I originally had
> > this for the --source param to the 'part commmand':
> >
> > parser.add_option("-s", "--source", action = "append",
> >                    default = True, help = "define additional wks sources
> > [sourcename=/path/to/source], referenced in .wks part commands as
> > --source sourcename")
> >
> > For the current wic code, I punted on trying to come up with something
> > more general purpose like this (and as you can see it's not really
> > general purpose, in my case just allowing multiple filesystem sources
> > and not very well thought out at that), and simply defined 'rootfs' to
> > mean the entire /rootfs passed in using the -r option.
> >
> > I think there needs to be a way to specify arbitrary (user-defined as
> > well) sources and those sources could really be anything, including
> > package repositories in whatever form you need them to be.  I don't have
> > the answer at the moment, just that there needs to be a generic
> > mechanism for providing and making use of arbitrary sources - any input
> > you could provide for the use cases it sounds like you've given more
> > thought to handling would help move that along...
> >
> > Thanks,
> >
> > Tom
> >
> >> What is the long term plan for wic in regards to the above ?
> >>
> >> Br,
> >> David
> >>
> >>> This just happens to be the initial (easier) part of that, the external
> >>> tool, and I expect in 1.6 to be doing a lot of the harder part,
> >>> integration with the build system.
> >>
> >>
> >>> The most important part, I think, is that this provides a high-level
> >>> user-oriented 'language' (the kickstart files) that users can use to
> >>> define custom images, rather than having to muck around in class files
> >>> or variable settings, or write specialized scripts such as mkefidisk.sh
> >>> for example.
> >>>
> >>> Making that available from a standalone tool such as 'wic' is
> >>> straightforward, doing the same from within the build system will
> >>> require more thought and work, but that's what I'm hoping to do in
> >>> 1.6...
> >>
> >>
> >>
> >>>
> >>> Tom
> >>>
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> Openembedded-core mailing list
> >>> Openembedded-core@lists.openembedded.org
> >>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
> >>>
> >>
> >
> >
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

[PATCH 2/3] wic: Add mic w/pykickstart

[Tom Zanussi]

This is the starting point for the implemention described in [YOCTO
3847] which came to the conclusion that it would make sense to use
kickstart syntax to implement image creation in OpenEmbedded.  I
subsequently realized that there was an existing tool that already
implemented image creation using kickstart syntax, the Tizen/Meego mic
tool.  As such, it made sense to use that as a starting point - this
commit essentially just copies the relevant Python code from the MIC
tool to the scripts/lib dir, where it can be accessed by the
previously created wic tool.

Most of this will be removed or renamed by later commits, since we're
initially focusing on partitioning only.  Care should be taken so that
we can easily add back any additional functionality should we decide
later to expand the tool, though (we may also want to contribute our
local changes to the mic tool to the Tizen project if it makes sense,
and therefore should avoid gratuitous changes to the original code if
possible).

Added the /mic subdir from Tizen mic repo as a starting point:

 git clone git://review.tizen.org/tools/mic.git

 For reference, the top commit:

 commit 20164175ddc234a17b8a12c33d04b012347b1530
 Author: Gui Chen <gui.chen@intel.com>
 Date:   Sun Jun 30 22:32:16 2013 -0400

    bump up to 0.19.2

Also added the /plugins subdir, moved to under the /mic subdir (to
match the default plugin_dir location in mic.conf.in, which was
renamed to yocto-image.conf (moved and renamed by later patches) and
put into /scripts.

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
---
 scripts/lib/mic/3rdparty/pykickstart/base.py       |  466 ++++++
 .../mic/3rdparty/pykickstart/commands/__init__.py  |   26 +
 .../3rdparty/pykickstart/commands/authconfig.py    |   40 +
 .../mic/3rdparty/pykickstart/commands/autopart.py  |  119 ++
 .../mic/3rdparty/pykickstart/commands/autostep.py  |   55 +
 .../3rdparty/pykickstart/commands/bootloader.py    |  265 ++++
 .../mic/3rdparty/pykickstart/commands/clearpart.py |   86 ++
 .../mic/3rdparty/pykickstart/commands/device.py    |  125 ++
 .../3rdparty/pykickstart/commands/deviceprobe.py   |   40 +
 .../3rdparty/pykickstart/commands/displaymode.py   |   68 +
 .../mic/3rdparty/pykickstart/commands/dmraid.py    |   91 ++
 .../3rdparty/pykickstart/commands/driverdisk.py    |  184 +++
 .../lib/mic/3rdparty/pykickstart/commands/fcoe.py  |  114 ++
 .../mic/3rdparty/pykickstart/commands/firewall.py  |  193 +++
 .../mic/3rdparty/pykickstart/commands/firstboot.py |   62 +
 .../lib/mic/3rdparty/pykickstart/commands/group.py |   88 ++
 .../3rdparty/pykickstart/commands/ignoredisk.py    |  139 ++
 .../3rdparty/pykickstart/commands/interactive.py   |   58 +
 .../lib/mic/3rdparty/pykickstart/commands/iscsi.py |  133 ++
 .../mic/3rdparty/pykickstart/commands/iscsiname.py |   54 +
 .../lib/mic/3rdparty/pykickstart/commands/key.py   |   64 +
 .../mic/3rdparty/pykickstart/commands/keyboard.py  |   55 +
 .../lib/mic/3rdparty/pykickstart/commands/lang.py  |   60 +
 .../3rdparty/pykickstart/commands/langsupport.py   |   58 +
 .../mic/3rdparty/pykickstart/commands/lilocheck.py |   54 +
 .../mic/3rdparty/pykickstart/commands/logging.py   |   66 +
 .../mic/3rdparty/pykickstart/commands/logvol.py    |  304 ++++
 .../3rdparty/pykickstart/commands/mediacheck.py    |   53 +
 .../mic/3rdparty/pykickstart/commands/method.py    |  186 +++
 .../mic/3rdparty/pykickstart/commands/monitor.py   |  106 ++
 .../lib/mic/3rdparty/pykickstart/commands/mouse.py |   70 +
 .../mic/3rdparty/pykickstart/commands/multipath.py |  111 ++
 .../mic/3rdparty/pykickstart/commands/network.py   |  363 +++++
 .../mic/3rdparty/pykickstart/commands/partition.py |  353 +++++
 .../lib/mic/3rdparty/pykickstart/commands/raid.py  |  365 +++++
 .../mic/3rdparty/pykickstart/commands/reboot.py    |   79 +
 .../lib/mic/3rdparty/pykickstart/commands/repo.py  |  249 +++
 .../mic/3rdparty/pykickstart/commands/rescue.py    |   68 +
 .../mic/3rdparty/pykickstart/commands/rootpw.py    |   93 ++
 .../mic/3rdparty/pykickstart/commands/selinux.py   |   64 +
 .../mic/3rdparty/pykickstart/commands/services.py  |   71 +
 .../lib/mic/3rdparty/pykickstart/commands/skipx.py |   54 +
 .../lib/mic/3rdparty/pykickstart/commands/sshpw.py |  105 ++
 .../mic/3rdparty/pykickstart/commands/timezone.py  |   86 ++
 .../mic/3rdparty/pykickstart/commands/updates.py   |   60 +
 .../mic/3rdparty/pykickstart/commands/upgrade.py   |  106 ++
 .../lib/mic/3rdparty/pykickstart/commands/user.py  |  173 +++
 .../lib/mic/3rdparty/pykickstart/commands/vnc.py   |  114 ++
 .../mic/3rdparty/pykickstart/commands/volgroup.py  |  102 ++
 .../mic/3rdparty/pykickstart/commands/xconfig.py   |  184 +++
 .../mic/3rdparty/pykickstart/commands/zerombr.py   |   69 +
 .../lib/mic/3rdparty/pykickstart/commands/zfcp.py  |  134 ++
 scripts/lib/mic/3rdparty/pykickstart/constants.py  |   57 +
 scripts/lib/mic/3rdparty/pykickstart/errors.py     |  103 ++
 .../mic/3rdparty/pykickstart/handlers/control.py   | 1307 ++++++++++++++++
 .../lib/mic/3rdparty/pykickstart/handlers/f10.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/f11.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/f12.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/f13.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/f14.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/f15.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/f16.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/f7.py    |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/f8.py    |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/f9.py    |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/fc3.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/fc4.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/fc5.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/fc6.py   |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/rhel3.py |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/rhel4.py |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/rhel5.py |   24 +
 .../lib/mic/3rdparty/pykickstart/handlers/rhel6.py |   24 +
 scripts/lib/mic/3rdparty/pykickstart/ko.py         |   37 +
 scripts/lib/mic/3rdparty/pykickstart/options.py    |  204 +++
 scripts/lib/mic/3rdparty/pykickstart/parser.py     |  702 +++++++++
 scripts/lib/mic/3rdparty/pykickstart/sections.py   |  244 +++
 .../3rdparty/pykickstart/urlgrabber/__init__.py    |   53 +
 .../3rdparty/pykickstart/urlgrabber/byterange.py   |  463 ++++++
 .../mic/3rdparty/pykickstart/urlgrabber/grabber.py | 1477 ++++++++++++++++++
 .../3rdparty/pykickstart/urlgrabber/keepalive.py   |  617 ++++++++
 .../mic/3rdparty/pykickstart/urlgrabber/mirror.py  |  458 ++++++
 .../3rdparty/pykickstart/urlgrabber/progress.py    |  530 +++++++
 .../3rdparty/pykickstart/urlgrabber/sslfactory.py  |   90 ++
 scripts/lib/mic/3rdparty/pykickstart/version.py    |  197 +++
 scripts/lib/mic/__init__.py                        |    4 +
 scripts/lib/mic/__version__.py                     |    1 +
 scripts/lib/mic/bootstrap.py                       |  279 ++++
 scripts/lib/mic/chroot.py                          |  343 +++++
 scripts/lib/mic/conf.py                            |  239 +++
 scripts/lib/mic/creator.py                         |  354 +++++
 scripts/lib/mic/imager/baseimager.py               | 1335 ++++++++++++++++
 scripts/lib/mic/imager/fs.py                       |   99 ++
 scripts/lib/mic/imager/livecd.py                   |  750 +++++++++
 scripts/lib/mic/imager/liveusb.py                  |  308 ++++
 scripts/lib/mic/imager/loop.py                     |  418 ++++++
 scripts/lib/mic/imager/raw.py                      |  501 +++++++
 scripts/lib/mic/kickstart/__init__.py              |  892 +++++++++++
 .../lib/mic/kickstart/custom_commands/__init__.py  |   12 +
 .../lib/mic/kickstart/custom_commands/desktop.py   |   95 ++
 .../mic/kickstart/custom_commands/installerfw.py   |   63 +
 .../lib/mic/kickstart/custom_commands/micboot.py   |   49 +
 .../lib/mic/kickstart/custom_commands/micrepo.py   |  127 ++
 .../lib/mic/kickstart/custom_commands/partition.py |   57 +
 scripts/lib/mic/msger.py                           |  309 ++++
 scripts/lib/mic/plugin.py                          |   98 ++
 scripts/lib/mic/pluginbase.py                      |  101 ++
 scripts/lib/mic/plugins/backend/yumpkgmgr.py       |  490 ++++++
 scripts/lib/mic/plugins/backend/zypppkgmgr.py      |  973 ++++++++++++
 scripts/lib/mic/plugins/hook/empty_hook.py         |    3 +
 scripts/lib/mic/plugins/imager/fs_plugin.py        |  143 ++
 scripts/lib/mic/plugins/imager/livecd_plugin.py    |  255 ++++
 scripts/lib/mic/plugins/imager/liveusb_plugin.py   |  260 ++++
 scripts/lib/mic/plugins/imager/loop_plugin.py      |  255 ++++
 scripts/lib/mic/plugins/imager/raw_plugin.py       |  275 ++++
 scripts/lib/mic/rt_util.py                         |  223 +++
 scripts/lib/mic/test                               |    1 +
 scripts/lib/mic/utils/BmapCreate.py                |  298 ++++
 scripts/lib/mic/utils/Fiemap.py                    |  252 ++++
 scripts/lib/mic/utils/cmdln.py                     | 1586 ++++++++++++++++++++
 scripts/lib/mic/utils/errors.py                    |   71 +
 scripts/lib/mic/utils/fs_related.py                | 1029 +++++++++++++
 scripts/lib/mic/utils/gpt_parser.py                |  331 ++++
 scripts/lib/mic/utils/grabber.py                   |   97 ++
 scripts/lib/mic/utils/misc.py                      | 1067 +++++++++++++
 scripts/lib/mic/utils/partitionedfs.py             |  790 ++++++++++
 scripts/lib/mic/utils/proxy.py                     |  183 +++
 scripts/lib/mic/utils/rpmmisc.py                   |  600 ++++++++
 scripts/lib/mic/utils/runner.py                    |  109 ++
 scripts/yocto-image.conf                           |   35 +
 130 files changed, 29216 insertions(+)
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/__init__.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/base.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/__init__.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/authconfig.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/autopart.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/autostep.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/bootloader.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/clearpart.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/device.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/deviceprobe.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/displaymode.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/dmraid.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/driverdisk.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/fcoe.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/firewall.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/firstboot.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/group.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/ignoredisk.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/interactive.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/iscsi.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/iscsiname.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/key.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/keyboard.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/lang.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/langsupport.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/lilocheck.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/logging.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/logvol.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/mediacheck.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/method.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/monitor.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/mouse.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/multipath.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/network.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/partition.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/raid.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/reboot.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/repo.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/rescue.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/rootpw.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/selinux.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/services.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/skipx.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/sshpw.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/timezone.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/updates.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/upgrade.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/user.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/vnc.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/volgroup.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/xconfig.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/zerombr.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/zfcp.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/constants.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/errors.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/__init__.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/control.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f10.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f11.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f12.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f13.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f14.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f15.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f16.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f7.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f8.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f9.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/fc3.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/fc4.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/fc5.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/fc6.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/rhel3.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/rhel4.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/rhel5.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/rhel6.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/ko.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/options.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/parser.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/sections.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/urlgrabber/__init__.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/urlgrabber/byterange.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/urlgrabber/grabber.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/urlgrabber/keepalive.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/urlgrabber/mirror.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/urlgrabber/progress.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/urlgrabber/sslfactory.py
 create mode 100644 scripts/lib/mic/3rdparty/pykickstart/version.py
 create mode 100644 scripts/lib/mic/__init__.py
 create mode 100644 scripts/lib/mic/__version__.py
 create mode 100644 scripts/lib/mic/bootstrap.py
 create mode 100644 scripts/lib/mic/chroot.py
 create mode 100644 scripts/lib/mic/conf.py
 create mode 100644 scripts/lib/mic/creator.py
 create mode 100644 scripts/lib/mic/imager/__init__.py
 create mode 100644 scripts/lib/mic/imager/baseimager.py
 create mode 100644 scripts/lib/mic/imager/fs.py
 create mode 100644 scripts/lib/mic/imager/livecd.py
 create mode 100644 scripts/lib/mic/imager/liveusb.py
 create mode 100644 scripts/lib/mic/imager/loop.py
 create mode 100644 scripts/lib/mic/imager/raw.py
 create mode 100644 scripts/lib/mic/kickstart/__init__.py
 create mode 100644 scripts/lib/mic/kickstart/custom_commands/__init__.py
 create mode 100644 scripts/lib/mic/kickstart/custom_commands/desktop.py
 create mode 100644 scripts/lib/mic/kickstart/custom_commands/installerfw.py
 create mode 100644 scripts/lib/mic/kickstart/custom_commands/micboot.py
 create mode 100644 scripts/lib/mic/kickstart/custom_commands/micrepo.py
 create mode 100644 scripts/lib/mic/kickstart/custom_commands/partition.py
 create mode 100644 scripts/lib/mic/msger.py
 create mode 100644 scripts/lib/mic/plugin.py
 create mode 100644 scripts/lib/mic/pluginbase.py
 create mode 100644 scripts/lib/mic/plugins/backend/yumpkgmgr.py
 create mode 100755 scripts/lib/mic/plugins/backend/zypppkgmgr.py
 create mode 100644 scripts/lib/mic/plugins/hook/.py
 create mode 100644 scripts/lib/mic/plugins/hook/empty_hook.py
 create mode 100644 scripts/lib/mic/plugins/imager/fs_plugin.py
 create mode 100644 scripts/lib/mic/plugins/imager/livecd_plugin.py
 create mode 100644 scripts/lib/mic/plugins/imager/liveusb_plugin.py
 create mode 100644 scripts/lib/mic/plugins/imager/loop_plugin.py
 create mode 100644 scripts/lib/mic/plugins/imager/raw_plugin.py
 create mode 100644 scripts/lib/mic/rt_util.py
 create mode 100644 scripts/lib/mic/test
 create mode 100644 scripts/lib/mic/utils/BmapCreate.py
 create mode 100644 scripts/lib/mic/utils/Fiemap.py
 create mode 100644 scripts/lib/mic/utils/__init__.py
 create mode 100644 scripts/lib/mic/utils/cmdln.py
 create mode 100644 scripts/lib/mic/utils/errors.py
 create mode 100644 scripts/lib/mic/utils/fs_related.py
 create mode 100644 scripts/lib/mic/utils/gpt_parser.py
 create mode 100644 scripts/lib/mic/utils/grabber.py
 create mode 100644 scripts/lib/mic/utils/misc.py
 create mode 100644 scripts/lib/mic/utils/partitionedfs.py
 create mode 100644 scripts/lib/mic/utils/proxy.py
 create mode 100644 scripts/lib/mic/utils/rpmmisc.py
 create mode 100644 scripts/lib/mic/utils/runner.py
 create mode 100644 scripts/yocto-image.conf

[
Patch contents have been removed - too large to post.  You can see
the patch contents here:

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=tzanussi/wic-v1&id=734affa7ac65895aa9329a04cb31782c8495f810
]

-- 
1.7.11.4

[PATCH 3/3] wic: Add OpenEmbedded-specific implementation

[Tom Zanussi]

Reuses the mic/livecd infrastructure but heavily subclasses and
modifies it to adapt to the special needs of building images from
existing OpenEmbedded build artifacts.

In addition to the OE-specific mic objects and modifications to the
underlying infrastructure, this adds a mechanism to allow OE kickstart
files to be 'canned' and made available to users via the 'wic list
images' command.

Two initial OE kickstart files have been added as canned .wks files:
directdisk, which implements the same thing as the images created by
directdisk.bbclass, and mkefidisk, which can essentially be used as a
replacement for mkefidisk.sh.  Of course, since creation of these
images are now driven by .wks files rather than being hard-coded into
class files or scripts, they can be easily modified to generate
different variations on those images.  They also don't require root
priveleges, since they don't use mount to create the images.  They
don't however write to media like mkefidisk.sh does, but rather create
images that can be written onto media.

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
---
 scripts/lib/image/canned-wks/directdisk.wks        |  10 +
 scripts/lib/image/canned-wks/mkefidisk.wks         |  11 +
 scripts/lib/image/config/wic.conf                  |   7 +
 scripts/lib/image/engine.py                        |  24 ++
 scripts/lib/mic/conf.py                            |  54 +--
 scripts/lib/mic/creator.py                         |   3 -
 scripts/lib/mic/imager/baseimager.py               |  82 +---
 scripts/lib/mic/imager/direct.py                   | 472 +++++++++++++++++++++
 scripts/lib/mic/kickstart/__init__.py              |   6 +-
 .../lib/mic/kickstart/custom_commands/__init__.py  |   7 +-
 .../mic/kickstart/custom_commands/micpartition.py  |  57 +++
 .../lib/mic/kickstart/custom_commands/partition.py | 389 +++++++++++++++--
 scripts/lib/mic/plugin.py                          |   6 +-
 scripts/lib/mic/plugins/imager/direct_plugin.py    |  92 ++++
 scripts/lib/mic/utils/fs_related.py                |  30 +-
 scripts/lib/mic/utils/misc.py                      |   4 +-
 scripts/lib/mic/utils/oe/__init__.py               |  22 +
 scripts/lib/mic/utils/oe/misc.py                   | 108 +++++
 scripts/lib/mic/utils/partitionedfs.py             |  94 ++--
 scripts/yocto-image.conf                           |  35 --
 20 files changed, 1253 insertions(+), 260 deletions(-)
 create mode 100644 scripts/lib/image/canned-wks/directdisk.wks
 create mode 100644 scripts/lib/image/canned-wks/mkefidisk.wks
 create mode 100644 scripts/lib/image/config/wic.conf
 create mode 100644 scripts/lib/mic/imager/direct.py
 create mode 100644 scripts/lib/mic/kickstart/custom_commands/micpartition.py
 create mode 100644 scripts/lib/mic/plugins/imager/direct_plugin.py
 create mode 100644 scripts/lib/mic/utils/oe/__init__.py
 create mode 100644 scripts/lib/mic/utils/oe/misc.py
 delete mode 100644 scripts/yocto-image.conf

diff --git a/scripts/lib/image/canned-wks/directdisk.wks b/scripts/lib/image/canned-wks/directdisk.wks
new file mode 100644
index 0000000..d54b382
--- /dev/null
+++ b/scripts/lib/image/canned-wks/directdisk.wks
@@ -0,0 +1,10 @@
+# short-description: Create a 'pcbios' direct disk image
+# long-description: Creates a partitioned legacy BIOS disk image that the user
+# can directly dd to boot media.
+
+
+part /boot --source bootimg --ondisk sda --fstype=msdos --label boot --active --align 1024
+part / --source rootfs --ondisk sda --fstype=ext3 --label platform --align 1024
+
+bootloader  --timeout=0  --append="rootwait rootfstype=ext3 video=vesafb vga=0x318 console=tty0"
+
diff --git a/scripts/lib/image/canned-wks/mkefidisk.wks b/scripts/lib/image/canned-wks/mkefidisk.wks
new file mode 100644
index 0000000..8a3e1f6
--- /dev/null
+++ b/scripts/lib/image/canned-wks/mkefidisk.wks
@@ -0,0 +1,11 @@
+# short-description: Create an EFI disk image
+# long-description: Creates a partitioned EFI disk image that the user
+# can directly dd to boot media.
+
+part /boot --source bootimg --ondisk sda --fstype=efi --label msdos --active --align 1024
+
+part / --source rootfs --ondisk sda --fstype=ext3 --label platform --align 1024
+
+part swap --ondisk sda --size 44 --label swap1 --fstype=swap
+
+bootloader  --timeout=10  --append="rootwait rootfstype=ext3 console=ttyPCH0,115200 console=tty0 vmalloc=256MB snd-hda-intel.enable_msi=0"
diff --git a/scripts/lib/image/config/wic.conf b/scripts/lib/image/config/wic.conf
new file mode 100644
index 0000000..e96d6ae
--- /dev/null
+++ b/scripts/lib/image/config/wic.conf
@@ -0,0 +1,7 @@
+[common]
+; general settings
+distro_name = OpenEmbedded
+
+[create]
+; settings for create subcommand
+runtime=native
diff --git a/scripts/lib/image/engine.py b/scripts/lib/image/engine.py
index a9b530c..be29222 100644
--- a/scripts/lib/image/engine.py
+++ b/scripts/lib/image/engine.py
@@ -37,6 +37,12 @@ import subprocess
 import shutil
 
 import os, sys, errno
+from mic import msger, creator
+from mic.utils import cmdln, misc, errors
+from mic.conf import configmgr
+from mic.plugin import pluginmgr
+from mic.__version__ import VERSION
+from mic.utils.oe.misc import *
 
 
 def verify_build_env():
@@ -216,6 +222,24 @@ def wic_create(args, wks_file, rootfs_dir, bootimg_dir, kernel_dir,
         print "BUILDDIR not found, exiting. (Did you forget to source oe-init-build-env?)"
         sys.exit(1)
 
+    direct_args = list()
+    direct_args.insert(0, oe_builddir)
+    direct_args.insert(0, image_output_dir)
+    direct_args.insert(0, wks_file)
+    direct_args.insert(0, rootfs_dir)
+    direct_args.insert(0, bootimg_dir)
+    direct_args.insert(0, kernel_dir)
+    direct_args.insert(0, native_sysroot)
+    direct_args.insert(0, hdddir)
+    direct_args.insert(0, staging_data_dir)
+    direct_args.insert(0, "direct")
+
+    cr = creator.Creator()
+
+    cr.main(direct_args)
+
+    print "\nThe image(s) were created using OE kickstart file:\n  %s" % wks_file
+
 
 def wic_list(args, scripts_path, properties_file):
     """
diff --git a/scripts/lib/mic/conf.py b/scripts/lib/mic/conf.py
index e37334c..58fad51 100644
--- a/scripts/lib/mic/conf.py
+++ b/scripts/lib/mic/conf.py
@@ -23,29 +23,24 @@ from mic import kickstart
 from mic.utils import misc, runner, proxy, errors
 
 
-DEFAULT_GSITECONF = '/etc/mic/mic.conf'
-
-
 def get_siteconf():
     mic_path = os.path.dirname(__file__)
+    eos = mic_path.find('scripts') + len('scripts')
+    scripts_path = mic_path[:eos]
 
-    m = re.match(r"(?P<prefix>.*)\/lib(64)?\/.*", mic_path)
-    if m and m.group('prefix') != "/usr":
-        return os.path.join(m.group('prefix'), "etc/mic/mic.conf")
-
-    return DEFAULT_GSITECONF
+    return scripts_path + "/lib/image/config/wic.conf"
 
 class ConfigMgr(object):
     prefer_backends = ["zypp", "yum"]
 
     DEFAULTS = {'common': {
                     "distro_name": "Default Distribution",
-                    "plugin_dir": "/usr/lib/mic/plugins", # TODO use prefix also?
+                    "plugin_dir": "/usr/lib/wic/plugins", # TODO use prefix also?
                 },
                 'create': {
-                    "tmpdir": '/var/tmp/mic',
-                    "cachedir": '/var/tmp/mic/cache',
-                    "outdir": './mic-output',
+                    "tmpdir": '/var/tmp/wic',
+                    "cachedir": '/var/tmp/wic/cache',
+                    "outdir": './wic-output',
 
                     "arch": None, # None means auto-detect
                     "pkgmgr": "auto",
@@ -75,7 +70,7 @@ class ConfigMgr(object):
                     "shell": False,
                 },
                 'bootstrap': {
-                    "rootdir": '/var/tmp/mic-bootstrap',
+                    "rootdir": '/var/tmp/wic-bootstrap',
                     "packages": [],
                 },
                }
@@ -191,39 +186,6 @@ class ConfigMgr(object):
                                               self.create['name_prefix'],
                                               self.create['name_suffix'])
 
-        msger.info("Retrieving repo metadata:")
-        ksrepos = misc.get_repostrs_from_ks(ks)
-        if not ksrepos:
-            raise errors.KsError('no valid repos found in ks file')
-
-        for repo in ksrepos:
-            if 'baseurl' in repo and repo['baseurl'].startswith("file:"):
-                repourl = repo['baseurl'].replace('file:', '')
-                repourl = "/%s" % repourl.lstrip('/')
-                self.create['localrepos'].append(repourl)
-
-        self.create['repomd'] = misc.get_metadata_from_repos(
-                                                    ksrepos,
-                                                    self.create['cachedir'])
-        msger.raw(" DONE")
-
-        target_archlist, archlist = misc.get_arch(self.create['repomd'])
-        if self.create['arch']:
-            if self.create['arch'] not in archlist:
-                raise errors.ConfigError("Invalid arch %s for repository. "
-                                  "Valid arches: %s" \
-                                  % (self.create['arch'], ', '.join(archlist)))
-        else:
-            if len(target_archlist) == 1:
-                self.create['arch'] = str(target_archlist[0])
-                msger.info("\nUse detected arch %s." % target_archlist[0])
-            else:
-                raise errors.ConfigError("Please specify a valid arch, "
-                                         "the choice can be: %s" \
-                                         % ', '.join(archlist))
-
-        kickstart.resolve_groups(self.create, self.create['repomd'])
-
         # check selinux, it will block arm and btrfs image creation
         misc.selinux_check(self.create['arch'],
                            [p.fstype for p in ks.handler.partition.partitions])
diff --git a/scripts/lib/mic/creator.py b/scripts/lib/mic/creator.py
index af5fb82..f3d0de1 100644
--- a/scripts/lib/mic/creator.py
+++ b/scripts/lib/mic/creator.py
@@ -293,9 +293,6 @@ class Creator(cmdln.Cmdln):
         if len(argv) == 1:
             return ['help', argv[0]]
 
-        if os.geteuid() != 0:
-            raise msger.error("Root permission is required, abort")
-
         return argv
 
     def do_auto(self, subcmd, opts, *args):
diff --git a/scripts/lib/mic/imager/baseimager.py b/scripts/lib/mic/imager/baseimager.py
index 6efc6c1..4d6be29 100644
--- a/scripts/lib/mic/imager/baseimager.py
+++ b/scripts/lib/mic/imager/baseimager.py
@@ -1,4 +1,3 @@
-
 #!/usr/bin/python -tt
 #
 # Copyright (c) 2007 Red Hat  Inc.
@@ -69,9 +68,10 @@ class BaseImageCreator(object):
 
         self.ks = None
         self.name = "target"
-        self.tmpdir = "/var/tmp/mic"
-        self.cachedir = "/var/tmp/mic/cache"
-        self.workdir = "/var/tmp/mic/build"
+        self.tmpdir = "/var/tmp/wic"
+        self.cachedir = "/var/tmp/wic/cache"
+        self.workdir = "/var/tmp/wic/build"
+
         self.destdir = "."
         self.installerfw_prefix = "INSTALLERFW_"
         self.target_arch = "noarch"
@@ -463,7 +463,7 @@ class BaseImageCreator(object):
         env[self.installerfw_prefix + "DISTRO_NAME"] = self.distro_name
 
         # Name of the image creation tool
-        env[self.installerfw_prefix + "INSTALLER_NAME"] = "mic"
+        env[self.installerfw_prefix + "INSTALLER_NAME"] = "wic"
 
         # The real current location of the mounted file-systems
         if in_chroot:
@@ -668,7 +668,7 @@ class BaseImageCreator(object):
         if cachedir:
             self.cachedir = cachedir
         else:
-            self.cachedir = self.__builddir + "/mic-cache"
+            self.cachedir = self.__builddir + "/wic-cache"
         fs.makedirs(self.cachedir)
         return self.cachedir
 
@@ -751,52 +751,8 @@ class BaseImageCreator(object):
         self.__setup_tmpdir()
         self.__ensure_builddir()
 
-        # prevent popup dialog in Ubuntu(s)
-        misc.hide_loopdev_presentation()
-
-        fs.makedirs(self._instroot)
-        fs.makedirs(self._outdir)
-
         self._mount_instroot(base_on)
 
-        for d in ("/dev/pts",
-                  "/etc",
-                  "/boot",
-                  "/var/log",
-                  "/sys",
-                  "/proc",
-                  "/usr/bin"):
-            fs.makedirs(self._instroot + d)
-
-        if self.target_arch and self.target_arch.startswith("arm"):
-            self.qemu_emulator = misc.setup_qemu_emulator(self._instroot,
-                                                          self.target_arch)
-
-
-        self.get_cachedir(cachedir)
-
-        # bind mount system directories into _instroot
-        for (f, dest) in [("/sys", None),
-                          ("/proc", None),
-                          ("/proc/sys/fs/binfmt_misc", None),
-                          ("/dev/pts", None)]:
-            self.__bindmounts.append(
-                    fs.BindChrootMount(
-                        f, self._instroot, dest))
-
-        self._do_bindmounts()
-
-        self.__create_minimal_dev()
-
-        if os.path.exists(self._instroot + "/etc/mtab"):
-            os.unlink(self._instroot + "/etc/mtab")
-        os.symlink("../proc/mounts", self._instroot + "/etc/mtab")
-
-        self.__write_fstab()
-
-        # get size of available space in 'instroot' fs
-        self._root_fs_avail = misc.get_filesystem_avail(self._instroot)
-
     def unmount(self):
         """Unmounts the target filesystem.
 
@@ -805,34 +761,8 @@ class BaseImageCreator(object):
         from the install root.
 
         """
-        try:
-            mtab = self._instroot + "/etc/mtab"
-            if not os.path.islink(mtab):
-                os.unlink(self._instroot + "/etc/mtab")
-
-            if self.qemu_emulator:
-                os.unlink(self._instroot + self.qemu_emulator)
-        except OSError:
-            pass
-
-        self._undo_bindmounts()
-
-        """ Clean up yum garbage """
-        try:
-            instroot_pdir = os.path.dirname(self._instroot + self._instroot)
-            if os.path.exists(instroot_pdir):
-                shutil.rmtree(instroot_pdir, ignore_errors = True)
-            yumlibdir = self._instroot + "/var/lib/yum"
-            if os.path.exists(yumlibdir):
-                shutil.rmtree(yumlibdir, ignore_errors = True)
-        except OSError:
-            pass
-
         self._unmount_instroot()
 
-        # reset settings of popup dialog in Ubuntu(s)
-        misc.unhide_loopdev_presentation()
-
 
     def cleanup(self):
         """Unmounts the target filesystem and deletes temporary files.
diff --git a/scripts/lib/mic/imager/direct.py b/scripts/lib/mic/imager/direct.py
new file mode 100644
index 0000000..d24bc68
--- /dev/null
+++ b/scripts/lib/mic/imager/direct.py
@@ -0,0 +1,472 @@
+# ex:ts=4:sw=4:sts=4:et
+# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
+#
+# Copyright (c) 2013, Intel Corporation.
+# All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+# DESCRIPTION
+# This implements the 'direct' image creator class for 'wic', based
+# loosely on the raw image creator from 'mic'
+#
+# AUTHORS
+# Tom Zanussi <tom.zanussi (at] linux.intel.com>
+#
+
+import os
+import stat
+import shutil
+
+from mic import kickstart, msger
+from mic.utils import fs_related, runner, misc
+from mic.utils.partitionedfs import PartitionedMount
+from mic.utils.errors import CreatorError, MountError
+from mic.imager.baseimager import BaseImageCreator
+from mic.utils.oe.misc import *
+
+class DirectImageCreator(BaseImageCreator):
+    """
+    Installs a system into a file containing a partitioned disk image.
+
+    DirectImageCreator is an advanced ImageCreator subclass; an image
+    file is formatted with a partition table, each partition created
+    from a rootfs or other OpenEmbedded build artifact and dd'ed into
+    the virtual disk. The disk image can subsequently be dd'ed onto
+    media and used on actual hardware.
+    """
+
+    def __init__(self, oe_builddir, image_output_dir, rootfs_dir, bootimg_dir,
+                 kernel_dir, native_sysroot, hdddir, staging_data_dir,
+                 creatoropts=None, pkgmgr=None, compress_image=None,
+                 generate_bmap=None, fstab_entry="uuid"):
+        """
+        Initialize a DirectImageCreator instance.
+
+        This method takes the same arguments as ImageCreator.__init__()
+        """
+        BaseImageCreator.__init__(self, creatoropts, pkgmgr)
+
+        self.__instimage = None
+        self.__imgdir = None
+        self.__disks = {}
+        self.__disk_format = "direct"
+        self._disk_names = []
+        self._ptable_format = self.ks.handler.bootloader.ptable
+        self.use_uuid = fstab_entry == "uuid"
+        self.compress_image = compress_image
+        self.bmap_needed = generate_bmap
+
+        self.oe_builddir = oe_builddir
+        if image_output_dir:
+            self.tmpdir = image_output_dir
+            self.cachedir = "%s/cache" % image_output_dir
+        self.rootfs_dir = rootfs_dir
+        self.bootimg_dir = bootimg_dir
+        self.kernel_dir = kernel_dir
+        self.native_sysroot = native_sysroot
+        self.hdddir = hdddir
+        self.staging_data_dir = staging_data_dir
+        self.boot_type = ""
+
+    def __write_fstab(self):
+        """overriden to generate fstab (temporarily) in rootfs. This
+        is called from mount_instroot, make sure it doesn't get called
+        from BaseImage.mount()"""
+
+        image_rootfs = self.rootfs_dir
+
+        parts = self._get_parts()
+
+        fstab = image_rootfs + "/etc/fstab"
+
+        self._save_fstab(fstab)
+        fstab_lines = self._get_fstab(fstab, parts)
+        self._update_fstab(fstab_lines, parts)
+        self._write_fstab(fstab, fstab_lines)
+
+        return fstab
+
+    def _update_fstab(self, fstab_lines, parts):
+        """Assume partition order same as in wks"""
+        for num, p in enumerate(parts, 1):
+            if p.mountpoint == "/" or p.mountpoint == "/boot":
+                continue
+            if self._ptable_format == 'msdos' and num > 3:
+                device_name = "/dev/" + p.disk + str(num + 1)
+            else:
+                device_name = "/dev/" + p.disk + str(num)
+            fstab_entry = device_name + "\t" + p.mountpoint + "\t" + p.fstype + "\tdefaults\t0\t0\n"
+            fstab_lines.append(fstab_entry)
+
+    def _write_fstab(self, fstab, fstab_lines):
+        fstab = open(fstab, "w")
+        for line in fstab_lines:
+            fstab.write(line)
+        fstab.close()
+
+    def _save_fstab(self, fstab):
+        """Save the current fstab in rootfs"""
+        shutil.copyfile(fstab, fstab + ".orig")
+
+    def _restore_fstab(self, fstab):
+        """Restore the saved fstab in rootfs"""
+        shutil.move(fstab + ".orig", fstab)
+
+    def _get_fstab(self, fstab, parts):
+        """Return the desired contents of /etc/fstab."""
+        f = open(fstab, "r")
+        fstab_contents = f.readlines()
+        f.close()
+
+        return fstab_contents
+
+    def _get_parts(self):
+        if not self.ks:
+            raise CreatorError("Failed to get partition info, "
+                               "please check your kickstart setting.")
+
+        # Set a default partition if no partition is given out
+        if not self.ks.handler.partition.partitions:
+            partstr = "part / --size 1900 --ondisk sda --fstype=ext3"
+            args = partstr.split()
+            pd = self.ks.handler.partition.parse(args[1:])
+            if pd not in self.ks.handler.partition.partitions:
+                self.ks.handler.partition.partitions.append(pd)
+
+        # partitions list from kickstart file
+        return kickstart.get_partitions(self.ks)
+
+    def get_disk_names(self):
+        """ Returns a list of physical target disk names (e.g., 'sdb') which
+        will be created. """
+
+        if self._disk_names:
+            return self._disk_names
+
+        #get partition info from ks handler
+        parts = self._get_parts()
+
+        for i in range(len(parts)):
+            if parts[i].disk:
+                disk_name = parts[i].disk
+            else:
+                raise CreatorError("Failed to create disks, no --ondisk "
+                                   "specified in partition line of ks file")
+
+            if parts[i].mountpoint and not parts[i].fstype:
+                raise CreatorError("Failed to create disks, no --fstype "
+                                    "specified for partition with mountpoint "
+                                    "'%s' in the ks file")
+
+            self._disk_names.append(disk_name)
+
+        return self._disk_names
+
+    def _full_name(self, name, extention):
+        """ Construct full file name for a file we generate. """
+        return "%s-%s.%s" % (self.name, name, extention)
+
+    def _full_path(self, path, name, extention):
+        """ Construct full file path to a file we generate. """
+        return os.path.join(path, self._full_name(name, extention))
+
+    def get_boot_type(self):
+        """ Determine the boot type from fstype and mountpoint. """
+        parts = self._get_parts()
+
+        boot_type = ""
+
+        for p in parts:
+            if p.mountpoint == "/boot":
+                if p.fstype == "msdos":
+                    boot_type = "pcbios"
+                else:
+                    boot_type = p.fstype
+        return boot_type
+
+    #
+    # Actual implemention
+    #
+    def _mount_instroot(self, base_on = None):
+        """
+        For 'wic', we already have our build artifacts and don't want
+        to loop mount anything to install into, we just create
+        filesystems from the artifacts directly and combine them into
+        a partitioned image.
+
+        We still want to reuse as much of the basic mic machinery
+        though; despite the fact that we don't actually do loop or any
+        other kind of mounting we still want to do many of the same
+        things to prepare images, so we basically just adapt to the
+        basic framework and reinterpret what 'mounting' means in our
+        context.
+
+        _instroot would normally be something like
+        /var/tmp/wic/build/imgcreate-s_9AKQ/install_root, for
+        installing packages, etc.  We don't currently need to do that,
+        so we simplify life by just using /var/tmp/wic/build as our
+        workdir.
+        """
+        parts = self._get_parts()
+
+        self.__instimage = PartitionedMount(self._instroot)
+
+        fstab = self.__write_fstab()
+
+        self.boot_type = self.get_boot_type()
+
+        if not self.bootimg_dir:
+            if self.boot_type == "pcbios":
+                self.bootimg_dir = self.staging_data_dir
+            elif self.boot_type == "efi":
+                self.bootimg_dir = self.hdddir
+
+        if self.boot_type == "pcbios":
+            self._create_syslinux_config()
+        elif self.boot_type == "efi":
+            self._create_grubefi_config()
+        else:
+            raise CreatorError("Failed to detect boot type (no /boot partition?), "
+                               "please check your kickstart setting.")
+
+        for p in parts:
+            if p.fstype == "efi":
+                p.fstype = "msdos"
+            # need to create the filesystems in order to get their
+            # sizes before we can add them and do the layout.
+            # PartitionedMount.mount() actually calls __format_disks()
+            # to create the disk images and carve out the partitions,
+            # then self.install() calls PartitionedMount.install()
+            # which calls __install_partitition() for each partition
+            # to dd the fs into the partitions.  It would be nice to
+            # be able to use e.g. ExtDiskMount etc to create the
+            # filesystems, since that's where existing e.g. mkfs code
+            # is, but those are only created after __format_disks()
+            # which needs the partition sizes so needs them created
+            # before its called.  Well, the existing setup is geared
+            # to installing packages into mounted filesystems - maybe
+            # when/if we need to actually do package selection we
+            # should modify things to use those objects, but for now
+            # we can avoid that.
+            p.prepare(self.workdir, self.oe_builddir, self.boot_type,
+                      self.rootfs_dir, self.bootimg_dir, self.kernel_dir,
+                      self.native_sysroot)
+
+            self.__instimage.add_partition(int(p.size),
+                                           p.disk,
+                                           p.mountpoint,
+                                           p.source_file,
+                                           p.fstype,
+                                           p.label,
+                                           fsopts = p.fsopts,
+                                           boot = p.active,
+                                           align = p.align,
+                                           part_type = p.part_type)
+        self._restore_fstab(fstab)
+        self.__instimage.layout_partitions(self._ptable_format)
+
+        self.__imgdir = self.workdir
+        for disk_name, disk in self.__instimage.disks.items():
+            full_path = self._full_path(self.__imgdir, disk_name, "direct")
+            msger.debug("Adding disk %s as %s with size %s bytes" \
+                        % (disk_name, full_path, disk['min_size']))
+            disk_obj = fs_related.DiskImage(full_path, disk['min_size'])
+            self.__disks[disk_name] = disk_obj
+            self.__instimage.add_disk(disk_name, disk_obj)
+
+        self.__instimage.mount()
+
+    def install(self, repo_urls=None):
+        """
+        Install fs images into partitions
+        """
+        for disk_name, disk in self.__instimage.disks.items():
+            full_path = self._full_path(self.__imgdir, disk_name, "direct")
+            msger.debug("Installing disk %s as %s with size %s bytes" \
+                        % (disk_name, full_path, disk['min_size']))
+            self.__instimage.install(full_path)
+
+    def configure(self, repodata = None):
+        """
+        Configure the system image according to kickstart.
+
+        For now, it just prepares the image to be bootable by e.g.
+        creating and installing a bootloader configuration.
+        """
+        if self.boot_type == "pcbios":
+            self._install_syslinux()
+
+    def print_outimage_info(self):
+        """
+        Print the image(s) and artifacts used, for the user.
+        """
+        msg = "The new image(s) can be found here:\n"
+
+        for disk_name, disk in self.__instimage.disks.items():
+            full_path = self._full_path(self.__imgdir, disk_name, "direct")
+            msg += '  %s\n\n' % full_path
+
+        msg += 'The following build artifacts were used to create the image(s):\n'
+        msg += '  ROOTFS_DIR:      %s\n' % self.rootfs_dir
+        msg += '  BOOTIMG_DIR:     %s\n' % self.bootimg_dir
+        msg += '  KERNEL_DIR:      %s\n' % self.kernel_dir
+        msg += '  NATIVE_SYSROOT:  %s\n' % self.native_sysroot
+
+        msger.info(msg)
+
+    def _get_boot_config(self):
+        """
+        Return the rootdev/root_part_uuid (if specified by
+        --part-type)
+
+        Assume partition order same as in wks
+        """
+        rootdev = None
+        root_part_uuid = None
+        parts = self._get_parts()
+        for num, p in enumerate(parts, 1):
+            if p.mountpoint == "/":
+                if self._ptable_format == 'msdos' and num > 3:
+                    rootdev = "/dev/%s%-d" % (p.disk, num + 1)
+                else:
+                    rootdev = "/dev/%s%-d" % (p.disk, num)
+                root_part_uuid = p.part_type
+
+        return (rootdev, root_part_uuid)
+
+    def _create_syslinux_config(self):
+        hdddir = "%s/hdd/boot" % self.workdir
+        rm_cmd = "rm -rf " + self.workdir
+        exec_cmd(rm_cmd)
+
+        install_cmd = "install -d %s" % hdddir
+        tmp = exec_cmd(install_cmd)
+
+        splash = os.path.join(self.workdir, "/hdd/boot/splash.jpg")
+        if os.path.exists(splash):
+            splashline = "menu background splash.jpg"
+        else:
+            splashline = ""
+
+        (rootdev, root_part_uuid) = self._get_boot_config()
+        options = self.ks.handler.bootloader.appendLine
+
+        syslinux_conf = ""
+        syslinux_conf += "PROMPT 0\n"
+        timeout = kickstart.get_timeout(self.ks)
+        if not timeout:
+            timeout = 0
+        syslinux_conf += "TIMEOUT " + str(timeout) + "\n"
+        syslinux_conf += "\n"
+        syslinux_conf += "ALLOWOPTIONS 1\n"
+        syslinux_conf += "SERIAL 0 115200\n"
+        syslinux_conf += "\n"
+        if splashline:
+            syslinux_conf += "%s\n" % splashline
+        syslinux_conf += "DEFAULT boot\n"
+        syslinux_conf += "LABEL boot\n"
+
+        kernel = "/vmlinuz"
+        syslinux_conf += "KERNEL " + kernel + "\n"
+
+        if self._ptable_format == 'msdos':
+            rootstr = rootdev
+        else:
+            if not root_part_uuid:
+                raise MountError("Cannot find the root GPT partition UUID")
+            rootstr = "PARTUUID=%s" % root_part_uuid
+
+        syslinux_conf += "APPEND label=boot root=%s %s\n" % (rootstr, options)
+
+        msger.debug("Writing syslinux config %s/hdd/boot/syslinux.cfg" \
+                    % self.workdir)
+        cfg = open("%s/hdd/boot/syslinux.cfg" % self.workdir, "w")
+        cfg.write(syslinux_conf)
+        cfg.close()
+
+    def _create_grubefi_config(self):
+        hdddir = "%s/hdd/boot" % self.workdir
+        rm_cmd = "rm -rf %s" % self.workdir
+        exec_cmd(rm_cmd)
+
+        install_cmd = "install -d %s/EFI/BOOT" % hdddir
+        tmp = exec_cmd(install_cmd)
+
+        splash = os.path.join(self.workdir, "/EFI/boot/splash.jpg")
+        if os.path.exists(splash):
+            splashline = "menu background splash.jpg"
+        else:
+            splashline = ""
+
+        (rootdev, root_part_uuid) = self._get_boot_config()
+        options = self.ks.handler.bootloader.appendLine
+
+        grubefi_conf = ""
+        grubefi_conf += "serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1\n"
+        grubefi_conf += "default=boot\n"
+        timeout = kickstart.get_timeout(self.ks)
+        if not timeout:
+            timeout = 0
+        grubefi_conf += "timeout=%s\n" % timeout
+        grubefi_conf += "menuentry 'boot'{\n"
+
+        kernel = "/vmlinuz"
+
+        if self._ptable_format == 'msdos':
+            rootstr = rootdev
+        else:
+            if not root_part_uuid:
+                raise MountError("Cannot find the root GPT partition UUID")
+            rootstr = "PARTUUID=%s" % root_part_uuid
+
+        grubefi_conf += "linux %s root=%s rootwait %s\n" \
+            % (kernel, rootstr, options)
+        grubefi_conf += "}\n"
+        if splashline:
+            syslinux_conf += "%s\n" % splashline
+
+        msger.debug("Writing grubefi config %s/hdd/boot/EFI/BOOT/grub.cfg" \
+                        % self.workdir)
+        cfg = open("%s/hdd/boot/EFI/BOOT/grub.cfg" % self.workdir, "w")
+        cfg.write(grubefi_conf)
+        cfg.close()
+
+    def _install_syslinux(self):
+        mbrfile = "%s/syslinux/" % self.bootimg_dir
+        if self._ptable_format == 'gpt':
+            mbrfile += "gptmbr.bin"
+        else:
+            mbrfile += "mbr.bin"
+
+        if not os.path.exists(mbrfile):
+            msger.error("Couldn't find %s.  If using the -e option, do you have the right MACHINE set in local.conf?  If not, is the bootimg_dir path correct?" % mbrfile)
+
+        for disk_name, disk in self.__instimage.disks.items():
+            full_path = self._full_path(self.__imgdir, disk_name, "direct")
+            msger.debug("Installing MBR on disk %s as %s with size %s bytes" \
+                            % (disk_name, full_path, disk['min_size']))
+
+            rc = runner.show(['dd', 'if=%s' % mbrfile,
+                              'of=%s' % full_path, 'conv=notrunc'])
+            if rc != 0:
+                raise MountError("Unable to set MBR to %s" % full_path)
+
+    def _unmount_instroot(self):
+        if not self.__instimage is None:
+            try:
+                self.__instimage.cleanup()
+            except MountError, err:
+                msger.warning("%s" % err)
+
diff --git a/scripts/lib/mic/kickstart/__init__.py b/scripts/lib/mic/kickstart/__init__.py
index f9a5334..7e645ca 100644
--- a/scripts/lib/mic/kickstart/__init__.py
+++ b/scripts/lib/mic/kickstart/__init__.py
@@ -99,11 +99,11 @@ def read_kickstart(path):
     commandMap[using_version]["desktop"] = desktop.Mic_Desktop
     commandMap[using_version]["repo"] = micrepo.Mic_Repo
     commandMap[using_version]["bootloader"] = micboot.Mic_Bootloader
-    commandMap[using_version]["part"] = partition.Mic_Partition
-    commandMap[using_version]["partition"] = partition.Mic_Partition
+    commandMap[using_version]["part"] = partition.Wic_Partition
+    commandMap[using_version]["partition"] = partition.Wic_Partition
     commandMap[using_version]["installerfw"] = installerfw.Mic_installerfw
     dataMap[using_version]["RepoData"] = micrepo.Mic_RepoData
-    dataMap[using_version]["PartData"] = partition.Mic_PartData
+    dataMap[using_version]["PartData"] = partition.Wic_PartData
     superclass = ksversion.returnClassForVersion(version=using_version)
 
     class KSHandlers(superclass):
diff --git a/scripts/lib/mic/kickstart/custom_commands/__init__.py b/scripts/lib/mic/kickstart/custom_commands/__init__.py
index 5f4c440..6aed0ff 100644
--- a/scripts/lib/mic/kickstart/custom_commands/__init__.py
+++ b/scripts/lib/mic/kickstart/custom_commands/__init__.py
@@ -1,12 +1,17 @@
 from desktop import Mic_Desktop
 from micrepo import Mic_Repo, Mic_RepoData
-from partition import Mic_Partition
+from micpartition import Mic_Partition
+from micpartition import Mic_PartData
 from installerfw import Mic_installerfw
+from partition import Wic_Partition
 
 __all__ = (
     "Mic_Desktop",
     "Mic_Repo",
     "Mic_RepoData",
     "Mic_Partition",
+    "Mic_PartData",
     "Mic_installerfw",
+    "Wic_Partition",
+    "Wic_PartData",
 )
diff --git a/scripts/lib/mic/kickstart/custom_commands/micpartition.py b/scripts/lib/mic/kickstart/custom_commands/micpartition.py
new file mode 100644
index 0000000..59a87fb
--- /dev/null
+++ b/scripts/lib/mic/kickstart/custom_commands/micpartition.py
@@ -0,0 +1,57 @@
+#!/usr/bin/python -tt
+#
+# Marko Saukko <marko.saukko@cybercom.com>
+#
+# Copyright (C) 2011 Nokia Corporation and/or its subsidiary(-ies).
+#
+# This copyrighted material is made available to anyone wishing to use, modify,
+# copy, or redistribute it subject to the terms and conditions of the GNU
+# General Public License v.2. This program is distributed in the hope that it
+# will be useful, but WITHOUT ANY WARRANTY expressed or implied, including the
+# implied warranties of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+# See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program; if not, write to the Free Software Foundation, Inc., 51
+# Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+
+from pykickstart.commands.partition import *
+
+class Mic_PartData(FC4_PartData):
+    removedKeywords = FC4_PartData.removedKeywords
+    removedAttrs = FC4_PartData.removedAttrs
+
+    def __init__(self, *args, **kwargs):
+        FC4_PartData.__init__(self, *args, **kwargs)
+        self.deleteRemovedAttrs()
+        self.align = kwargs.get("align", None)
+        self.extopts = kwargs.get("extopts", None)
+        self.part_type = kwargs.get("part_type", None)
+
+    def _getArgsAsStr(self):
+        retval = FC4_PartData._getArgsAsStr(self)
+
+        if self.align:
+            retval += " --align"
+        if self.extopts:
+            retval += " --extoptions=%s" % self.extopts
+        if self.part_type:
+            retval += " --part-type=%s" % self.part_type
+
+        return retval
+
+class Mic_Partition(FC4_Partition):
+    removedKeywords = FC4_Partition.removedKeywords
+    removedAttrs = FC4_Partition.removedAttrs
+
+    def _getParser(self):
+        op = FC4_Partition._getParser(self)
+        # The alignment value is given in kBytes. e.g., value 8 means that
+        # the partition is aligned to start from 8096 byte boundary.
+        op.add_option("--align", type="int", action="store", dest="align",
+                      default=None)
+        op.add_option("--extoptions", type="string", action="store", dest="extopts",
+                      default=None)
+        op.add_option("--part-type", type="string", action="store", dest="part_type",
+                      default=None)
+        return op
diff --git a/scripts/lib/mic/kickstart/custom_commands/partition.py b/scripts/lib/mic/kickstart/custom_commands/partition.py
index 59a87fb..302cace 100644
--- a/scripts/lib/mic/kickstart/custom_commands/partition.py
+++ b/scripts/lib/mic/kickstart/custom_commands/partition.py
@@ -1,57 +1,370 @@
-#!/usr/bin/python -tt
+# ex:ts=4:sw=4:sts=4:et
+# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
 #
-# Marko Saukko <marko.saukko@cybercom.com>
+# Copyright (c) 2013, Intel Corporation.
+# All rights reserved.
 #
-# Copyright (C) 2011 Nokia Corporation and/or its subsidiary(-ies).
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
 #
-# This copyrighted material is made available to anyone wishing to use, modify,
-# copy, or redistribute it subject to the terms and conditions of the GNU
-# General Public License v.2. This program is distributed in the hope that it
-# will be useful, but WITHOUT ANY WARRANTY expressed or implied, including the
-# implied warranties of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
-# See the GNU General Public License for more details.
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+# DESCRIPTION
+# This module provides the OpenEmbedded partition object definitions.
+#
+# AUTHORS
+# Tom Zanussi <tom.zanussi (at] linux.intel.com>
 #
-# You should have received a copy of the GNU General Public License along with
-# this program; if not, write to the Free Software Foundation, Inc., 51
-# Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+
+import shutil
 
 from pykickstart.commands.partition import *
+from mic.utils.oe.misc import *
+
+from mic.kickstart.custom_commands import *
 
-class Mic_PartData(FC4_PartData):
-    removedKeywords = FC4_PartData.removedKeywords
-    removedAttrs = FC4_PartData.removedAttrs
+BOOTDD_EXTRA_SPACE = 16384
+
+class Wic_PartData(Mic_PartData):
+    removedKeywords = Mic_PartData.removedKeywords
+    removedAttrs = Mic_PartData.removedAttrs
 
     def __init__(self, *args, **kwargs):
-        FC4_PartData.__init__(self, *args, **kwargs)
+        Mic_PartData.__init__(self, *args, **kwargs)
         self.deleteRemovedAttrs()
-        self.align = kwargs.get("align", None)
-        self.extopts = kwargs.get("extopts", None)
-        self.part_type = kwargs.get("part_type", None)
+        self.source = kwargs.get("source", None)
+        self.source_file = ""
+        self.size = 0
 
     def _getArgsAsStr(self):
-        retval = FC4_PartData._getArgsAsStr(self)
+        retval = Mic_PartData._getArgsAsStr(self)
 
-        if self.align:
-            retval += " --align"
-        if self.extopts:
-            retval += " --extoptions=%s" % self.extopts
-        if self.part_type:
-            retval += " --part-type=%s" % self.part_type
+        if self.source:
+            retval += " --source=%s" % self.source
 
         return retval
 
-class Mic_Partition(FC4_Partition):
-    removedKeywords = FC4_Partition.removedKeywords
-    removedAttrs = FC4_Partition.removedAttrs
+    def prepare(self, cr_workdir, oe_builddir, boot_type, rootfs_dir,
+                bootimg_dir, kernel_dir, native_sysroot):
+        """
+        Prepare content for individual partitions, depending on
+        partition command parameters.
+        """
+        if not self.source:
+            if self.fstype and self.fstype == "swap":
+                self.prepare_swap_partition(cr_workdir, oe_builddir,
+                                            native_sysroot)
+            elif self.fstype:
+                self.prepare_empty_partition(cr_workdir, oe_builddir,
+                                             native_sysroot)
+            return
+
+        if self.source == "bootimg" and boot_type == "pcbios":
+            self.prepare_bootimg_pcbios(cr_workdir, oe_builddir, bootimg_dir,
+                                        kernel_dir, native_sysroot)
+        elif self.source == "bootimg" and boot_type == "efi":
+            self.prepare_bootimg_efi(cr_workdir, oe_builddir, bootimg_dir,
+                                     kernel_dir, native_sysroot)
+        elif self.source.startswith("rootfs"):
+            self.prepare_rootfs(cr_workdir, oe_builddir, rootfs_dir,
+                                native_sysroot)
+
+    def prepare_bootimg_pcbios(self, cr_workdir, oe_builddir, bootimg_dir,
+                               kernel_dir, native_sysroot):
+        """
+        Prepare content for a legacy bios boot partition.
+        """
+        staging_kernel_dir = kernel_dir
+        staging_data_dir = bootimg_dir
+
+        hdddir = "%s/hdd/boot" % cr_workdir
+
+        install_cmd = "install -m 0644 %s/bzImage %s/vmlinuz" \
+            % (staging_kernel_dir, hdddir)
+        tmp = exec_cmd(install_cmd)
+
+        install_cmd = "install -m 444 %s/syslinux/ldlinux.sys %s/ldlinux.sys" \
+            % (staging_data_dir, hdddir)
+        tmp = exec_cmd(install_cmd)
+
+        du_cmd = "du -bks %s" % hdddir
+        rc, out = exec_cmd(du_cmd)
+        blocks = int(out.split()[0])
+
+        blocks += BOOTDD_EXTRA_SPACE
+
+        # Ensure total sectors is an integral number of sectors per
+        # track or mcopy will complain. Sectors are 512 bytes, and we
+        # generate images with 32 sectors per track. This calculation is
+        # done in blocks, thus the mod by 16 instead of 32.
+        blocks += (16 - (blocks % 16))
+
+        # dosfs image, created by mkdosfs
+        bootimg = "%s/boot.img" % cr_workdir
+
+        dosfs_cmd = "mkdosfs -n boot -S 512 -C %s %d" % (bootimg, blocks)
+        exec_native_cmd(dosfs_cmd, native_sysroot)
+
+        mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir)
+        exec_native_cmd(mcopy_cmd, native_sysroot)
+
+        syslinux_cmd = "syslinux %s" % bootimg
+        exec_native_cmd(syslinux_cmd, native_sysroot)
+
+        chmod_cmd = "chmod 644 %s" % bootimg
+        exec_cmd(chmod_cmd)
+
+        du_cmd = "du -Lbms %s" % bootimg
+        rc, out = exec_cmd(du_cmd)
+        bootimg_size = out.split()[0]
+
+        self.size = bootimg_size
+        self.source_file = bootimg
+
+    def prepare_bootimg_efi(self, cr_workdir, oe_builddir, bootimg_dir,
+                            kernel_dir, native_sysroot):
+        """
+        Prepare content for an EFI (grub) boot partition.
+        """
+        staging_kernel_dir = kernel_dir
+        staging_data_dir = bootimg_dir
+
+        hdddir = "%s/hdd/boot" % cr_workdir
+
+        install_cmd = "install -m 0644 %s/bzImage %s/vmlinuz" % \
+            (staging_kernel_dir, hdddir)
+        tmp = exec_cmd(install_cmd)
+
+        shutil.copyfile("%s/hdd/boot/EFI/BOOT/grub.cfg" % cr_workdir,
+                        "%s/grub.cfg" % cr_workdir)
+
+        cp_cmd = "cp %s/EFI/BOOT/* %s/EFI/BOOT" % (staging_data_dir, hdddir)
+        exec_cmd(cp_cmd, True)
+
+        shutil.move("%s/grub.cfg" % cr_workdir,
+                    "%s/hdd/boot/EFI/BOOT/grub.cfg" % cr_workdir)
+
+        du_cmd = "du -bks %s" % hdddir
+        rc, out = exec_cmd(du_cmd)
+        blocks = int(out.split()[0])
+
+        blocks += BOOTDD_EXTRA_SPACE
+
+        # Ensure total sectors is an integral number of sectors per
+        # track or mcopy will complain. Sectors are 512 bytes, and we
+        # generate images with 32 sectors per track. This calculation is
+        # done in blocks, thus the mod by 16 instead of 32.
+        blocks += (16 - (blocks % 16))
+
+        # dosfs image, created by mkdosfs
+        bootimg = "%s/boot.img" % cr_workdir
+
+        dosfs_cmd = "mkdosfs -n efi -C %s %d" % (bootimg, blocks)
+        exec_native_cmd(dosfs_cmd, native_sysroot)
+
+        mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir)
+        exec_native_cmd(mcopy_cmd, native_sysroot)
+
+        chmod_cmd = "chmod 644 %s" % bootimg
+        exec_cmd(chmod_cmd)
+
+        du_cmd = "du -Lbms %s" % bootimg
+        rc, out = exec_cmd(du_cmd)
+        bootimg_size = out.split()[0]
+
+        self.size = bootimg_size
+        self.source_file = bootimg
+
+    def prepare_rootfs_from_fs_image(self, cr_workdir, oe_builddir,
+                                     rootfs_dir):
+        """
+        Handle an already-created partition e.g. xxx.ext3
+        """
+        rootfs = oe_builddir
+        du_cmd = "du -Lbms %s" % rootfs
+        rc, out = exec_cmd(du_cmd)
+        rootfs_size = out.split()[0]
+
+        self.size = rootfs_size
+        self.source_file = rootfs
+
+    def prepare_rootfs(self, cr_workdir, oe_builddir, rootfs_dir,
+                       native_sysroot):
+        """
+        Prepare content for a rootfs partition i.e. create a partition
+        and fill it from a /rootfs dir.
+
+        Currently handles ext2/3/4 and btrfs.
+        """
+        if self.fstype.startswith("ext"):
+            return self.prepare_rootfs_ext(cr_workdir, oe_builddir,
+                                           rootfs_dir, native_sysroot)
+        elif self.fstype.startswith("btrfs"):
+            return self.prepare_rootfs_btrfs(cr_workdir, oe_builddir,
+                                             rootfs_dir, native_sysroot)
+
+    def prepare_rootfs_ext(self, cr_workdir, oe_builddir, rootfs_dir,
+                           native_sysroot):
+        """
+        Prepare content for an ext2/3/4 rootfs partition.
+        """
+        populate_script = "%s/usr/bin/populate-extfs.sh" % native_sysroot
+        image_extra_space = 10240
+
+        image_rootfs = rootfs_dir
+        rootfs = "%s/rootfs.%s" % (cr_workdir, self.fstype)
+
+        du_cmd = "du -ks %s" % image_rootfs
+        rc, out = exec_cmd(du_cmd)
+        actual_rootfs_size = out.split()[0]
+
+        rootfs_size = int(actual_rootfs_size) + image_extra_space
+
+        dd_cmd = "dd if=/dev/zero of=%s bs=1024 seek=%d count=0 bs=1k" % \
+            (rootfs, rootfs_size)
+        rc, out = exec_cmd(dd_cmd)
+
+        extra_imagecmd = "-i 8192"
+
+        mkfs_cmd = "mkfs.%s -F %s %s" % (self.fstype, extra_imagecmd, rootfs)
+        rc, out = exec_native_cmd(mkfs_cmd, native_sysroot)
+
+        populate_cmd = populate_script + " " + image_rootfs + " " + rootfs
+        rc, out = exec_native_cmd(populate_cmd, native_sysroot)
+
+        # get the rootfs size in the right units for kickstart (Mb)
+        du_cmd = "du -Lbms %s" % rootfs
+        rc, out = exec_cmd(du_cmd)
+        rootfs_size = out.split()[0]
+
+        self.size = rootfs_size
+        self.source_file = rootfs
+
+        return 0
+
+    def prepare_rootfs_btrfs(self, cr_workdir, oe_builddir, rootfs_dir,
+                             native_sysroot):
+        """
+        Prepare content for a btrfs rootfs partition.
+
+        Currently handles ext2/3/4 and btrfs.
+        """
+        image_extra_space = 10240
+
+        image_rootfs = rootfs_dir
+        rootfs = "%s/rootfs.%s" % (cr_workdir, self.fstype)
+
+        du_cmd = "du -ks %s" % image_rootfs
+        rc, out = exec_cmd(du_cmd)
+        actual_rootfs_size = out.split()[0]
+
+        rootfs_size = int(actual_rootfs_size) + image_extra_space
+
+        dd_cmd = "dd if=/dev/zero of=%s bs=1024 seek=%d count=0 bs=1k" % \
+            (rootfs, rootfs_size)
+        rc, out = exec_cmd(dd_cmd)
+
+        mkfs_cmd = "mkfs.%s -b %d -r %s %s" % \
+            (self.fstype, rootfs_size * 1024, image_rootfs, rootfs)
+        rc, out = exec_native_cmd(mkfs_cmd, native_sysroot)
+
+        # get the rootfs size in the right units for kickstart (Mb)
+        du_cmd = "du -Lbms %s" % rootfs
+        rc, out = exec_cmd(du_cmd)
+        rootfs_size = out.split()[0]
+
+        self.size = rootfs_size
+        self.source_file = rootfs
+
+    def prepare_empty_partition(self, cr_workdir, oe_builddir, native_sysroot):
+        """
+        Prepare an empty partition.
+        """
+        if self.fstype.startswith("ext"):
+            return self.prepare_empty_partition_ext(cr_workdir, oe_builddir,
+                                                    native_sysroot)
+        elif self.fstype.startswith("btrfs"):
+            return self.prepare_empty_partition_btrfs(cr_workdir, oe_builddir,
+                                                      native_sysroot)
+
+    def prepare_empty_partition_ext(self, cr_workdir, oe_builddir,
+                                    native_sysroot):
+        """
+        Prepare an empty ext2/3/4 partition.
+        """
+        fs = "%s/fs.%s" % (cr_workdir, self.fstype)
+
+        dd_cmd = "dd if=/dev/zero of=%s bs=1M seek=%d count=0" % \
+            (fs, self.size)
+        rc, out = exec_cmd(dd_cmd)
+
+        extra_imagecmd = "-i 8192"
+
+        mkfs_cmd = "mkfs.%s -F %s %s" % (self.fstype, extra_imagecmd, fs)
+        rc, out = exec_native_cmd(mkfs_cmd, native_sysroot)
+
+        self.source_file = fs
+
+        return 0
+
+    def prepare_empty_partition_btrfs(self, cr_workdir, oe_builddir,
+                                      native_sysroot):
+        """
+        Prepare an empty btrfs partition.
+        """
+        fs = "%s/fs.%s" % (cr_workdir, self.fstype)
+
+        dd_cmd = "dd if=/dev/zero of=%s bs=1M seek=%d count=0" % \
+            (fs, self.size)
+        rc, out = exec_cmd(dd_cmd)
+
+        mkfs_cmd = "mkfs.%s -b %d %s" % (self.fstype, self.size * 1024, rootfs)
+        rc, out = exec_native_cmd(mkfs_cmd, native_sysroot)
+
+        mkfs_cmd = "mkfs.%s -F %s %s" % (self.fstype, extra_imagecmd, fs)
+        rc, out = exec_native_cmd(mkfs_cmd, native_sysroot)
+
+        self.source_file = fs
+
+        return 0
+
+    def prepare_swap_partition(self, cr_workdir, oe_builddir, native_sysroot):
+        """
+        Prepare a swap partition.
+        """
+        fs = "%s/fs.%s" % (cr_workdir, self.fstype)
+
+        dd_cmd = "dd if=/dev/zero of=%s bs=1M seek=%d count=0" % \
+            (fs, self.size)
+        rc, out = exec_cmd(dd_cmd)
+
+        import uuid
+        label_str = ""
+        if self.label:
+            label_str = "-L %s" % self.label
+        mkswap_cmd = "mkswap %s -U %s %s" % (label_str, str(uuid.uuid1()), fs)
+        rc, out = exec_native_cmd(mkswap_cmd, native_sysroot)
+
+        self.source_file = fs
+
+        return 0
+
+class Wic_Partition(Mic_Partition):
+    removedKeywords = Mic_Partition.removedKeywords
+    removedAttrs = Mic_Partition.removedAttrs
 
     def _getParser(self):
-        op = FC4_Partition._getParser(self)
-        # The alignment value is given in kBytes. e.g., value 8 means that
-        # the partition is aligned to start from 8096 byte boundary.
-        op.add_option("--align", type="int", action="store", dest="align",
-                      default=None)
-        op.add_option("--extoptions", type="string", action="store", dest="extopts",
-                      default=None)
-        op.add_option("--part-type", type="string", action="store", dest="part_type",
-                      default=None)
+        op = Mic_Partition._getParser(self)
+        # use specified source file to fill the partition
+        # and calculate partition size
+        op.add_option("--source", type="string", action="store",
+                      dest="source", default=None)
         return op
diff --git a/scripts/lib/mic/plugin.py b/scripts/lib/mic/plugin.py
index 18c93ad..7c296e9 100644
--- a/scripts/lib/mic/plugin.py
+++ b/scripts/lib/mic/plugin.py
@@ -40,7 +40,11 @@ class PluginMgr(object):
         return cls._instance
 
     def __init__(self):
-        self.plugin_dir = configmgr.common['plugin_dir']
+        mic_path = os.path.dirname(__file__)
+        eos = mic_path.find('scripts') + len('scripts')
+        scripts_path = mic_path[:eos]
+
+        self.plugin_dir = scripts_path + "/lib/mic/plugins"
 
     def append_dirs(self, dirs):
         for path in dirs:
diff --git a/scripts/lib/mic/plugins/imager/direct_plugin.py b/scripts/lib/mic/plugins/imager/direct_plugin.py
new file mode 100644
index 0000000..53381e5
--- /dev/null
+++ b/scripts/lib/mic/plugins/imager/direct_plugin.py
@@ -0,0 +1,92 @@
+# ex:ts=4:sw=4:sts=4:et
+# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
+#
+# Copyright (c) 2013, Intel Corporation.
+# All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+# DESCRIPTION
+# This implements the 'direct' imager plugin class for 'wic', based
+# loosely on the raw imager plugin from 'mic'
+#
+# AUTHORS
+# Tom Zanussi <tom.zanussi (at] linux.intel.com>
+#
+
+import os
+import shutil
+import re
+import tempfile
+
+from mic import chroot, msger, rt_util
+from mic.utils import misc, fs_related, errors, runner, cmdln
+from mic.conf import configmgr
+from mic.plugin import pluginmgr
+from mic.utils.partitionedfs import PartitionedMount
+
+import mic.imager.direct as direct
+from mic.pluginbase import ImagerPlugin
+
+class DirectPlugin(ImagerPlugin):
+    name = 'direct'
+
+    @classmethod
+    def do_create(self, subcmd, opts, *args):
+        """
+        Create direct image, called from creator as 'direct' cmd
+        """
+        if len(args) != 9:
+            raise errors.Usage("Extra arguments given")
+
+        staging_data_dir = args[0]
+        hdddir = args[1]
+        native_sysroot = args[2]
+        kernel_dir = args[3]
+        bootimg_dir = args[4]
+        rootfs_dir = args[5]
+
+        creatoropts = configmgr.create
+        ksconf = args[6]
+
+        image_output_dir = args[7]
+        oe_builddir = args[8]
+
+        configmgr._ksconf = ksconf
+
+        creator = direct.DirectImageCreator(oe_builddir,
+                                            image_output_dir,
+                                            rootfs_dir,
+                                            bootimg_dir,
+                                            kernel_dir,
+                                            native_sysroot,
+                                            hdddir,
+                                            staging_data_dir,
+                                            creatoropts,
+                                            None,
+                                            None,
+                                            None)
+
+        try:
+            creator.mount(None, creatoropts["cachedir"])
+            creator.install()
+            creator.configure(creatoropts["repomd"])
+            creator.print_outimage_info()
+
+        except errors.CreatorError:
+            raise
+        finally:
+            creator.cleanup()
+
+        return 0
diff --git a/scripts/lib/mic/utils/fs_related.py b/scripts/lib/mic/utils/fs_related.py
index b9b9a97..6161735 100644
--- a/scripts/lib/mic/utils/fs_related.py
+++ b/scripts/lib/mic/utils/fs_related.py
@@ -29,7 +29,7 @@ import uuid
 from mic import msger
 from mic.utils import runner
 from mic.utils.errors import *
-
+from mic.utils.oe.misc import *
 
 def find_binary_inchroot(binary, chroot):
     paths = ["/usr/sbin",
@@ -280,6 +280,34 @@ class RawDisk(Disk):
     def exists(self):
         return True
 
+
+class DiskImage(Disk):
+    """
+    A Disk backed by a file.
+    """
+    def __init__(self, image_file, size):
+        Disk.__init__(self, size)
+        self.image_file = image_file
+
+    def exists(self):
+        return os.path.exists(self.image_file)
+
+    def create(self):
+        if self.device is not None:
+            return
+
+        blocks = self.size / 1024
+        if self.size - blocks * 1024:
+            blocks += 1
+
+        # create disk image
+        dd_cmd = "dd if=/dev/zero of=%s bs=1024 seek=%d count=1" % \
+            (self.image_file, blocks)
+        rc, out = exec_cmd(dd_cmd)
+
+        self.device = self.image_file
+
+
 class LoopbackDisk(Disk):
     """A Disk backed by a file via the loop module."""
     def __init__(self, lofile, size):
diff --git a/scripts/lib/mic/utils/misc.py b/scripts/lib/mic/utils/misc.py
index 6302434..67ddef2 100644
--- a/scripts/lib/mic/utils/misc.py
+++ b/scripts/lib/mic/utils/misc.py
@@ -512,8 +512,8 @@ def uncompress_squashfs(squashfsimg, outdir):
     if (rc != 0):
         raise SquashfsError("Failed to uncompress %s." % squashfsimg)
 
-def mkdtemp(dir = "/var/tmp", prefix = "mic-tmp-"):
-    """ FIXME: use the dir in mic.conf instead """
+def mkdtemp(dir = "/var/tmp", prefix = "wic-tmp-"):
+    """ FIXME: use the dir in wic.conf instead """
 
     makedirs(dir)
     return tempfile.mkdtemp(dir = dir, prefix = prefix)
diff --git a/scripts/lib/mic/utils/oe/__init__.py b/scripts/lib/mic/utils/oe/__init__.py
new file mode 100644
index 0000000..d10e802
--- /dev/null
+++ b/scripts/lib/mic/utils/oe/__init__.py
@@ -0,0 +1,22 @@
+#
+# OpenEmbedded mic utils library
+#
+# Copyright (c) 2013, Intel Corporation.
+# All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+# AUTHORS
+# Tom Zanussi <tom.zanussi (at] linux.intel.com>
+#
diff --git a/scripts/lib/mic/utils/oe/misc.py b/scripts/lib/mic/utils/oe/misc.py
new file mode 100644
index 0000000..9edaa23
--- /dev/null
+++ b/scripts/lib/mic/utils/oe/misc.py
@@ -0,0 +1,108 @@
+# ex:ts=4:sw=4:sts=4:et
+# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
+#
+# Copyright (c) 2013, Intel Corporation.
+# All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+# DESCRIPTION
+# This module provides a place to collect various mic-related utils
+# for the OpenEmbedded Image Tools.
+#
+# AUTHORS
+# Tom Zanussi <tom.zanussi (at] linux.intel.com>
+#
+
+from mic import msger
+from mic.utils import runner
+
+def exec_cmd(cmd_and_args, as_shell = False, catch = 3):
+    """
+    Execute command, catching stderr, stdout
+
+    Need to execute as_shell if the command uses wildcards
+    """
+    msger.debug("exec_cmd: %s" % cmd_and_args)
+    args = cmd_and_args.split()
+    msger.debug(args)
+
+    if (as_shell):
+        rc, out = runner.runtool(cmd_and_args, catch)
+    else:
+        rc, out = runner.runtool(args, catch)
+    out = out.strip()
+    msger.debug("exec_cmd: output for %s (rc = %d): %s" % \
+                    (cmd_and_args, rc, out))
+
+    if rc != 0:
+        # We don't throw exception when return code is not 0, because
+        # parted always fails to reload part table with loop devices. This
+        # prevents us from distinguishing real errors based on return
+        # code.
+        msger.debug("WARNING: %s returned '%s' instead of 0" % (args[0], rc))
+
+    return (rc, out)
+
+
+def exec_cmd_quiet(cmd_and_args, as_shell = False):
+    """
+    Execute command, catching nothing in the output
+
+    Need to execute as_shell if the command uses wildcards
+    """
+    return exec_cmd(cmd_and_args, as_shell, 0)
+
+
+def exec_native_cmd(cmd_and_args, native_sysroot, catch = 3):
+    """
+    Execute native command, catching stderr, stdout
+
+    Need to execute as_shell if the command uses wildcards
+
+    Always need to execute native commands as_shell
+    """
+    native_paths = \
+        "export PATH=%s/sbin:PATH=%s/usr/sbin:PATH=%s/usr/bin:$PATH" % \
+        (native_sysroot, native_sysroot, native_sysroot)
+    native_cmd_and_args = "%s;%s" % (native_paths, cmd_and_args)
+    msger.debug("exec_native_cmd: %s" % cmd_and_args)
+
+    args = cmd_and_args.split()
+    msger.debug(args)
+
+    return exec_cmd(native_cmd_and_args, True, catch)
+
+
+def exec_native_cmd_quiet(cmd_and_args, native_sysroot):
+    """
+    Execute native command, catching nothing in the output
+
+    Need to execute as_shell if the command uses wildcards
+
+    Always need to execute native commands as_shell
+    """
+    return exec_native_cmd(cmd_and_args, native_sysroot, 0)
+
+
+# kickstart doesn't support variable substution in commands, so this
+# is our current simplistic scheme for supporting that
+
+wks_vars = dict()
+
+def get_wks_var(key):
+    return wks_vars[key]
+
+def add_wks_var(key, val):
+    wks_vars[key] = val
diff --git a/scripts/lib/mic/utils/partitionedfs.py b/scripts/lib/mic/utils/partitionedfs.py
index 0475844..e8cded2 100644
--- a/scripts/lib/mic/utils/partitionedfs.py
+++ b/scripts/lib/mic/utils/partitionedfs.py
@@ -25,6 +25,7 @@ from mic.utils import runner
 from mic.utils.errors import MountError
 from mic.utils.fs_related import *
 from mic.utils.gpt_parser import GptParser
+from mic.utils.oe.misc import *
 
 # Overhead of the MBR partitioning scheme (just one sector)
 MBR_OVERHEAD = 1
@@ -93,7 +94,7 @@ class PartitionedMount(Mount):
         self.partitions.append(part)
         self.__add_disk(part['disk_name'])
 
-    def add_partition(self, size, disk_name, mountpoint, fstype = None,
+    def add_partition(self, size, disk_name, mountpoint, source_file = None, fstype = None,
                       label=None, fsopts = None, boot = False, align = None,
                       part_type = None):
         """ Add the next partition. Prtitions have to be added in the
@@ -141,6 +142,7 @@ class PartitionedMount(Mount):
             part = { 'ks_pnum' : ks_pnum, # Partition number in the KS file
                      'size': size, # In sectors
                      'mountpoint': mountpoint, # Mount relative to chroot
+                     'source_file': source_file, # partition contents
                      'fstype': fstype, # Filesystem type
                      'fsopts': fsopts, # Filesystem mount options
                      'label': label, # Partition label
@@ -723,67 +725,51 @@ class PartitionedMount(Mount):
 
         self.snapshot_created = True
 
+    def __install_partition(self, num, source_file, start, size):
+        """
+        Install source_file contents into a partition.
+        """
+        if not source_file: # nothing to install
+            return
+
+        # Start is included in the size so need to substract one from the end.
+        end = start + size - 1
+        msger.debug("Installed %s in partition %d, sectors %d-%d, size %d sectors" % (source_file, num, start, end, size))
+
+        dd_cmd = "dd if=%s of=%s bs=%d seek=%d count=%d conv=notrunc" % \
+            (source_file, self.image_file, self.sector_size, start, size)
+        rc, out = exec_cmd(dd_cmd)
+
+
+    def install(self, image_file):
+        msger.debug("Installing partitions")
+
+        self.image_file = image_file
+
+        for p in self.partitions:
+            d = self.disks[p['disk_name']]
+            if d['ptable_format'] == "msdos" and p['num'] == 5:
+                # The last sector of the 3rd partition was reserved for the EBR
+                # of the first _logical_ partition. This is why the extended
+                # partition should start one sector before the first logical
+                # partition.
+                self.__install_partition(p['num'], p['source_file'],
+                                         p['start'] - 1,
+                                         d['offset'] - p['start'])
+
+            self.__install_partition(p['num'], p['source_file'],
+                                     p['start'], p['size'])
+
     def mount(self):
         for dev in self.disks.keys():
             d = self.disks[dev]
             d['disk'].create()
 
         self.__format_disks()
-        self.__map_partitions()
-        self.__calculate_mountorder()
 
-        for mp in self.mountOrder:
-            p = None
-            for p1 in self.partitions:
-                if p1['mountpoint'] == mp:
-                    p = p1
-                    break
-
-            if not p['label']:
-                if p['mountpoint'] == "/":
-                    p['label'] = 'platform'
-                else:
-                    p['label'] = mp.split('/')[-1]
-
-            if mp == 'swap':
-                import uuid
-                p['uuid'] = str(uuid.uuid1())
-                runner.show([self.mkswap,
-                             '-L', p['label'],
-                             '-U', p['uuid'],
-                             p['device']])
-                continue
+        self.__calculate_mountorder()
 
-            rmmountdir = False
-            if p['mountpoint'] == "/":
-                rmmountdir = True
-            if p['fstype'] == "vfat" or p['fstype'] == "msdos":
-                myDiskMount = VfatDiskMount
-            elif p['fstype'] in ("ext2", "ext3", "ext4"):
-                myDiskMount = ExtDiskMount
-            elif p['fstype'] == "btrfs":
-                myDiskMount = BtrfsDiskMount
-            else:
-                raise MountError("Fail to support file system " + p['fstype'])
-
-            if p['fstype'] == "btrfs" and not p['fsopts']:
-                p['fsopts'] = "subvolid=0"
-
-            pdisk = myDiskMount(RawDisk(p['size'] * self.sector_size, p['device']),
-                                 self.mountdir + p['mountpoint'],
-                                 p['fstype'],
-                                 4096,
-                                 p['label'],
-                                 rmmountdir,
-                                 self.skipformat,
-                                 fsopts = p['fsopts'])
-            pdisk.mount(pdisk.fsopts)
-            if p['fstype'] == "btrfs" and p['mountpoint'] == "/":
-                if not self.skipformat:
-                    self.__create_subvolumes(p, pdisk)
-                self.__mount_subvolumes(p, pdisk)
-            p['mount'] = pdisk
-            p['uuid'] = pdisk.uuid
+        return
 
     def resparse(self, size = None):
         # Can't re-sparse a disk image - too hard
diff --git a/scripts/yocto-image.conf b/scripts/yocto-image.conf
deleted file mode 100644
index 3d41eff..0000000
--- a/scripts/yocto-image.conf
+++ /dev/null
@@ -1,35 +0,0 @@
-[common]
-; general settings
-distro_name = Tizen
-
-plugin_dir = @PREFIX@/lib/mic/plugins
-
-[create]
-; settings for create subcommand
-tmpdir= /var/tmp/mic
-cachedir= /var/tmp/mic/cache
-outdir= ./mic-output
-runtime=bootstrap
-
-pkgmgr = auto
-
-# to set global proxy for repos
-#proxy = http://proxy.yourcompany.com:8080/
-#no_proxy = localhost,127.0.0.0/8,.yourcompany.com
-
-# prefix will be added in front of generated files
-#name_prefix = output
-
-# to skip all ssl verification for repos
-#ssl_verify = no
-
-[convert]
-; settings for convert subcommand
-
-[chroot]
-; settings for chroot subcommand
-
-[bootstrap]
-rootdir=/var/tmp/mic-bootstrap
-packages=mic-bootstrap-x86-arm
-
-- 
1.7.11.4

Re: [PATCH 0/3] 'wic'- OpenEmbedded Image Creator

[David Nystrom]

[-- Attachment #1: Type: text/plain, Size: 35691 bytes --]

On Sep 27, 2013 4:17 AM, "Tom Zanussi" <tom.zanussi@linux.intel.com> wrote:
>
> This patchset implements a new command named 'wic' (for OpenEmbedded
> Image Creator).  Please see [YOCTO #3847] for extensive background on
> what's implemented here.

Nice !

Would it be possible to add the MIC as a recipe for native and nativesdk,
with wic patches applied(easier to maintain), and could also enable wic to
use a package repo when built for nativesdk.

Br,
David

> This is a completely independent standalone utility that initially
> provides easier-to-use and more flexible replacements for a couple
> bits of existing functionality in oe-core: directdisk.bbclass and
> mkefidisk.sh.  The more interesting aspect of the current patchset
> though is that the replaced (this patchset doesn't actually replace
> those scripts, however) scripts are implemented by a general-purpose
> partitioning 'language' based on Redhat kickstart syntax (with the
> underlying code borrowed from Tizen mic, which in turn was borrowed
> from Meego mic, in turn borrowed from Fedora livecd, etc.).
>
> Though this patchset only uses the kickstart syntax related to
> partitioning and bootloaders and only for creating images, because the
> code is based on the mic/pykickstart code, future deployment efforts
> such as those partially described by [YOCTO #4106], but also others
> including package selection (from e.g. binary feeds) and deployment
> configuration of users/network/services, etc, could be implemented
> under this framework, considering that all of those are implemented
> in some form by the base system.
>
> The current patchset has successfully been used to generate and boot
> images from core-image-minimal build artifacts using the 'mkefidisk'
> kickstart file (in scripts/lib/image/canned-wks) on minnow and the
> 'directdisk' kickstart file using both core-image-minimal and
> core-image-sato artifacts for crownbay.
>
> The 'wic' command generates partitioned images from existing
> OpenEmbedded build artifacts.  Image generation is driven by
> partitioning commands contained in an 'Openembedded kickstart' (.wks)
> file specified either directly on the command-line or as one of a
> selection of canned .wks files (see 'wic list images').  When applied
> to a given set of build artifacts, the result is an image or set of
> images that can be directly written onto media and used on a
> particular system.
>
> It can be used in 'raw' mode, where artifacts are explicitly specified
> via command-line arguments (see example below), or it can be used in a
> more easily usable 'cooked' mode which uses the current MACHINE
> setting and a specified image name to automatically locate the
> artifacts used to create the image.
>
> OE kickstart files (.wks) can of course be specified directly on the
> command-line, but the user can also choose from a set of 'canned' .wks
> files available via the 'wic list images' command (example below).
>
> In any case, the prerequisite for generating any image is to have the
> build artifacts already available.  The below examples assume the user
> has already build a 'core-image-minimal' for a specific machine
> (future versions won't require this redundant step, but for now that's
> typically how build artifacts get generated).
>
> The other prerequisite is to source the build environment:
>
> $ source oe-init-build-env
>
> To start out with, we'll generate an image from one of the canned .wks
> files.  The following generates a list of availailable images:
>
> $ wic list images
>   mkefidisk             Create an EFI disk image
>   directdisk            Create a 'pcbios' direct disk image
>
> You can get more information about any of the available images by typing
> 'wic list xxx help', where 'xxx' is one of the image names:
>
> $ wic list mkefidisk help
>
> Creates a partitioned EFI disk image that the user
> can directly dd to boot media.
>
> At any time, you can get help on the 'wic' command or any subcommand
> (currently 'list' and 'create').  For instance, to get the description
> of 'wic create' command and its parameters:
>
> $ wic create
>
> Usage:
>
>  Create a new OpenEmbedded image
>
>  usage: wic create <wks file or image name> [-o <DIRNAME> | --outdir
<DIRNAME>]
>             [-i <JSON PROPERTY FILE> | --infile <JSON PROPERTY_FILE>]
>             [-e | --image-name] [-r, --rootfs-dir] [-b, --bootimg-dir]
>             [-k, --kernel-dir] [-n, --native-sysroot] [-s,
--skip-build-check]
>
>  This command creates an OpenEmbedded image based on the 'OE kickstart
>  commands' found in the <wks file>.
>
>  The -o option can be used to place the image in a directory with a
>  different name and location.
>
>  See 'wic help create' for more detailed instructions.
>
> ...
>
> As mentioned in the command, you can get even more detailed information
> by adding 'help' to the above:
>
> $ wic help create
>
> NAME
>     wic create - Create a new OpenEmbedded image
>
> SYNOPSIS
>     wic create <wks file or image name> [-o <DIRNAME> | --outdir
> <DIRNAME>]
>         [-i <JSON PROPERTY FILE> | --infile <JSON PROPERTY_FILE>]
>         [-e | --image-name] [-r, --rootfs-dir] [-b, --bootimg-dir]
>         [-k, --kernel-dir] [-n, --native-sysroot] [-s,
> --skip-build-check]
>
> DESCRIPTION
>     This command creates an OpenEmbedded image based on the 'OE
>     kickstart commands' found in the <wks file>.
>
>     In order to do this, wic needs to know the locations of the
>     various build artifacts required to build the image.
>
>     Users can explicitly specify the build artifact locations using
>     the -r, -b, -k, and -n options.  See below for details on where
>     the corresponding artifacts are typically found in a normal
>     OpenEmbedded build.
>
>     Alternatively, users can use the -e option to have 'mic' determine
>     those locations for a given image.  If the -e option is used, the
>     user needs to have set the appropriate MACHINE variable in
>     local.conf, and have sourced the build environment.
>
>     The -e option is used to specify the name of the image to use the
>     artifacts from e.g. core-image-sato.
>
>     The -r option is used to specify the path to the /rootfs dir to
>     use as the .wks rootfs source.
>
>     The -b option is used to specify the path to the dir containing
>     the boot artifacts (e.g. /EFI or /syslinux dirs) to use as the
>     .wks bootimg source.
>
>     The -k option is used to specify the path to the dir containing
>     the kernel to use in the .wks bootimg.
>
>     The -n option is used to specify the path to the native sysroot
>     containing the tools to use to build the image.
>
>     The -s option is used to skip the build check.  The build check is
>     a simple sanity check used to determine whether the user has
>     sourced the build environment so that the -e option can operate
>     correctly.  If the user has specified the build artifact locations
>     explicitly, 'wic' assumes the user knows what he or she is doing
>     and skips the build check.
>
>     When 'wic -e' is used, the locations for the build artifacts
>     values are determined by 'wic -e' from the output of the 'bitbake
>     -e' command given an image name e.g. 'core-image-minimal' and a
>     given machine set in local.conf.  In that case, the image is
>     created as if the following 'bitbake -e' variables were used:
>
>     -r:        IMAGE_ROOTFS
>     -k:        STAGING_KERNEL_DIR
>     -n:        STAGING_DIR_NATIVE
>     -b:        HDDDIR and STAGING_DATA_DIR (handlers decide which to
> use)
>
>     If 'wic -e' is not used, the user needs to select the appropriate
>     value for -b (as well as -r, -k, and -n).
>
>     The -o option can be used to place the image in a directory with a
>     different name and location.
>
>     As an alternative to the wks file, the image-specific properties
>     that define the values that will be used to generate a particular
>     image can be specified on the command-line using the -i option and
>     supplying a JSON object consisting of the set of name:value pairs
>     needed by image creation.
>
>     The set of properties available for a given image type can be
>     listed using the 'wic list' command.
>
> So, the easiest way to create an image is to use the -e option with a
> canned .wks file.  To use the -e option, you need to specify the image
> used to generate the artifacts and you actually need to have the MACHINE
> used to build them specified in your local.conf (these requirements
> aren't necessary if you aren't using the -e options.  Below, we generate
> a directdisk image, pointing the process at the core-image-minimal
> artifacts for the current MACHINE:
>
> $ wic create directdisk -e core-image-minimal
>
> Checking basic build environment...
> Done.
>
> Creating image(s)...
>
> Info: The new image(s) can be found here:
>   /var/tmp/wic/build/directdisk-201309252350-sda.direct
>
> The following build artifacts were used to create the image(s):
>
> ROOTFS_DIR:
 /home/trz/yocto/yocto-image/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs
> BOOTIMG_DIR:
/home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/share
> KERNEL_DIR:
 /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel
> NATIVE_SYSROOT:
 /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel
>
> The image(s) were created using OE kickstart file:
>   /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/directdisk.wks
>
> The output shows the name and location of the image created, and so that
> you know exactly what was used to generate the image, each of the
> artifacts and the kickstart file used.
>
> Similarly, I can create a 'mkefidisk' image in the same way (notice that
> I'm using a different machine - because I'm using the -e option, I
> needed to change the MACHINE in my local.conf):
>
> $ wic create mkefidisk -e core-image-minimal
> Checking basic build environment...
> Done.
>
> Creating image(s)...
>
> Info: The new image(s) can be found here:
>   /var/tmp/wic/build/mkefidisk-201309260027-sda.direct
>
> The following build artifacts were used to create the image(s):
>
> ROOTFS_DIR:
 /home/trz/yocto/yocto-image/build/tmp/work/minnow-poky-linux/core-image-minimal/1.0-r0/rootfs
> BOOTIMG_DIR:
/home/trz/yocto/yocto-image/build/tmp/work/minnow-poky-linux/core-image-minimal/1.0-r0/core-image-minimal-1.0/hddimg
> KERNEL_DIR:
 /home/trz/yocto/yocto-image/build/tmp/sysroots/minnow/usr/src/kernel
> NATIVE_SYSROOT:
 /home/trz/yocto/yocto-image/build/tmp/sysroots/minnow/usr/src/kernel
>
> The image(s) were created using OE kickstart file:
>   /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/mkefidisk.wks
>
> Here's an example that doesn't take the easy way out and manually
> specifies each build artifact, along with a non-canned .wks file, and
> also uses the -o option to have wic create the output somewhere other
> than the default /var/tmp/wic:
>
> $ wic create ~/test.wks -o /home/trz/testwic
> --rootfs-dir
/home/trz/yocto/yocto-image/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs
--bootimg-dir
/home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/share
--kernel-dir
/home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel
--native-sysroot /home/trz/yocto/yocto-image/build/tmp/sysroots/x86_64-linux
>
> Creating image(s)...
>
> Info: The new image(s) can be found here:
>   /home/trz/testwic/build/test-201309260032-sda.direct
>
> The following build artifacts were used to create the image(s):
>
> ROOTFS_DIR:
 /home/trz/yocto/yocto-image/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs
> BOOTIMG_DIR:
/home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/share
> KERNEL_DIR:
 /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel
> NATIVE_SYSROOT:
 /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel
>
> The image(s) were created using OE kickstart file:
>   /home/trz/test.wks
>
> Finally, here's an example of the actual partition language commands
> used to generate the mkefidisk image i.e. these are the contents of the
> mkefidisk.wks OE kickstart file:
>
> # short-description: Create an EFI disk image
> # long-description: Creates a partitioned EFI disk image that the user
> # can directly dd to boot media.
>
> part /boot --source bootimg --ondisk sda --fstype=efi --label msdos
--active --align 1024
>
> part / --source rootfs --ondisk sda --fstype=ext3 --label platform
--align 1024
>
> part swap --ondisk sda --size 44 --label swap1 --fstype=swap
>
> bootloader  --timeout=10  --append="rootwait rootfstype=ext3
console=ttyPCH0,115200 console=tty0 vmalloc=256MB
snd-hda-intel.enable_msi=0"
>
> The initial 'wic' implementation here supports only the basic
> kickstart partitioning commands: 'partition' (or 'part' for short) and
> 'bootloader'.
>
> Currently, only the options used in the 'canned' scripts (in
> scripts/lib/image/canned-wks) are supported.
>
> They're listed below and mostly follow the syntax and meaning of the
> standard kickstart options for those commands:
>
> http://fedoraproject.org/wiki/Anaconda/Kickstart#part_or_partition
> http://fedoraproject.org/wiki/Anaconda/Kickstart#bootloader
>
> For the 'partition' ('part') command, here are the supported options:
>
> --source
>         bootimg
>         rootfs
> --ondisk sda
> --size size (in mB)
> --fstype
>         msdos
>         efi
>         ext4
>         ext3
>         ext2
>         btrfs
>         swap
> --label label
> --active
> ---align (in kB)
>
> The --source option is a wic-specific option that can currently have
> one of two values, 'bootimg' or 'rootfs'.
>
> If '--source rootfs' is used, it tells the wic command to create a
> partition as large as needed to fill with the contents of /rootfs
> (specified by the -r 'wic' option) and to fill it with the contents of
> /rootfs.
>
> If '--source bootimg' is used, it tells the wic command to create a
> partition as large as needed to fill with the contents of the boot
> partition (specified by the -b 'wic' option).  Exactly what those
> contents are depend on the value of the --fstype option for that
> partition.  If '--fstype=efi' is specified, the boot artifacts
> contained in HDDDIR are used, and if '--fstype=msdos' is specified,
> the boot artifacts found in STAGING_DATADIR are used.
>
> The '--align' option is a mic-specific option that says to start a
> partition on an x kB boundary.
>
> For the 'bootloader' command, these are the supported options:
>
> --timeout
> --append
>
> Future updates will implement more options - using anything not
> explicitly supported can result in unpredicable results.
>
>
> The following changes since commit
b049d532f6e0ab9e458e486f81b00be47ee69acf:
>
>   testimage: Exclude BB_ORIGENV variable (2013-09-26 17:27:02 +0100)
>
> are available in the git repository at:
>
>   git://git.yoctoproject.org/poky-contrib.git tzanussi/wic-v1
>
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=tzanussi/wic-v1
>
> Tom Zanussi (3):
>   wic: Initial code for wic (OpenEmbedded Image Creator)
>   wic: Add mic w/pykickstart
>   wic: Add OpenEmbedded-specific implementation
>
>  scripts/lib/image/__init__.py                      |   22 +
>  scripts/lib/image/canned-wks/directdisk.wks        |   10 +
>  scripts/lib/image/canned-wks/mkefidisk.wks         |   11 +
>  scripts/lib/image/config/wic.conf                  |    7 +
>  scripts/lib/image/engine.py                        |  280 ++++
>  scripts/lib/image/help.py                          |  311 ++++
>  scripts/lib/mic/3rdparty/pykickstart/base.py       |  466 ++++++
>  .../mic/3rdparty/pykickstart/commands/__init__.py  |   26 +
>  .../3rdparty/pykickstart/commands/authconfig.py    |   40 +
>  .../mic/3rdparty/pykickstart/commands/autopart.py  |  119 ++
>  .../mic/3rdparty/pykickstart/commands/autostep.py  |   55 +
>  .../3rdparty/pykickstart/commands/bootloader.py    |  265 ++++
>  .../mic/3rdparty/pykickstart/commands/clearpart.py |   86 ++
>  .../mic/3rdparty/pykickstart/commands/device.py    |  125 ++
>  .../3rdparty/pykickstart/commands/deviceprobe.py   |   40 +
>  .../3rdparty/pykickstart/commands/displaymode.py   |   68 +
>  .../mic/3rdparty/pykickstart/commands/dmraid.py    |   91 ++
>  .../3rdparty/pykickstart/commands/driverdisk.py    |  184 +++
>  .../lib/mic/3rdparty/pykickstart/commands/fcoe.py  |  114 ++
>  .../mic/3rdparty/pykickstart/commands/firewall.py  |  193 +++
>  .../mic/3rdparty/pykickstart/commands/firstboot.py |   62 +
>  .../lib/mic/3rdparty/pykickstart/commands/group.py |   88 ++
>  .../3rdparty/pykickstart/commands/ignoredisk.py    |  139 ++
>  .../3rdparty/pykickstart/commands/interactive.py   |   58 +
>  .../lib/mic/3rdparty/pykickstart/commands/iscsi.py |  133 ++
>  .../mic/3rdparty/pykickstart/commands/iscsiname.py |   54 +
>  .../lib/mic/3rdparty/pykickstart/commands/key.py   |   64 +
>  .../mic/3rdparty/pykickstart/commands/keyboard.py  |   55 +
>  .../lib/mic/3rdparty/pykickstart/commands/lang.py  |   60 +
>  .../3rdparty/pykickstart/commands/langsupport.py   |   58 +
>  .../mic/3rdparty/pykickstart/commands/lilocheck.py |   54 +
>  .../mic/3rdparty/pykickstart/commands/logging.py   |   66 +
>  .../mic/3rdparty/pykickstart/commands/logvol.py    |  304 ++++
>  .../3rdparty/pykickstart/commands/mediacheck.py    |   53 +
>  .../mic/3rdparty/pykickstart/commands/method.py    |  186 +++
>  .../mic/3rdparty/pykickstart/commands/monitor.py   |  106 ++
>  .../lib/mic/3rdparty/pykickstart/commands/mouse.py |   70 +
>  .../mic/3rdparty/pykickstart/commands/multipath.py |  111 ++
>  .../mic/3rdparty/pykickstart/commands/network.py   |  363 +++++
>  .../mic/3rdparty/pykickstart/commands/partition.py |  353 +++++
>  .../lib/mic/3rdparty/pykickstart/commands/raid.py  |  365 +++++
>  .../mic/3rdparty/pykickstart/commands/reboot.py    |   79 +
>  .../lib/mic/3rdparty/pykickstart/commands/repo.py  |  249 +++
>  .../mic/3rdparty/pykickstart/commands/rescue.py    |   68 +
>  .../mic/3rdparty/pykickstart/commands/rootpw.py    |   93 ++
>  .../mic/3rdparty/pykickstart/commands/selinux.py   |   64 +
>  .../mic/3rdparty/pykickstart/commands/services.py  |   71 +
>  .../lib/mic/3rdparty/pykickstart/commands/skipx.py |   54 +
>  .../lib/mic/3rdparty/pykickstart/commands/sshpw.py |  105 ++
>  .../mic/3rdparty/pykickstart/commands/timezone.py  |   86 ++
>  .../mic/3rdparty/pykickstart/commands/updates.py   |   60 +
>  .../mic/3rdparty/pykickstart/commands/upgrade.py   |  106 ++
>  .../lib/mic/3rdparty/pykickstart/commands/user.py  |  173 +++
>  .../lib/mic/3rdparty/pykickstart/commands/vnc.py   |  114 ++
>  .../mic/3rdparty/pykickstart/commands/volgroup.py  |  102 ++
>  .../mic/3rdparty/pykickstart/commands/xconfig.py   |  184 +++
>  .../mic/3rdparty/pykickstart/commands/zerombr.py   |   69 +
>  .../lib/mic/3rdparty/pykickstart/commands/zfcp.py  |  134 ++
>  scripts/lib/mic/3rdparty/pykickstart/constants.py  |   57 +
>  scripts/lib/mic/3rdparty/pykickstart/errors.py     |  103 ++
>  .../mic/3rdparty/pykickstart/handlers/control.py   | 1307
++++++++++++++++
>  .../lib/mic/3rdparty/pykickstart/handlers/f10.py   |   24 +
>  .../lib/mic/3rdparty/pykickstart/handlers/f11.py   |   24 +
>  .../lib/mic/3rdparty/pykickstart/handlers/f12.py   |   24 +
>  .../lib/mic/3rdparty/pykickstart/handlers/f13.py   |   24 +
>  .../lib/mic/3rdparty/pykickstart/handlers/f14.py   |   24 +
>  .../lib/mic/3rdparty/pykickstart/handlers/f15.py   |   24 +
>  .../lib/mic/3rdparty/pykickstart/handlers/f16.py   |   24 +
>  .../lib/mic/3rdparty/pykickstart/handlers/f7.py    |   24 +
>  .../lib/mic/3rdparty/pykickstart/handlers/f8.py    |   24 +
>  .../lib/mic/3rdparty/pykickstart/handlers/f9.py    |   24 +
>  .../lib/mic/3rdparty/pykickstart/handlers/fc3.py   |   24 +
>  .../lib/mic/3rdparty/pykickstart/handlers/fc4.py   |   24 +
>  .../lib/mic/3rdparty/pykickstart/handlers/fc5.py   |   24 +
>  .../lib/mic/3rdparty/pykickstart/handlers/fc6.py   |   24 +
>  .../lib/mic/3rdparty/pykickstart/handlers/rhel3.py |   24 +
>  .../lib/mic/3rdparty/pykickstart/handlers/rhel4.py |   24 +
>  .../lib/mic/3rdparty/pykickstart/handlers/rhel5.py |   24 +
>  .../lib/mic/3rdparty/pykickstart/handlers/rhel6.py |   24 +
>  scripts/lib/mic/3rdparty/pykickstart/ko.py         |   37 +
>  scripts/lib/mic/3rdparty/pykickstart/options.py    |  204 +++
>  scripts/lib/mic/3rdparty/pykickstart/parser.py     |  702 +++++++++
>  scripts/lib/mic/3rdparty/pykickstart/sections.py   |  244 +++
>  .../3rdparty/pykickstart/urlgrabber/__init__.py    |   53 +
>  .../3rdparty/pykickstart/urlgrabber/byterange.py   |  463 ++++++
>  .../mic/3rdparty/pykickstart/urlgrabber/grabber.py | 1477
++++++++++++++++++
>  .../3rdparty/pykickstart/urlgrabber/keepalive.py   |  617 ++++++++
>  .../mic/3rdparty/pykickstart/urlgrabber/mirror.py  |  458 ++++++
>  .../3rdparty/pykickstart/urlgrabber/progress.py    |  530 +++++++
>  .../3rdparty/pykickstart/urlgrabber/sslfactory.py  |   90 ++
>  scripts/lib/mic/3rdparty/pykickstart/version.py    |  197 +++
>  scripts/lib/mic/__init__.py                        |    4 +
>  scripts/lib/mic/__version__.py                     |    1 +
>  scripts/lib/mic/bootstrap.py                       |  279 ++++
>  scripts/lib/mic/chroot.py                          |  343 +++++
>  scripts/lib/mic/conf.py                            |  201 +++
>  scripts/lib/mic/creator.py                         |  351 +++++
>  scripts/lib/mic/imager/baseimager.py               | 1265
++++++++++++++++
>  scripts/lib/mic/imager/direct.py                   |  472 ++++++
>  scripts/lib/mic/imager/fs.py                       |   99 ++
>  scripts/lib/mic/imager/livecd.py                   |  750 +++++++++
>  scripts/lib/mic/imager/liveusb.py                  |  308 ++++
>  scripts/lib/mic/imager/loop.py                     |  418 ++++++
>  scripts/lib/mic/imager/raw.py                      |  501 +++++++
>  scripts/lib/mic/kickstart/__init__.py              |  892 +++++++++++
>  .../lib/mic/kickstart/custom_commands/__init__.py  |   17 +
>  .../lib/mic/kickstart/custom_commands/desktop.py   |   95 ++
>  .../mic/kickstart/custom_commands/installerfw.py   |   63 +
>  .../lib/mic/kickstart/custom_commands/micboot.py   |   49 +
>  .../mic/kickstart/custom_commands/micpartition.py  |   57 +
>  .../lib/mic/kickstart/custom_commands/micrepo.py   |  127 ++
>  .../lib/mic/kickstart/custom_commands/partition.py |  370 +++++
>  scripts/lib/mic/msger.py                           |  309 ++++
>  scripts/lib/mic/plugin.py                          |  102 ++
>  scripts/lib/mic/pluginbase.py                      |  101 ++
>  scripts/lib/mic/plugins/backend/yumpkgmgr.py       |  490 ++++++
>  scripts/lib/mic/plugins/backend/zypppkgmgr.py      |  973 ++++++++++++
>  scripts/lib/mic/plugins/hook/empty_hook.py         |    3 +
>  scripts/lib/mic/plugins/imager/direct_plugin.py    |   92 ++
>  scripts/lib/mic/plugins/imager/fs_plugin.py        |  143 ++
>  scripts/lib/mic/plugins/imager/livecd_plugin.py    |  255 ++++
>  scripts/lib/mic/plugins/imager/liveusb_plugin.py   |  260 ++++
>  scripts/lib/mic/plugins/imager/loop_plugin.py      |  255 ++++
>  scripts/lib/mic/plugins/imager/raw_plugin.py       |  275 ++++
>  scripts/lib/mic/rt_util.py                         |  223 +++
>  scripts/lib/mic/test                               |    1 +
>  scripts/lib/mic/utils/BmapCreate.py                |  298 ++++
>  scripts/lib/mic/utils/Fiemap.py                    |  252 ++++
>  scripts/lib/mic/utils/cmdln.py                     | 1586
++++++++++++++++++++
>  scripts/lib/mic/utils/errors.py                    |   71 +
>  scripts/lib/mic/utils/fs_related.py                | 1057 +++++++++++++
>  scripts/lib/mic/utils/gpt_parser.py                |  331 ++++
>  scripts/lib/mic/utils/grabber.py                   |   97 ++
>  scripts/lib/mic/utils/misc.py                      | 1067 +++++++++++++
>  scripts/lib/mic/utils/oe/__init__.py               |   22 +
>  scripts/lib/mic/utils/oe/misc.py                   |  108 ++
>  scripts/lib/mic/utils/partitionedfs.py             |  776 ++++++++++
>  scripts/lib/mic/utils/proxy.py                     |  183 +++
>  scripts/lib/mic/utils/rpmmisc.py                   |  600 ++++++++
>  scripts/lib/mic/utils/runner.py                    |  109 ++
>  scripts/wic                                        |  185 +++
>  141 files changed, 30983 insertions(+)
>  create mode 100644 scripts/lib/image/__init__.py
>  create mode 100644 scripts/lib/image/canned-wks/directdisk.wks
>  create mode 100644 scripts/lib/image/canned-wks/mkefidisk.wks
>  create mode 100644 scripts/lib/image/config/wic.conf
>  create mode 100644 scripts/lib/image/engine.py
>  create mode 100644 scripts/lib/image/help.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/__init__.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/base.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/__init__.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/authconfig.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/autopart.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/autostep.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/bootloader.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/clearpart.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/device.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/deviceprobe.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/displaymode.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/dmraid.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/driverdisk.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/fcoe.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/firewall.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/firstboot.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/group.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/ignoredisk.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/interactive.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/iscsi.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/iscsiname.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/key.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/keyboard.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/lang.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/langsupport.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/lilocheck.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/logging.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/logvol.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/mediacheck.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/method.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/monitor.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/mouse.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/multipath.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/network.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/partition.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/raid.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/reboot.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/repo.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/rescue.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/rootpw.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/selinux.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/services.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/skipx.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/sshpw.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/timezone.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/updates.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/upgrade.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/user.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/vnc.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/volgroup.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/xconfig.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/commands/zerombr.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/commands/zfcp.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/constants.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/errors.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/handlers/__init__.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/handlers/control.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f10.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f11.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f12.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f13.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f14.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f15.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f16.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f7.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f8.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/f9.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/fc3.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/fc4.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/fc5.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/fc6.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/rhel3.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/rhel4.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/rhel5.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/handlers/rhel6.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/ko.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/options.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/parser.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/sections.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/urlgrabber/__init__.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/urlgrabber/byterange.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/urlgrabber/grabber.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/urlgrabber/keepalive.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/urlgrabber/mirror.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/urlgrabber/progress.py
>  create mode 100644
scripts/lib/mic/3rdparty/pykickstart/urlgrabber/sslfactory.py
>  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/version.py
>  create mode 100644 scripts/lib/mic/__init__.py
>  create mode 100644 scripts/lib/mic/__version__.py
>  create mode 100644 scripts/lib/mic/bootstrap.py
>  create mode 100644 scripts/lib/mic/chroot.py
>  create mode 100644 scripts/lib/mic/conf.py
>  create mode 100644 scripts/lib/mic/creator.py
>  create mode 100644 scripts/lib/mic/imager/__init__.py
>  create mode 100644 scripts/lib/mic/imager/baseimager.py
>  create mode 100644 scripts/lib/mic/imager/direct.py
>  create mode 100644 scripts/lib/mic/imager/fs.py
>  create mode 100644 scripts/lib/mic/imager/livecd.py
>  create mode 100644 scripts/lib/mic/imager/liveusb.py
>  create mode 100644 scripts/lib/mic/imager/loop.py
>  create mode 100644 scripts/lib/mic/imager/raw.py
>  create mode 100644 scripts/lib/mic/kickstart/__init__.py
>  create mode 100644 scripts/lib/mic/kickstart/custom_commands/__init__.py
>  create mode 100644 scripts/lib/mic/kickstart/custom_commands/desktop.py
>  create mode 100644
scripts/lib/mic/kickstart/custom_commands/installerfw.py
>  create mode 100644 scripts/lib/mic/kickstart/custom_commands/micboot.py
>  create mode 100644
scripts/lib/mic/kickstart/custom_commands/micpartition.py
>  create mode 100644 scripts/lib/mic/kickstart/custom_commands/micrepo.py
>  create mode 100644 scripts/lib/mic/kickstart/custom_commands/partition.py
>  create mode 100644 scripts/lib/mic/msger.py
>  create mode 100644 scripts/lib/mic/plugin.py
>  create mode 100644 scripts/lib/mic/pluginbase.py
>  create mode 100644 scripts/lib/mic/plugins/backend/yumpkgmgr.py
>  create mode 100755 scripts/lib/mic/plugins/backend/zypppkgmgr.py
>  create mode 100644 scripts/lib/mic/plugins/hook/.py
>  create mode 100644 scripts/lib/mic/plugins/hook/empty_hook.py
>  create mode 100644 scripts/lib/mic/plugins/imager/direct_plugin.py
>  create mode 100644 scripts/lib/mic/plugins/imager/fs_plugin.py
>  create mode 100644 scripts/lib/mic/plugins/imager/livecd_plugin.py
>  create mode 100644 scripts/lib/mic/plugins/imager/liveusb_plugin.py
>  create mode 100644 scripts/lib/mic/plugins/imager/loop_plugin.py
>  create mode 100644 scripts/lib/mic/plugins/imager/raw_plugin.py
>  create mode 100644 scripts/lib/mic/rt_util.py
>  create mode 100644 scripts/lib/mic/test
>  create mode 100644 scripts/lib/mic/utils/BmapCreate.py
>  create mode 100644 scripts/lib/mic/utils/Fiemap.py
>  create mode 100644 scripts/lib/mic/utils/__init__.py
>  create mode 100644 scripts/lib/mic/utils/cmdln.py
>  create mode 100644 scripts/lib/mic/utils/errors.py
>  create mode 100644 scripts/lib/mic/utils/fs_related.py
>  create mode 100644 scripts/lib/mic/utils/gpt_parser.py
>  create mode 100644 scripts/lib/mic/utils/grabber.py
>  create mode 100644 scripts/lib/mic/utils/misc.py
>  create mode 100644 scripts/lib/mic/utils/oe/__init__.py
>  create mode 100644 scripts/lib/mic/utils/oe/misc.py
>  create mode 100644 scripts/lib/mic/utils/partitionedfs.py
>  create mode 100644 scripts/lib/mic/utils/proxy.py
>  create mode 100644 scripts/lib/mic/utils/rpmmisc.py
>  create mode 100644 scripts/lib/mic/utils/runner.py
>  create mode 100755 scripts/wic
>
> --
> 1.7.11.4
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

[-- Attachment #2: Type: text/html, Size: 41308 bytes --]

Re: [PATCH 0/3] 'wic'- OpenEmbedded Image Creator

[Tom Zanussi]

On Fri, 2013-09-27 at 08:48 +0200, David Nystrom wrote:
> 
> On Sep 27, 2013 4:17 AM, "Tom Zanussi" <tom.zanussi@linux.intel.com>
> wrote:
> >
> > This patchset implements a new command named 'wic' (for OpenEmbedded
> > Image Creator).  Please see [YOCTO #3847] for extensive background
> on
> > what's implemented here.
> 
> Nice !
> 
> Would it be possible to add the MIC as a recipe for native and
> nativesdk, with wic patches applied(easier to maintain), and could
> also enable wic to use a package repo when built for nativesdk.
> 

Yes, I think something like that would make sense - for 1.5, the design
goal was to make it a completely standalone thing, with any integration
work and other enhancements coming in 1.6...

Tom

> Br,
> David
> 
> 
> 
> > This is a completely independent standalone utility that initially
> > provides easier-to-use and more flexible replacements for a couple
> > bits of existing functionality in oe-core: directdisk.bbclass and
> > mkefidisk.sh.  The more interesting aspect of the current patchset
> > though is that the replaced (this patchset doesn't actually replace
> > those scripts, however) scripts are implemented by a general-purpose
> > partitioning 'language' based on Redhat kickstart syntax (with the
> > underlying code borrowed from Tizen mic, which in turn was borrowed
> > from Meego mic, in turn borrowed from Fedora livecd, etc.).
> >
> > Though this patchset only uses the kickstart syntax related to
> > partitioning and bootloaders and only for creating images, because
> the
> > code is based on the mic/pykickstart code, future deployment efforts
> > such as those partially described by [YOCTO #4106], but also others
> > including package selection (from e.g. binary feeds) and deployment
> > configuration of users/network/services, etc, could be implemented
> > under this framework, considering that all of those are implemented
> > in some form by the base system.
> >
> > The current patchset has successfully been used to generate and boot
> > images from core-image-minimal build artifacts using the 'mkefidisk'
> > kickstart file (in scripts/lib/image/canned-wks) on minnow and the
> > 'directdisk' kickstart file using both core-image-minimal and
> > core-image-sato artifacts for crownbay.
> >
> > The 'wic' command generates partitioned images from existing
> > OpenEmbedded build artifacts.  Image generation is driven by
> > partitioning commands contained in an 'Openembedded
> kickstart' (.wks)
> > file specified either directly on the command-line or as one of a
> > selection of canned .wks files (see 'wic list images').  When
> applied
> > to a given set of build artifacts, the result is an image or set of
> > images that can be directly written onto media and used on a
> > particular system.
> >
> > It can be used in 'raw' mode, where artifacts are explicitly
> specified
> > via command-line arguments (see example below), or it can be used in
> a
> > more easily usable 'cooked' mode which uses the current MACHINE
> > setting and a specified image name to automatically locate the
> > artifacts used to create the image.
> >
> > OE kickstart files (.wks) can of course be specified directly on the
> > command-line, but the user can also choose from a set of
> 'canned' .wks
> > files available via the 'wic list images' command (example below).
> >
> > In any case, the prerequisite for generating any image is to have
> the
> > build artifacts already available.  The below examples assume the
> user
> > has already build a 'core-image-minimal' for a specific machine
> > (future versions won't require this redundant step, but for now
> that's
> > typically how build artifacts get generated).
> >
> > The other prerequisite is to source the build environment:
> >
> > $ source oe-init-build-env
> >
> > To start out with, we'll generate an image from one of the
> canned .wks
> > files.  The following generates a list of availailable images:
> >
> > $ wic list images
> >   mkefidisk             Create an EFI disk image
> >   directdisk            Create a 'pcbios' direct disk image
> >
> > You can get more information about any of the available images by
> typing
> > 'wic list xxx help', where 'xxx' is one of the image names:
> >
> > $ wic list mkefidisk help
> >
> > Creates a partitioned EFI disk image that the user
> > can directly dd to boot media.
> >
> > At any time, you can get help on the 'wic' command or any subcommand
> > (currently 'list' and 'create').  For instance, to get the
> description
> > of 'wic create' command and its parameters:
> >
> > $ wic create
> >
> > Usage:
> >
> >  Create a new OpenEmbedded image
> >
> >  usage: wic create <wks file or image name> [-o <DIRNAME> | --outdir
> <DIRNAME>]
> >             [-i <JSON PROPERTY FILE> | --infile <JSON
> PROPERTY_FILE>]
> >             [-e | --image-name] [-r, --rootfs-dir] [-b,
> --bootimg-dir]
> >             [-k, --kernel-dir] [-n, --native-sysroot] [-s,
> --skip-build-check]
> >
> >  This command creates an OpenEmbedded image based on the 'OE
> kickstart
> >  commands' found in the <wks file>.
> >
> >  The -o option can be used to place the image in a directory with a
> >  different name and location.
> >
> >  See 'wic help create' for more detailed instructions.
> >
> > ...
> >
> > As mentioned in the command, you can get even more detailed
> information
> > by adding 'help' to the above:
> >
> > $ wic help create
> >
> > NAME
> >     wic create - Create a new OpenEmbedded image
> >
> > SYNOPSIS
> >     wic create <wks file or image name> [-o <DIRNAME> | --outdir
> > <DIRNAME>]
> >         [-i <JSON PROPERTY FILE> | --infile <JSON PROPERTY_FILE>]
> >         [-e | --image-name] [-r, --rootfs-dir] [-b, --bootimg-dir]
> >         [-k, --kernel-dir] [-n, --native-sysroot] [-s,
> > --skip-build-check]
> >
> > DESCRIPTION
> >     This command creates an OpenEmbedded image based on the 'OE
> >     kickstart commands' found in the <wks file>.
> >
> >     In order to do this, wic needs to know the locations of the
> >     various build artifacts required to build the image.
> >
> >     Users can explicitly specify the build artifact locations using
> >     the -r, -b, -k, and -n options.  See below for details on where
> >     the corresponding artifacts are typically found in a normal
> >     OpenEmbedded build.
> >
> >     Alternatively, users can use the -e option to have 'mic'
> determine
> >     those locations for a given image.  If the -e option is used,
> the
> >     user needs to have set the appropriate MACHINE variable in
> >     local.conf, and have sourced the build environment.
> >
> >     The -e option is used to specify the name of the image to use
> the
> >     artifacts from e.g. core-image-sato.
> >
> >     The -r option is used to specify the path to the /rootfs dir to
> >     use as the .wks rootfs source.
> >
> >     The -b option is used to specify the path to the dir containing
> >     the boot artifacts (e.g. /EFI or /syslinux dirs) to use as the
> >     .wks bootimg source.
> >
> >     The -k option is used to specify the path to the dir containing
> >     the kernel to use in the .wks bootimg.
> >
> >     The -n option is used to specify the path to the native sysroot
> >     containing the tools to use to build the image.
> >
> >     The -s option is used to skip the build check.  The build check
> is
> >     a simple sanity check used to determine whether the user has
> >     sourced the build environment so that the -e option can operate
> >     correctly.  If the user has specified the build artifact
> locations
> >     explicitly, 'wic' assumes the user knows what he or she is doing
> >     and skips the build check.
> >
> >     When 'wic -e' is used, the locations for the build artifacts
> >     values are determined by 'wic -e' from the output of the
> 'bitbake
> >     -e' command given an image name e.g. 'core-image-minimal' and a
> >     given machine set in local.conf.  In that case, the image is
> >     created as if the following 'bitbake -e' variables were used:
> >
> >     -r:        IMAGE_ROOTFS
> >     -k:        STAGING_KERNEL_DIR
> >     -n:        STAGING_DIR_NATIVE
> >     -b:        HDDDIR and STAGING_DATA_DIR (handlers decide which to
> > use)
> >
> >     If 'wic -e' is not used, the user needs to select the
> appropriate
> >     value for -b (as well as -r, -k, and -n).
> >
> >     The -o option can be used to place the image in a directory with
> a
> >     different name and location.
> >
> >     As an alternative to the wks file, the image-specific properties
> >     that define the values that will be used to generate a
> particular
> >     image can be specified on the command-line using the -i option
> and
> >     supplying a JSON object consisting of the set of name:value
> pairs
> >     needed by image creation.
> >
> >     The set of properties available for a given image type can be
> >     listed using the 'wic list' command.
> >
> > So, the easiest way to create an image is to use the -e option with
> a
> > canned .wks file.  To use the -e option, you need to specify the
> image
> > used to generate the artifacts and you actually need to have the
> MACHINE
> > used to build them specified in your local.conf (these requirements
> > aren't necessary if you aren't using the -e options.  Below, we
> generate
> > a directdisk image, pointing the process at the core-image-minimal
> > artifacts for the current MACHINE:
> >
> > $ wic create directdisk -e core-image-minimal
> >
> > Checking basic build environment...
> > Done.
> >
> > Creating image(s)...
> >
> > Info: The new image(s) can be found here:
> >   /var/tmp/wic/build/directdisk-201309252350-sda.direct
> >
> > The following build artifacts were used to create the image(s):
> >
> > ROOTFS_DIR:
>  /home/trz/yocto/yocto-image/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs
> > BOOTIMG_DIR:
>   /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/share
> > KERNEL_DIR:
>  /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel
> > NATIVE_SYSROOT:
>  /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel
> >
> > The image(s) were created using OE kickstart file:
> >
>   /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/directdisk.wks
> >
> > The output shows the name and location of the image created, and so
> that
> > you know exactly what was used to generate the image, each of the
> > artifacts and the kickstart file used.
> >
> > Similarly, I can create a 'mkefidisk' image in the same way (notice
> that
> > I'm using a different machine - because I'm using the -e option, I
> > needed to change the MACHINE in my local.conf):
> >
> > $ wic create mkefidisk -e core-image-minimal
> > Checking basic build environment...
> > Done.
> >
> > Creating image(s)...
> >
> > Info: The new image(s) can be found here:
> >   /var/tmp/wic/build/mkefidisk-201309260027-sda.direct
> >
> > The following build artifacts were used to create the image(s):
> >
> > ROOTFS_DIR:
>  /home/trz/yocto/yocto-image/build/tmp/work/minnow-poky-linux/core-image-minimal/1.0-r0/rootfs
> > BOOTIMG_DIR:
>   /home/trz/yocto/yocto-image/build/tmp/work/minnow-poky-linux/core-image-minimal/1.0-r0/core-image-minimal-1.0/hddimg
> > KERNEL_DIR:
>  /home/trz/yocto/yocto-image/build/tmp/sysroots/minnow/usr/src/kernel
> > NATIVE_SYSROOT:
>  /home/trz/yocto/yocto-image/build/tmp/sysroots/minnow/usr/src/kernel
> >
> > The image(s) were created using OE kickstart file:
> >
>   /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/mkefidisk.wks
> >
> > Here's an example that doesn't take the easy way out and manually
> > specifies each build artifact, along with a non-canned .wks file,
> and
> > also uses the -o option to have wic create the output somewhere
> other
> > than the default /var/tmp/wic:
> >
> > $ wic create ~/test.wks -o /home/trz/testwic
> >
> --rootfs-dir /home/trz/yocto/yocto-image/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs --bootimg-dir /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/share --kernel-dir /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel --native-sysroot /home/trz/yocto/yocto-image/build/tmp/sysroots/x86_64-linux
> >
> > Creating image(s)...
> >
> > Info: The new image(s) can be found here:
> >   /home/trz/testwic/build/test-201309260032-sda.direct
> >
> > The following build artifacts were used to create the image(s):
> >
> > ROOTFS_DIR:
>  /home/trz/yocto/yocto-image/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs
> > BOOTIMG_DIR:
>   /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/share
> > KERNEL_DIR:
>  /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel
> > NATIVE_SYSROOT:
>  /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel
> >
> > The image(s) were created using OE kickstart file:
> >   /home/trz/test.wks
> >
> > Finally, here's an example of the actual partition language commands
> > used to generate the mkefidisk image i.e. these are the contents of
> the
> > mkefidisk.wks OE kickstart file:
> >
> > # short-description: Create an EFI disk image
> > # long-description: Creates a partitioned EFI disk image that the
> user
> > # can directly dd to boot media.
> >
> > part /boot --source bootimg --ondisk sda --fstype=efi --label msdos
> --active --align 1024
> >
> > part / --source rootfs --ondisk sda --fstype=ext3 --label platform
> --align 1024
> >
> > part swap --ondisk sda --size 44 --label swap1 --fstype=swap
> >
> > bootloader  --timeout=10  --append="rootwait rootfstype=ext3
> console=ttyPCH0,115200 console=tty0 vmalloc=256MB
> snd-hda-intel.enable_msi=0"
> >
> > The initial 'wic' implementation here supports only the basic
> > kickstart partitioning commands: 'partition' (or 'part' for short)
> and
> > 'bootloader'.
> >
> > Currently, only the options used in the 'canned' scripts (in
> > scripts/lib/image/canned-wks) are supported.
> >
> > They're listed below and mostly follow the syntax and meaning of the
> > standard kickstart options for those commands:
> >
> > http://fedoraproject.org/wiki/Anaconda/Kickstart#part_or_partition
> > http://fedoraproject.org/wiki/Anaconda/Kickstart#bootloader
> >
> > For the 'partition' ('part') command, here are the supported
> options:
> >
> > --source
> >         bootimg
> >         rootfs
> > --ondisk sda
> > --size size (in mB)
> > --fstype
> >         msdos
> >         efi
> >         ext4
> >         ext3
> >         ext2
> >         btrfs
> >         swap
> > --label label
> > --active
> > ---align (in kB)
> >
> > The --source option is a wic-specific option that can currently have
> > one of two values, 'bootimg' or 'rootfs'.
> >
> > If '--source rootfs' is used, it tells the wic command to create a
> > partition as large as needed to fill with the contents of /rootfs
> > (specified by the -r 'wic' option) and to fill it with the contents
> of
> > /rootfs.
> >
> > If '--source bootimg' is used, it tells the wic command to create a
> > partition as large as needed to fill with the contents of the boot
> > partition (specified by the -b 'wic' option).  Exactly what those
> > contents are depend on the value of the --fstype option for that
> > partition.  If '--fstype=efi' is specified, the boot artifacts
> > contained in HDDDIR are used, and if '--fstype=msdos' is specified,
> > the boot artifacts found in STAGING_DATADIR are used.
> >
> > The '--align' option is a mic-specific option that says to start a
> > partition on an x kB boundary.
> >
> > For the 'bootloader' command, these are the supported options:
> >
> > --timeout
> > --append
> >
> > Future updates will implement more options - using anything not
> > explicitly supported can result in unpredicable results.
> >
> >
> > The following changes since commit
> b049d532f6e0ab9e458e486f81b00be47ee69acf:
> >
> >   testimage: Exclude BB_ORIGENV variable (2013-09-26 17:27:02 +0100)
> >
> > are available in the git repository at:
> >
> >   git://git.yoctoproject.org/poky-contrib.git tzanussi/wic-v1
> >
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=tzanussi/wic-v1
> >
> > Tom Zanussi (3):
> >   wic: Initial code for wic (OpenEmbedded Image Creator)
> >   wic: Add mic w/pykickstart
> >   wic: Add OpenEmbedded-specific implementation
> >
> >  scripts/lib/image/__init__.py                      |   22 +
> >  scripts/lib/image/canned-wks/directdisk.wks        |   10 +
> >  scripts/lib/image/canned-wks/mkefidisk.wks         |   11 +
> >  scripts/lib/image/config/wic.conf                  |    7 +
> >  scripts/lib/image/engine.py                        |  280 ++++
> >  scripts/lib/image/help.py                          |  311 ++++
> >  scripts/lib/mic/3rdparty/pykickstart/base.py       |  466 ++++++
> >  .../mic/3rdparty/pykickstart/commands/__init__.py  |   26 +
> >  .../3rdparty/pykickstart/commands/authconfig.py    |   40 +
> >  .../mic/3rdparty/pykickstart/commands/autopart.py  |  119 ++
> >  .../mic/3rdparty/pykickstart/commands/autostep.py  |   55 +
> >  .../3rdparty/pykickstart/commands/bootloader.py    |  265 ++++
> >  .../mic/3rdparty/pykickstart/commands/clearpart.py |   86 ++
> >  .../mic/3rdparty/pykickstart/commands/device.py    |  125 ++
> >  .../3rdparty/pykickstart/commands/deviceprobe.py   |   40 +
> >  .../3rdparty/pykickstart/commands/displaymode.py   |   68 +
> >  .../mic/3rdparty/pykickstart/commands/dmraid.py    |   91 ++
> >  .../3rdparty/pykickstart/commands/driverdisk.py    |  184 +++
> >  .../lib/mic/3rdparty/pykickstart/commands/fcoe.py  |  114 ++
> >  .../mic/3rdparty/pykickstart/commands/firewall.py  |  193 +++
> >  .../mic/3rdparty/pykickstart/commands/firstboot.py |   62 +
> >  .../lib/mic/3rdparty/pykickstart/commands/group.py |   88 ++
> >  .../3rdparty/pykickstart/commands/ignoredisk.py    |  139 ++
> >  .../3rdparty/pykickstart/commands/interactive.py   |   58 +
> >  .../lib/mic/3rdparty/pykickstart/commands/iscsi.py |  133 ++
> >  .../mic/3rdparty/pykickstart/commands/iscsiname.py |   54 +
> >  .../lib/mic/3rdparty/pykickstart/commands/key.py   |   64 +
> >  .../mic/3rdparty/pykickstart/commands/keyboard.py  |   55 +
> >  .../lib/mic/3rdparty/pykickstart/commands/lang.py  |   60 +
> >  .../3rdparty/pykickstart/commands/langsupport.py   |   58 +
> >  .../mic/3rdparty/pykickstart/commands/lilocheck.py |   54 +
> >  .../mic/3rdparty/pykickstart/commands/logging.py   |   66 +
> >  .../mic/3rdparty/pykickstart/commands/logvol.py    |  304 ++++
> >  .../3rdparty/pykickstart/commands/mediacheck.py    |   53 +
> >  .../mic/3rdparty/pykickstart/commands/method.py    |  186 +++
> >  .../mic/3rdparty/pykickstart/commands/monitor.py   |  106 ++
> >  .../lib/mic/3rdparty/pykickstart/commands/mouse.py |   70 +
> >  .../mic/3rdparty/pykickstart/commands/multipath.py |  111 ++
> >  .../mic/3rdparty/pykickstart/commands/network.py   |  363 +++++
> >  .../mic/3rdparty/pykickstart/commands/partition.py |  353 +++++
> >  .../lib/mic/3rdparty/pykickstart/commands/raid.py  |  365 +++++
> >  .../mic/3rdparty/pykickstart/commands/reboot.py    |   79 +
> >  .../lib/mic/3rdparty/pykickstart/commands/repo.py  |  249 +++
> >  .../mic/3rdparty/pykickstart/commands/rescue.py    |   68 +
> >  .../mic/3rdparty/pykickstart/commands/rootpw.py    |   93 ++
> >  .../mic/3rdparty/pykickstart/commands/selinux.py   |   64 +
> >  .../mic/3rdparty/pykickstart/commands/services.py  |   71 +
> >  .../lib/mic/3rdparty/pykickstart/commands/skipx.py |   54 +
> >  .../lib/mic/3rdparty/pykickstart/commands/sshpw.py |  105 ++
> >  .../mic/3rdparty/pykickstart/commands/timezone.py  |   86 ++
> >  .../mic/3rdparty/pykickstart/commands/updates.py   |   60 +
> >  .../mic/3rdparty/pykickstart/commands/upgrade.py   |  106 ++
> >  .../lib/mic/3rdparty/pykickstart/commands/user.py  |  173 +++
> >  .../lib/mic/3rdparty/pykickstart/commands/vnc.py   |  114 ++
> >  .../mic/3rdparty/pykickstart/commands/volgroup.py  |  102 ++
> >  .../mic/3rdparty/pykickstart/commands/xconfig.py   |  184 +++
> >  .../mic/3rdparty/pykickstart/commands/zerombr.py   |   69 +
> >  .../lib/mic/3rdparty/pykickstart/commands/zfcp.py  |  134 ++
> >  scripts/lib/mic/3rdparty/pykickstart/constants.py  |   57 +
> >  scripts/lib/mic/3rdparty/pykickstart/errors.py     |  103 ++
> >  .../mic/3rdparty/pykickstart/handlers/control.py   | 1307
> ++++++++++++++++
> >  .../lib/mic/3rdparty/pykickstart/handlers/f10.py   |   24 +
> >  .../lib/mic/3rdparty/pykickstart/handlers/f11.py   |   24 +
> >  .../lib/mic/3rdparty/pykickstart/handlers/f12.py   |   24 +
> >  .../lib/mic/3rdparty/pykickstart/handlers/f13.py   |   24 +
> >  .../lib/mic/3rdparty/pykickstart/handlers/f14.py   |   24 +
> >  .../lib/mic/3rdparty/pykickstart/handlers/f15.py   |   24 +
> >  .../lib/mic/3rdparty/pykickstart/handlers/f16.py   |   24 +
> >  .../lib/mic/3rdparty/pykickstart/handlers/f7.py    |   24 +
> >  .../lib/mic/3rdparty/pykickstart/handlers/f8.py    |   24 +
> >  .../lib/mic/3rdparty/pykickstart/handlers/f9.py    |   24 +
> >  .../lib/mic/3rdparty/pykickstart/handlers/fc3.py   |   24 +
> >  .../lib/mic/3rdparty/pykickstart/handlers/fc4.py   |   24 +
> >  .../lib/mic/3rdparty/pykickstart/handlers/fc5.py   |   24 +
> >  .../lib/mic/3rdparty/pykickstart/handlers/fc6.py   |   24 +
> >  .../lib/mic/3rdparty/pykickstart/handlers/rhel3.py |   24 +
> >  .../lib/mic/3rdparty/pykickstart/handlers/rhel4.py |   24 +
> >  .../lib/mic/3rdparty/pykickstart/handlers/rhel5.py |   24 +
> >  .../lib/mic/3rdparty/pykickstart/handlers/rhel6.py |   24 +
> >  scripts/lib/mic/3rdparty/pykickstart/ko.py         |   37 +
> >  scripts/lib/mic/3rdparty/pykickstart/options.py    |  204 +++
> >  scripts/lib/mic/3rdparty/pykickstart/parser.py     |  702 +++++++++
> >  scripts/lib/mic/3rdparty/pykickstart/sections.py   |  244 +++
> >  .../3rdparty/pykickstart/urlgrabber/__init__.py    |   53 +
> >  .../3rdparty/pykickstart/urlgrabber/byterange.py   |  463 ++++++
> >  .../mic/3rdparty/pykickstart/urlgrabber/grabber.py | 1477
> ++++++++++++++++++
> >  .../3rdparty/pykickstart/urlgrabber/keepalive.py   |  617 ++++++++
> >  .../mic/3rdparty/pykickstart/urlgrabber/mirror.py  |  458 ++++++
> >  .../3rdparty/pykickstart/urlgrabber/progress.py    |  530 +++++++
> >  .../3rdparty/pykickstart/urlgrabber/sslfactory.py  |   90 ++
> >  scripts/lib/mic/3rdparty/pykickstart/version.py    |  197 +++
> >  scripts/lib/mic/__init__.py                        |    4 +
> >  scripts/lib/mic/__version__.py                     |    1 +
> >  scripts/lib/mic/bootstrap.py                       |  279 ++++
> >  scripts/lib/mic/chroot.py                          |  343 +++++
> >  scripts/lib/mic/conf.py                            |  201 +++
> >  scripts/lib/mic/creator.py                         |  351 +++++
> >  scripts/lib/mic/imager/baseimager.py               | 1265
> ++++++++++++++++
> >  scripts/lib/mic/imager/direct.py                   |  472 ++++++
> >  scripts/lib/mic/imager/fs.py                       |   99 ++
> >  scripts/lib/mic/imager/livecd.py                   |  750 +++++++++
> >  scripts/lib/mic/imager/liveusb.py                  |  308 ++++
> >  scripts/lib/mic/imager/loop.py                     |  418 ++++++
> >  scripts/lib/mic/imager/raw.py                      |  501 +++++++
> >  scripts/lib/mic/kickstart/__init__.py              |  892
> +++++++++++
> >  .../lib/mic/kickstart/custom_commands/__init__.py  |   17 +
> >  .../lib/mic/kickstart/custom_commands/desktop.py   |   95 ++
> >  .../mic/kickstart/custom_commands/installerfw.py   |   63 +
> >  .../lib/mic/kickstart/custom_commands/micboot.py   |   49 +
> >  .../mic/kickstart/custom_commands/micpartition.py  |   57 +
> >  .../lib/mic/kickstart/custom_commands/micrepo.py   |  127 ++
> >  .../lib/mic/kickstart/custom_commands/partition.py |  370 +++++
> >  scripts/lib/mic/msger.py                           |  309 ++++
> >  scripts/lib/mic/plugin.py                          |  102 ++
> >  scripts/lib/mic/pluginbase.py                      |  101 ++
> >  scripts/lib/mic/plugins/backend/yumpkgmgr.py       |  490 ++++++
> >  scripts/lib/mic/plugins/backend/zypppkgmgr.py      |  973
> ++++++++++++
> >  scripts/lib/mic/plugins/hook/empty_hook.py         |    3 +
> >  scripts/lib/mic/plugins/imager/direct_plugin.py    |   92 ++
> >  scripts/lib/mic/plugins/imager/fs_plugin.py        |  143 ++
> >  scripts/lib/mic/plugins/imager/livecd_plugin.py    |  255 ++++
> >  scripts/lib/mic/plugins/imager/liveusb_plugin.py   |  260 ++++
> >  scripts/lib/mic/plugins/imager/loop_plugin.py      |  255 ++++
> >  scripts/lib/mic/plugins/imager/raw_plugin.py       |  275 ++++
> >  scripts/lib/mic/rt_util.py                         |  223 +++
> >  scripts/lib/mic/test                               |    1 +
> >  scripts/lib/mic/utils/BmapCreate.py                |  298 ++++
> >  scripts/lib/mic/utils/Fiemap.py                    |  252 ++++
> >  scripts/lib/mic/utils/cmdln.py                     | 1586
> ++++++++++++++++++++
> >  scripts/lib/mic/utils/errors.py                    |   71 +
> >  scripts/lib/mic/utils/fs_related.py                | 1057
> +++++++++++++
> >  scripts/lib/mic/utils/gpt_parser.py                |  331 ++++
> >  scripts/lib/mic/utils/grabber.py                   |   97 ++
> >  scripts/lib/mic/utils/misc.py                      | 1067
> +++++++++++++
> >  scripts/lib/mic/utils/oe/__init__.py               |   22 +
> >  scripts/lib/mic/utils/oe/misc.py                   |  108 ++
> >  scripts/lib/mic/utils/partitionedfs.py             |  776
> ++++++++++
> >  scripts/lib/mic/utils/proxy.py                     |  183 +++
> >  scripts/lib/mic/utils/rpmmisc.py                   |  600 ++++++++
> >  scripts/lib/mic/utils/runner.py                    |  109 ++
> >  scripts/wic                                        |  185 +++
> >  141 files changed, 30983 insertions(+)
> >  create mode 100644 scripts/lib/image/__init__.py
> >  create mode 100644 scripts/lib/image/canned-wks/directdisk.wks
> >  create mode 100644 scripts/lib/image/canned-wks/mkefidisk.wks
> >  create mode 100644 scripts/lib/image/config/wic.conf
> >  create mode 100644 scripts/lib/image/engine.py
> >  create mode 100644 scripts/lib/image/help.py
> >  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/__init__.py
> >  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/base.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/__init__.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/authconfig.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/autopart.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/autostep.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/bootloader.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/clearpart.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/device.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/deviceprobe.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/displaymode.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/dmraid.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/driverdisk.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/fcoe.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/firewall.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/firstboot.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/group.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/ignoredisk.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/interactive.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/iscsi.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/iscsiname.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/key.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/keyboard.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/lang.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/langsupport.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/lilocheck.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/logging.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/logvol.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/mediacheck.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/method.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/monitor.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/mouse.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/multipath.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/network.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/partition.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/raid.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/reboot.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/repo.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/rescue.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/rootpw.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/selinux.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/services.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/skipx.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/sshpw.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/timezone.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/updates.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/upgrade.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/user.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/vnc.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/volgroup.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/xconfig.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/zerombr.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/commands/zfcp.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/constants.py
> >  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/errors.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/__init__.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/control.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/f10.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/f11.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/f12.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/f13.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/f14.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/f15.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/f16.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/f7.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/f8.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/f9.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/fc3.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/fc4.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/fc5.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/fc6.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/rhel3.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/rhel4.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/rhel5.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/handlers/rhel6.py
> >  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/ko.py
> >  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/options.py
> >  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/parser.py
> >  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/sections.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/urlgrabber/__init__.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/urlgrabber/byterange.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/urlgrabber/grabber.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/urlgrabber/keepalive.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/urlgrabber/mirror.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/urlgrabber/progress.py
> >  create mode 100644
> scripts/lib/mic/3rdparty/pykickstart/urlgrabber/sslfactory.py
> >  create mode 100644 scripts/lib/mic/3rdparty/pykickstart/version.py
> >  create mode 100644 scripts/lib/mic/__init__.py
> >  create mode 100644 scripts/lib/mic/__version__.py
> >  create mode 100644 scripts/lib/mic/bootstrap.py
> >  create mode 100644 scripts/lib/mic/chroot.py
> >  create mode 100644 scripts/lib/mic/conf.py
> >  create mode 100644 scripts/lib/mic/creator.py
> >  create mode 100644 scripts/lib/mic/imager/__init__.py
> >  create mode 100644 scripts/lib/mic/imager/baseimager.py
> >  create mode 100644 scripts/lib/mic/imager/direct.py
> >  create mode 100644 scripts/lib/mic/imager/fs.py
> >  create mode 100644 scripts/lib/mic/imager/livecd.py
> >  create mode 100644 scripts/lib/mic/imager/liveusb.py
> >  create mode 100644 scripts/lib/mic/imager/loop.py
> >  create mode 100644 scripts/lib/mic/imager/raw.py
> >  create mode 100644 scripts/lib/mic/kickstart/__init__.py
> >  create mode 100644
> scripts/lib/mic/kickstart/custom_commands/__init__.py
> >  create mode 100644
> scripts/lib/mic/kickstart/custom_commands/desktop.py
> >  create mode 100644
> scripts/lib/mic/kickstart/custom_commands/installerfw.py
> >  create mode 100644
> scripts/lib/mic/kickstart/custom_commands/micboot.py
> >  create mode 100644
> scripts/lib/mic/kickstart/custom_commands/micpartition.py
> >  create mode 100644
> scripts/lib/mic/kickstart/custom_commands/micrepo.py
> >  create mode 100644
> scripts/lib/mic/kickstart/custom_commands/partition.py
> >  create mode 100644 scripts/lib/mic/msger.py
> >  create mode 100644 scripts/lib/mic/plugin.py
> >  create mode 100644 scripts/lib/mic/pluginbase.py
> >  create mode 100644 scripts/lib/mic/plugins/backend/yumpkgmgr.py
> >  create mode 100755 scripts/lib/mic/plugins/backend/zypppkgmgr.py
> >  create mode 100644 scripts/lib/mic/plugins/hook/.py
> >  create mode 100644 scripts/lib/mic/plugins/hook/empty_hook.py
> >  create mode 100644 scripts/lib/mic/plugins/imager/direct_plugin.py
> >  create mode 100644 scripts/lib/mic/plugins/imager/fs_plugin.py
> >  create mode 100644 scripts/lib/mic/plugins/imager/livecd_plugin.py
> >  create mode 100644 scripts/lib/mic/plugins/imager/liveusb_plugin.py
> >  create mode 100644 scripts/lib/mic/plugins/imager/loop_plugin.py
> >  create mode 100644 scripts/lib/mic/plugins/imager/raw_plugin.py
> >  create mode 100644 scripts/lib/mic/rt_util.py
> >  create mode 100644 scripts/lib/mic/test
> >  create mode 100644 scripts/lib/mic/utils/BmapCreate.py
> >  create mode 100644 scripts/lib/mic/utils/Fiemap.py
> >  create mode 100644 scripts/lib/mic/utils/__init__.py
> >  create mode 100644 scripts/lib/mic/utils/cmdln.py
> >  create mode 100644 scripts/lib/mic/utils/errors.py
> >  create mode 100644 scripts/lib/mic/utils/fs_related.py
> >  create mode 100644 scripts/lib/mic/utils/gpt_parser.py
> >  create mode 100644 scripts/lib/mic/utils/grabber.py
> >  create mode 100644 scripts/lib/mic/utils/misc.py
> >  create mode 100644 scripts/lib/mic/utils/oe/__init__.py
> >  create mode 100644 scripts/lib/mic/utils/oe/misc.py
> >  create mode 100644 scripts/lib/mic/utils/partitionedfs.py
> >  create mode 100644 scripts/lib/mic/utils/proxy.py
> >  create mode 100644 scripts/lib/mic/utils/rpmmisc.py
> >  create mode 100644 scripts/lib/mic/utils/runner.py
> >  create mode 100755 scripts/wic
> >
> > --
> > 1.7.11.4
> >
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> 
> 

Re: [PATCH 0/3] 'wic'- OpenEmbedded Image Creator

[Trevor Woerner]

On 26 September 2013 22:17, Tom Zanussi <tom.zanussi@linux.intel.com> wrote:
> This patchset implements a new command named 'wic' (for OpenEmbedded
> Image Creator).  Please see [YOCTO #3847] for extensive background on
> what's implemented here.

Wow Tom, this is really AWESOME work! And thanks for the excellent
detailed description of the current system which you documented in
[YOCTO #3847]!

One question comes to mind: if I have a package which installs files
to two (or more) different directories and I have decided (for
whatever reason) that those directories should be on separate
partitions, is this supported?

Re: [PATCH 0/3] 'wic'- OpenEmbedded Image Creator

[Tom Zanussi]

On Sat, 2013-09-28 at 13:10 -0400, Trevor Woerner wrote:
> On 26 September 2013 22:17, Tom Zanussi <tom.zanussi@linux.intel.com> wrote:
> > This patchset implements a new command named 'wic' (for OpenEmbedded
> > Image Creator).  Please see [YOCTO #3847] for extensive background on
> > what's implemented here.
> 
> Wow Tom, this is really AWESOME work! And thanks for the excellent
> detailed description of the current system which you documented in
> [YOCTO #3847]!
> 

Thanks!

> One question comes to mind: if I have a package which installs files
> to two (or more) different directories and I have decided (for
> whatever reason) that those directories should be on separate
> partitions, is this supported?

No, the current code doesn't, but what I had planned on doing was adding
hyphenated suffixes onto the --source rootfs param which would be
treated as subdividing the rootfs (passed in using -r) and creating
separate partitions with those contents e.g.

--source rootfs-home
--source rootfs-usr-local

So in the above, the contents of /home would be split out into a
separate partition (and subtracted from rootfs) as well as /usr/local,
and fstab entries would be created to mount those partitions as /home
and /usr/local...

Tom 

> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core