[wic][PATCH v2 0/9] UUID support

[wic][PATCH v2 0/9] UUID support

[Ed Bartosh]

Hi,

Please, review implementation of partition UUID support in wic.

It contains two options: --part-type and --use-uuid
--part-type allows to set partition type UUID. It was incorrectly
implemted in wic and now it's hopefully fixed.

--use-uuid makes wic to generate random globally unique identifier
(GUID) for the partition and configure bootloader to boot from it
using root=PARTUUID=<GUID> in kernel command line.

This technique makes boot process more deterministic and reliable
for at least two reasons:
- It allows to boot the same image from different devices
- It fixes possible boot issues caused by using device names in
  kernel commandline as GUID doesn't change unlike device names.


Ed Bartosh (9):
  wic: Add --use-uuid partition option
  wic: Generate random uuid for partition
  wic: Fix format string
  wic: Refactored getting root device name
  wic: Add parameter 'uuid' to Image.add_partition method
  wic: Set type GUID and UUID for partition
  wic: Use partition UUID in directdisk-gpt
  wic: oe-selftest: Configure testing of gpt/UUID image
  wic: Add help for --part-type and --use-uuid options

 meta/lib/oeqa/selftest/wic.py                      |  4 +--
 scripts/lib/image/canned-wks/directdisk-gpt.wks    |  2 +-
 scripts/lib/image/help.py                          |  9 +++++++
 scripts/lib/wic/imager/direct.py                   | 31 ++++++++++------------
 .../lib/wic/kickstart/custom_commands/partition.py | 21 ++++++++++++++-
 scripts/lib/wic/plugins/source/bootimg-efi.py      |  6 ++---
 scripts/lib/wic/plugins/source/bootimg-pcbios.py   |  3 +--
 .../lib/wic/plugins/source/rootfs_pcbios_ext.py    |  3 +--
 scripts/lib/wic/utils/partitionedfs.py             | 20 ++++++++++++--
 9 files changed, 68 insertions(+), 31 deletions(-)

--
Ed

[wic][PATCH v2 1/9] wic: Add --use-uuid partition option

[Ed Bartosh]

Added --use-uuid option to the configuration of wks parser.
Processing of this option will be implemented in the following
commits.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/wic/kickstart/custom_commands/partition.py b/scripts/lib/wic/kickstart/custom_commands/partition.py
index 5d033bb..7d04738 100644
--- a/scripts/lib/wic/kickstart/custom_commands/partition.py
+++ b/scripts/lib/wic/kickstart/custom_commands/partition.py
@@ -51,6 +51,7 @@ class Wic_PartData(Mic_PartData):
         self.no_table = kwargs.get("no-table", False)
         self.extra_space = kwargs.get("extra-space", "10M")
         self.overhead_factor = kwargs.get("overhead-factor", 1.3)
+        self.use_uuid = kwargs.get("use-uuid", False)
         self.source_file = ""
         self.size = 0
 
@@ -65,6 +66,8 @@ class Wic_PartData(Mic_PartData):
                 retval += " --rootfs-dir=%s" % self.rootfs
         if self.no_table:
             retval += " --no-table"
+        if self.use_uuid:
+            retval += " --use-uuid"
         retval += " --extra-space=%d" % self.extra_space
         retval += " --overhead-factor=%f" % self.overhead_factor
 
@@ -561,4 +564,7 @@ class Wic_Partition(Mic_Partition):
         op.add_option("--overhead-factor", dest="overhead_factor",
                       action="callback", callback=overhead_cb, type="float",
                       nargs=1, default=1.3)
+        op.add_option("--use-uuid", dest="use_uuid", action="store_true",
+                      default=False)
+
         return op
-- 
2.1.4

[wic][PATCH v2 2/9] wic: Generate random uuid for partition

[Ed Bartosh]

'uuid' attribute of partition object is set to generated uuid
when --use-uuid option is used for partition in .wks file.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/wic/kickstart/custom_commands/partition.py b/scripts/lib/wic/kickstart/custom_commands/partition.py
index 7d04738..f3c5545 100644
--- a/scripts/lib/wic/kickstart/custom_commands/partition.py
+++ b/scripts/lib/wic/kickstart/custom_commands/partition.py
@@ -26,6 +26,7 @@
 
 import os
 import tempfile
+import uuid
 
 from pykickstart.commands.partition import *
 from wic.utils.oe.misc import *
@@ -51,6 +52,8 @@ class Wic_PartData(Mic_PartData):
         self.no_table = kwargs.get("no-table", False)
         self.extra_space = kwargs.get("extra-space", "10M")
         self.overhead_factor = kwargs.get("overhead-factor", 1.3)
+        self._use_uuid = False
+        self.uuid = None
         self.use_uuid = kwargs.get("use-uuid", False)
         self.source_file = ""
         self.size = 0
@@ -73,6 +76,16 @@ class Wic_PartData(Mic_PartData):
 
         return retval
 
+    @property
+    def use_uuid(self):
+        return self._use_uuid
+
+    @use_uuid.setter
+    def use_uuid(self, value):
+        self._use_uuid = value
+        if value and not self.uuid:
+            self.uuid = str(uuid.uuid4())
+
     def get_rootfs(self):
         """
         Acessor for rootfs dir
-- 
2.1.4

[wic][PATCH v2 3/9] wic: Fix format string

[Ed Bartosh]

wic crashes with "TypeError: %d format: a number is required, not str"
Due to incorrect format used for Wic_PartData.extra_size attribute.
Using %s instead of %d should fix the crash.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/wic/kickstart/custom_commands/partition.py b/scripts/lib/wic/kickstart/custom_commands/partition.py
index f3c5545..40c2772 100644
--- a/scripts/lib/wic/kickstart/custom_commands/partition.py
+++ b/scripts/lib/wic/kickstart/custom_commands/partition.py
@@ -71,7 +71,7 @@ class Wic_PartData(Mic_PartData):
             retval += " --no-table"
         if self.use_uuid:
             retval += " --use-uuid"
-        retval += " --extra-space=%d" % self.extra_space
+        retval += " --extra-space=%s" % self.extra_space
         retval += " --overhead-factor=%f" % self.overhead_factor
 
         return retval
-- 
2.1.4

[wic][PATCH v2 4/9] wic: Refactored getting root device name

[Ed Bartosh]

Replaced DirectImageCreator._get_boot_config private method
with a 'rootdev' property.
Simplified the code and API.
Used 'uuid' property instead of incorrectly used 'part_type'.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/wic/imager/direct.py b/scripts/lib/wic/imager/direct.py
index 83f9688..36150c9 100644
--- a/scripts/lib/wic/imager/direct.py
+++ b/scripts/lib/wic/imager/direct.py
@@ -346,27 +346,23 @@ class DirectImageCreator(BaseImageCreator):
 
         msger.info(msg)
 
-    def _get_boot_config(self):
+    @property
+    def rootdev(self):
         """
-        Return the rootdev/root_part_uuid (if specified by
-        --part-type)
+        Get root device name to use as a 'root' parameter
+        in kernel command line.
 
         Assume partition order same as in wks
         """
-        rootdev = None
-        root_part_uuid = None
         parts = self._get_parts()
-        for num, p in enumerate(parts, 1):
-            if p.mountpoint == "/":
-                part = ''
-                if p.disk.startswith('mmcblk'):
-                    part = 'p'
-
-                pnum = self.__get_part_num(num, parts)
-                rootdev = "/dev/%s%s%-d" % (p.disk, part, pnum)
-                root_part_uuid = p.part_type
-
-        return (rootdev, root_part_uuid)
+        for num, part in enumerate(parts, 1):
+            if part.mountpoint == "/":
+                if part.uuid:
+                    return "PARTUUID=%s" % part.uuid
+                else:
+                    suffix = 'p' if part.disk.startswith('mmcblk') else ''
+                    pnum = self.__get_part_num(num, parts)
+                    return "/dev/%s%s%-d" % (part.disk, suffix, pnum)
 
     def _cleanup(self):
         if not self.__image is None:
diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py b/scripts/lib/wic/plugins/source/bootimg-efi.py
index 400e3a2..39ce9f3 100644
--- a/scripts/lib/wic/plugins/source/bootimg-efi.py
+++ b/scripts/lib/wic/plugins/source/bootimg-efi.py
@@ -47,7 +47,6 @@ class BootimgEFIPlugin(SourcePlugin):
         else:
             splashline = ""
 
-        (rootdev, root_part_uuid) = cr._get_boot_config()
         options = cr.ks.handler.bootloader.appendLine
 
         grubefi_conf = ""
@@ -62,7 +61,7 @@ class BootimgEFIPlugin(SourcePlugin):
         kernel = "/bzImage"
 
         if cr.ptable_format in ('msdos', 'gpt'):
-            rootstr = rootdev
+            rootstr = cr.rootdev
         else:
             raise ImageError("Unsupported partition table format found")
 
@@ -87,7 +86,6 @@ class BootimgEFIPlugin(SourcePlugin):
         install_cmd = "install -d %s/loader/entries" % hdddir
         exec_cmd(install_cmd)
 
-        (rootdev, root_part_uuid) = cr._get_boot_config()
         options = cr.ks.handler.bootloader.appendLine
 
         timeout = kickstart.get_timeout(cr.ks)
@@ -107,7 +105,7 @@ class BootimgEFIPlugin(SourcePlugin):
         kernel = "/bzImage"
 
         if cr.ptable_format in ('msdos', 'gpt'):
-            rootstr = rootdev
+            rootstr = cr.rootdev
         else:
             raise ImageError("Unsupported partition table format found")
 
diff --git a/scripts/lib/wic/plugins/source/bootimg-pcbios.py b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
index 1c3c6e6..dd49480 100644
--- a/scripts/lib/wic/plugins/source/bootimg-pcbios.py
+++ b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
@@ -83,7 +83,6 @@ class BootimgPcbiosPlugin(SourcePlugin):
         else:
             splashline = ""
 
-        (rootdev, root_part_uuid) = cr._get_boot_config()
         options = cr.ks.handler.bootloader.appendLine
 
         syslinux_conf = ""
@@ -105,7 +104,7 @@ class BootimgPcbiosPlugin(SourcePlugin):
         syslinux_conf += "KERNEL " + kernel + "\n"
 
         if cr.ptable_format in ('msdos', 'gpt'):
-            rootstr = rootdev
+            rootstr = cr.rootdev
         else:
             raise ImageError("Unsupported partition table format found")
 
diff --git a/scripts/lib/wic/plugins/source/rootfs_pcbios_ext.py b/scripts/lib/wic/plugins/source/rootfs_pcbios_ext.py
index 50b2213..90dac05 100644
--- a/scripts/lib/wic/plugins/source/rootfs_pcbios_ext.py
+++ b/scripts/lib/wic/plugins/source/rootfs_pcbios_ext.py
@@ -82,7 +82,6 @@ class RootfsPlugin(SourcePlugin):
 
         Called before do_prepare_partition()
         """
-        rootdev = image_creator._get_boot_config()[0]
         options = image_creator.ks.handler.bootloader.appendLine
 
         syslinux_conf = ""
@@ -102,7 +101,7 @@ class RootfsPlugin(SourcePlugin):
         syslinux_conf += "  KERNEL /boot/bzImage\n"
 
         if image_creator.ptable_format in ('msdos', 'gpt'):
-            rootstr = rootdev
+            rootstr = image_creator.rootdev
         else:
             raise ImageError("Unsupported partition table format found")
 
-- 
2.1.4

[wic][PATCH v2 5/9] wic: Add parameter 'uuid' to Image.add_partition method

[Ed Bartosh]

With this parameter it's possible to pass generated UUID
into Image class to set it for partition when it's created.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/wic/imager/direct.py b/scripts/lib/wic/imager/direct.py
index 36150c9..2290ecd 100644
--- a/scripts/lib/wic/imager/direct.py
+++ b/scripts/lib/wic/imager/direct.py
@@ -272,7 +272,8 @@ class DirectImageCreator(BaseImageCreator):
                                        boot=p.active,
                                        align=p.align,
                                        no_table=p.no_table,
-                                       part_type=p.part_type)
+                                       part_type=p.part_type,
+                                       uuid=p.uuid)
 
         self._restore_fstab(fstab)
 
diff --git a/scripts/lib/wic/utils/partitionedfs.py b/scripts/lib/wic/utils/partitionedfs.py
index 902548f..a6e2e4f 100644
--- a/scripts/lib/wic/utils/partitionedfs.py
+++ b/scripts/lib/wic/utils/partitionedfs.py
@@ -86,7 +86,7 @@ class Image(object):
 
     def add_partition(self, size, disk_name, mountpoint, source_file=None, fstype=None,
                       label=None, fsopts=None, boot=False, align=None, no_table=False,
-                      part_type=None):
+                      part_type=None, uuid=None):
         """ Add the next partition. Prtitions have to be added in the
         first-to-last order. """
 
@@ -110,7 +110,8 @@ class Image(object):
                     'boot': boot, # Bootable flag
                     'align': align, # Partition alignment
                     'no_table' : no_table, # Partition does not appear in partition table
-                    'part_type' : part_type} # Partition type
+                    'part_type' : part_type, # Partition type
+                    'uuid': uuid} # Partition UUID
 
             self.__add_partition(part)
 
-- 
2.1.4

[wic][PATCH v2 6/9] wic: Set type GUID and UUID for partition

[Ed Bartosh]

Set type GUID and UUID for partition using sgdisk utility.

Type GUID can be specified for partition in .wks with
--part-type option.

UUID is generated when --use-uuid option is specified for
partition.

[YOCTO #7716]

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/wic/utils/partitionedfs.py b/scripts/lib/wic/utils/partitionedfs.py
index a6e2e4f..1eb1f01 100644
--- a/scripts/lib/wic/utils/partitionedfs.py
+++ b/scripts/lib/wic/utils/partitionedfs.py
@@ -294,6 +294,21 @@ class Image(object):
             self.__create_partition(d['disk'].device, p['type'],
                                     parted_fs_type, p['start'], p['size'])
 
+            if p['part_type']:
+                msger.debug("partition %d: set type UID to %s" % \
+                            (p['num'], p['part_type']))
+                exec_native_cmd("sgdisk --typecode=%d:%s %s" % \
+                                         (p['num'], p['part_type'],
+                                          d['disk'].device), self.native_sysroot)
+
+            if p['uuid']:
+                msger.debug("partition %d: set UUID to %s" % \
+                            (p['num'], p['uuid']))
+                exec_native_cmd("sgdisk --partition-guid=%d:%s %s" % \
+                                         (p['num'], p['uuid'],
+                                          d['disk'].device),
+                                        self.native_sysroot)
+
             if p['boot']:
                 flag_name = "legacy_boot" if d['ptable_format'] == 'gpt' else "boot"
                 msger.debug("Set '%s' flag for partition '%s' on disk '%s'" % \
-- 
2.1.4

[wic][PATCH v2 7/9] wic: Use partition UUID in directdisk-gpt

[Ed Bartosh]

Used --use-uuid option for root partition in directdisk-gpt.wks
This is useful to have example of image with UUID support.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/image/canned-wks/directdisk-gpt.wks b/scripts/lib/image/canned-wks/directdisk-gpt.wks
index 76fda1f..2355259 100644
--- a/scripts/lib/image/canned-wks/directdisk-gpt.wks
+++ b/scripts/lib/image/canned-wks/directdisk-gpt.wks
@@ -4,7 +4,7 @@
 
 
 part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024
-part / --source rootfs --ondisk sda --fstype=ext3 --label platform --align 1024
+part / --source rootfs --ondisk sda --fstype=ext3 --label platform --align 1024 --use-uuid
 
 bootloader  --ptable gpt --timeout=0  --append="rootwait rootfstype=ext3 video=vesafb vga=0x318 console=tty0"
 
-- 
2.1.4

[wic][PATCH v2 8/9] wic: oe-selftest: Configure testing of gpt/UUID image

[Ed Bartosh]

Added runtime dependency to gptfdisk-native to wic test suite to
be able to test modified directdisk-gpt with UUID support.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/meta/lib/oeqa/selftest/wic.py b/meta/lib/oeqa/selftest/wic.py
index e97dd1d..358f09e 100644
--- a/meta/lib/oeqa/selftest/wic.py
+++ b/meta/lib/oeqa/selftest/wic.py
@@ -39,7 +39,7 @@ class Wic(oeSelfTest):
     @classmethod
     def setUpClass(cls):
         """Build wic runtime dependencies and images used in the tests."""
-        bitbake('syslinux syslinux-native parted-native '
+        bitbake('syslinux syslinux-native parted-native gptfdisk-native '
                 'dosfstools-native mtools-native core-image-minimal')
 
     def setUp(self):
@@ -75,7 +75,7 @@ class Wic(oeSelfTest):
         self.assertEqual(1, len(glob(self.resultdir + "directdisk-*.direct")))
 
     def test06_gpt_image(self):
-        """Test creation of core-image-minimal with gpt table"""
+        """Test creation of core-image-minimal with gpt table and UUID boot"""
         self.assertEqual(0, runCmd("wic create directdisk-gpt "
                                    "--image-name core-image-minimal").status)
         self.assertEqual(1, len(glob(self.resultdir + "directdisk-*.direct")))
-- 
2.1.4

[wic][PATCH v2 9/9] wic: Add help for --part-type and --use-uuid options

[Ed Bartosh]

Added help for two wks partition options specific to
GUID partition table and globally unique identificators (GUID).

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

diff --git a/scripts/lib/image/help.py b/scripts/lib/image/help.py
index e365a07..14f0ff1 100644
--- a/scripts/lib/image/help.py
+++ b/scripts/lib/image/help.py
@@ -757,6 +757,15 @@ DESCRIPTION
                             equal to 1.
                             The default value is 1.3.
 
+         --part-type: This option is specific to wic. It specifies partition
+                      type GUID for GPT partitions.
+                      List of partition type GUIDS can be found here:
+                      http://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_type_GUIDs
+
+         --use-uuid: This option is specific to wic. It makes wic to generate
+                     random globally unique identifier (GUID) for the partition
+                     and use it in bootloader configuration to specify root partition.
+
     * bootloader
 
       This command allows the user to specify various bootloader
-- 
2.1.4