From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0803AC4725D for ; Fri, 19 Jan 2024 11:40:27 +0000 (UTC) Subject: [PATCH] [WIC] Configurable addressing of GTP main table To: openembedded-core@lists.openembedded.org From: davide.cardillo@seco.com X-Originating-Location: Florence, Tuscany, IT (94.89.17.99) X-Originating-Platform: Linux Chrome 120 User-Agent: GROUPS.IO Web Poster MIME-Version: 1.0 Date: Fri, 19 Jan 2024 00:23:46 -0800 Message-ID: Content-Type: multipart/alternative; boundary="h5EEC8WXheWC5siKFw19" List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 19 Jan 2024 11:40:27 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/194032 --h5EEC8WXheWC5siKFw19 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Issue: the usage of a gpt table with i.MX processor is not compatible by default. This is due to a memory conflict: the main partition table is placed into the same memory space used to flash the header of the U-Boot. This last can not be moved since the ROM code of the i.MX6 starts to read to that fixed address (0x400). This problem can be solved using a feature of the GTP partition table for witch the main table can be moved to a desired memory area. The WIC library present in poky manages the GTP table through some standard Linux commands (gdisk and sgdisk) which ones have already the opportunity to perform this task. This patch introduces this feature with the addition of some wic's options, used also in wks file. New partition placement behavior: Since now all partitions are placed after the gpt table presents into the first 1KB of memory (including both the gpt header at the first byte of memory and the main partition table staring at 0x400). Now the main gpt table can be placed in other address spaces and a partitio= n can be placed also before the gpt table. Recap: - by default, all partitions are placed sequentially, one after the other. The first partition implicitly starts after the GPT partition table. - adding a flag to the item of a partition into the wks file, the partition can be placed, starting from a fixed memory offset (neglecting the memory space occupied by the GTP table). Combining the moving of the main partition table and the absolute (not yet relative) positioning of a partition, now it is possible to place one or mo= re partitions before the GPT table. New flags: - main-ptable-offset (size in KB): to add as property of the bootloader ite= m. Specifies the main partition table location in KB. - fixed-align (boolean): to add as property of a partition. Specifies that = the partition must to placed with absolute offset, used in conjunction with --a= lign property Example: setting used for a wic for an i.MX6 processor, where the main partition tab= le is placed with the absolute offset of 4MB bootloader --ptable gpt --main-ptable-offset 4096K part / --source rawcopy --sourceparams=3D"file=3Du-boot.imx" --ondisk mmcbl= k --align 1 --size 1M --no-table --fixed-align Test environment: This patch is a part of Edgehog OS (https://git.seco.com/edgehog) a custom Yocto distribution by SECO SpA (www.seco.com), used on multiple architectures (both ARM and x86). This solution is widely used by all SECO customers who use hardware solution based on NXP i.MX6 processors. Signed-off-by: Davide Cardillo Reviewed-by: Tobias Poganiuch Tested-by: Andrea Da Col Tested-by: Daniel Rinaldi --- scripts/lib/wic/ksparser.py =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0| =C2=A0=C2=A02 + scripts/lib/wic/partition.py =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0| =C2=A0=C2=A01 + scripts/lib/wic/plugins/imager/direct.py | 102 +++++++++++++++++++++-- scripts/wic =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0| =C2=A0=C2=A02 +- 4 files changed, 98 insertions(+), 9 deletions(-) diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py index 0df9eb0d05..45db2eaa5d 100644 --- a/scripts/lib/wic/ksparser.py +++ b/scripts/lib/wic/ksparser.py @@ -148,6 +148,7 @@ class KickStart(): part =3D subparsers.add_parser('part') part.add_argument('mountpoint', nargs=3D'?') part.add_argument('--active', action=3D'store_true') + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0part.add_argument('--fixed-alig= n', action=3D'store_true') part.add_argument('--align', type=3Dint) part.add_argument('--offset', type=3Dsizetype("K", True)) part.add_argument('--exclude-path', nargs=3D'+') @@ -194,6 +195,7 @@ class KickStart(): default=3D'msdos') bootloader.add_argument('--timeout', type=3Dint) bootloader.add_argument('--source') + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0bootloader.add_argument('--main= -ptable-offset', type=3Dsizetype("K"), default=3D0) include =3D subparsers.add_parser('include') include.add_argument('path', type=3Dcannedpathtype) diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py index 5275da6ed3..1bd2b457cd 100644 --- a/scripts/lib/wic/partition.py +++ b/scripts/lib/wic/partition.py @@ -26,6 +26,7 @@ class Partition(): self.args =3D args self.active =3D args.active self.align =3D args.align + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self.fixed_align =3D args.fixed= _align self.disk =3D args.disk self.device =3D None self.extra_space =3D args.extra_space diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plu= gins/imager/direct.py index 165fc2979f..efa4d7804e 100644 --- a/scripts/lib/wic/plugins/imager/direct.py +++ b/scripts/lib/wic/plugins/imager/direct.py @@ -16,6 +16,7 @@ import random import shutil import tempfile import uuid +import operator from time import strftime @@ -66,6 +67,7 @@ class DirectPlugin(ImagerPlugin): self.workdir =3D self.setup_workdir(options.workdir) self._image =3D None self.ptable_format =3D self.ks.bootloader.ptable + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self.main_ptable_offset =3D sel= f.ks.bootloader.main_ptable_offset self.parts =3D self.ks.partitions # as a convenience, set source to the boot partition source @@ -77,6 +79,7 @@ class DirectPlugin(ImagerPlugin): image_path =3D self._full_path(self.workdir, self.parts[0].disk, "direct") self._image =3D PartitionedImage(image_path, self.ptable_format, + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0self.main_ptable_offset, self.parts, self.native_sysroot, options.extra_space) @@ -294,15 +297,27 @@ GPT_OVERHEAD =3D 34 # Size of a sector in bytes SECTOR_SIZE =3D 512 +class memoryRegion(): + =C2=A0=C2=A0=C2=A0def __init__(self): + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self.start_sector =3D 1 + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self.end_sector =3D 1 + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self.part_num =3D 0 + + =C2=A0=C2=A0=C2=A0def __init__(self, part_num, start_sec, end_sec): + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self.part_num =3D part_num + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self.start_sector =3D start_sec + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self.end_sector =3D end_sec + class PartitionedImage(): """ Partitioned image in a file. """ - =C2=A0=C2=A0=C2=A0def __init__(self, path, ptable_format, partitions, nat= ive_sysroot=3DNone, extra_space=3D0): + =C2=A0=C2=A0=C2=A0def __init__(self, path, ptable_format, main_ptable_off= set, partitions, native_sysroot=3DNone, extra_space=3D0): self.path =3D path =C2=A0# Path to the image file self.numpart =3D 0 =C2=A0# Number of allocated partitions self.realpart =3D 0 # Number of partitions in the partition table + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self.fixedalign_part =3D 0 # Nu= mber of partitions with absolute alignment self.primary_part_num =3D 0 =C2=A0# Number of primary partitions (msdos) self.extendedpart =3D 0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0# Create extended par= tition before this logical partition (msdos) self.extended_size_sec =3D 0 # Size of exteded partition (msdos) @@ -311,11 +326,13 @@ class PartitionedImage(): self.min_size =3D 0 # Minimum required disk size to fit # all partitions (in bytes) self.ptable_format =3D ptable_format =C2=A0# Partition table format + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self.main_ptable_offset =3D mai= n_ptable_offset # for GPT table, offset # Disk system identifier self.identifier =3D random.SystemRandom().randint(1, 0xffffffff) self.partitions =3D partitions self.partimages =3D [] + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self.used_memory_region =3D [] # Size of a sector used in calculations self.sector_size =3D SECTOR_SIZE self.native_sysroot =3D native_sysroot @@ -368,6 +385,30 @@ class PartitionedImage(): # Converting kB to sectors for parted part.size_sec =3D part.disk_size * 1024 // self.sector_size + =C2=A0=C2=A0=C2=A0def _add_busy_region(self, part_num, start_sec, end_sec= ): + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self.used_memory_region.append(= memoryRegion(part_num, start_sec, end_sec)) + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self.used_memory_region =3D sor= ted(self.used_memory_region, key=3Doperator.attrgetter('start_sector')) + + =C2=A0=C2=A0=C2=A0def _print_busy_region_list(self): + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0list =3D "Busy memory region (i= n sector)\n" + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0list +=3D "\tpart#\tstart\tEnd\= n" + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0for num in range(len(self.used_= memory_region)): + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0list += =3D"\t%s\t%s\t%s\n" % (self.used_memory_region[num].part_num, \ + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0self.used_memory_region[num].start_sector, self.used_memo= ry_region[num].end_sector) + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0logger.debug(list) + + =C2=A0=C2=A0=C2=A0def _check_memory_region(self, start_sec, end_sec ): + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0""" Check if the specified regi= on is already flagges as used. Returns + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0zero if not used, 1 otherwise""= " + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0for num in range(len(self.used_= memory_region)): + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0region = =3D self.used_memory_region[num] + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0if ( ( = region.start_sector < start_sec and start_sec < region.end_sector) or \ + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0( region.start_sector < end_sec a= nd end_sec < region.end_sector) ): + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0return 1 + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0if ( st= art_sec < region.start_sector and end_sec > region.end_sector ): + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0return 1 + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0return 0 + def layout_partitions(self): """ Layout the partitions, meaning calculate the position of every partition on the disk. The 'ptable_format' parameter defines the @@ -379,6 +420,35 @@ class PartitionedImage(): # partitions not listed in the table are not included. num_real_partitions =3D len([p for p in self.partitions if not p.no_table]) + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0# The partitions flagged as fix= ed alignment partition are placed into image + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0# before the others. Instead, r= elative-aligned partitions are placed in a + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0# serialized fashion, with cont= rol of the intersection of memory areas. + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0# The base address of the relat= ive-aligned partitions is, as before, + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0# the maximum address accupied = by the partition table + + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0# Flag as used the momoery requ= ired for partition table + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self._add_busy_region(-1, 0, MB= R_OVERHEAD) + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0if self.ptable_format =3D=3D "g= pt": + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0offset = =3D self.main_ptable_offset * 1024 // self.sector_size + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self._a= dd_busy_region(-1, offset, offset + GPT_OVERHEAD) + + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0# Search all partition with fix= ed position into the image + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0for num in range(len(self.parti= tions)): + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0part = =3D self.partitions[num] + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0if not = part.fixed_align: + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0continue + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0if not = part.no_table: + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0raise WicError("A partition with fixed alignment must hav= e no_table flag set") + + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0start_s= ector =3D (part.align * 1024 // self.sector_size) + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0end_sec= tor =3D start_sector + part.size_sec - 1 + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0if self= ._check_memory_region(start_sector, end_sector) =3D=3D 1: + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0self._print_busy_region_list() + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0raise WicError("A partition wants to use an already used = memory region (sectors %d - %d)" \ + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0% (start_= sector, end_sector)) + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self._a= dd_busy_region(num, start_sector, end_sector) + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0part.st= art =3D start_sector + # Go through partitions in the order they are added in .ks file for num in range(len(self.partitions)): part =3D self.partitions[num] @@ -403,7 +473,7 @@ class PartitionedImage(): if self.ptable_format =3D=3D "msdos": overhead =3D MBR_OVERHEAD elif self.ptable_format =3D=3D "gpt": - =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0overhead =3D GPT_OVERHEAD + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0overhead =3D (self.main_ptable_of= fset * 1024 // self.sector_size) + GPT_OVERHEAD # Skip one sector required for the partitioning scheme overhead self.offset +=3D overhead @@ -418,7 +488,7 @@ class PartitionedImage(): self.offset +=3D 2 align_sectors =3D 0 - =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0if part= .align: + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0if not = part.fixed_align and part.align: # If not first partition and we do have alignment set we need # to align the partition. # FIXME: This leaves a empty spaces to the disk. To fill the @@ -440,7 +510,7 @@ class PartitionedImage(): # increase the offset so we actually start the partition on right alignment self.offset +=3D align_sectors - =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0if part= .offset is not None: + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0if not = part.fixed_align and part.offset is not None: offset =3D part.offset // self.sector_size if offset * self.sector_size !=3D part.offset: @@ -455,14 +525,20 @@ class PartitionedImage(): self.offset =3D offset - =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0part.st= art =3D self.offset - =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self.of= fset +=3D part.size_sec - if not part.no_table: part.num =3D self.realpart else: part.num =3D 0 + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0if not = part.fixed_align: + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0part.start =3D self.offset + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0self.offset +=3D part.size_sec + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0if self._check_memory_region(part.start, self.offset) =3D= =3D 1: + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self._print_busy_region_list() + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0raise WicError("A partition wants= to use an already used memory region (sectors %d - %d)" \ + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0% (part.s= tart, self.offset)) + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0self._add_busy_region(part.num, part.start, self.offset) + if self.ptable_format =3D=3D "msdos" and not part.no_table: if part.type =3D=3D 'logical': self.logical_part_cnt +=3D 1 @@ -482,12 +558,14 @@ class PartitionedImage(): "sectors (%d bytes).", part.mountpoint, part.disk, part.num, part.start, self.offset - 1, part.size_sec, part.size_sec * self.sector_size) + + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self._print_busy_region_list() # Once all the partitions have been layed out, we can calculate the # minumim disk size self.min_size =3D self.offset if self.ptable_format =3D=3D "gpt": - =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self.mi= n_size +=3D GPT_OVERHEAD + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0self.mi= n_size +=3D (self.main_ptable_offset * 1024 // self.sector_size) + GPT_OVER= HEAD self.min_size *=3D self.sector_size self.min_size +=3D self.extra_space @@ -606,6 +684,14 @@ class PartitionedImage(): exec_native_cmd("sfdisk --part-type %s %s %s" % \ (self.path, part.num, part.system_id), self.native_sysroot) + + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0if self.ptable_format =3D=3D "g= pt" and self.main_ptable_offset > 0: + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0main_pt= able_sectors =3D self.main_ptable_offset * 1024 // self.sector_size + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0logger.= debug("Move the main GPT partition table forward by %s sector(s)", main_pta= ble_sectors) + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0cmd =3D= "sgdisk -j %d %s" % (main_ptable_sectors, self.path) + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0#cmd = =3D "(echo -e 'x'; echo -e 'j' ; echo -e '%d'; echo -e 'w'; echo -e 'Y') | = gdisk %s" % (main_ptable_sectors, self.path) + =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0exec_na= tive_cmd(cmd, self.native_sysroot) + def cleanup(self): pass diff --git a/scripts/wic b/scripts/wic index 06e0b48db0..0c59a550ef 100755 --- a/scripts/wic +++ b/scripts/wic @@ -59,7 +59,7 @@ from wic import help as hlp def wic_logger(): - =C2=A0=C2=A0=C2=A0"""Create and convfigure wic logger.""" + =C2=A0=C2=A0=C2=A0"""Create and configure wic logger.""" logger =3D logging.getLogger('wic') logger.setLevel(logging.INFO) -- 2.39.2 --h5EEC8WXheWC5siKFw19 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable
Issue:
the usage of a g= pt table with i.MX processor is not compatible by default.
This is du= e to a memory conflict: the main partition table is placed into
the s= ame memory space used to flash the header of the U-Boot. This last
ca= n not be moved since the ROM code of the i.MX6 starts to read to that
fixed address (0x400).
This problem can be solved using a feature of= the GTP partition table for
witch the main table can be moved to a d= esired memory area.
The WIC library present in poky manages the GTP t= able through some
standard Linux commands (gdisk and sgdisk) which on= es have already the
opportunity to perform this task.
This patc= h introduces this feature with the addition of some wic's options,
us= ed also in wks file.

New partition placement behavior:
Si= nce now all partitions are placed after the gpt table presents into the first 1KB of memory (including both the gpt header at the first byte of =
memory and the main partition table staring at 0x400).
Now the = main gpt table can be placed in other address spaces and a partition
= can be placed also before the gpt table. Recap:
- by default, all par= titions are placed sequentially, one after the other.
The first parti= tion implicitly starts after the GPT partition table.
- adding a flag= to the item of a partition into the wks file, the partition
can be p= laced, starting from a fixed memory offset (neglecting the memory
spa= ce occupied by the GTP table).
Combining the moving of the main parti= tion table and the absolute (not yet
relative) positioning of a parti= tion, now it is possible to place one or more
   parti= tions before the GPT table.

New flags:
- main-ptable-offs= et (size in KB): to add as property of the bootloader item.
Specifies= the main partition table location in KB.
- fixed-align (boolean): to= add as property of a partition. Specifies that the
partition must to= placed with absolute offset, used in conjunction with --align
proper= ty

Example:
setting used for a wic for an i.MX6 processor= , where the main partition table is
placed with the absolute offset o= f 4MB

bootloader --ptable gpt --main-ptable-offset 4096K
= part / --source rawcopy --sourceparams=3D"file=3Du-boot.imx" --ondisk mmcbl= k --align 1 --size 1M --no-table --fixed-align

Test environment= :
This patch is a part of Edgehog OS (https://git.seco.com/edgehog) a=
custom Yocto distribution by SECO SpA (www.seco.com), used on multip= le
architectures (both ARM and x86).
This solution is widely us= ed by all SECO customers who use hardware
solution based on NXP i.MX6= processors.

Signed-off-by: Davide Cardillo <davide.cardillo= @seco.com>
Reviewed-by: Tobias Poganiuch <tobias.poganiuch@seco= .com>
Tested-by: Andrea Da Col <andrea.dacol@seco.com>
Tested-by: Daniel Rinaldi <daniel.rinaldi@seco.com>
---
= scripts/lib/wic/ksparser.py         = ;     |   2 +
scripts/lib/wic/part= ition.py            =  |   1 +
scripts/lib/wic/plugins/imager/direct.py | 10= 2 +++++++++++++++++++++--
scripts/wic      &= nbsp;           &nbs= p;           |  = ; 2 +-
4 files changed, 98 insertions(+), 9 deletions(-)
<= br />diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py=
index 0df9eb0d05..45db2eaa5d 100644
--- a/scripts/lib/wic/kspa= rser.py
+++ b/scripts/lib/wic/ksparser.py
@@ -148,6 +148,7 @@ c= lass KickStart():
        par= t =3D subparsers.add_parser('part')
     &nb= sp;  part.add_argument('mountpoint', nargs=3D'?')
 &nb= sp;      part.add_argument('--active', action= =3D'store_true')
+        part.add= _argument('--fixed-align', action=3D'store_true')
   &= nbsp;    part.add_argument('--align', type=3Dint)
        part.add_argument('--offse= t', type=3Dsizetype("K", True))
      &= nbsp; part.add_argument('--exclude-path', nargs=3D'+')
@@ -194,6= +195,7 @@ class KickStart():
      &nb= sp;            =             &nb= sp;default=3D'msdos')
        = ;bootloader.add_argument('--timeout', type=3Dint)
   &= nbsp;    bootloader.add_argument('--source')
+ &n= bsp;      bootloader.add_argument('--main-pta= ble-offset', type=3Dsizetype("K"), default=3D0)
 
 &nb= sp;      include =3D subparsers.add_parser('i= nclude')
        include.add_= argument('path', type=3Dcannedpathtype)
diff --git a/scripts/lib/wic/= partition.py b/scripts/lib/wic/partition.py
index 5275da6ed3..1bd2b45= 7cd 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib= /wic/partition.py
@@ -26,6 +26,7 @@ class Partition():
 &n= bsp;      self.args =3D args
 &nbs= p;      self.active =3D args.active
&nb= sp;       self.align =3D args.align
+        self.fixed_align =3D args.fix= ed_align
        self.disk = =3D args.disk
        self.de= vice =3D None
        self.ex= tra_space =3D args.extra_space
diff --git a/scripts/lib/wic/plugins/i= mager/direct.py b/scripts/lib/wic/plugins/imager/direct.py
index 165f= c2979f..efa4d7804e 100644
--- a/scripts/lib/wic/plugins/imager/direct= .py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -16,6 +16= ,7 @@ import random
import shutil
import tempfile
import = uuid
+import operator
 
from time import strftime  
@@ -66,6 +67,7 @@ class DirectPlugin(ImagerPlugin):
&n= bsp;       self.workdir =3D self.setup_w= orkdir(options.workdir)
       &nb= sp;self._image =3D None
       &nb= sp;self.ptable_format =3D self.ks.bootloader.ptable
+   &nb= sp;    self.main_ptable_offset =3D self.ks.bootloader.m= ain_ptable_offset
        sel= f.parts =3D self.ks.partitions
 
    &n= bsp;   # as a convenience, set source to the boot partition = source
@@ -77,6 +79,7 @@ class DirectPlugin(ImagerPlugin):
&nbs= p;
        image_path =3D self= ._full_path(self.workdir, self.parts[0].disk, "direct")
  &= nbsp;     self._image =3D PartitionedImage(image_p= ath, self.ptable_format,
+        =             &nb= sp;            =       self.main_ptable_offset,
 &n= bsp;            = ;            &n= bsp;            = ;self.parts, self.native_sysroot,
      = ;            &n= bsp;            = ;        options.extra_space)
 
@@ -294,15 +297,27 @@ GPT_OVERHEAD =3D 34
# Size of a se= ctor in bytes
SECTOR_SIZE =3D 512
 
+class memoryRegi= on():
+    def __init__(self):
+   &nb= sp;    self.start_sector =3D 1
+   &nbs= p;    self.end_sector =3D 1
+    &= nbsp;   self.part_num =3D 0
+
+   &nbs= p;def __init__(self, part_num, start_sec, end_sec):
+   &nb= sp;    self.part_num =3D part_num
+   &= nbsp;    self.start_sector =3D start_sec
+  =       self.end_sector =3D end_sec
+ class PartitionedImage():
    """
 =    Partitioned image in a file.
   &nbs= p;"""
 
-    def __init__(self, path, ptable= _format, partitions, native_sysroot=3DNone, extra_space=3D0):
+  = ;  def __init__(self, path, ptable_format, main_ptable_offset, pa= rtitions, native_sysroot=3DNone, extra_space=3D0):
   =      self.path =3D path  # Path to the image = file
        self.numpart =3D= 0  # Number of allocated partitions
    &nb= sp;   self.realpart =3D 0 # Number of partitions in the part= ition table
+        self.fixedali= gn_part =3D 0 # Number of partitions with absolute alignment
 &n= bsp;      self.primary_part_num =3D 0  #= Number of primary partitions (msdos)
     &= nbsp;  self.extendedpart =3D 0      # Cr= eate extended partition before this logical partition (msdos)
 &= nbsp;      self.extended_size_sec =3D 0 # Siz= e of exteded partition (msdos)
@@ -311,11 +326,13 @@ class Partitione= dImage():
        self.min_si= ze =3D 0 # Minimum required disk size to fit
    =             &nb= sp;         # all partitions (= in bytes)
        self.ptable= _format =3D ptable_format  # Partition table format
+  &nbs= p;     self.main_ptable_offset =3D main_ptable_off= set # for GPT table, offset
       = ; # Disk system identifier
      &= nbsp; self.identifier =3D random.SystemRandom().randint(1, 0xffffffff)=
 
        self.par= titions =3D partitions
       &nbs= p;self.partimages =3D []
+        = self.used_memory_region =3D []
      &n= bsp; # Size of a sector used in calculations
   &= nbsp;    self.sector_size =3D SECTOR_SIZE
 &= nbsp;      self.native_sysroot =3D native_sys= root
@@ -368,6 +385,30 @@ class PartitionedImage():
  = ;          # Converting k= B to sectors for parted
       &nb= sp;    part.size_sec =3D part.disk_size * 1024 // self.= sector_size
 
+    def _add_busy_region(self= , part_num, start_sec, end_sec):
+      &nbs= p; self.used_memory_region.append(memoryRegion(part_num, start_sec, en= d_sec))
+        self.used_memory_= region =3D sorted(self.used_memory_region, key=3Doperator.attrgetter('start= _sector'))
+
+    def _print_busy_region_list(se= lf):
+        list =3D "Busy memor= y region (in sector)\n"
+        l= ist +=3D "\tpart#\tstart\tEnd\n"
+      &nbs= p; for num in range(len(self.used_memory_region)):
+   = ;         list +=3D"\t%s\t%s\t= %s\n" % (self.used_memory_region[num].part_num, \
+    = ;            se= lf.used_memory_region[num].start_sector, self.used_memory_region[num].end_s= ector)
+        logger.debug(list)=
+
+    def _check_memory_region(self, start_sec= , end_sec ):
+        """ Check if= the specified region is already flagges as used. Returns
+  &nb= sp;     zero if not used, 1 otherwise"""
+ &= nbsp;      for num in range(len(self.used_mem= ory_region)):
+          = ;  region =3D self.used_memory_region[num]
+   &n= bsp;        if ( ( region.start_sec= tor < start_sec and start_sec < region.end_sector) or \
+  = ;            &n= bsp;     ( region.start_sector < end_sec and en= d_sec < region.end_sector) ):
+      &nbs= p;         return 1
+ &n= bsp;          if ( start_= sec < region.start_sector and end_sec > region.end_sector ):
+ =             &nb= sp;  return 1
+        r= eturn 0
+
    def layout_partitions(self): =
        """ Layout the partit= ions, meaning calculate the position of every
    = ;    partition on the disk. The 'ptable_format' paramet= er defines the
@@ -379,6 +420,35 @@ class PartitionedImage():
&= nbsp;       # partitions not listed in t= he table are not included.
       =  num_real_partitions =3D len([p for p in self.partitions if not p.no_t= able])
 
+        # The = partitions flagged as fixed alignment partition are placed into image
+        # before the others. Instead, = relative-aligned partitions are placed in a
+     = ;   # serialized fashion, with control of the intersection o= f memory areas.
+        # The bas= e address of the relative-aligned partitions is, as before,
+  &= nbsp;     # the maximum address accupied by the pa= rtition table
+
+        # F= lag as used the momoery required for partition table
+   &n= bsp;    self._add_busy_region(-1, 0, MBR_OVERHEAD)
+        if self.ptable_format =3D=3D = "gpt":
+           =  offset =3D self.main_ptable_offset * 1024 // self.sector_size
+=            self._ad= d_busy_region(-1, offset, offset + GPT_OVERHEAD)
+
+  &nbs= p;     # Search all partition with fixed position = into the image
+        for num in= range(len(self.partitions)):
+       &= nbsp;    part =3D self.partitions[num]
+  &n= bsp;         if not part.fixed= _align:
+           = ;     continue
+     &nb= sp;      if not part.no_table:
+  =             &nb= sp; raise WicError("A partition with fixed alignment must have no_tabl= e flag set")
+
+        &nbs= p;   start_sector =3D (part.align * 1024 // self.sector_size= )
+            = ;end_sector =3D start_sector + part.size_sec - 1
+    =         if self._check_memory_regio= n(start_sector, end_sector) =3D=3D 1:
+      = ;          self._print_bu= sy_region_list()
+         &n= bsp;      raise WicError("A partition wants t= o use an already used memory region (sectors %d - %d)" \
+  &nbs= p;            &= nbsp;        % (start_sector, end_s= ector))
+           = ; self._add_busy_region(num, start_sector, end_sector)
+  &= nbsp;         part.start =3D s= tart_sector
+
        #= Go through partitions in the order they are added in .ks file
 =        for num in range(len(self.partiti= ons)):
          &n= bsp; part =3D self.partitions[num]
@@ -403,7 +473,7 @@ class Par= titionedImage():
        &nbs= p;       if self.ptable_format =3D=3D "m= sdos":
          &n= bsp;         overhead =3D MBR_= OVERHEAD
          =       elif self.ptable_format =3D=3D "gpt": <= br />-            &n= bsp;       overhead =3D GPT_OVERHEAD +            &nbs= p;       overhead =3D (self.main_ptable_= offset * 1024 // self.sector_size) + GPT_OVERHEAD
 
 &= nbsp;           &nbs= p;  # Skip one sector required for the partitioning scheme overhe= ad
           =      self.offset +=3D overhead
@@ -418,7 +48= 8,7 @@ class PartitionedImage():
      =             &nb= sp; self.offset +=3D 2
 
     = ;       align_sectors =3D 0
- &nbs= p;          if part.align= :
+            = ;if not part.fixed_align and part.align:
    &nbs= p;           # If no= t first partition and we do have alignment set we need
  &n= bsp;            = ; # to align the partition.
      =           # FIXME: This l= eaves a empty spaces to the disk. To fill the
@@ -440,7 +510,7 @@ cla= ss PartitionedImage():
       &nbs= p;            #= increase the offset so we actually start the partition on right alignment =
           &nb= sp;        self.offset +=3D align_s= ectors
 
-         =    if part.offset is not None:
+    &nb= sp;       if not part.fixed_align and pa= rt.offset is not None:
       &nbs= p;        offset =3D part.offset //= self.sector_size
 
      &nb= sp;         if offset * self.s= ector_size !=3D part.offset:
@@ -455,14 +525,20 @@ class PartitionedI= mage():
 
        &= nbsp;       self.offset =3D offset
 
-           =  part.start =3D self.offset
-      &nbs= p;     self.offset +=3D part.size_sec
-
            i= f not part.no_table:
        =         part.num =3D self.realpart =
           &nb= sp;else:
          =       part.num =3D 0
 
+ &nbs= p;          if not part.f= ixed_align:
+          &= nbsp;     part.start =3D self.offset
+  = ;            &n= bsp; self.offset +=3D part.size_sec
+     &n= bsp;          if self._ch= eck_memory_region(part.start, self.offset) =3D=3D 1:
+   &n= bsp;            = ;    self._print_busy_region_list()
+   = ;            &n= bsp;    raise WicError("A partition wants to use an alr= eady used memory region (sectors %d - %d)" \
+    &nbs= p;            &= nbsp;      % (part.start, self.offset))
+             =    self._add_busy_region(part.num, part.start, self.offset) =
+
          &= nbsp; if self.ptable_format =3D=3D "msdos" and not part.no_table:
            &= nbsp;   if part.type =3D=3D 'logical':
  &nb= sp;            =      self.logical_part_cnt +=3D 1
@@ -482,12= +558,14 @@ class PartitionedImage():
     &= nbsp;           &nbs= p;       "sectors (%d bytes).", part.mou= ntpoint, part.disk,
        &= nbsp;           &nbs= p;    part.num, part.start, self.offset - 1, part.size_= sec,
          &nbs= p;            &= nbsp; part.size_sec * self.sector_size)
+    &nbs= p;        
+    = ;    self._print_busy_region_list()
 
&= nbsp;       # Once all the partitions ha= ve been layed out, we can calculate the
     = ;   # minumim disk size
     =    self.min_size =3D self.offset
   &nb= sp;    if self.ptable_format =3D=3D "gpt":
- &nbs= p;          self.min_size= +=3D GPT_OVERHEAD
+         =    self.min_size +=3D (self.main_ptable_offset * 1024 // sel= f.sector_size) + GPT_OVERHEAD
 
    &nb= sp;   self.min_size *=3D self.sector_size
  =       self.min_size +=3D self.extra_space @@ -606,6 +684,14 @@ class PartitionedImage():
   &= nbsp;           &nbs= p;exec_native_cmd("sfdisk --part-type %s %s %s" % \
   = ;            &n= bsp;            = ;    (self.path, part.num, part.system_id),
 = ;            &n= bsp;            = ;      self.native_sysroot)
+  &nb= sp;            =   
+        if self.ptabl= e_format =3D=3D "gpt" and self.main_ptable_offset > 0:
+  &nb= sp;         main_ptable_sector= s =3D self.main_ptable_offset * 1024 // self.sector_size
+  &nbs= p;         logger.debug("Move = the main GPT partition table forward by %s sector(s)", main_ptable_sectors)=
+            = cmd =3D "sgdisk -j %d %s" % (main_ptable_sectors, self.path)
+  =           #cmd =3D "(echo= -e 'x'; echo -e 'j' ; echo -e '%d'; echo -e 'w'; echo -e 'Y') | gdisk %s" = % (main_ptable_sectors, self.path)
+      &n= bsp;     exec_native_cmd(cmd, self.native_sysroot)=
+
 
    def cleanup(self):
        pass
diff --git a/sc= ripts/wic b/scripts/wic
index 06e0b48db0..0c59a550ef 100755
---= a/scripts/wic
+++ b/scripts/wic
@@ -59,7 +59,7 @@ from wic imp= ort help as hlp
 
 
def wic_logger():
- &nb= sp;  """Create and convfigure wic logger."""
+   =  """Create and configure wic logger."""
    = logger =3D logging.getLogger('wic')
    logger.se= tLevel(logging.INFO)
 
--  
2.39.2
 --h5EEC8WXheWC5siKFw19--