[PATCH 0/3] arm64: Add a build target for Flat Image Tree

[PATCH 0/3] arm64: Add a build target for Flat Image Tree

[Simon Glass]

Flat Image Tree (FIT) is a widely used file format for packaging a
kernel and associated devicetree files. This series adds support for
building a FIT as part of the kernel build. This makes it easy to try
out the kernel - just load the FIT onto your tftp server and it will
run automatically on any supported arm64 board.

The script is written in Python, since it is easy to build a FIT using
the Python libfdt bindings. For now, no attempt is made to compress
files in parallel, so build the 900-odd files takes a while, about
6 seconds with my testing.

The series also includes a few minor clean-up patches.


Simon Glass (3):
  kbuild: Correct missing architecture-specific hyphens
  kbuild: arm64: Add BOOT_TARGETS variable
  arm64: boot: Support Flat Image Tree

 MAINTAINERS              |   7 +
 Makefile                 |  10 +-
 arch/arm64/Makefile      |   7 +-
 arch/arm64/boot/Makefile |   8 +-
 scripts/Makefile.lib     |  16 ++-
 scripts/make_fit.py      | 285 +++++++++++++++++++++++++++++++++++++++
 6 files changed, 324 insertions(+), 9 deletions(-)
 create mode 100755 scripts/make_fit.py

-- 
2.42.0.758.gaed0368e0e-goog


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 1/3] kbuild: Correct missing architecture-specific hyphens

[Simon Glass]

These should add a hyphen to indicate that it makes a adjective. Fix
them.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 Makefile | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/Makefile b/Makefile
index 690cde550acd..b63796e6f94a 100644
--- a/Makefile
+++ b/Makefile
@@ -675,7 +675,7 @@ ifdef config-build
 # *config targets only - make sure prerequisites are updated, and descend
 # in scripts/kconfig to make the *config target
 
-# Read arch specific Makefile to set KBUILD_DEFCONFIG as needed.
+# Read arch-specific Makefile to set KBUILD_DEFCONFIG as needed.
 # KBUILD_DEFCONFIG may point out an alternative default configuration
 # used for 'make defconfig'
 include $(srctree)/arch/$(SRCARCH)/Makefile
@@ -689,7 +689,7 @@ config: outputmakefile scripts_basic FORCE
 
 else #!config-build
 # ===========================================================================
-# Build targets only - this includes vmlinux, arch specific targets, clean
+# Build targets only - this includes vmlinux, arch-specific targets, clean
 # targets and others. In general all targets except *config targets.
 
 # If building an external module we do not care about the all: rule
@@ -1637,9 +1637,9 @@ help:
 	@echo  'Documentation targets:'
 	@$(MAKE) -f $(srctree)/Documentation/Makefile dochelp
 	@echo  ''
-	@echo  'Architecture specific targets ($(SRCARCH)):'
+	@echo  'Architecture-specific targets ($(SRCARCH)):'
 	@$(or $(archhelp),\
-		echo '  No architecture specific help defined for $(SRCARCH)')
+		echo '  No architecture-specific help defined for $(SRCARCH)')
 	@echo  ''
 	@$(if $(boards), \
 		$(foreach b, $(boards), \
@@ -1681,7 +1681,7 @@ help-boards: $(help-board-dirs)
 boards-per-dir = $(sort $(notdir $(wildcard $(srctree)/arch/$(SRCARCH)/configs/$*/*_defconfig)))
 
 $(help-board-dirs): help-%:
-	@echo  'Architecture specific targets ($(SRCARCH) $*):'
+	@echo  'Architecture-specific targets ($(SRCARCH) $*):'
 	@$(if $(boards-per-dir), \
 		$(foreach b, $(boards-per-dir), \
 		printf "  %-24s - Build for %s\\n" $*/$(b) $(subst _defconfig,,$(b));) \
-- 
2.42.0.758.gaed0368e0e-goog


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 1/3] kbuild: Correct missing architecture-specific hyphens

[Masahiro Yamada]

On Thu, Oct 26, 2023 at 4:27 PM Simon Glass <sjg@chromium.org> wrote:
>
> These should add a hyphen to indicate that it makes a adjective. Fix
> them.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---


This is trivial.
Applied to linux-kbuild. Thanks.


git grep -i 'arch specific'

 or

git grep -i 'architecture specific'

finds similar patterns, but presumably we are not
keen on fixing them tree-wide.





-- 
Best Regards
Masahiro Yamada

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 1/3] kbuild: Correct missing architecture-specific hyphens

[Randy Dunlap]

On 10/28/23 02:50, Masahiro Yamada wrote:
> On Thu, Oct 26, 2023 at 4:27 PM Simon Glass <sjg@chromium.org> wrote:
>>
>> These should add a hyphen to indicate that it makes a adjective. Fix
>> them.
>>
>> Signed-off-by: Simon Glass <sjg@chromium.org>
>> ---
> 
> 
> This is trivial.
> Applied to linux-kbuild. Thanks.
> 
> 
> git grep -i 'arch specific'
> 
>  or
> 
> git grep -i 'architecture specific'
> 
> finds similar patterns, but presumably we are not
> keen on fixing them tree-wide.

or '32 bit', '64 bit', but I agree with "not keen on
fixing them tree-wide."


-- 
~Randy

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 2/3] kbuild: arm64: Add BOOT_TARGETS variable

[Simon Glass]

Add a new variable containing a list of possible targets. Mark them as
phony. This matches the approach taken for arch/arm

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 arch/arm64/Makefile | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 4bd85cc0d32b..7b77b63e978f 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -150,6 +150,10 @@ libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
 # Default target when executing plain make
 boot		:= arch/arm64/boot
 
+BOOT_TARGETS	:= Image.gz vmlinuz.efi
+
+PHONY += $(BOOT_TARGETS)
+
 ifeq ($(CONFIG_EFI_ZBOOT),)
 KBUILD_IMAGE	:= $(boot)/Image.gz
 else
@@ -159,7 +163,7 @@ endif
 all:	$(notdir $(KBUILD_IMAGE))
 
 
-Image vmlinuz.efi: vmlinux
+$(BOOT_TARGETS): vmlinux
 	$(Q)$(MAKE) $(build)=$(boot) $(boot)/$@
 
 Image.%: Image
-- 
2.42.0.758.gaed0368e0e-goog


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 3/3] arm64: boot: Support Flat Image Tree

[Simon Glass]

Add a script which produces a Flat Image Tree (FIT), a single file
containing the built kernel and associated devicetree files.
Compression defaults to gzip which gives a good balance of size and
performance.

The files compress from about 85MB to 24MB using this approach.

The FIT can be used by bootloaders which support it, such as U-Boot
and Linuxboot. It permits automatic selection of the correct
devicetree, matching the compatible string of the running board with
the closest compatible string in the FIT. There is no need for
filenames or other workarounds.

Add a 'make image.fit' build target for arm64, as well.

The FIT can be examined using 'dumpimage -l'.

This features requires pylibfdt (use 'pip install libfdt'). It also
requires compression utilities for the algorithm being used. Supported
compression options are the same as the Image.xxx files. For now there
is no way to change the compression other than by editing the rule for
$(obj)/image.fit

While FIT supports a ramdisk / initrd, no attempt is made to support
this here, since it must be built separately from the Linux build.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 MAINTAINERS              |   7 +
 arch/arm64/Makefile      |   3 +-
 arch/arm64/boot/Makefile |   8 +-
 scripts/Makefile.lib     |  16 ++-
 scripts/make_fit.py      | 285 +++++++++++++++++++++++++++++++++++++++
 5 files changed, 315 insertions(+), 4 deletions(-)
 create mode 100755 scripts/make_fit.py

diff --git a/MAINTAINERS b/MAINTAINERS
index 2d13bbd69adb..d6955ebc3c24 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1575,6 +1575,13 @@ F:	Documentation/process/maintainer-soc*.rst
 F:	arch/arm/boot/dts/Makefile
 F:	arch/arm64/boot/dts/Makefile
 
+ARM64 FIT SUPPORT
+M:	Simon Glass <sjg@chromium.org>
+L:	linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
+S:	Maintained
+F:	arch/arm64/boot/Makefile
+F:	scripts/make_fit.py
+
 ARM ARCHITECTED TIMER DRIVER
 M:	Mark Rutland <mark.rutland@arm.com>
 M:	Marc Zyngier <maz@kernel.org>
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 7b77b63e978f..d8290dcab6b6 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -150,7 +150,7 @@ libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
 # Default target when executing plain make
 boot		:= arch/arm64/boot
 
-BOOT_TARGETS	:= Image.gz vmlinuz.efi
+BOOT_TARGETS	:= Image.gz vmlinuz.efi image.fit
 
 PHONY += $(BOOT_TARGETS)
 
@@ -215,6 +215,7 @@ virtconfig:
 define archhelp
   echo  '* Image.gz      - Compressed kernel image (arch/$(ARCH)/boot/Image.gz)'
   echo  '  Image         - Uncompressed kernel image (arch/$(ARCH)/boot/Image)'
+$(if $(CONFIG_EFI_ZBOOT),,echo  '  image.fit     - Flat Image Tree (arch/$(ARCH)/boot/image.fit)')
   echo  '  install       - Install uncompressed kernel'
   echo  '  zinstall      - Install compressed kernel'
   echo  '                  Install using (your) ~/bin/installkernel or'
diff --git a/arch/arm64/boot/Makefile b/arch/arm64/boot/Makefile
index 1761f5972443..a6e5b20b22bd 100644
--- a/arch/arm64/boot/Makefile
+++ b/arch/arm64/boot/Makefile
@@ -16,7 +16,8 @@
 
 OBJCOPYFLAGS_Image :=-O binary -R .note -R .note.gnu.build-id -R .comment -S
 
-targets := Image Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo Image.zst
+targets := Image Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo \
+	Image.zst image.fit
 
 $(obj)/Image: vmlinux FORCE
 	$(call if_changed,objcopy)
@@ -39,6 +40,11 @@ $(obj)/Image.lzo: $(obj)/Image FORCE
 $(obj)/Image.zst: $(obj)/Image FORCE
 	$(call if_changed,zstd)
 
+ifndef CONFIG_EFI_ZBOOT
+$(obj)/image.fit: $(obj)/Image $(obj)/dts FORCE
+	$(call if_changed,fit,gzip)
+endif
+
 EFI_ZBOOT_PAYLOAD	:= Image
 EFI_ZBOOT_BFD_TARGET	:= elf64-littleaarch64
 EFI_ZBOOT_MACH_TYPE	:= ARM64
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 68d0134bdbf9..4e4364ad641a 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -487,14 +487,26 @@ UIMAGE_OPTS-y ?=
 UIMAGE_TYPE ?= kernel
 UIMAGE_LOADADDR ?= arch_must_set_this
 UIMAGE_ENTRYADDR ?= $(UIMAGE_LOADADDR)
-UIMAGE_NAME ?= 'Linux-$(KERNELRELEASE)'
+UIMAGE_NAME ?= "Linux-$(KERNELRELEASE)"
 
 quiet_cmd_uimage = UIMAGE  $@
       cmd_uimage = $(BASH) $(MKIMAGE) -A $(UIMAGE_ARCH) -O linux \
 			-C $(UIMAGE_COMPRESSION) $(UIMAGE_OPTS-y) \
 			-T $(UIMAGE_TYPE) \
 			-a $(UIMAGE_LOADADDR) -e $(UIMAGE_ENTRYADDR) \
-			-n $(UIMAGE_NAME) -d $< $@
+			-n "$(UIMAGE_NAME)" -d $< $@
+
+# Flat Image Tree (FIT)
+# This allows for packaging of a kernel and all devicetrees files, using
+# compression.
+# ---------------------------------------------------------------------------
+
+MAKE_FIT := $(srctree)/scripts/make_fit.py
+
+quiet_cmd_fit = FIT     $@
+      cmd_fit = $(MAKE_FIT) -f $@ --arch $(UIMAGE_ARCH) --os linux \
+			--name "$(UIMAGE_NAME)" \
+			--compress $(UIMAGE_COMPRESSION) $^
 
 # XZ
 # ---------------------------------------------------------------------------
diff --git a/scripts/make_fit.py b/scripts/make_fit.py
new file mode 100755
index 000000000000..07500d8638e9
--- /dev/null
+++ b/scripts/make_fit.py
@@ -0,0 +1,285 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0+
+#
+# Copyright 2023 Google LLC
+# Written by Simon Glass <sjg@chromium.org>
+#
+
+"""Build a FIT containing a lot of devicetree files
+
+Usage:
+    make_fit.py -A arm64 -n 'Linux-6.6' -O linux
+        -f arch/arm64/boot/image.fit /tmp/kern/arch/arm64/boot/Image
+        /tmp/kern/arch/arm64/boot/dts/ -E -c gzip
+
+Creates a FIT containing the supplied list of files. The kernel must
+be specified first. The DTB files are specified by way of a directory
+tree containing them.
+
+Use -E to generate an external FIT (where the data is placed after the
+FIT data structure). This allows parsing of the data without loading
+the entire FIT.
+
+Use -c to compress the data, using bzip2, gzip, lz4, lzma, lzo and
+zstd
+
+The resulting FIT can be booted by bootloaders which support FIT, such
+as U-Boot, Linuxboot, Tianocore, etc.
+
+Note that this tool does not yet support adding a ramdisk / initrd.
+"""
+
+import argparse
+import collections
+import os
+import subprocess
+import sys
+import tempfile
+import time
+
+import libfdt
+
+
+# Tool extension and the name of the command-line tools
+CompTool = collections.namedtuple('CompTool', 'ext,tools')
+
+COMP_TOOLS = {
+    'bzip2': CompTool('.bz2', 'bzip2'),
+    'gzip': CompTool('.gz', 'pigz,gzip'),
+    'lz4': CompTool('.lz4', 'lz4'),
+    'lzma': CompTool('.lzma', 'lzma'),
+    'lzo': CompTool('.lzo', 'lzop'),
+    'zstd': CompTool('.zstd', 'zstd'),
+}
+
+def parse_args():
+    """Parse the program ArgumentParser
+
+    Returns:
+        Namespace object containing the arguments
+    """
+    epilog = 'Build a FIT from a directory tree containing .dtb files'
+    parser = argparse.ArgumentParser(epilog=epilog)
+    parser.add_argument('-A', '--arch', type=str, required=True,
+          help='Specifies the architecture')
+    parser.add_argument('-c', '--compress', type=str, default='none',
+          help='Specifies the compression')
+    parser.add_argument('-E', '--external', action='store_true',
+          help='Convert the FIT to use external data')
+    parser.add_argument('-n', '--name', type=str, required=True,
+          help='Specifies the name')
+    parser.add_argument('-O', '--os', type=str, required=True,
+          help='Specifies the operating system')
+    parser.add_argument('-f', '--fit', type=str, required=True,
+          help='Specifies the output file (.fit)')
+    parser.add_argument('srcdir', type=str, nargs='*',
+          help='Specifies the directory tree that contains .dtb files')
+
+    return parser.parse_args()
+
+def setup_fit(fsw):
+    """Make a start on writing the FIT
+
+    Outputs the root properties and the 'images' node
+
+    Args:
+        fsw (libfdt.FdtSw): Object to use for writing
+    """
+    fsw.INC_SIZE = 65536
+    fsw.finish_reservemap()
+    fsw.begin_node('')
+    fsw.property_string('description', 'DTB set')
+    fsw.property_u32('#address-cells', 1)
+
+    fsw.property_u32('timestamp', int(time.time()))
+    fsw.begin_node('images')
+
+
+def write_kernel(fsw, data, arch, fit_os, name, compress):
+    """Write out the kernel image
+
+    Writes a kernel node along with the required properties
+
+    Args:
+        fsw (libfdt.FdtSw): Object to use for writing
+        data (bytes): Data to write (possibly compressed)
+        arch (str): FIT architecture, e.g. 'arm64'
+        fit_os (str): Operating Systems, e.g. 'linux'
+        name (str): Name of OS, e.g. 'Linux-6.6.0-rc7'
+        compress (str): Compression algorithm to use, e.g. 'gzip'
+    """
+    with fsw.add_node('kernel'):
+        fsw.property_string('description', name)
+        fsw.property_string('type', 'kernel_noload')
+        fsw.property_string('arch', arch)
+        fsw.property_string('os', fit_os)
+        fsw.property_string('compression', compress)
+        fsw.property('data', data)
+        fsw.property_u32('load', 0)
+        fsw.property_u32('entry', 0)
+
+
+def finish_fit(fsw, entries):
+    """Finish the FIT ready for use
+
+    Writes the /configurations node and subnodes
+
+    Args:
+        fsw (libfdt.FdtSw): Object to use for writing
+        entries (list of tuple): List of configurations:
+            str: Description of model
+            str: Compatible stringlist
+    """
+    fsw.end_node()
+    seq = 0
+    with fsw.add_node('configurations'):
+        for model, compat in entries:
+            seq += 1
+            with fsw.add_node(f'conf-{seq}'):
+                fsw.property('compatible', bytes(compat))
+                fsw.property_string('description', model)
+                fsw.property_string('fdt', f'fdt-{seq}')
+                fsw.property_string('kernel', 'kernel')
+    fsw.end_node()
+
+
+def compress_data(inf, compress):
+    """Compress data using a selected algorithm
+
+    Args:
+        inf (IOBase): Filename containing the data to compress
+        compress (str): Compression algorithm, e.g. 'gzip'
+
+    Return:
+        bytes: Compressed data
+    """
+    if compress == 'none':
+        return inf.read()
+
+    comp = COMP_TOOLS.get(compress)
+    if not comp:
+        raise ValueError(f"Unknown compression algorithm '{compress}'")
+
+    with tempfile.NamedTemporaryFile() as comp_fname:
+        with open(comp_fname.name, 'wb') as outf:
+            done = False
+            for tool in comp.tools.split(','):
+                try:
+                    subprocess.call([tool, '-c'], stdin=inf, stdout=outf)
+                    done = True
+                    break
+                except FileNotFoundError:
+                    pass
+            if not done:
+                raise ValueError(f'Missing tool(s): {comp.tools}\n')
+            with open(comp_fname.name, 'rb') as compf:
+                comp_data = compf.read()
+    return comp_data
+
+
+def output_dtb(fsw, seq, fname, compress):
+    """Write out a single devicetree to the FIT
+
+    Args:
+        fsw (libfdt.FdtSw): Object to use for writing
+        seq (int): Sequence number (1 for first)
+        fmame (str): Filename containing the DTB
+        compress (str): Compressed algorithm, e.g. 'gzip'
+
+    Returns:
+        tuple:
+            str: Model name
+            bytes: Compatible stringlist
+    """
+    with fsw.add_node(f'fdt-{seq}'):
+        # Get the compatible / model information
+        with open(fname, 'rb') as inf:
+            data = inf.read()
+        fdt = libfdt.FdtRo(data)
+        model = fdt.getprop(0, 'model').as_str()
+        compat = fdt.getprop(0, 'compatible')
+
+        fsw.property_string('description', model)
+        fsw.property_string('type', 'flat_dt')
+        fsw.property_string('arch', 'arm64')
+        fsw.property_string('compression', compress)
+        fsw.property('compatible', bytes(compat))
+
+        with open(fname, 'rb') as inf:
+            compressed = compress_data(inf, compress)
+        fsw.property('data', compressed)
+    return model, compat
+
+
+def build_fit(args):
+    """Build the FIT from the provided files and arguments
+
+    Args:
+        args: Namespace object containing program arguments
+
+    Returns:
+        tuple:
+            bytes: FIT data
+            int: Number of configurations generated
+            size: Total uncompressed size of data
+    """
+    fsw = libfdt.FdtSw()
+    setup_fit(fsw)
+    seq = 0
+    size = 0
+    entries = []
+    for path in args.srcdir:
+        # Handle devicetree files
+        if os.path.isdir(path):
+            for dirpath, _, fnames in os.walk(path):
+                for fname in fnames:
+                    if os.path.splitext(fname)[1] != '.dtb':
+                        continue
+                    pathname = os.path.join(dirpath, fname)
+                    seq += 1
+                    size += os.path.getsize(pathname)
+                    model, compat = output_dtb(fsw, seq, pathname,
+                                               args.compress)
+                    entries.append([model, compat])
+
+        # Handle the kernel
+        elif path != 'FORCE':
+            with open(path, 'rb') as inf:
+                comp_data = compress_data(inf, args.compress)
+            size += os.path.getsize(path)
+            write_kernel(fsw, comp_data, args.arch, args.os,
+                            args.name, args.compress)
+
+    finish_fit(fsw, entries)
+
+    return fsw.as_fdt().as_bytearray(), seq, size
+
+
+def run_make_fit():
+    """Run the tool's main logic"""
+    args = parse_args()
+
+    out_data, count, size = build_fit(args)
+    with open(args.fit, 'wb') as outf:
+        outf.write(out_data)
+
+    ext_fit_size = None
+    if args.external:
+        mkimage = os.environ.get('MKIMAGE', 'mkimage')
+        subprocess.check_call([mkimage, '-E', '-F', args.fit],
+                              stdout=subprocess.DEVNULL)
+
+        with open(args.fit, 'rb') as inf:
+            data = inf.read()
+        ext_fit = libfdt.FdtRo(data)
+        ext_fit_size = ext_fit.totalsize()
+
+    comp_size = len(out_data)
+    print(f'FIT size {comp_size:#x}/{comp_size / 1024 / 1024:.1f} MB', end='')
+    if ext_fit_size:
+        print(f', header {ext_fit_size:#x}/{ext_fit_size / 1024:.1f} KB', end='')
+    print(f', {count} files, uncompressed {size / 1024 / 1024:.1f} MB')
+
+
+if __name__ == "__main__":
+    sys.exit(run_make_fit())
-- 
2.42.0.758.gaed0368e0e-goog


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 3/3] arm64: boot: Support Flat Image Tree

[Masahiro Yamada]

On Thu, Oct 26, 2023 at 4:28 PM Simon Glass <sjg@chromium.org> wrote:
>
> Add a script which produces a Flat Image Tree (FIT), a single file
> containing the built kernel and associated devicetree files.
> Compression defaults to gzip which gives a good balance of size and
> performance.
>
> The files compress from about 85MB to 24MB using this approach.
>
> The FIT can be used by bootloaders which support it, such as U-Boot
> and Linuxboot. It permits automatic selection of the correct
> devicetree, matching the compatible string of the running board with
> the closest compatible string in the FIT. There is no need for
> filenames or other workarounds.
>
> Add a 'make image.fit' build target for arm64, as well.
>
> The FIT can be examined using 'dumpimage -l'.
>
> This features requires pylibfdt (use 'pip install libfdt'). It also
> requires compression utilities for the algorithm being used. Supported
> compression options are the same as the Image.xxx files. For now there
> is no way to change the compression other than by editing the rule for
> $(obj)/image.fit
>
> While FIT supports a ramdisk / initrd, no attempt is made to support
> this here, since it must be built separately from the Linux build.


Is this useful?
For arm64, initrd is likely used.

FIT should be created in unbrellea projects
such as OpenEmbedded,  Buildroot, etc.




> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
>
>  MAINTAINERS              |   7 +
>  arch/arm64/Makefile      |   3 +-
>  arch/arm64/boot/Makefile |   8 +-
>  scripts/Makefile.lib     |  16 ++-
>  scripts/make_fit.py      | 285 +++++++++++++++++++++++++++++++++++++++
>  5 files changed, 315 insertions(+), 4 deletions(-)
>  create mode 100755 scripts/make_fit.py
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 2d13bbd69adb..d6955ebc3c24 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1575,6 +1575,13 @@ F:       Documentation/process/maintainer-soc*.rst
>  F:     arch/arm/boot/dts/Makefile
>  F:     arch/arm64/boot/dts/Makefile
>
> +ARM64 FIT SUPPORT
> +M:     Simon Glass <sjg@chromium.org>
> +L:     linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
> +S:     Maintained
> +F:     arch/arm64/boot/Makefile
> +F:     scripts/make_fit.py
> +
>  ARM ARCHITECTED TIMER DRIVER
>  M:     Mark Rutland <mark.rutland@arm.com>
>  M:     Marc Zyngier <maz@kernel.org>
> diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
> index 7b77b63e978f..d8290dcab6b6 100644
> --- a/arch/arm64/Makefile
> +++ b/arch/arm64/Makefile
> @@ -150,7 +150,7 @@ libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
>  # Default target when executing plain make
>  boot           := arch/arm64/boot
>
> -BOOT_TARGETS   := Image.gz vmlinuz.efi
> +BOOT_TARGETS   := Image.gz vmlinuz.efi image.fit
>
>  PHONY += $(BOOT_TARGETS)
>
> @@ -215,6 +215,7 @@ virtconfig:
>  define archhelp
>    echo  '* Image.gz      - Compressed kernel image (arch/$(ARCH)/boot/Image.gz)'
>    echo  '  Image         - Uncompressed kernel image (arch/$(ARCH)/boot/Image)'
> +$(if $(CONFIG_EFI_ZBOOT),,echo  '  image.fit     - Flat Image Tree (arch/$(ARCH)/boot/image.fit)')
>    echo  '  install       - Install uncompressed kernel'
>    echo  '  zinstall      - Install compressed kernel'
>    echo  '                  Install using (your) ~/bin/installkernel or'
> diff --git a/arch/arm64/boot/Makefile b/arch/arm64/boot/Makefile
> index 1761f5972443..a6e5b20b22bd 100644
> --- a/arch/arm64/boot/Makefile
> +++ b/arch/arm64/boot/Makefile
> @@ -16,7 +16,8 @@
>
>  OBJCOPYFLAGS_Image :=-O binary -R .note -R .note.gnu.build-id -R .comment -S
>
> -targets := Image Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo Image.zst
> +targets := Image Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo \
> +       Image.zst image.fit
>
>  $(obj)/Image: vmlinux FORCE
>         $(call if_changed,objcopy)
> @@ -39,6 +40,11 @@ $(obj)/Image.lzo: $(obj)/Image FORCE
>  $(obj)/Image.zst: $(obj)/Image FORCE
>         $(call if_changed,zstd)
>
> +ifndef CONFIG_EFI_ZBOOT
> +$(obj)/image.fit: $(obj)/Image $(obj)/dts FORCE



This is wrong.

$(obj)/dts is a directory.

There is no point for comparing timestamps
between $(obj)/image.fit and $(obj)/dts.

Updates of *.dtb do not result in the
update of the $(obj)/dts timestamp.


if_changed never works correctly.








> +       $(call if_changed,fit,gzip)
> +endif
> +
>  EFI_ZBOOT_PAYLOAD      := Image
>  EFI_ZBOOT_BFD_TARGET   := elf64-littleaarch64
>  EFI_ZBOOT_MACH_TYPE    := ARM64
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index 68d0134bdbf9..4e4364ad641a 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -487,14 +487,26 @@ UIMAGE_OPTS-y ?=
>  UIMAGE_TYPE ?= kernel
>  UIMAGE_LOADADDR ?= arch_must_set_this
>  UIMAGE_ENTRYADDR ?= $(UIMAGE_LOADADDR)
> -UIMAGE_NAME ?= 'Linux-$(KERNELRELEASE)'
> +UIMAGE_NAME ?= "Linux-$(KERNELRELEASE)"


Unneeded noise change.







--
Best Regards

Masahiro Yamada

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 3/3] arm64: boot: Support Flat Image Tree

[Simon Glass]

Hi Masahiro,

On Fri, Oct 27, 2023 at 2:25 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> On Thu, Oct 26, 2023 at 4:28 PM Simon Glass <sjg@chromium.org> wrote:
> >
> > Add a script which produces a Flat Image Tree (FIT), a single file
> > containing the built kernel and associated devicetree files.
> > Compression defaults to gzip which gives a good balance of size and
> > performance.
> >
> > The files compress from about 85MB to 24MB using this approach.
> >
> > The FIT can be used by bootloaders which support it, such as U-Boot
> > and Linuxboot. It permits automatic selection of the correct
> > devicetree, matching the compatible string of the running board with
> > the closest compatible string in the FIT. There is no need for
> > filenames or other workarounds.
> >
> > Add a 'make image.fit' build target for arm64, as well.
> >
> > The FIT can be examined using 'dumpimage -l'.
> >
> > This features requires pylibfdt (use 'pip install libfdt'). It also
> > requires compression utilities for the algorithm being used. Supported
> > compression options are the same as the Image.xxx files. For now there
> > is no way to change the compression other than by editing the rule for
> > $(obj)/image.fit
> >
> > While FIT supports a ramdisk / initrd, no attempt is made to support
> > this here, since it must be built separately from the Linux build.
>
>
> Is this useful?
> For arm64, initrd is likely used.

It depends on your kernel configuration, though. I have found it
convenient to enable the drivers necessary to boot.

>
> FIT should be created in unbrellea projects
> such as OpenEmbedded,  Buildroot, etc.

Sure, but that is a separate issue, isn't it? We already support
various boot targets in arm64 but not one that includes the DTs, so
far as I can see. The old arm 'uImage' target is pretty out-of-date
now.

>
>
>
>
> > Signed-off-by: Simon Glass <sjg@chromium.org>
> > ---
> >
> >  MAINTAINERS              |   7 +
> >  arch/arm64/Makefile      |   3 +-
> >  arch/arm64/boot/Makefile |   8 +-
> >  scripts/Makefile.lib     |  16 ++-
> >  scripts/make_fit.py      | 285 +++++++++++++++++++++++++++++++++++++++
> >  5 files changed, 315 insertions(+), 4 deletions(-)
> >  create mode 100755 scripts/make_fit.py
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 2d13bbd69adb..d6955ebc3c24 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -1575,6 +1575,13 @@ F:       Documentation/process/maintainer-soc*.rst
> >  F:     arch/arm/boot/dts/Makefile
> >  F:     arch/arm64/boot/dts/Makefile
> >
> > +ARM64 FIT SUPPORT
> > +M:     Simon Glass <sjg@chromium.org>
> > +L:     linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
> > +S:     Maintained
> > +F:     arch/arm64/boot/Makefile
> > +F:     scripts/make_fit.py
> > +
> >  ARM ARCHITECTED TIMER DRIVER
> >  M:     Mark Rutland <mark.rutland@arm.com>
> >  M:     Marc Zyngier <maz@kernel.org>
> > diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
> > index 7b77b63e978f..d8290dcab6b6 100644
> > --- a/arch/arm64/Makefile
> > +++ b/arch/arm64/Makefile
> > @@ -150,7 +150,7 @@ libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
> >  # Default target when executing plain make
> >  boot           := arch/arm64/boot
> >
> > -BOOT_TARGETS   := Image.gz vmlinuz.efi
> > +BOOT_TARGETS   := Image.gz vmlinuz.efi image.fit
> >
> >  PHONY += $(BOOT_TARGETS)
> >
> > @@ -215,6 +215,7 @@ virtconfig:
> >  define archhelp
> >    echo  '* Image.gz      - Compressed kernel image (arch/$(ARCH)/boot/Image.gz)'
> >    echo  '  Image         - Uncompressed kernel image (arch/$(ARCH)/boot/Image)'
> > +$(if $(CONFIG_EFI_ZBOOT),,echo  '  image.fit     - Flat Image Tree (arch/$(ARCH)/boot/image.fit)')
> >    echo  '  install       - Install uncompressed kernel'
> >    echo  '  zinstall      - Install compressed kernel'
> >    echo  '                  Install using (your) ~/bin/installkernel or'
> > diff --git a/arch/arm64/boot/Makefile b/arch/arm64/boot/Makefile
> > index 1761f5972443..a6e5b20b22bd 100644
> > --- a/arch/arm64/boot/Makefile
> > +++ b/arch/arm64/boot/Makefile
> > @@ -16,7 +16,8 @@
> >
> >  OBJCOPYFLAGS_Image :=-O binary -R .note -R .note.gnu.build-id -R .comment -S
> >
> > -targets := Image Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo Image.zst
> > +targets := Image Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo \
> > +       Image.zst image.fit
> >
> >  $(obj)/Image: vmlinux FORCE
> >         $(call if_changed,objcopy)
> > @@ -39,6 +40,11 @@ $(obj)/Image.lzo: $(obj)/Image FORCE
> >  $(obj)/Image.zst: $(obj)/Image FORCE
> >         $(call if_changed,zstd)
> >
> > +ifndef CONFIG_EFI_ZBOOT
> > +$(obj)/image.fit: $(obj)/Image $(obj)/dts FORCE
>
>
>
> This is wrong.
>
> $(obj)/dts is a directory.
>
> There is no point for comparing timestamps
> between $(obj)/image.fit and $(obj)/dts.
>
> Updates of *.dtb do not result in the
> update of the $(obj)/dts timestamp.
>
>
> if_changed never works correctly.

Yes I am aware that this doesn't actually deal with changes. I'll see
if I can figure out another way.

>
>
>
>
>
>
>
>
> > +       $(call if_changed,fit,gzip)
> > +endif
> > +
> >  EFI_ZBOOT_PAYLOAD      := Image
> >  EFI_ZBOOT_BFD_TARGET   := elf64-littleaarch64
> >  EFI_ZBOOT_MACH_TYPE    := ARM64
> > diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> > index 68d0134bdbf9..4e4364ad641a 100644
> > --- a/scripts/Makefile.lib
> > +++ b/scripts/Makefile.lib
> > @@ -487,14 +487,26 @@ UIMAGE_OPTS-y ?=
> >  UIMAGE_TYPE ?= kernel
> >  UIMAGE_LOADADDR ?= arch_must_set_this
> >  UIMAGE_ENTRYADDR ?= $(UIMAGE_LOADADDR)
> > -UIMAGE_NAME ?= 'Linux-$(KERNELRELEASE)'
> > +UIMAGE_NAME ?= "Linux-$(KERNELRELEASE)"
>
>
> Unneeded noise change.

I'll move that to a separate patch. We don't want the single quotes in the name.

Regards,
Simon

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 3/3] arm64: boot: Support Flat Image Tree

[Russell King (Oracle)]

On Sun, Oct 29, 2023 at 05:46:12AM +1300, Simon Glass wrote:
> Hi Masahiro,
> 
> Sure, but that is a separate issue, isn't it? We already support
> various boot targets in arm64 but not one that includes the DTs, so
> far as I can see. The old arm 'uImage' target is pretty out-of-date
> now.

Does that mean it can be removed? ;)

I've NAK'd FIT support on 32-bit Arm in the past, and I remain of the
opinion that boot loader specific packaging of the kernel should not
be in the kernel but should be external to it - even more so given the
multi-platform nature of 32-bit Arm kernels.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 3/3] arm64: boot: Support Flat Image Tree

[Tom Rini]

On Mon, Oct 30, 2023 at 03:35:34PM +0000, Russell King (Oracle) wrote:
> On Sun, Oct 29, 2023 at 05:46:12AM +1300, Simon Glass wrote:
> > Hi Masahiro,
> > 
> > Sure, but that is a separate issue, isn't it? We already support
> > various boot targets in arm64 but not one that includes the DTs, so
> > far as I can see. The old arm 'uImage' target is pretty out-of-date
> > now.
> 
> Does that mean it can be removed? ;)
> 
> I've NAK'd FIT support on 32-bit Arm in the past, and I remain of the
> opinion that boot loader specific packaging of the kernel should not
> be in the kernel but should be external to it - even more so given the
> multi-platform nature of 32-bit Arm kernels.

I'll point it out here rather than Simon. As part of
https://github.com/open-source-firmware FIT is a standard and not "boot
loader specific". And one of the points of a FIT image is that you can
easily support multi-platform kernels in a single file (without
optimizing things further, at a cost in tens of milliseconds on a Pi 3
anyhow) and with user-controlled security.

-- 
Tom

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 3/3] arm64: boot: Support Flat Image Tree

[Masahiro Yamada]

On Tue, Oct 31, 2023 at 1:12 AM Tom Rini <trini@konsulko.com> wrote:
>
> On Mon, Oct 30, 2023 at 03:35:34PM +0000, Russell King (Oracle) wrote:
> > On Sun, Oct 29, 2023 at 05:46:12AM +1300, Simon Glass wrote:
> > > Hi Masahiro,
> > >
> > > Sure, but that is a separate issue, isn't it? We already support
> > > various boot targets in arm64 but not one that includes the DTs, so
> > > far as I can see. The old arm 'uImage' target is pretty out-of-date
> > > now.
> >
> > Does that mean it can be removed? ;)
> >
> > I've NAK'd FIT support on 32-bit Arm in the past, and I remain of the
> > opinion that boot loader specific packaging of the kernel should not
> > be in the kernel but should be external to it - even more so given the
> > multi-platform nature of 32-bit Arm kernels.
>
> I'll point it out here rather than Simon. As part of
> https://github.com/open-source-firmware FIT is a standard and not "boot
> loader specific". And one of the points of a FIT image is that you can
> easily support multi-platform kernels in a single file (without
> optimizing things further, at a cost in tens of milliseconds on a Pi 3
> anyhow) and with user-controlled security.
>
> --
> Tom



It is a copy of the document in U-Boot.

The file was split into two, but the content is the same.


[original in U-Boot]
https://github.com/u-boot/u-boot/blob/v2023.10/doc/usage/fit/source_file_format.rst


[flat-image-tree]
https://github.com/open-source-firmware/flat-image-tree/blob/v0.8/source/chapter1-introduction.rst
https://github.com/open-source-firmware/flat-image-tree/blob/v0.8/source/chapter2-source-file-format.rst





--
Best Regards
Masahiro Yamada

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 3/3] arm64: boot: Support Flat Image Tree

[Tom Rini]

On Tue, Oct 31, 2023 at 04:03:18PM +0900, Masahiro Yamada wrote:
> On Tue, Oct 31, 2023 at 1:12 AM Tom Rini <trini@konsulko.com> wrote:
> >
> > On Mon, Oct 30, 2023 at 03:35:34PM +0000, Russell King (Oracle) wrote:
> > > On Sun, Oct 29, 2023 at 05:46:12AM +1300, Simon Glass wrote:
> > > > Hi Masahiro,
> > > >
> > > > Sure, but that is a separate issue, isn't it? We already support
> > > > various boot targets in arm64 but not one that includes the DTs, so
> > > > far as I can see. The old arm 'uImage' target is pretty out-of-date
> > > > now.
> > >
> > > Does that mean it can be removed? ;)
> > >
> > > I've NAK'd FIT support on 32-bit Arm in the past, and I remain of the
> > > opinion that boot loader specific packaging of the kernel should not
> > > be in the kernel but should be external to it - even more so given the
> > > multi-platform nature of 32-bit Arm kernels.
> >
> > I'll point it out here rather than Simon. As part of
> > https://github.com/open-source-firmware FIT is a standard and not "boot
> > loader specific". And one of the points of a FIT image is that you can
> > easily support multi-platform kernels in a single file (without
> > optimizing things further, at a cost in tens of milliseconds on a Pi 3
> > anyhow) and with user-controlled security.
> >
> > --
> > Tom
> 
> 
> 
> It is a copy of the document in U-Boot.
> 
> The file was split into two, but the content is the same.
> 
> 
> [original in U-Boot]
> https://github.com/u-boot/u-boot/blob/v2023.10/doc/usage/fit/source_file_format.rst
> 
> 
> [flat-image-tree]
> https://github.com/open-source-firmware/flat-image-tree/blob/v0.8/source/chapter1-introduction.rst
> https://github.com/open-source-firmware/flat-image-tree/blob/v0.8/source/chapter2-source-file-format.rst

Yes, it would have been a bad idea to change a 15 year old format as
part of getting it included in some standards, and we'd also recently
cleaned it up to rST. Similar comments would I expect be true of turning
grub.cfg in to extlinux.conf and all of the organizations that has moved
along, and anything else that wasn't developed by committee at some
Standards organization.

-- 
Tom

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel