linux-kbuild.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v2 0/2] scripts: introduce containerized builds
@ 2025-12-18 12:49 Guillaume Tucker
  2025-12-18 12:49 ` [PATCH v2 1/2] scripts: add tool to run " Guillaume Tucker
  2025-12-18 12:49 ` [PATCH v2 2/2] Documentation: dev-tools: add container.rst page Guillaume Tucker
  0 siblings, 2 replies; 10+ messages in thread
From: Guillaume Tucker @ 2025-12-18 12:49 UTC (permalink / raw)
  To: Nathan Chancellor, Miguel Ojeda, David Gow, Onur Özkan
  Cc: Guillaume Tucker, Arnd Bergmann, linux-kernel, rust-for-linux,
	linux-kbuild, automated-testing, workflows, llvm

This proposal emerged from an email discussion and a talk at Plumbers
last year:

    https://lore.kernel.org/all/affb7aff-dc9b-4263-bbd4-a7965c19ac4e@gtucker.io/

The aim is to facilitate reproducing builds for CI bots as well as
developers using containers.  Here's an illustrative example with a
kernel.org toolchain in a Docker image from tuxmake:

    $ scripts/container -i tuxmake/korg-clang-21 make LLVM=1 defconfig
      HOSTCC  scripts/basic/fixdep
      HOSTCC  scripts/kconfig/conf.o
    [...]
      HOSTCC  scripts/kconfig/util.o
      HOSTLD  scripts/kconfig/conf
    *** Default configuration is based on 'x86_64_defconfig'
    #
    # configuration written to .config
    #

and a follow-up command to build the kernel with the verbose flag
turned on to show DEBUG log messages from the container tool:

    $ scripts/container -i tuxmake/korg-clang-21 -v -- make LLVM=1 -j8
    [container DEBUG] runtime: docker
    [container DEBUG] image: tuxmake/korg-clang-21
    [container DEBUG] container: c5a88761-f55a-4027-84c9-bc3c6dc9c4cd
      GEN     arch/x86/include/generated/asm/orc_hash.h
      HOSTCC  scripts/basic/fixdep
      SYSHDR  arch/x86/include/generated/uapi/asm/unistd_32.h
    [...]
      BUILD   arch/x86/boot/bzImage
    Kernel: arch/x86/boot/bzImage is ready  (#1)

As a next step to make this tool more useful, I'm in the process of
preparing reference container images with kernel.org toolchains and no
third-party dependencies other than the base Debian distro:

    https://gitlab.com/gtucker/korg-containers

Say, to run KUnit using the latest kernel.org GCC toolchain:

    scripts/container \
        -i registry.gitlab.com/gtucker/korg-containers/gcc:kunit -- \
        tools/testing/kunit/kunit.py \
            run \
            --arch=x86_64 \
            --cross_compile=x86_64-linux-

This patch series also include a documentation page with all the
relevant details about how to use the tool and the images currently
available.

---
Changes in v2:
- Drop default image but make -i option required
- Look for Docker and Podman if no runtime specified
- Catch SIGINT from user to abort container with Docker
- Explicitly name each container with a UUID
- Update documentation accordingly

---

Guillaume Tucker (2):
  scripts: add tool to run containerized builds
  Documentation: dev-tools: add container.rst page

 Documentation/dev-tools/container.rst | 175 +++++++++++++++++++++++
 Documentation/dev-tools/index.rst     |   1 +
 scripts/container                     | 194 ++++++++++++++++++++++++++
 3 files changed, 370 insertions(+)
 create mode 100644 Documentation/dev-tools/container.rst
 create mode 100755 scripts/container

-- 
2.47.3


^ permalink raw reply	[flat|nested] 10+ messages in thread

* [PATCH v2 1/2] scripts: add tool to run containerized builds
  2025-12-18 12:49 [PATCH v2 0/2] scripts: introduce containerized builds Guillaume Tucker
@ 2025-12-18 12:49 ` Guillaume Tucker
  2025-12-19 21:27   ` Nathan Chancellor
  2025-12-21 20:19   ` Guillaume Tucker
  2025-12-18 12:49 ` [PATCH v2 2/2] Documentation: dev-tools: add container.rst page Guillaume Tucker
  1 sibling, 2 replies; 10+ messages in thread
From: Guillaume Tucker @ 2025-12-18 12:49 UTC (permalink / raw)
  To: Nathan Chancellor, Miguel Ojeda, David Gow, Onur Özkan
  Cc: Guillaume Tucker, Arnd Bergmann, linux-kernel, rust-for-linux,
	linux-kbuild, automated-testing, workflows, llvm

Add a 'scripts/container' tool written in Python to run any command in
the source tree from within a container.  This can typically be used
to call 'make' with a compiler toolchain image to run reproducible
builds but any arbitrary command can be run too.  Only Docker and
Podman are supported for this initial version.

Cc: Nathan Chancellor <nathan@kernel.org>
Cc: Miguel Ojeda <ojeda@kernel.org>
Cc: David Gow <davidgow@google.com>
Cc: "Onur Özkan" <work@onurozkan.dev>
Link: https://lore.kernel.org/all/affb7aff-dc9b-4263-bbd4-a7965c19ac4e@gtucker.io/
Signed-off-by: Guillaume Tucker <gtucker@gtucker.io>
---
 scripts/container | 194 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 194 insertions(+)
 create mode 100755 scripts/container

diff --git a/scripts/container b/scripts/container
new file mode 100755
index 000000000000..2d0143c7d43e
--- /dev/null
+++ b/scripts/container
@@ -0,0 +1,194 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright (C) 2025 Guillaume Tucker
+
+"""Containerized builds"""
+
+import abc
+import argparse
+import logging
+import os
+import shutil
+import subprocess
+import sys
+import uuid
+
+
+class ContainerRuntime(abc.ABC):
+    """Base class for a container runtime implementation"""
+
+    name = None  # Property defined in each implementation class
+
+    def __init__(self, args, logger):
+        self._uid = args.uid or os.getuid()
+        self._gid = args.gid or args.uid or os.getgid()
+        self._env_file = args.env_file
+        self._logger = logger
+
+    @classmethod
+    def is_present(cls):
+        """Determine whether the runtime is present on the system"""
+        return shutil.which(cls.name) is not None
+
+    @abc.abstractmethod
+    def _do_run(self, image, cmd, container_name):
+        """Runtime-specific handler to run a command in a container"""
+
+    @abc.abstractmethod
+    def _do_abort(self, container_name):
+        """Runtime-specific handler to abort a command in running container"""
+
+    def run(self, image, cmd):
+        """Run a command in a runtime container"""
+        container_name = str(uuid.uuid4())
+        self._logger.debug("container: %s", container_name)
+        try:
+            return self._do_run(image, cmd, container_name)
+        except KeyboardInterrupt:
+            self._logger.error("user aborted")
+            self._do_abort(container_name)
+            return 1
+
+
+class DockerRuntime(ContainerRuntime):
+    """Run a command in a Docker container"""
+
+    name = 'docker'
+
+    def _do_run(self, image, cmd, container_name):
+        cmdline = [
+            'docker', 'run',
+            '--name', container_name,
+            '--rm',
+            '--tty',
+            '--volume', f'{os.getcwd()}:/src',
+            '--workdir', '/src',
+            '--user', f'{self._uid}:{self._gid}'
+        ]
+        if self._env_file:
+            cmdline += ['--env-file', self._env_file]
+        cmdline.append(image)
+        cmdline += cmd
+        return subprocess.call(cmdline)
+
+    def _do_abort(self, container_name):
+        subprocess.call(['docker', 'kill', container_name])
+
+
+class PodmanRuntime(ContainerRuntime):
+    """Run a command in a Podman container"""
+
+    name = 'podman'
+
+    def _do_run(self, image, cmd, container_name):
+        cmdline = [
+            'podman', 'run',
+            '--name', container_name,
+            '--rm',
+            '--tty',
+            '--interactive',
+            '--volume', f'{os.getcwd()}:/src',
+            '--workdir', '/src',
+            '--userns', f'keep-id:uid={self._uid},gid={self._gid}',
+        ]
+        if self._env_file:
+            cmdline += ['--env-file', self._env_file]
+        cmdline.append(image)
+        cmdline += cmd
+        return subprocess.call(cmdline)
+
+    def _do_abort(self, container_name):
+        pass  # Signals are handled by Podman in interactive mode
+
+
+class Runtimes:
+    """List of all supported runtimes"""
+
+    runtimes = [DockerRuntime, PodmanRuntime]
+
+    @classmethod
+    def get_names(cls):
+        """Get a list of all the runtime names"""
+        return list(runtime.name for runtime in cls.runtimes)
+
+    @classmethod
+    def get(cls, name):
+        """Get a single runtime class matching the given name"""
+        for runtime in cls.runtimes:
+            if runtime.name == name:
+                if not runtime.is_present():
+                    raise ValueError(f"runtime not found: {name}")
+                return runtime
+        raise ValueError(f"unknown runtime: {runtime}")
+
+    @classmethod
+    def find(cls):
+        """Find the first runtime present on the system"""
+        for runtime in cls.runtimes:
+            if runtime.is_present():
+                return runtime
+        raise ValueError("no runtime found")
+
+
+def _get_logger(verbose):
+    """Set up a logger with the appropriate level"""
+    logger = logging.getLogger('container')
+    handler = logging.StreamHandler()
+    handler.setFormatter(logging.Formatter(
+        fmt='[container {levelname}] {message}', style='{'
+    ))
+    logger.addHandler(handler)
+    logger.setLevel(logging.DEBUG if verbose is True else logging.INFO)
+    return logger
+
+
+def main(args):
+    """Main entry point for the container tool"""
+    logger = _get_logger(args.verbose)
+    try:
+        cls = Runtimes.get(args.runtime) if args.runtime else Runtimes.find()
+    except ValueError as ex:
+        logger.error(ex)
+        return 1
+    logger.debug("runtime: %s", cls.name)
+    logger.debug("image: %s", args.image)
+    return cls(args, logger).run(args.image, args.cmd)
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(
+        'container',
+        description="Containerized builds.  See the dev-tools/container "
+        "kernel documentation section for more details."
+    )
+    parser.add_argument(
+        '-e', '--env-file',
+        help="Path to an environment file to load in the container."
+    )
+    parser.add_argument(
+        '-g', '--gid',
+        help="Group ID to use inside the container."
+    )
+    parser.add_argument(
+        '-i', '--image', required=True,
+        help="Container image name."
+    )
+    parser.add_argument(
+        '-r', '--runtime', choices=Runtimes.get_names(),
+        help="Container runtime name.  If not specified, the first one found "
+        "on the system will be used i.e. Docker if present, otherwise Podman."
+    )
+    parser.add_argument(
+        '-u', '--uid',
+        help="User ID to use inside the container.  If the -g option is not "
+        "specified, the user ID will also be set as the group ID."
+    )
+    parser.add_argument(
+        '-v', '--verbose', action='store_true',
+        help="Enable verbose output."
+    )
+    parser.add_argument(
+        'cmd', nargs='+',
+        help="Command to run in the container"
+    )
+    sys.exit(main(parser.parse_args(sys.argv[1:])))
-- 
2.47.3


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCH v2 2/2] Documentation: dev-tools: add container.rst page
  2025-12-18 12:49 [PATCH v2 0/2] scripts: introduce containerized builds Guillaume Tucker
  2025-12-18 12:49 ` [PATCH v2 1/2] scripts: add tool to run " Guillaume Tucker
@ 2025-12-18 12:49 ` Guillaume Tucker
  2025-12-21 20:13   ` Guillaume Tucker
  1 sibling, 1 reply; 10+ messages in thread
From: Guillaume Tucker @ 2025-12-18 12:49 UTC (permalink / raw)
  To: Nathan Chancellor, Miguel Ojeda, David Gow, Onur Özkan
  Cc: Guillaume Tucker, Arnd Bergmann, linux-kernel, rust-for-linux,
	linux-kbuild, automated-testing, workflows, llvm

Add a dev-tools/container.rst documentation page for the
scripts/container tool.  This covers the basic usage with additional
information about environment variables and user IDs.  It also
includes a number of practical examples with a reference to the
experimental kernel.org toolchain images.

Cc: Nathan Chancellor <nathan@kernel.org>
Cc: Miguel Ojeda <ojeda@kernel.org>
Cc: David Gow <davidgow@google.com>
Cc: "Onur Özkan" <work@onurozkan.dev>
Signed-off-by: Guillaume Tucker <gtucker@gtucker.io>
---
 Documentation/dev-tools/container.rst | 175 ++++++++++++++++++++++++++
 Documentation/dev-tools/index.rst     |   1 +
 2 files changed, 176 insertions(+)
 create mode 100644 Documentation/dev-tools/container.rst

diff --git a/Documentation/dev-tools/container.rst b/Documentation/dev-tools/container.rst
new file mode 100644
index 000000000000..5254feae02c2
--- /dev/null
+++ b/Documentation/dev-tools/container.rst
@@ -0,0 +1,175 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+.. Copyright (C) 2025 Guillaume Tucker
+
+====================
+Containerized Builds
+====================
+
+The ``container`` tool can be used to run any command in the kernel source tree
+from within a container.  Doing so facilitates reproducing builds across
+various platforms, for example when a test bot has reported an issue which
+requires a specific version of a compiler or an external test suite.  While
+this can already be done by users who are familiar with containers, having a
+dedicated tool in the kernel tree lowers the barrier to entry by solving common
+problems once and for all (e.g. user id management).  It also makes it easier
+to share an exact command line leading to a particular result.  The main use
+case is likely to be kernel builds but virtually anything can be run: KUnit,
+checkpatch etc. provided a suitable image is available.
+
+
+Options
+=======
+
+Command line syntax::
+
+  scripts/container -i IMAGE [OPTION]... CMD...
+
+Available options:
+
+``-e, --env-file ENV_FILE``
+
+    Path to an environment file to load in the container.
+
+``-g, --gid GID``
+
+    Group id to use inside the container.
+
+``-i, --image IMAGE``
+
+    Container image name (required).
+
+``-r, --runtime RUNTIME``
+
+    Container runtime name.  Supported runtimes: ``docker``, ``podman``.
+
+    If not specified, the first one found on the system will be used
+    i.e. Docker if present, otherwise Podman.
+
+``-u, --uid UID``
+
+    User id to use inside the container.
+
+    If the ``-g`` option is not specified, the user id will also be used for
+    the group id.
+
+``-v, --verbose``
+
+    Enable verbose output.
+
+``-h, --help``
+
+    Show the help message and exit.
+
+
+Usage
+=====
+
+It's entirely up to the user to choose which image to use and the ``CMD``
+arguments are passed directly as an arbitrary command line to run in the
+container.  The tool will take care of mounting the source tree as the current
+working directory and adjust the user and group id as needed.
+
+The container image which would typically include a compiler toolchain is
+provided by the user and selected via the ``-i`` option.  The container runtime
+can be selected with the ``-r`` option, which can be either ``docker`` or
+``podman``.  If none is specified, the first one found on the system will be
+used.  Support for other runtimes may be added later depending on their
+popularity among users.
+
+
+Environment Variables
+=====================
+
+Environment variables are not propagated to the container so they have to be
+either defined in the image itself or via the ``-e`` option using an
+environment file.  In some cases it makes more sense to have them defined in
+the Containerfile used to create the image.  For example, a Clang-only compiler
+toolchain image may have ``LLVM=1`` defined.  The local environment file is
+more useful for user-specific variables added during development.
+
+Please note that ``make`` options can still be passed on the command line, so
+while this can't be done since the first argument needs to be the executable::
+
+  scripts/container -i tuxmake/korg-clang LLVM=1 make
+
+this will work::
+
+  scripts/container -i tuxmake/korg-clang make LLVM=1
+
+
+User IDs
+========
+
+This is an area where the behaviour will vary slightly depending on the
+container runtime.  The goal is to run commands as the user invoking the tool.
+With Podman, a namespace is created to map the current user id to a different
+one in the container (1000 by default).  With Docker, while this is also
+possible with recent versions it requires a special feature to be enabled in
+the daemon so it's not used here for simplicity.  Instead, the container is run
+with the current user id directly.  In both cases, this will provide the same
+file permissions for the kernel source tree mounted as a volume.  The only
+difference is that when using Docker without a namespace, the user id may not
+be the same as the default one set in the image.
+
+Say, we're using an image which sets up a default user with id 1000 and the
+current user calling the ``container`` tool has id 1234.  The kernel source
+tree was checked out by this same user so the files belong to user 1234.  With
+Podman, the container will be running as user id 1000 with a mapping to id 1234
+so that the files from the mounted volume appear to belong to id 1000 inside
+the container.  With Docker and no namespace, the container will be running
+with user id 1234 which can access the files in the volume but not in the user
+1000 home directory.  This shouldn't be an issue when running commands only in
+the kernel tree but it is worth highlighting here as it might matter for
+special corner cases.
+
+
+Examples
+========
+
+The shortest example is to run a basic kernel build using Docker and a tuxmake
+Clang image::
+
+  scripts/container -i tuxmake/korg-clang -- make LLVM=1 defconfig
+  scripts/container -i tuxmake/korg-clang -- make LLVM=1 -j$(nproc)
+
+.. note::
+
+   When running a command with options within the container, it should be
+   separated with a double dash ``--`` to not confuse them with the
+   ``container`` tool options.  Plain commands with no options don't strictly
+   require the double dashes e.g.::
+
+     scripts/container -i tuxmake/korg-clang make mrproper
+
+To run ``checkpatch.pl`` in a ``patches`` directory with a generic image::
+
+  scripts/container -i perl:slim-trixie scripts/checkpatch.pl patches/*
+
+The examples below refer to ``kernel.org`` images which are based on the
+`kernel.org compiler toolchains
+<https://mirrors.edge.kernel.org/pub/tools/>`__.  These aren't (yet) officially
+available in any public registry but users can build their own locally instead
+using this `experimental repository
+<https://gitlab.com/gtucker/korg-containers>`__ by running ``make
+PREFIX=kernel.org/``.
+
+To build just ``bzImage`` using Clang::
+
+  scripts/container -i kernel.org/clang -- make bzImage -j$(nproc)
+
+Same with GCC 15 as a particular version tag::
+
+  scripts/container -i kernel.org/gcc:15 -- make bzImage -j$(nproc)
+
+To run KUnit::
+
+  scripts/container -i kernel.org/gcc:kunit -- \
+      tools/testing/kunit/kunit.py \
+          run \
+          --arch=x86_64 \
+          --cross_compile=x86_64-linux-
+
+To build the HTML documentation, which requires the ``kdocs`` image built with
+``make PREFIX=kernel.org/ extra`` as it's not a compiler toolchain::
+
+  scripts/container -i kernel.org/kdocs make htmldocs
diff --git a/Documentation/dev-tools/index.rst b/Documentation/dev-tools/index.rst
index 4b8425e348ab..527a0e4cf2ed 100644
--- a/Documentation/dev-tools/index.rst
+++ b/Documentation/dev-tools/index.rst
@@ -38,6 +38,7 @@ Documentation/process/debugging/index.rst
    gpio-sloppy-logic-analyzer
    autofdo
    propeller
+   container
 
 
 .. only::  subproject and html
-- 
2.47.3


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: [PATCH v2 1/2] scripts: add tool to run containerized builds
  2025-12-18 12:49 ` [PATCH v2 1/2] scripts: add tool to run " Guillaume Tucker
@ 2025-12-19 21:27   ` Nathan Chancellor
  2025-12-21 20:09     ` Guillaume Tucker
  2025-12-21 20:19   ` Guillaume Tucker
  1 sibling, 1 reply; 10+ messages in thread
From: Nathan Chancellor @ 2025-12-19 21:27 UTC (permalink / raw)
  To: Guillaume Tucker
  Cc: Miguel Ojeda, David Gow, Onur Özkan, Arnd Bergmann,
	linux-kernel, rust-for-linux, linux-kbuild, automated-testing,
	workflows, llvm

On Thu, Dec 18, 2025 at 01:49:52PM +0100, Guillaume Tucker wrote:
...
> +    def __init__(self, args, logger):

Adding something like

    self._args = [
        '--rm',
        '--tty',
        '--volume', f'{os.getcwd()}:/src',
        '--workdir', '/src',
    ]

here then adding an __init__() in the subclasses to append the runtime
specific arguments would allow _do_run() to be moved into
ContainerRuntime(). Otherwise, this looks pretty good and extensible.

> +        self._uid = args.uid or os.getuid()
> +        self._gid = args.gid or args.uid or os.getgid()
> +        self._env_file = args.env_file
> +        self._logger = logger
> +
> +    @classmethod
> +    def is_present(cls):
> +        """Determine whether the runtime is present on the system"""
> +        return shutil.which(cls.name) is not None
> +
> +    @abc.abstractmethod
> +    def _do_run(self, image, cmd, container_name):
> +        """Runtime-specific handler to run a command in a container"""
> +
> +    @abc.abstractmethod
> +    def _do_abort(self, container_name):
> +        """Runtime-specific handler to abort a command in running container"""
> +
> +    def run(self, image, cmd):
> +        """Run a command in a runtime container"""
> +        container_name = str(uuid.uuid4())
> +        self._logger.debug("container: %s", container_name)
> +        try:
> +            return self._do_run(image, cmd, container_name)
> +        except KeyboardInterrupt:
> +            self._logger.error("user aborted")
> +            self._do_abort(container_name)
> +            return 1
> +
> +
> +class DockerRuntime(ContainerRuntime):
> +    """Run a command in a Docker container"""
> +
> +    name = 'docker'
> +
> +    def _do_run(self, image, cmd, container_name):
> +        cmdline = [
> +            'docker', 'run',
> +            '--name', container_name,
> +            '--rm',
> +            '--tty',
> +            '--volume', f'{os.getcwd()}:/src',
> +            '--workdir', '/src',
> +            '--user', f'{self._uid}:{self._gid}'
> +        ]
> +        if self._env_file:
> +            cmdline += ['--env-file', self._env_file]
> +        cmdline.append(image)
> +        cmdline += cmd
> +        return subprocess.call(cmdline)
> +
> +    def _do_abort(self, container_name):
> +        subprocess.call(['docker', 'kill', container_name])
> +
> +
> +class PodmanRuntime(ContainerRuntime):
> +    """Run a command in a Podman container"""
> +
> +    name = 'podman'
> +
> +    def _do_run(self, image, cmd, container_name):
> +        cmdline = [
> +            'podman', 'run',
> +            '--name', container_name,
> +            '--rm',
> +            '--tty',
> +            '--interactive',
> +            '--volume', f'{os.getcwd()}:/src',
> +            '--workdir', '/src',
> +            '--userns', f'keep-id:uid={self._uid},gid={self._gid}',
> +        ]
> +        if self._env_file:
> +            cmdline += ['--env-file', self._env_file]
> +        cmdline.append(image)
> +        cmdline += cmd
> +        return subprocess.call(cmdline)
> +
> +    def _do_abort(self, container_name):
> +        pass  # Signals are handled by Podman in interactive mode

Cheers,
Nathan

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH v2 1/2] scripts: add tool to run containerized builds
  2025-12-19 21:27   ` Nathan Chancellor
@ 2025-12-21 20:09     ` Guillaume Tucker
  0 siblings, 0 replies; 10+ messages in thread
From: Guillaume Tucker @ 2025-12-21 20:09 UTC (permalink / raw)
  To: Nathan Chancellor
  Cc: Miguel Ojeda, David Gow, Onur Özkan, Arnd Bergmann,
	linux-kernel, rust-for-linux, linux-kbuild, automated-testing,
	workflows, llvm

On 19/12/2025 10:27 pm, Nathan Chancellor wrote:
> On Thu, Dec 18, 2025 at 01:49:52PM +0100, Guillaume Tucker wrote:
> ...
>> +    def __init__(self, args, logger):
> Adding something like
> 
>      self._args = [
>          '--rm',
>          '--tty',
>          '--volume', f'{os.getcwd()}:/src',
>          '--workdir', '/src',
>      ]
> 
> here then adding an __init__() in the subclasses to append the runtime
> specific arguments would allow _do_run() to be moved into
> ContainerRuntime(). Otherwise, this looks pretty good and extensible.

Yes, I left these very similar parts as-is on purpose to make it very
clear what the command line arguments are for each container runtime.
It's a good idea to refactor this though as you mention, and I'll add
a debug log message to print the command line instead.

I'm also anticipating that other runtimes will be quite different,
Podman and Docker just so happen to have many options in common.
Things like runc or containerd are very different beasts so I don't
want to over-generalise.  But this is straightforward enough for now.

So I'll rework this a bit in a v3.

Cheers,
Guillaume

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH v2 2/2] Documentation: dev-tools: add container.rst page
  2025-12-18 12:49 ` [PATCH v2 2/2] Documentation: dev-tools: add container.rst page Guillaume Tucker
@ 2025-12-21 20:13   ` Guillaume Tucker
  0 siblings, 0 replies; 10+ messages in thread
From: Guillaume Tucker @ 2025-12-21 20:13 UTC (permalink / raw)
  To: Nathan Chancellor, Miguel Ojeda, David Gow, Onur Özkan
  Cc: Arnd Bergmann, linux-kernel, rust-for-linux, linux-kbuild,
	automated-testing, workflows, llvm

Hi Nathan,

On 18/12/2025 1:49 pm, Guillaume Tucker wrote:
> +User IDs
> +========
> +
> +This is an area where the behaviour will vary slightly depending on the
> +container runtime.  The goal is to run commands as the user invoking the tool.
> +With Podman, a namespace is created to map the current user id to a different
> +one in the container (1000 by default).  With Docker, while this is also
> +possible with recent versions it requires a special feature to be enabled in
> +the daemon so it's not used here for simplicity.  Instead, the container is run
> +with the current user id directly.  In both cases, this will provide the same
> +file permissions for the kernel source tree mounted as a volume.  The only
> +difference is that when using Docker without a namespace, the user id may not
> +be the same as the default one set in the image.
> +
> +Say, we're using an image which sets up a default user with id 1000 and the
> +current user calling the ``container`` tool has id 1234.  The kernel source
> +tree was checked out by this same user so the files belong to user 1234.  With
> +Podman, the container will be running as user id 1000 with a mapping to id 1234
> +so that the files from the mounted volume appear to belong to id 1000 inside
> +the container.  With Docker and no namespace, the container will be running
> +with user id 1234 which can access the files in the volume but not in the user
> +1000 home directory.  This shouldn't be an issue when running commands only in
> +the kernel tree but it is worth highlighting here as it might matter for
> +special corner cases.

This part of the docs explains why things are a bit different between
Podman and Docker.  In both cases, it should "just work" from a user
point of view - just with some special corner cases.  Let me know if
you thing the documentation needs to be improved.

I may add a runtime check as a follow-up to detect if namespaces are
enabled in Docker and if so use them, but to get started I wanted to
keep things as simple as possible.

Cheers,
Guillaume

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH v2 1/2] scripts: add tool to run containerized builds
  2025-12-18 12:49 ` [PATCH v2 1/2] scripts: add tool to run " Guillaume Tucker
  2025-12-19 21:27   ` Nathan Chancellor
@ 2025-12-21 20:19   ` Guillaume Tucker
  2025-12-22  3:30     ` Miguel Ojeda
  1 sibling, 1 reply; 10+ messages in thread
From: Guillaume Tucker @ 2025-12-21 20:19 UTC (permalink / raw)
  To: Nathan Chancellor, Miguel Ojeda, David Gow, Onur Özkan
  Cc: Arnd Bergmann, linux-kernel, rust-for-linux, linux-kbuild,
	automated-testing, workflows, llvm

Hi Nathan,

On 18/12/2025 1:49 pm, Guillaume Tucker wrote:
> +if __name__ == '__main__':
> +    parser = argparse.ArgumentParser(
> +        'container',
> +        description="Containerized builds.  See the dev-tools/container "
> +        "kernel documentation section for more details."
> +    )

Another piece of feedback from your v1 review was to add a link to
the documentation.  As it's not published yet I just mentioned the
section name here in the v2 - but I can anticipate what the final URL
will be i.e.:

     https://www.kernel.org/doc/html/latest/dev-tools/container

So I'll tweak this as well in the v3 unless anyone suggests
otherwise.

Cheers,
Guillaume




^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH v2 1/2] scripts: add tool to run containerized builds
  2025-12-21 20:19   ` Guillaume Tucker
@ 2025-12-22  3:30     ` Miguel Ojeda
  2025-12-22  9:11       ` Guillaume Tucker
  0 siblings, 1 reply; 10+ messages in thread
From: Miguel Ojeda @ 2025-12-22  3:30 UTC (permalink / raw)
  To: Guillaume Tucker
  Cc: Nathan Chancellor, Miguel Ojeda, David Gow, Onur Özkan,
	Arnd Bergmann, linux-kernel, rust-for-linux, linux-kbuild,
	automated-testing, workflows, llvm

On Sun, Dec 21, 2025 at 9:19 PM Guillaume Tucker <gtucker@gtucker.io> wrote:
>
> Another piece of feedback from your v1 review was to add a link to
> the documentation.  As it's not published yet I just mentioned the
> section name here in the v2 - but I can anticipate what the final URL
> will be i.e.:
>
>      https://www.kernel.org/doc/html/latest/dev-tools/container

A docs.kernel.org URL instead may look better?

Cheers,
Miguel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH v2 1/2] scripts: add tool to run containerized builds
  2025-12-22  3:30     ` Miguel Ojeda
@ 2025-12-22  9:11       ` Guillaume Tucker
  2025-12-22 16:12         ` Konstantin Ryabitsev
  0 siblings, 1 reply; 10+ messages in thread
From: Guillaume Tucker @ 2025-12-22  9:11 UTC (permalink / raw)
  To: Miguel Ojeda
  Cc: Nathan Chancellor, Miguel Ojeda, David Gow, Onur Özkan,
	Arnd Bergmann, linux-kernel, rust-for-linux, linux-kbuild,
	automated-testing, workflows, llvm

Hi Miguel,

On 22/12/2025 04:30, Miguel Ojeda wrote:
> On Sun, Dec 21, 2025 at 9:19 PM Guillaume Tucker <gtucker@gtucker.io> wrote:
>>
>> Another piece of feedback from your v1 review was to add a link to
>> the documentation.  As it's not published yet I just mentioned the
>> section name here in the v2 - but I can anticipate what the final URL
>> will be i.e.:
>>
>>      https://www.kernel.org/doc/html/latest/dev-tools/container
> 
> A docs.kernel.org URL instead may look better?

Ah yes, good point.  Except right now it's returning HTTP 403...
Hopefully that's just a temporary technical glitch.

Cheers,
Guillaume


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH v2 1/2] scripts: add tool to run containerized builds
  2025-12-22  9:11       ` Guillaume Tucker
@ 2025-12-22 16:12         ` Konstantin Ryabitsev
  0 siblings, 0 replies; 10+ messages in thread
From: Konstantin Ryabitsev @ 2025-12-22 16:12 UTC (permalink / raw)
  To: Guillaume Tucker
  Cc: Miguel Ojeda, Nathan Chancellor, Miguel Ojeda, David Gow,
	Onur Özkan, Arnd Bergmann, linux-kernel, rust-for-linux,
	linux-kbuild, automated-testing, workflows, llvm

On Mon, Dec 22, 2025 at 10:11:25AM +0100, Guillaume Tucker wrote:
> Ah yes, good point.  Except right now it's returning HTTP 403...
> Hopefully that's just a temporary technical glitch.

(it was)

-K

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2025-12-22 16:12 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-12-18 12:49 [PATCH v2 0/2] scripts: introduce containerized builds Guillaume Tucker
2025-12-18 12:49 ` [PATCH v2 1/2] scripts: add tool to run " Guillaume Tucker
2025-12-19 21:27   ` Nathan Chancellor
2025-12-21 20:09     ` Guillaume Tucker
2025-12-21 20:19   ` Guillaume Tucker
2025-12-22  3:30     ` Miguel Ojeda
2025-12-22  9:11       ` Guillaume Tucker
2025-12-22 16:12         ` Konstantin Ryabitsev
2025-12-18 12:49 ` [PATCH v2 2/2] Documentation: dev-tools: add container.rst page Guillaume Tucker
2025-12-21 20:13   ` Guillaume Tucker

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).