Re: [OE-core] [PATCH v2 15/25] runqemu-extract-sdk: refactor in Python

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Sun, 2026-08-30 at 23:48 +0200, Adrian Freihofer via lists.openembedded.org wrote:
> From: Adrian Freihofer <adrian.freihofer@siemens.com>
> 
> Move the pseudo-backed rootfs extraction logic into a reusable Python
> module while preserving the existing command interface.
> Install the module with qemu-helper so the wrapper also works from an SDK.
> 
> AI-Generated: Uses GitHub Copilot
> 
> Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
> ---
>  .../qemu/nativesdk-qemu-helper_1.0.bb         |   3 +
>  scripts/lib/runqemu_utils.py                  | 124 ++++++++++++++++++
>  scripts/runqemu-extract-sdk                   | 103 +--------------
>  3 files changed, 133 insertions(+), 97 deletions(-)
>  create mode 100644 scripts/lib/runqemu_utils.py
> 
> diff --git a/meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb b/meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb
> index e1f343989a..16fc07fadf 100644
> --- a/meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb
> +++ b/meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb
> @@ -9,6 +9,7 @@ RDEPENDS:${PN} = "nativesdk-qemu nativesdk-unfs3 nativesdk-pseudo \
>  LIC_FILES_CHKSUM = "file://${COREBASE}/scripts/runqemu;beginline=5;endline=10;md5=ac2b489a58739c7628a2604698db5e7f"
>  
>  SRC_URI = "file://${COREBASE}/scripts/runqemu \
> +           file://${COREBASE}/scripts/lib/runqemu_utils.py \
>             file://${COREBASE}/scripts/runqemu-addptable2image \
>             file://${COREBASE}/scripts/runqemu-gen-tapdevs \
>             file://${COREBASE}/scripts/runqemu-ifup \
> @@ -30,4 +31,6 @@ do_install() {
>  	install -d ${D}${bindir}
>  	install -m 0755 ${S}${COREBASE}/scripts/oe-* ${D}${bindir}/
>  	install -m 0755 ${S}${COREBASE}/scripts/runqemu* ${D}${bindir}/
> +	# The runqemu-* wrappers import this module from their own directory.
> +	install -m 0644 ${S}${COREBASE}/scripts/lib/runqemu_utils.py ${D}${bindir}/
>  }
> diff --git a/scripts/lib/runqemu_utils.py b/scripts/lib/runqemu_utils.py
> new file mode 100644
> index 0000000000..dff0ca8d3a
> --- /dev/null
> +++ b/scripts/lib/runqemu_utils.py
> @@ -0,0 +1,124 @@
> +#!/usr/bin/env python3
> +#
> +# Helpers shared by runqemu and NFS-rootfs preparation tools.
> +#
> +# SPDX-License-Identifier: GPL-2.0-only
> +
> +"""Extract and export rootfs tarballs for NFS booting."""
> +
> +import os
> +import subprocess
> +import sys
> +
> +
> +class RunQemuRootfsError(Exception):
> +    """Raised when an NFS rootfs cannot be prepared or exported."""
> +
> +
> +def native_environment():
> +    """Return the qemu-helper-native environment set by oe-find-native-sysroot."""
> +    command = '''
> +        helper=$(command -v oe-find-native-sysroot) || exit 127
> +        . "$helper" qemu-helper-native >/dev/null
> +        env -0
> +        printf 'PSEUDO=%s\\0OECORE_NATIVE_SYSROOT=%s\\0' "$PSEUDO" "$OECORE_NATIVE_SYSROOT"
> +    '''
> +    result = subprocess.run(['bash', '-c', command], capture_output=True)
> +    if result.returncode:
> +        if result.returncode == 127:
> +            raise RunQemuRootfsError(
> +                'Unable to find the oe-find-native-sysroot script.\n'
> +                'Did you forget to source your build system environment setup script?')
> +        raise RunQemuRootfsError(
> +            result.stderr.decode(errors='replace').strip() or
> +            'Unable to set up the qemu-helper-native sysroot')
> +
> +    environment = {}
> +    for entry in result.stdout.split(b'\0'):
> +        if b'=' in entry:
> +            key, value = entry.split(b'=', 1)
> +            environment[key.decode()] = value.decode()
> +    return environment

This takes a relatively simple shell script and turns it into python
but the python is very machine generated and probably not great for
readability or maintainability in the future.

For example, in the above we're using the shell to find oe-find-native-
sysroot. Instead we could use shutil.which(). That would make the code
a lot more pythonic and readable. It also special cases some variables
specifically so it isn't quite a generic function but it is named
generically.

> +def _tar_options(rootfs_tarball):
> +    if rootfs_tarball.endswith('.tar.xz'):
> +        return ['--numeric-owner', '-xJf']
> +    if rootfs_tarball.endswith('.tar.bz2'):
> +        return ['--numeric-owner', '-xjf']
> +    if rootfs_tarball.endswith('.tar.gz'):
> +        return ['--numeric-owner', '-xzf']
> +    if rootfs_tarball.endswith('.tar.zst'):
> +        return ['--numeric-owner', '--zstd', '-xf']
> +    if rootfs_tarball.endswith('.tar'):
> +        return ['--numeric-owner', '-xf']
> +    raise RunQemuRootfsError(
> +        'Unable to determine sdk tarball format\n'
> +        'Accepted types: .tar / .tar.gz / .tar.bz2 / .tar.xz / .tar.zst')

A human doing this would create a table and not duplicate the data. Its
a pain to maintain in future. I appreciate it matches the shell code
but if we're going to do it, we may as well do it well.

> +def pseudo_state_dir(rootfs_dir):
> +    """Return the pseudo database location associated with an extracted rootfs."""
> +    rootfs_dir = os.path.realpath(rootfs_dir)
> +    return os.path.join(os.path.dirname(rootfs_dir),
> +                        os.path.basename(rootfs_dir) + '.pseudo_state')


os.path.join(os.path.dirname(rootfs_dir), os.path.basename(rootfs_dir)) ? 

really? :)

I think this needs a bit more human TLC before it could be merged...

Cheers,

Richard