* [PATCH v2] tests/docker: ensure container command is probed at most once
@ 2026-07-15 10:42 Daniel P. Berrangé
2026-07-15 16:07 ` Pierrick Bouvier
0 siblings, 1 reply; 6+ messages in thread
From: Daniel P. Berrangé @ 2026-07-15 10:42 UTC (permalink / raw)
To: qemu-devel
Cc: Pierrick Bouvier, Paolo Bonzini, Alex Bennée,
Daniel P. Berrangé
The '--engine' arg accepts either 'podman' or 'docker', which is
not sufficiently granular to map directly to a command. This
means that docker.py still has to then probe the exact command
to use.
Meanwhile the 'probe' command prints out the full command to use
but this cannot be passed back to docker.py to avoid probing
again, so the caching is only useful in the few case where we
run a container directly bypassing docker.py.
Address this by replacing --engine with --command for docker.py.
This in turn requires the --container-engine configure arg to be
replaced with --container-command.
With these changes the container command is probed at most once
during configure and never again, while running in an unconfigured
tree will still probe on demand.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
In v2:
* Don't set container_command to "no" if 'probe' finds
not viable binaries.
configure | 25 +++++++------
tests/docker/Makefile.include | 13 ++++---
tests/docker/docker.py | 67 ++++++++++++-----------------------
3 files changed, 40 insertions(+), 65 deletions(-)
diff --git a/configure b/configure
index d8bc10060e..5a8e2fd278 100755
--- a/configure
+++ b/configure
@@ -172,7 +172,7 @@ fi
# some defaults, based on the host environment
# default parameters
-container_engine="auto"
+container_command=""
cpu=""
cross_compile="no"
cross_prefix=""
@@ -734,7 +734,7 @@ for opt do
;;
--disable-containers) use_containers="no"
;;
- --container-engine=*) container_engine="$optarg"
+ --container-command=*) container_command="$optarg"
;;
--rust-target-triple=*) rust_target_triple="$optarg"
;;
@@ -869,7 +869,7 @@ Advanced options (experts only):
--enable-debug enable common debug build options
--cpu=CPU Build for host CPU [$cpu]
--disable-containers don't use containers for cross-building
- --container-engine=TYPE which container engine to use [$container_engine]
+ --container-command=CMD which container command to use [autodetect]
--gdb=GDB-path gdb to use for gdbstub tests [$gdb_bin]
--wasm64-32bit-address-limit Restrict wasm64 address space to 32-bit (default
is to use the whole 64-bit range).
@@ -1291,12 +1291,12 @@ fi
##########################################
# functions to probe cross compilers
-runc="no"
-if test $use_containers = "yes" && (has "docker" || has "podman"); then
- runc=$($python "$source_path"/tests/docker/docker.py --engine "$container_engine" probe)
- if test "$runc" != "no"; then
- docker_py="$python $source_path/tests/docker/docker.py --engine $container_engine"
- fi
+if test "$container_command" = ""; then
+ container_command=$($python "$source_path"/tests/docker/docker.py probe)
+ test "$container_command" == "no" && container_command=""
+fi
+if test $use_containers = "yes" && test "$container_command" != ""; then
+ docker_py="$python $source_path/tests/docker/docker.py --command $container_command"
fi
# cross compilers defaults, can be overridden with --cross-cc-ARCH
@@ -1415,7 +1415,7 @@ probe_target_compiler() {
esac
for host in $container_hosts; do
- test "$runc" != no || continue
+ test "$container_command" != "" || continue
test "$host" = "$cpu" || continue
case $target_arch in
# debian-all-test-cross architectures
@@ -1736,9 +1736,8 @@ echo all: >> $config_host_mak
echo "SRC_PATH=$source_path" >> $config_host_mak
echo "TARGET_DIRS=$target_list" >> $config_host_mak
echo "GDB=$gdb_bin" >> $config_host_mak
-if test "$runc" != no; then
- echo "RUNC=$runc" >> $config_host_mak
- echo "CONTAINER_ENGINE=$container_engine" >> $config_host_mak
+if test "$container_command" != ""; then
+ echo "CONTAINER_COMMAND=$container_command" >> $config_host_mak
fi
echo "SUBDIRS=$subdirs" >> $config_host_mak
if test "$rust" != disabled; then
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 4725c39807..0adddb6a5c 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -16,9 +16,8 @@ DOCKER_DEFAULT_REGISTRY := registry.gitlab.com/qemu-project/qemu
endif
DOCKER_REGISTRY := $(if $(REGISTRY),$(REGISTRY),$(DOCKER_DEFAULT_REGISTRY))
-CONTAINER_ENGINE = auto
-DOCKER_SCRIPT=$(SRC_PATH)/tests/docker/docker.py --engine $(CONTAINER_ENGINE)
-RUNC ?= $(shell $(DOCKER_SCRIPT) probe)
+CONTAINER_COMMAND ?= $(shell $(SRC_PATH)/tests/docker/docker.py probe)
+DOCKER_SCRIPT=$(SRC_PATH)/tests/docker/docker.py --command "$(CONTAINER_COMMAND)"
CUR_TIME := $(shell date +%Y-%m-%d-%H.%M.%S.$$$$)
DOCKER_SRC_COPY := $(BUILD_DIR)/docker-src.$(CUR_TIME)
@@ -41,7 +40,7 @@ docker-qemu-src: $(DOCKER_SRC_COPY)
# General rule for building docker images.
docker-image-%: $(DOCKER_FILES_DIR)/%.docker
$(call quiet-command, \
- DOCKER_BUILDKIT=1 $(RUNC) build \
+ DOCKER_BUILDKIT=1 $(CONTAINER_COMMAND) build \
$(if $(DOCKER_V),,--quiet) \
$(if $(NOCACHE),--no-cache, \
$(if $(DOCKER_REGISTRY),--cache-from $(DOCKER_REGISTRY)/qemu/$*)) \
@@ -152,7 +151,7 @@ $(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGES)), \
)
docker:
- @echo 'Build QEMU and run tests inside $(RUNC) containers'
+ @echo 'Build QEMU and run tests inside $(CONTAINER_COMMAND) containers'
@echo
@echo 'Available targets:'
@echo
@@ -219,10 +218,10 @@ docker-run: docker-qemu-src
$(IMAGE) --executable $(EXECUTABLE), \
" COPYING $(EXECUTABLE) to $(IMAGE)"))
$(call quiet-command, \
- $(RUNC) run \
+ $(CONTAINER_COMMAND) run \
--rm \
$(if $(NOUSER),, \
- $(if $(filter docker,$(RUNC)), \
+ $(if $(filter docker,$(CONTAINER_COMMAND)), \
-u $(UID), \
--userns keep-id \
) \
diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 9e18b984f4..d2f39b5645 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -35,27 +35,6 @@
DEVNULL = open(os.devnull, 'wb')
-class EngineEnum(enum.IntEnum):
- AUTO = 1
- DOCKER = 2
- PODMAN = 3
-
- def __str__(self):
- return self.name.lower()
-
- def __repr__(self):
- return str(self)
-
- @staticmethod
- def argparse(s):
- try:
- return EngineEnum[s.upper()]
- except KeyError:
- return s
-
-
-USE_ENGINE = EngineEnum.AUTO
-
def _bytes_checksum(bytes):
"""Calculate a digest string unique to the text content"""
return hashlib.sha1(bytes).hexdigest()
@@ -73,12 +52,11 @@ def _file_checksum(filename):
def _guess_engine_command():
""" Guess a working engine command or raise exception if not found"""
- commands = []
-
- if USE_ENGINE in [EngineEnum.AUTO, EngineEnum.PODMAN]:
- commands += [["podman"], ["podman-remote"], ["podman", "--remote"]]
- if USE_ENGINE in [EngineEnum.AUTO, EngineEnum.DOCKER]:
- commands += [["docker"], ["sudo", "-n", "docker"]]
+ commands = [["podman"],
+ ["podman-remote"],
+ ["podman", "--remote"],
+ ["docker"],
+ ["sudo", "-n", "docker"]]
for cmd in commands:
try:
# 'version' is not sufficient to prove a working binary
@@ -222,8 +200,11 @@ def _dockerfile_verify_flat(df):
class Docker(object):
""" Running Docker commands """
- def __init__(self):
- self._command = _guess_engine_command()
+ def __init__(self, commandstr=None):
+ if commandstr is None:
+ self._command = _guess_engine_command()
+ else:
+ self._command = commandstr.split(" ")
if ("docker" in self._command and
"TRAVIS" not in os.environ and
@@ -411,8 +392,8 @@ def args(self, parser):
help="Run container using the current user's uid")
def run(self, args, argv):
- return Docker().run(argv, args.keep, quiet=args.quiet,
- as_user=args.run_as_current_user)
+ return Docker(args.command).run(argv, args.keep, quiet=args.quiet,
+ as_user=args.run_as_current_user)
class BuildCommand(SubCommand):
@@ -445,7 +426,7 @@ def run(self, args, argv):
dockerfile = _read_dockerfile(args.dockerfile)
tag = args.tag
- dkr = Docker()
+ dkr = Docker(args.command)
if "--no-cache" not in argv and \
dkr.image_matches_dockerfile(tag, dockerfile):
if not args.quiet:
@@ -512,7 +493,7 @@ def args(self, parser):
help="Docker registry")
def run(self, args, argv):
- dkr = Docker()
+ dkr = Docker(args.command)
dkr.command(cmd="pull", quiet=args.quiet,
argv=["%s/%s" % (args.registry, args.tag)])
dkr.command(cmd="tag", quiet=args.quiet,
@@ -590,7 +571,7 @@ def run(self, args, argv):
tmp.seek(0)
# Run the build with our tarball context
- dkr = Docker()
+ dkr = Docker(args.command)
dkr.update_image(args.tag, tmp, quiet=args.quiet)
return 0
@@ -601,7 +582,7 @@ class CleanCommand(SubCommand):
name = "clean"
def run(self, args, argv):
- Docker().clean()
+ Docker(args.command).clean()
return 0
@@ -610,7 +591,7 @@ class ImagesCommand(SubCommand):
name = "images"
def run(self, args, argv):
- return Docker().command("images", argv, args.quiet)
+ return Docker(args.command).command("images", argv, args.quiet)
class ProbeCommand(SubCommand):
@@ -619,7 +600,7 @@ class ProbeCommand(SubCommand):
def run(self, args, argv):
try:
- docker = Docker()
+ docker = Docker(args.command)
print(" ".join(docker._command))
except Exception:
print("no")
@@ -651,18 +632,16 @@ def run(self, args, argv):
cmd += ["-v", "%s:%s:ro,z" % (p, p)]
cmd += [args.image, args.cc]
cmd += argv
- return Docker().run(cmd, False, quiet=args.quiet,
- as_user=True)
+ return Docker(args.command).run(cmd, False, quiet=args.quiet,
+ as_user=True)
def main():
- global USE_ENGINE
-
parser = argparse.ArgumentParser(description="A Docker helper",
usage="%s <subcommand> ..." %
os.path.basename(sys.argv[0]))
- parser.add_argument("--engine", type=EngineEnum.argparse, choices=list(EngineEnum),
- help="specify which container engine to use")
+ parser.add_argument("--command",
+ help="specify which container engine command to use")
subparsers = parser.add_subparsers(title="subcommands", help=None)
for cls in SubCommand.__subclasses__():
cmd = cls()
@@ -671,8 +650,6 @@ def main():
cmd.args(subp)
subp.set_defaults(cmdobj=cmd)
args, argv = parser.parse_known_args()
- if args.engine:
- USE_ENGINE = args.engine
return args.cmdobj.run(args, argv)
--
2.55.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v2] tests/docker: ensure container command is probed at most once
2026-07-15 10:42 [PATCH v2] tests/docker: ensure container command is probed at most once Daniel P. Berrangé
@ 2026-07-15 16:07 ` Pierrick Bouvier
2026-07-15 16:16 ` Daniel P. Berrangé
0 siblings, 1 reply; 6+ messages in thread
From: Pierrick Bouvier @ 2026-07-15 16:07 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel; +Cc: Paolo Bonzini, Alex Bennée
On 7/15/2026 3:42 AM, Daniel P. Berrangé wrote:
> The '--engine' arg accepts either 'podman' or 'docker', which is
> not sufficiently granular to map directly to a command. This
> means that docker.py still has to then probe the exact command
> to use.
>
> Meanwhile the 'probe' command prints out the full command to use
> but this cannot be passed back to docker.py to avoid probing
> again, so the caching is only useful in the few case where we
> run a container directly bypassing docker.py.
>
> Address this by replacing --engine with --command for docker.py.
>
> This in turn requires the --container-engine configure arg to be
> replaced with --container-command.
>
> With these changes the container command is probed at most once
> during configure and never again, while running in an unconfigured
> tree will still probe on demand.
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>
> In v2:
>
> * Don't set container_command to "no" if 'probe' finds
> not viable binaries.
>
> configure | 25 +++++++------
> tests/docker/Makefile.include | 13 ++++---
> tests/docker/docker.py | 67 ++++++++++++-----------------------
> 3 files changed, 40 insertions(+), 65 deletions(-)
>
> diff --git a/configure b/configure
> index d8bc10060e..5a8e2fd278 100755
> --- a/configure
> +++ b/configure
> @@ -172,7 +172,7 @@ fi
> # some defaults, based on the host environment
>
> # default parameters
> -container_engine="auto"
> +container_command=""
> cpu=""
> cross_compile="no"
> cross_prefix=""
> @@ -734,7 +734,7 @@ for opt do
> ;;
> --disable-containers) use_containers="no"
> ;;
> - --container-engine=*) container_engine="$optarg"
> + --container-command=*) container_command="$optarg"
> ;;
> --rust-target-triple=*) rust_target_triple="$optarg"
> ;;
> @@ -869,7 +869,7 @@ Advanced options (experts only):
> --enable-debug enable common debug build options
> --cpu=CPU Build for host CPU [$cpu]
> --disable-containers don't use containers for cross-building
> - --container-engine=TYPE which container engine to use [$container_engine]
> + --container-command=CMD which container command to use [autodetect]
> --gdb=GDB-path gdb to use for gdbstub tests [$gdb_bin]
> --wasm64-32bit-address-limit Restrict wasm64 address space to 32-bit (default
> is to use the whole 64-bit range).
> @@ -1291,12 +1291,12 @@ fi
> ##########################################
> # functions to probe cross compilers
>
> -runc="no"
> -if test $use_containers = "yes" && (has "docker" || has "podman"); then
> - runc=$($python "$source_path"/tests/docker/docker.py --engine "$container_engine" probe)
> - if test "$runc" != "no"; then
> - docker_py="$python $source_path/tests/docker/docker.py --engine $container_engine"
> - fi
> +if test "$container_command" = ""; then
> + container_command=$($python "$source_path"/tests/docker/docker.py probe)
> + test "$container_command" == "no" && container_command=""
> +fi
> +if test $use_containers = "yes" && test "$container_command" != ""; then
> + docker_py="$python $source_path/tests/docker/docker.py --command $container_command"
> fi
>
> # cross compilers defaults, can be overridden with --cross-cc-ARCH
> @@ -1415,7 +1415,7 @@ probe_target_compiler() {
> esac
>
> for host in $container_hosts; do
> - test "$runc" != no || continue
> + test "$container_command" != "" || continue
> test "$host" = "$cpu" || continue
> case $target_arch in
> # debian-all-test-cross architectures
> @@ -1736,9 +1736,8 @@ echo all: >> $config_host_mak
> echo "SRC_PATH=$source_path" >> $config_host_mak
> echo "TARGET_DIRS=$target_list" >> $config_host_mak
> echo "GDB=$gdb_bin" >> $config_host_mak
> -if test "$runc" != no; then
> - echo "RUNC=$runc" >> $config_host_mak
> - echo "CONTAINER_ENGINE=$container_engine" >> $config_host_mak
> +if test "$container_command" != ""; then
> + echo "CONTAINER_COMMAND=$container_command" >> $config_host_mak
> fi
> echo "SUBDIRS=$subdirs" >> $config_host_mak
> if test "$rust" != disabled; then
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index 4725c39807..0adddb6a5c 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -16,9 +16,8 @@ DOCKER_DEFAULT_REGISTRY := registry.gitlab.com/qemu-project/qemu
> endif
> DOCKER_REGISTRY := $(if $(REGISTRY),$(REGISTRY),$(DOCKER_DEFAULT_REGISTRY))
>
> -CONTAINER_ENGINE = auto
> -DOCKER_SCRIPT=$(SRC_PATH)/tests/docker/docker.py --engine $(CONTAINER_ENGINE)
> -RUNC ?= $(shell $(DOCKER_SCRIPT) probe)
> +CONTAINER_COMMAND ?= $(shell $(SRC_PATH)/tests/docker/docker.py probe)
> +DOCKER_SCRIPT=$(SRC_PATH)/tests/docker/docker.py --command "$(CONTAINER_COMMAND)"
>
> CUR_TIME := $(shell date +%Y-%m-%d-%H.%M.%S.$$$$)
> DOCKER_SRC_COPY := $(BUILD_DIR)/docker-src.$(CUR_TIME)
> @@ -41,7 +40,7 @@ docker-qemu-src: $(DOCKER_SRC_COPY)
> # General rule for building docker images.
> docker-image-%: $(DOCKER_FILES_DIR)/%.docker
> $(call quiet-command, \
> - DOCKER_BUILDKIT=1 $(RUNC) build \
> + DOCKER_BUILDKIT=1 $(CONTAINER_COMMAND) build \
> $(if $(DOCKER_V),,--quiet) \
> $(if $(NOCACHE),--no-cache, \
> $(if $(DOCKER_REGISTRY),--cache-from $(DOCKER_REGISTRY)/qemu/$*)) \
> @@ -152,7 +151,7 @@ $(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGES)), \
> )
>
> docker:
> - @echo 'Build QEMU and run tests inside $(RUNC) containers'
> + @echo 'Build QEMU and run tests inside $(CONTAINER_COMMAND) containers'
> @echo
> @echo 'Available targets:'
> @echo
> @@ -219,10 +218,10 @@ docker-run: docker-qemu-src
> $(IMAGE) --executable $(EXECUTABLE), \
> " COPYING $(EXECUTABLE) to $(IMAGE)"))
> $(call quiet-command, \
> - $(RUNC) run \
> + $(CONTAINER_COMMAND) run \
> --rm \
> $(if $(NOUSER),, \
> - $(if $(filter docker,$(RUNC)), \
> + $(if $(filter docker,$(CONTAINER_COMMAND)), \
> -u $(UID), \
> --userns keep-id \
> ) \
> diff --git a/tests/docker/docker.py b/tests/docker/docker.py
> index 9e18b984f4..d2f39b5645 100755
> --- a/tests/docker/docker.py
> +++ b/tests/docker/docker.py
> @@ -35,27 +35,6 @@
>
> DEVNULL = open(os.devnull, 'wb')
>
> -class EngineEnum(enum.IntEnum):
> - AUTO = 1
> - DOCKER = 2
> - PODMAN = 3
> -
> - def __str__(self):
> - return self.name.lower()
> -
> - def __repr__(self):
> - return str(self)
> -
> - @staticmethod
> - def argparse(s):
> - try:
> - return EngineEnum[s.upper()]
> - except KeyError:
> - return s
> -
> -
> -USE_ENGINE = EngineEnum.AUTO
> -
> def _bytes_checksum(bytes):
> """Calculate a digest string unique to the text content"""
> return hashlib.sha1(bytes).hexdigest()
> @@ -73,12 +52,11 @@ def _file_checksum(filename):
>
> def _guess_engine_command():
> """ Guess a working engine command or raise exception if not found"""
> - commands = []
> -
> - if USE_ENGINE in [EngineEnum.AUTO, EngineEnum.PODMAN]:
> - commands += [["podman"], ["podman-remote"], ["podman", "--remote"]]
> - if USE_ENGINE in [EngineEnum.AUTO, EngineEnum.DOCKER]:
> - commands += [["docker"], ["sudo", "-n", "docker"]]
> + commands = [["podman"],
> + ["podman-remote"],
> + ["podman", "--remote"],
> + ["docker"],
> + ["sudo", "-n", "docker"]]
> for cmd in commands:
> try:
> # 'version' is not sufficient to prove a working binary
> @@ -222,8 +200,11 @@ def _dockerfile_verify_flat(df):
>
> class Docker(object):
> """ Running Docker commands """
> - def __init__(self):
> - self._command = _guess_engine_command()
> + def __init__(self, commandstr=None):
> + if commandstr is None:
> + self._command = _guess_engine_command()
> + else:
> + self._command = commandstr.split(" ")
>
> if ("docker" in self._command and
> "TRAVIS" not in os.environ and
> @@ -411,8 +392,8 @@ def args(self, parser):
> help="Run container using the current user's uid")
>
> def run(self, args, argv):
> - return Docker().run(argv, args.keep, quiet=args.quiet,
> - as_user=args.run_as_current_user)
> + return Docker(args.command).run(argv, args.keep, quiet=args.quiet,
> + as_user=args.run_as_current_user)
>
>
> class BuildCommand(SubCommand):
> @@ -445,7 +426,7 @@ def run(self, args, argv):
> dockerfile = _read_dockerfile(args.dockerfile)
> tag = args.tag
>
> - dkr = Docker()
> + dkr = Docker(args.command)
> if "--no-cache" not in argv and \
> dkr.image_matches_dockerfile(tag, dockerfile):
> if not args.quiet:
> @@ -512,7 +493,7 @@ def args(self, parser):
> help="Docker registry")
>
> def run(self, args, argv):
> - dkr = Docker()
> + dkr = Docker(args.command)
> dkr.command(cmd="pull", quiet=args.quiet,
> argv=["%s/%s" % (args.registry, args.tag)])
> dkr.command(cmd="tag", quiet=args.quiet,
> @@ -590,7 +571,7 @@ def run(self, args, argv):
> tmp.seek(0)
>
> # Run the build with our tarball context
> - dkr = Docker()
> + dkr = Docker(args.command)
> dkr.update_image(args.tag, tmp, quiet=args.quiet)
>
> return 0
> @@ -601,7 +582,7 @@ class CleanCommand(SubCommand):
> name = "clean"
>
> def run(self, args, argv):
> - Docker().clean()
> + Docker(args.command).clean()
> return 0
>
>
> @@ -610,7 +591,7 @@ class ImagesCommand(SubCommand):
> name = "images"
>
> def run(self, args, argv):
> - return Docker().command("images", argv, args.quiet)
> + return Docker(args.command).command("images", argv, args.quiet)
>
>
> class ProbeCommand(SubCommand):
> @@ -619,7 +600,7 @@ class ProbeCommand(SubCommand):
>
> def run(self, args, argv):
> try:
> - docker = Docker()
> + docker = Docker(args.command)
> print(" ".join(docker._command))
> except Exception:
> print("no")
> @@ -651,18 +632,16 @@ def run(self, args, argv):
> cmd += ["-v", "%s:%s:ro,z" % (p, p)]
> cmd += [args.image, args.cc]
> cmd += argv
> - return Docker().run(cmd, False, quiet=args.quiet,
> - as_user=True)
> + return Docker(args.command).run(cmd, False, quiet=args.quiet,
> + as_user=True)
>
>
> def main():
> - global USE_ENGINE
> -
> parser = argparse.ArgumentParser(description="A Docker helper",
> usage="%s <subcommand> ..." %
> os.path.basename(sys.argv[0]))
> - parser.add_argument("--engine", type=EngineEnum.argparse, choices=list(EngineEnum),
> - help="specify which container engine to use")
> + parser.add_argument("--command",
> + help="specify which container engine command to use")
> subparsers = parser.add_subparsers(title="subcommands", help=None)
> for cls in SubCommand.__subclasses__():
> cmd = cls()
> @@ -671,8 +650,6 @@ def main():
> cmd.args(subp)
> subp.set_defaults(cmdobj=cmd)
> args, argv = parser.parse_known_args()
> - if args.engine:
> - USE_ENGINE = args.engine
> return args.cmdobj.run(args, argv)
>
>
I still observe the same error when running make-tcg without
podman/docker installed.
bash: line 1: no: command not found
And I will suggest again to include in your series:
https://lore.kernel.org/qemu-devel/20260710214343.2065491-27-pierrick.bouvier@oss.qualcomm.com/
and detect if probe command worked or failed, and adapt behavior based
on it. Existing behavior to compare to a string "no" is a very
unfamiliar pattern for any command. `which` does not return no for instance.
You can test next patch using podman (which it seems you have):
```
podman build \
-f tests/docker/dockerfiles/debian-all-test-cross.docker -t all
podman run -it --rm all -w $(pwd) -v $(pwd):$(pwd) all \
bash -c "./configure && cd build && make all -j32 &&make check-tcg -j32"
```
Regards,
Pierrick
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2] tests/docker: ensure container command is probed at most once
2026-07-15 16:07 ` Pierrick Bouvier
@ 2026-07-15 16:16 ` Daniel P. Berrangé
2026-07-15 16:22 ` Pierrick Bouvier
0 siblings, 1 reply; 6+ messages in thread
From: Daniel P. Berrangé @ 2026-07-15 16:16 UTC (permalink / raw)
To: Pierrick Bouvier; +Cc: qemu-devel, Paolo Bonzini, Alex Bennée
On Wed, Jul 15, 2026 at 09:07:18AM -0700, Pierrick Bouvier wrote:
> On 7/15/2026 3:42 AM, Daniel P. Berrangé wrote:
> > The '--engine' arg accepts either 'podman' or 'docker', which is
> > not sufficiently granular to map directly to a command. This
> > means that docker.py still has to then probe the exact command
> > to use.
> >
> > Meanwhile the 'probe' command prints out the full command to use
> > but this cannot be passed back to docker.py to avoid probing
> > again, so the caching is only useful in the few case where we
> > run a container directly bypassing docker.py.
> >
> > Address this by replacing --engine with --command for docker.py.
> >
> > This in turn requires the --container-engine configure arg to be
> > replaced with --container-command.
> >
> > With these changes the container command is probed at most once
> > during configure and never again, while running in an unconfigured
> > tree will still probe on demand.
> >
> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > ---
> >
> > In v2:
> >
> > * Don't set container_command to "no" if 'probe' finds
> > not viable binaries.
> >
> > configure | 25 +++++++------
> > tests/docker/Makefile.include | 13 ++++---
> > tests/docker/docker.py | 67 ++++++++++++-----------------------
> > 3 files changed, 40 insertions(+), 65 deletions(-)
>
> I still observe the same error when running make-tcg without
> podman/docker installed.
> bash: line 1: no: command not found
>
> And I will suggest again to include in your series:
> https://lore.kernel.org/qemu-devel/20260710214343.2065491-27-pierrick.bouvier@oss.qualcomm.com/
> and detect if probe command worked or failed, and adapt behavior based
> on it. Existing behavior to compare to a string "no" is a very
> unfamiliar pattern for any command. `which` does not return no for instance.
That didn't do anything for me when I tested it, and this current
patch did fix the problem. Are you testing on top of a vanilla
git master, or did you have other patches that aren't merged
upstream.
When /usr/bin/podman was absent
$ ./configure --target-list=s390x-linux-user
...
$ grep CONTAINER build/config-host.mak
<no match>
$ make run-tcg-tests-s390x-linux-user
changing dir to build for make "run-tcg-tests-s390x-linux-user"...
make[1]: Entering directory '/home/berrange/src/virt/qemu/build'
ninja: no work to do.
/home/berrange/src/virt/qemu/build/pyvenv/bin/meson introspect --targets --tests --benchmarks | /home/berrange/src/virt/qemu/build/pyvenv/bin/python3 -B scripts/mtest2make.py > Makefile.mtest
make[1]: *** No rule to make target 'run-tcg-tests-s390x-linux-user'. Stop.
configure has correctly skipped setup of the tcg cross
compiler targets.
When /usr/bin/podman was re-instaled it once again
worked
$ ./configure --target-list=s390x-linux-user
...
$ grep CONTAINER build/config-host.mak
CONTAINER_COMMAND=podman
$ make run-tcg-tests-s390x-linux-user
changing dir to build for make "run-tcg-tests-s390x-linux-user"...
make[1]: Entering directory '/home/berrange/src/virt/qemu/build'
ninja: no work to do.
/home/berrange/src/virt/qemu/build/pyvenv/bin/meson introspect --targets --tests --benchmarks | /home/berrange/src/virt/qemu/build/pyvenv/bin/python3 -B scripts/mtest2make.py > Makefile.mtest
BUILD debian-s390x-cross
BUILD s390x-linux-user guest-tests
tests/tcg/s390x-linux-user: -march=z14 detected
tests/tcg/s390x-linux-user: -march=z15 detected
...tests running...
> You can test next patch using podman (which it seems you have):
> ```
> podman build \
> -f tests/docker/dockerfiles/debian-all-test-cross.docker -t all
> podman run -it --rm all -w $(pwd) -v $(pwd):$(pwd) all \
> bash -c "./configure && cd build && make all -j32 &&make check-tcg -j32"
> ```
I've run 'make check-tcg' and it correctly skips all tests when
podman is absent
$ make check-tcg
changing dir to build for make "check-tcg"...
make[1]: Entering directory '/home/berrange/src/virt/qemu/build'
make[1]: Nothing to be done for 'check-tcg'.
make[1]: Leaving directory '/home/berrange/src/virt/qemu/build'
With regards,
Daniel
--
|: https://berrange.com ~~ https://hachyderm.io/@berrange :|
|: https://libvirt.org ~~ https://entangle-photo.org :|
|: https://pixelfed.art/berrange ~~ https://fstop138.berrange.com :|
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2] tests/docker: ensure container command is probed at most once
2026-07-15 16:16 ` Daniel P. Berrangé
@ 2026-07-15 16:22 ` Pierrick Bouvier
2026-07-15 16:31 ` Daniel P. Berrangé
0 siblings, 1 reply; 6+ messages in thread
From: Pierrick Bouvier @ 2026-07-15 16:22 UTC (permalink / raw)
To: Daniel P. Berrangé; +Cc: qemu-devel, Paolo Bonzini, Alex Bennée
On 7/15/2026 9:16 AM, Daniel P. Berrangé wrote:
> On Wed, Jul 15, 2026 at 09:07:18AM -0700, Pierrick Bouvier wrote:
>> On 7/15/2026 3:42 AM, Daniel P. Berrangé wrote:
>>> The '--engine' arg accepts either 'podman' or 'docker', which is
>>> not sufficiently granular to map directly to a command. This
>>> means that docker.py still has to then probe the exact command
>>> to use.
>>>
>>> Meanwhile the 'probe' command prints out the full command to use
>>> but this cannot be passed back to docker.py to avoid probing
>>> again, so the caching is only useful in the few case where we
>>> run a container directly bypassing docker.py.
>>>
>>> Address this by replacing --engine with --command for docker.py.
>>>
>>> This in turn requires the --container-engine configure arg to be
>>> replaced with --container-command.
>>>
>>> With these changes the container command is probed at most once
>>> during configure and never again, while running in an unconfigured
>>> tree will still probe on demand.
>>>
>>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>>> ---
>>>
>>> In v2:
>>>
>>> * Don't set container_command to "no" if 'probe' finds
>>> not viable binaries.
>>>
>>> configure | 25 +++++++------
>>> tests/docker/Makefile.include | 13 ++++---
>>> tests/docker/docker.py | 67 ++++++++++++-----------------------
>>> 3 files changed, 40 insertions(+), 65 deletions(-)
>
>
>>
>> I still observe the same error when running make-tcg without
>> podman/docker installed.
>> bash: line 1: no: command not found
>>
>> And I will suggest again to include in your series:
>> https://lore.kernel.org/qemu-devel/20260710214343.2065491-27-pierrick.bouvier@oss.qualcomm.com/
>> and detect if probe command worked or failed, and adapt behavior based
>> on it. Existing behavior to compare to a string "no" is a very
>> unfamiliar pattern for any command. `which` does not return no for instance.
>
> That didn't do anything for me when I tested it, and this current
> patch did fix the problem. Are you testing on top of a vanilla
> git master, or did you have other patches that aren't merged
> upstream.
>
Originally caught on GitHub.
If you can survive to have to log on this evil platform:
https://github.com/p-b-o/qemu-ci/actions/runs/29410261308/job/87335442476
Which I reproduced on my machine locally using the command I posted at
the end of my email.
Top of master + this patch, without any other patch.
> When /usr/bin/podman was absent
>
> $ ./configure --target-list=s390x-linux-user
> ...
> $ grep CONTAINER build/config-host.mak
> <no match>
>
> $ make run-tcg-tests-s390x-linux-user
> changing dir to build for make "run-tcg-tests-s390x-linux-user"...
> make[1]: Entering directory '/home/berrange/src/virt/qemu/build'
> ninja: no work to do.
> /home/berrange/src/virt/qemu/build/pyvenv/bin/meson introspect --targets --tests --benchmarks | /home/berrange/src/virt/qemu/build/pyvenv/bin/python3 -B scripts/mtest2make.py > Makefile.mtest
> make[1]: *** No rule to make target 'run-tcg-tests-s390x-linux-user'. Stop.
>
> configure has correctly skipped setup of the tcg cross
> compiler targets.
>
> When /usr/bin/podman was re-instaled it once again
> worked
>
>
> $ ./configure --target-list=s390x-linux-user
> ...
> $ grep CONTAINER build/config-host.mak
> CONTAINER_COMMAND=podman
>
> $ make run-tcg-tests-s390x-linux-user
> changing dir to build for make "run-tcg-tests-s390x-linux-user"...
> make[1]: Entering directory '/home/berrange/src/virt/qemu/build'
> ninja: no work to do.
> /home/berrange/src/virt/qemu/build/pyvenv/bin/meson introspect --targets --tests --benchmarks | /home/berrange/src/virt/qemu/build/pyvenv/bin/python3 -B scripts/mtest2make.py > Makefile.mtest
> BUILD debian-s390x-cross
> BUILD s390x-linux-user guest-tests
> tests/tcg/s390x-linux-user: -march=z14 detected
> tests/tcg/s390x-linux-user: -march=z15 detected
> ...tests running...
>
>> You can test next patch using podman (which it seems you have):
>> ```
>> podman build \
>> -f tests/docker/dockerfiles/debian-all-test-cross.docker -t all
>> podman run -it --rm all -w $(pwd) -v $(pwd):$(pwd) all \
>> bash -c "./configure && cd build && make all -j32 &&make check-tcg -j32"
>> ```
>
Try to run the exact command above, and we'll see if you can reproduce
or not.
> I've run 'make check-tcg' and it correctly skips all tests when
> podman is absent
>
> $ make check-tcg
> changing dir to build for make "check-tcg"...
> make[1]: Entering directory '/home/berrange/src/virt/qemu/build'
> make[1]: Nothing to be done for 'check-tcg'.
> make[1]: Leaving directory '/home/berrange/src/virt/qemu/build'
>
>
> With regards,
> Daniel
Regards,
Pierrick
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2] tests/docker: ensure container command is probed at most once
2026-07-15 16:22 ` Pierrick Bouvier
@ 2026-07-15 16:31 ` Daniel P. Berrangé
2026-07-15 16:40 ` Pierrick Bouvier
0 siblings, 1 reply; 6+ messages in thread
From: Daniel P. Berrangé @ 2026-07-15 16:31 UTC (permalink / raw)
To: Pierrick Bouvier; +Cc: qemu-devel, Paolo Bonzini, Alex Bennée
On Wed, Jul 15, 2026 at 09:22:05AM -0700, Pierrick Bouvier wrote:
> On 7/15/2026 9:16 AM, Daniel P. Berrangé wrote:
> > On Wed, Jul 15, 2026 at 09:07:18AM -0700, Pierrick Bouvier wrote:
> >> On 7/15/2026 3:42 AM, Daniel P. Berrangé wrote:
> >>> The '--engine' arg accepts either 'podman' or 'docker', which is
> >>> not sufficiently granular to map directly to a command. This
> >>> means that docker.py still has to then probe the exact command
> >>> to use.
> >>>
> >>> Meanwhile the 'probe' command prints out the full command to use
> >>> but this cannot be passed back to docker.py to avoid probing
> >>> again, so the caching is only useful in the few case where we
> >>> run a container directly bypassing docker.py.
> >>>
> >>> Address this by replacing --engine with --command for docker.py.
> >>>
> >>> This in turn requires the --container-engine configure arg to be
> >>> replaced with --container-command.
> >>>
> >>> With these changes the container command is probed at most once
> >>> during configure and never again, while running in an unconfigured
> >>> tree will still probe on demand.
> >>>
> >>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> >>> ---
> >>>
> >>> In v2:
> >>>
> >>> * Don't set container_command to "no" if 'probe' finds
> >>> not viable binaries.
> >>>
> >>> configure | 25 +++++++------
> >>> tests/docker/Makefile.include | 13 ++++---
> >>> tests/docker/docker.py | 67 ++++++++++++-----------------------
> >>> 3 files changed, 40 insertions(+), 65 deletions(-)
> >
> >
> >>
> >> I still observe the same error when running make-tcg without
> >> podman/docker installed.
> >> bash: line 1: no: command not found
> >>
> >> And I will suggest again to include in your series:
> >> https://lore.kernel.org/qemu-devel/20260710214343.2065491-27-pierrick.bouvier@oss.qualcomm.com/
> >> and detect if probe command worked or failed, and adapt behavior based
> >> on it. Existing behavior to compare to a string "no" is a very
> >> unfamiliar pattern for any command. `which` does not return no for instance.
> >
> > That didn't do anything for me when I tested it, and this current
> > patch did fix the problem. Are you testing on top of a vanilla
> > git master, or did you have other patches that aren't merged
> > upstream.
> >
>
> Originally caught on GitHub.
> If you can survive to have to log on this evil platform:
> https://github.com/p-b-o/qemu-ci/actions/runs/29410261308/job/87335442476
>
> Which I reproduced on my machine locally using the command I posted at
> the end of my email.
> Top of master + this patch, without any other patch.
>
> > When /usr/bin/podman was absent
> >
> > $ ./configure --target-list=s390x-linux-user
> > ...
> > $ grep CONTAINER build/config-host.mak
> > <no match>
> >
> > $ make run-tcg-tests-s390x-linux-user
> > changing dir to build for make "run-tcg-tests-s390x-linux-user"...
> > make[1]: Entering directory '/home/berrange/src/virt/qemu/build'
> > ninja: no work to do.
> > /home/berrange/src/virt/qemu/build/pyvenv/bin/meson introspect --targets --tests --benchmarks | /home/berrange/src/virt/qemu/build/pyvenv/bin/python3 -B scripts/mtest2make.py > Makefile.mtest
> > make[1]: *** No rule to make target 'run-tcg-tests-s390x-linux-user'. Stop.
> >
> > configure has correctly skipped setup of the tcg cross
> > compiler targets.
> >
> > When /usr/bin/podman was re-instaled it once again
> > worked
> >
> >
> > $ ./configure --target-list=s390x-linux-user
> > ...
> > $ grep CONTAINER build/config-host.mak
> > CONTAINER_COMMAND=podman
> >
> > $ make run-tcg-tests-s390x-linux-user
> > changing dir to build for make "run-tcg-tests-s390x-linux-user"...
> > make[1]: Entering directory '/home/berrange/src/virt/qemu/build'
> > ninja: no work to do.
> > /home/berrange/src/virt/qemu/build/pyvenv/bin/meson introspect --targets --tests --benchmarks | /home/berrange/src/virt/qemu/build/pyvenv/bin/python3 -B scripts/mtest2make.py > Makefile.mtest
> > BUILD debian-s390x-cross
> > BUILD s390x-linux-user guest-tests
> > tests/tcg/s390x-linux-user: -march=z14 detected
> > tests/tcg/s390x-linux-user: -march=z15 detected
> > ...tests running...
> >
> >> You can test next patch using podman (which it seems you have):
> >> ```
> >> podman build \
> >> -f tests/docker/dockerfiles/debian-all-test-cross.docker -t all
> >> podman run -it --rm all -w $(pwd) -v $(pwd):$(pwd) all \
> >> bash -c "./configure && cd build && make all -j32 &&make check-tcg -j32"
> >> ```
> >
>
> Try to run the exact command above, and we'll see if you can reproduce
> or not.
Oh it is the bash-ism test "==" vs "="
With regards,
Daniel
--
|: https://berrange.com ~~ https://hachyderm.io/@berrange :|
|: https://libvirt.org ~~ https://entangle-photo.org :|
|: https://pixelfed.art/berrange ~~ https://fstop138.berrange.com :|
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2] tests/docker: ensure container command is probed at most once
2026-07-15 16:31 ` Daniel P. Berrangé
@ 2026-07-15 16:40 ` Pierrick Bouvier
0 siblings, 0 replies; 6+ messages in thread
From: Pierrick Bouvier @ 2026-07-15 16:40 UTC (permalink / raw)
To: Daniel P. Berrangé; +Cc: qemu-devel, Paolo Bonzini, Alex Bennée
On 7/15/2026 9:31 AM, Daniel P. Berrangé wrote:
> On Wed, Jul 15, 2026 at 09:22:05AM -0700, Pierrick Bouvier wrote:
>> On 7/15/2026 9:16 AM, Daniel P. Berrangé wrote:
>>> On Wed, Jul 15, 2026 at 09:07:18AM -0700, Pierrick Bouvier wrote:
>>>> On 7/15/2026 3:42 AM, Daniel P. Berrangé wrote:
>>>>> The '--engine' arg accepts either 'podman' or 'docker', which is
>>>>> not sufficiently granular to map directly to a command. This
>>>>> means that docker.py still has to then probe the exact command
>>>>> to use.
>>>>>
>>>>> Meanwhile the 'probe' command prints out the full command to use
>>>>> but this cannot be passed back to docker.py to avoid probing
>>>>> again, so the caching is only useful in the few case where we
>>>>> run a container directly bypassing docker.py.
>>>>>
>>>>> Address this by replacing --engine with --command for docker.py.
>>>>>
>>>>> This in turn requires the --container-engine configure arg to be
>>>>> replaced with --container-command.
>>>>>
>>>>> With these changes the container command is probed at most once
>>>>> during configure and never again, while running in an unconfigured
>>>>> tree will still probe on demand.
>>>>>
>>>>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>>>>> ---
>>>>>
>>>>> In v2:
>>>>>
>>>>> * Don't set container_command to "no" if 'probe' finds
>>>>> not viable binaries.
>>>>>
>>>>> configure | 25 +++++++------
>>>>> tests/docker/Makefile.include | 13 ++++---
>>>>> tests/docker/docker.py | 67 ++++++++++++-----------------------
>>>>> 3 files changed, 40 insertions(+), 65 deletions(-)
>>>
>>>
>>>>
>>>> I still observe the same error when running make-tcg without
>>>> podman/docker installed.
>>>> bash: line 1: no: command not found
>>>>
>>>> And I will suggest again to include in your series:
>>>> https://lore.kernel.org/qemu-devel/20260710214343.2065491-27-pierrick.bouvier@oss.qualcomm.com/
>>>> and detect if probe command worked or failed, and adapt behavior based
>>>> on it. Existing behavior to compare to a string "no" is a very
>>>> unfamiliar pattern for any command. `which` does not return no for instance.
>>>
>>> That didn't do anything for me when I tested it, and this current
>>> patch did fix the problem. Are you testing on top of a vanilla
>>> git master, or did you have other patches that aren't merged
>>> upstream.
>>>
>>
>> Originally caught on GitHub.
>> If you can survive to have to log on this evil platform:
>> https://github.com/p-b-o/qemu-ci/actions/runs/29410261308/job/87335442476
>>
>> Which I reproduced on my machine locally using the command I posted at
>> the end of my email.
>> Top of master + this patch, without any other patch.
>>
>>> When /usr/bin/podman was absent
>>>
>>> $ ./configure --target-list=s390x-linux-user
>>> ...
>>> $ grep CONTAINER build/config-host.mak
>>> <no match>
>>>
>>> $ make run-tcg-tests-s390x-linux-user
>>> changing dir to build for make "run-tcg-tests-s390x-linux-user"...
>>> make[1]: Entering directory '/home/berrange/src/virt/qemu/build'
>>> ninja: no work to do.
>>> /home/berrange/src/virt/qemu/build/pyvenv/bin/meson introspect --targets --tests --benchmarks | /home/berrange/src/virt/qemu/build/pyvenv/bin/python3 -B scripts/mtest2make.py > Makefile.mtest
>>> make[1]: *** No rule to make target 'run-tcg-tests-s390x-linux-user'. Stop.
>>>
>>> configure has correctly skipped setup of the tcg cross
>>> compiler targets.
>>>
>>> When /usr/bin/podman was re-instaled it once again
>>> worked
>>>
>>>
>>> $ ./configure --target-list=s390x-linux-user
>>> ...
>>> $ grep CONTAINER build/config-host.mak
>>> CONTAINER_COMMAND=podman
>>>
>>> $ make run-tcg-tests-s390x-linux-user
>>> changing dir to build for make "run-tcg-tests-s390x-linux-user"...
>>> make[1]: Entering directory '/home/berrange/src/virt/qemu/build'
>>> ninja: no work to do.
>>> /home/berrange/src/virt/qemu/build/pyvenv/bin/meson introspect --targets --tests --benchmarks | /home/berrange/src/virt/qemu/build/pyvenv/bin/python3 -B scripts/mtest2make.py > Makefile.mtest
>>> BUILD debian-s390x-cross
>>> BUILD s390x-linux-user guest-tests
>>> tests/tcg/s390x-linux-user: -march=z14 detected
>>> tests/tcg/s390x-linux-user: -march=z15 detected
>>> ...tests running...
>>>
>>>> You can test next patch using podman (which it seems you have):
>>>> ```
>>>> podman build \
>>>> -f tests/docker/dockerfiles/debian-all-test-cross.docker -t all
>>>> podman run -it --rm all -w $(pwd) -v $(pwd):$(pwd) all \
>>>> bash -c "./configure && cd build && make all -j32 &&make check-tcg -j32"
>>>> ```
>>>
>>
>> Try to run the exact command above, and we'll see if you can reproduce
>> or not.
>
> Oh it is the bash-ism test "==" vs "="
>
Maybe next time, you can try the repro before saying "it works on my
machine"™️ :). Someone with your experience knows that something
unexpected can always happen.
> With regards,
> Daniel
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-07-15 16:41 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-07-15 10:42 [PATCH v2] tests/docker: ensure container command is probed at most once Daniel P. Berrangé
2026-07-15 16:07 ` Pierrick Bouvier
2026-07-15 16:16 ` Daniel P. Berrangé
2026-07-15 16:22 ` Pierrick Bouvier
2026-07-15 16:31 ` Daniel P. Berrangé
2026-07-15 16:40 ` Pierrick Bouvier
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.