From: John Snow <jsnow@redhat.com>
To: "Alex Bennée" <alex.bennee@linaro.org>, qemu-devel@nongnu.org
Cc: fam@euphon.net, berrange@redhat.com, stefanb@linux.vnet.ibm.com,
richard.henderson@linaro.org, f4bug@amsat.org, cota@braap.org,
stefanha@redhat.com, pbonzini@redhat.com,
marcandre.lureau@redhat.com,
"Philippe Mathieu-Daudé" <philmd@redhat.com>,
aurelien@aurel32.net
Subject: Re: [Qemu-devel] [PATCH v1 03/42] tests/docker: fix "cc" command to work with podman
Date: Wed, 4 Sep 2019 19:31:32 -0400 [thread overview]
Message-ID: <b25bd465-bc39-936f-5c6b-114e2926aba4@redhat.com> (raw)
In-Reply-To: <20190904203013.9028-4-alex.bennee@linaro.org>
On 9/4/19 4:29 PM, Alex Bennée wrote:
> Podman requires a little bit of additional magic to the uid mapping
> which was already done for the normal RunCommand. We simplify the
> logic by pushing it directly into the Docker::run method to avoid
> instantiating an extra Docker() object and ensure the CC command
> always runs as the current user.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> tests/docker/docker.py | 30 +++++++++++++++---------------
> tests/tcg/Makefile.include | 2 +-
> 2 files changed, 16 insertions(+), 16 deletions(-)
>
> diff --git a/tests/docker/docker.py b/tests/docker/docker.py
> index e23209f71ee..8f391eb278b 100755
> --- a/tests/docker/docker.py
> +++ b/tests/docker/docker.py
> @@ -318,10 +318,20 @@ class Docker(object):
> return False
> return checksum == _text_checksum(_dockerfile_preprocess(dockerfile))
>
> - def run(self, cmd, keep, quiet):
> + def run(self, cmd, keep, quiet, as_user=False):
> label = uuid.uuid1().hex
> if not keep:
> self._instances.append(label)
> +
> + if as_user:
> + uid = os.getuid()
> + cmd = [ "-u", str(uid) ] + cmd
> + # podman requires a bit more fiddling
> + if self._command[0] == "podman":
> + cmd = [ "--uidmap", "%d:0:1" % uid,
> + "--uidmap", "0:1:%d" % uid,
> + "--uidmap", "%d:%d:64536" % (uid + 1, uid + 1)] + cmd
> +
I was having problems with constructs like these recently. I think we
either need to use --userns=keep-id (vastly preferred) or adjust 64536
there to read as "65536 - uid" because not everyone will have a UID of 1000.
(My UID is over 20,000 and for whatever reason, this causes podman to
crash very badly when using this hackaround.)
> ret = self._do_check(["run", "--label",
> "com.qemu.instance.uuid=" + label] + cmd,
> quiet=quiet)
> @@ -364,15 +374,8 @@ class RunCommand(SubCommand):
> help="Run container using the current user's uid")
>
> def run(self, args, argv):
> - if args.run_as_current_user:
> - uid = os.getuid()
> - argv = [ "-u", str(uid) ] + argv
> - docker = Docker()
> - if docker._command[0] == "podman":
> - argv = [ "--uidmap", "%d:0:1" % uid,
> - "--uidmap", "0:1:%d" % uid,
> - "--uidmap", "%d:%d:64536" % (uid + 1, uid + 1)] + argv
> - return Docker().run(argv, args.keep, quiet=args.quiet)
> + return Docker().run(argv, args.keep, quiet=args.quiet,
> + as_user=args.run_as_current_user)
>
>
> class BuildCommand(SubCommand):
> @@ -556,8 +559,6 @@ class CcCommand(SubCommand):
> help="The docker image in which to run cc")
> parser.add_argument("--cc", default="cc",
> help="The compiler executable to call")
> - parser.add_argument("--user",
> - help="The user-id to run under")
> parser.add_argument("--source-path", "-s", nargs="*", dest="paths",
> help="""Extra paths to (ro) mount into container for
> reading sources""")
> @@ -571,11 +572,10 @@ class CcCommand(SubCommand):
> if args.paths:
> for p in args.paths:
> cmd += ["-v", "%s:%s:ro,z" % (p, p)]
> - if args.user:
> - cmd += ["-u", args.user]
> cmd += [args.image, args.cc]
> cmd += argv
> - return Docker().command("run", cmd, args.quiet)
> + return Docker().run(cmd, False, quiet=args.quiet,
> + as_user=True)
>
>
> class CheckCommand(SubCommand):
> diff --git a/tests/tcg/Makefile.include b/tests/tcg/Makefile.include
> index 73b5626fc5f..210f8428237 100644
> --- a/tests/tcg/Makefile.include
> +++ b/tests/tcg/Makefile.include
> @@ -41,7 +41,7 @@ ifneq ($(DOCKER_IMAGE),)
> # We also need the Docker make rules to depend on
> include $(SRC_PATH)/tests/docker/Makefile.include
>
> -DOCKER_COMPILE_CMD="$(DOCKER_SCRIPT) cc --user $(shell id -u) \
> +DOCKER_COMPILE_CMD="$(DOCKER_SCRIPT) cc \
> --cc $(DOCKER_CROSS_COMPILER) \
> -i qemu:$(DOCKER_IMAGE) \
> -s $(SRC_PATH) -- "
>
--
—js
next prev parent reply other threads:[~2019-09-04 23:33 UTC|newest]
Thread overview: 74+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-04 20:29 [Qemu-devel] [PATCH v1 00/42] current testing/next queue (podman, docker, ci) Alex Bennée
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 01/42] hw/misc: Mark most objects as "common" code to speed up compilation a litte bit Alex Bennée
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 02/42] configure: clean-up container cross compile detect Alex Bennée
2019-09-05 17:53 ` David Hildenbrand
2019-09-05 18:35 ` Alex Bennée
2019-09-05 18:37 ` David Hildenbrand
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 03/42] tests/docker: fix "cc" command to work with podman Alex Bennée
2019-09-04 23:31 ` John Snow [this message]
2019-09-05 9:51 ` Alex Bennée
2019-09-05 17:18 ` John Snow
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 04/42] tests/tcg: use EXTRA_CFLAGS everywhere Alex Bennée
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 05/42] tests/tcg: cleanup Makefile inclusions Alex Bennée
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 06/42] tests/tcg: move configuration to a sub-shell script Alex Bennée
2019-09-05 10:29 ` Alex Bennée
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 07/42] tests/tcg: add .gitignore for in source builds Alex Bennée
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 08/42] tests/docker: move DEF_TARGET_LIST setting to common.rc Alex Bennée
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 09/42] tests/docker: set DEF_TARGET_LIST for some containers Alex Bennée
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 10/42] tests/docker: add Buster to DOCKER_PARTIAL_IMAGES Alex Bennée
2019-09-05 10:34 ` Philippe Mathieu-Daudé
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 11/42] tests/docker: move our arm64 cross compile to Buster Alex Bennée
2019-09-05 10:35 ` Philippe Mathieu-Daudé
2019-09-07 11:26 ` Philippe Mathieu-Daudé
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 12/42] tests/docker: move our powerpc " Alex Bennée
2019-09-07 11:32 ` Philippe Mathieu-Daudé
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 13/42] tests/docker: move our Alpha " Alex Bennée
2019-09-07 11:23 ` Philippe Mathieu-Daudé
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 14/42] tests/docker: move our HPPA " Alex Bennée
2019-09-07 11:22 ` Philippe Mathieu-Daudé
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 15/42] tests/docker: move our m68k " Alex Bennée
2019-09-07 11:28 ` Philippe Mathieu-Daudé
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 16/42] tests/docker: move our sparc64 " Alex Bennée
2019-09-07 11:27 ` Philippe Mathieu-Daudé
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 17/42] tests/docker: move our sh4 " Alex Bennée
2019-09-07 11:23 ` Philippe Mathieu-Daudé
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 18/42] tests/docker: move our mips64 " Alex Bennée
2019-09-06 19:05 ` Aleksandar Markovic
2019-09-07 11:33 ` Philippe Mathieu-Daudé
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 19/42] tests/docker: move our riscv64 " Alex Bennée
2019-09-07 11:24 ` Philippe Mathieu-Daudé
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 20/42] tests/docker: move our ppc64 " Alex Bennée
2019-09-07 11:29 ` Philippe Mathieu-Daudé
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 21/42] tests/docker: drop debian-sid image Alex Bennée
2019-09-07 20:02 ` Philippe Mathieu-Daudé
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 22/42] tests/docker: drop powerpc-user image for QEMU cross builds Alex Bennée
2019-09-07 20:21 ` Philippe Mathieu-Daudé
2019-09-09 9:54 ` Alex Bennée
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 23/42] tests/docker: add debian-xtensa-cross to DEBIAN_PARTIAL_IMAGES Alex Bennée
2019-09-07 12:06 ` Philippe Mathieu-Daudé
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 24/42] tests/docker: add debian9-mxe " Alex Bennée
2019-09-05 10:36 ` Philippe Mathieu-Daudé
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 25/42] tests/docker: avoid $SHELL invoke bash directly Alex Bennée
2019-09-05 10:40 ` Philippe Mathieu-Daudé
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 26/42] tests/docker: add debian-amd64-cross for non-x86 hosts Alex Bennée
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 27/42] tests/docker: use --arch-only for installing deps Alex Bennée
2019-09-07 11:40 ` Philippe Mathieu-Daudé
2019-09-04 20:29 ` [Qemu-devel] [PATCH v1 28/42] tests/docker: add more images to PARTIAL_IMAGES when not on x86_64 Alex Bennée
2019-09-04 20:30 ` [Qemu-devel] [PATCH v1 29/42] configure: check if --no-pie is supported first Alex Bennée
2019-09-04 20:30 ` [Qemu-devel] [PATCH v1 30/42] .travis.yml: Enable multiple caching features Alex Bennée
2019-09-04 20:30 ` [Qemu-devel] [PATCH v1 31/42] .travis.yml: Increase cache timeout from 3min to 20min Alex Bennée
2019-09-04 20:30 ` [Qemu-devel] [PATCH v1 32/42] .travis.yml: Cache Python PIP packages Alex Bennée
2019-09-04 20:30 ` [Qemu-devel] [PATCH v1 33/42] .travis.yml: Cache Avocado cache Alex Bennée
2019-09-04 20:30 ` [Qemu-devel] [PATCH v1 34/42] .travis.yml: Improve ccache use Alex Bennée
2019-09-04 20:30 ` [Qemu-devel] [PATCH v1 35/42] .travis.yml: Enable ccache on OSX Alex Bennée
2019-09-04 20:30 ` [Qemu-devel] [PATCH v1 36/42] .travis.yml: Document how the build matrix use caches Alex Bennée
2019-09-04 20:30 ` [Qemu-devel] [PATCH v1 37/42] .travis.yml: Cache Linux/GCC 'debug profile' jobs together Alex Bennée
2019-09-04 20:30 ` [Qemu-devel] [PATCH v1 38/42] .travis.yml: Cache Linux/GCC 'non-debug " Alex Bennée
2019-09-04 20:30 ` [Qemu-devel] [PATCH v1 39/42] .travis.yml: Cache Linux/Clang " Alex Bennée
2019-09-04 20:30 ` [Qemu-devel] [PATCH v1 40/42] Fedora images: use URLs from stable "archives.fedoraproject.org" Alex Bennée
2019-09-05 10:33 ` Philippe Mathieu-Daudé
2019-09-04 20:30 ` [Qemu-devel] [PATCH v1 41/42] tests/docker: --disable-libssh on ubuntu1804 builds Alex Bennée
2019-09-05 10:32 ` Philippe Mathieu-Daudé
2019-09-04 20:30 ` [Qemu-devel] [PATCH v1 42/42] tests/docker: don't always encoding for subprocess.check_output Alex Bennée
2019-09-04 22:16 ` [Qemu-devel] [PATCH v1 00/42] current testing/next queue (podman, docker, ci) no-reply
2019-09-06 10:20 ` Alex Bennée
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b25bd465-bc39-936f-5c6b-114e2926aba4@redhat.com \
--to=jsnow@redhat.com \
--cc=alex.bennee@linaro.org \
--cc=aurelien@aurel32.net \
--cc=berrange@redhat.com \
--cc=cota@braap.org \
--cc=f4bug@amsat.org \
--cc=fam@euphon.net \
--cc=marcandre.lureau@redhat.com \
--cc=pbonzini@redhat.com \
--cc=philmd@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=richard.henderson@linaro.org \
--cc=stefanb@linux.vnet.ibm.com \
--cc=stefanha@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).