[Qemu-devel] [RFC PATCH for 2.10 0/3] some docker fixes

[Qemu-devel] [RFC PATCH for 2.10 0/3] some docker fixes

[Alex Bennée]

Hi,

I noticed some problems while running my local builds and while trying
to reproduce Travis failures. They apply on top of the currently in
flight pull-req pull-ci-updates-for-softfreeze-180717-2

Alex Bennée (3):
  docker: ensure NOUSER for travis images
  docker: fix dirty/stash detection  on some systems
  docker: ignore submodules when checking diff

 tests/docker/Makefile.include | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

-- 
2.13.0

[Qemu-devel] [RFC PATCH for 2.10 1/3] docker: ensure NOUSER for travis images

[Alex Bennée]

While adding the current user is a useful default behaviour for
creating new images it is not appropriate for Travis which already has
a default user.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index aaab1a4208..d7dafdbd27 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -71,6 +71,7 @@ docker-image-debian-ppc64el-cross: docker-image-debian9
 docker-image-debian-s390x-cross: docker-image-debian9
 docker-image-debian-win32-cross: docker-image-debian8-mxe
 docker-image-debian-win64-cross: docker-image-debian8-mxe
+docker-image-travis: NOUSER=1
 
 # Expand all the pre-requistes for each docker image and test combination
 $(foreach i,$(DOCKER_IMAGES), \
-- 
2.13.0

Re: [Qemu-devel] [RFC PATCH for 2.10 1/3] docker: ensure NOUSER for travis images

[Philippe Mathieu-Daudé]

On 07/20/2017 10:47 AM, Alex Bennée wrote:
> While adding the current user is a useful default behaviour for
> creating new images it is not appropriate for Travis which already has
> a default user.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>   tests/docker/Makefile.include | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index aaab1a4208..d7dafdbd27 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -71,6 +71,7 @@ docker-image-debian-ppc64el-cross: docker-image-debian9
>   docker-image-debian-s390x-cross: docker-image-debian9
>   docker-image-debian-win32-cross: docker-image-debian8-mxe
>   docker-image-debian-win64-cross: docker-image-debian8-mxe
> +docker-image-travis: NOUSER=1

Cool you kept it ordered :)

I'm surprised we need to install the full LaTeX stack to be able to 
compile the device-tree-compiler...

Reading 
https://docs.travis-ci.com/user/environment-variables#default-environment-variables 
I think it'd be a better match if we also use those default environment 
variables, at least:

DEBIAN_FRONTEND=noninteractive
LANG=en_US.UTF-8
LC_ALL=en_US.UTF-8

what do you think?

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

>   
>   # Expand all the pre-requistes for each docker image and test combination
>   $(foreach i,$(DOCKER_IMAGES), \
> 

Re: [Qemu-devel] [RFC PATCH for 2.10 1/3] docker: ensure NOUSER for travis images

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> On 07/20/2017 10:47 AM, Alex Bennée wrote:
>> While adding the current user is a useful default behaviour for
>> creating new images it is not appropriate for Travis which already has
>> a default user.
>>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> ---
>>   tests/docker/Makefile.include | 1 +
>>   1 file changed, 1 insertion(+)
>>
>> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
>> index aaab1a4208..d7dafdbd27 100644
>> --- a/tests/docker/Makefile.include
>> +++ b/tests/docker/Makefile.include
>> @@ -71,6 +71,7 @@ docker-image-debian-ppc64el-cross: docker-image-debian9
>>   docker-image-debian-s390x-cross: docker-image-debian9
>>   docker-image-debian-win32-cross: docker-image-debian8-mxe
>>   docker-image-debian-win64-cross: docker-image-debian8-mxe
>> +docker-image-travis: NOUSER=1
>
> Cool you kept it ordered :)
>
> I'm surprised we need to install the full LaTeX stack to be able to
> compile the device-tree-compiler...

Hmm I think installing build-deps is a little extreme as we are not
re-building the device-tree-compiler but using it.

>
> Reading
> https://docs.travis-ci.com/user/environment-variables#default-environment-variables
> I think it'd be a better match if we also use those default
> environment variables, at least:
>
> DEBIAN_FRONTEND=noninteractive
> LANG=en_US.UTF-8
> LC_ALL=en_US.UTF-8
>
> what do you think?

Sure - it does reduce the noise somewhat.

>
> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>
>>     # Expand all the pre-requistes for each docker image and test
>> combination
>>   $(foreach i,$(DOCKER_IMAGES), \
>>


--
Alex Bennée

[Qemu-devel] [RFC PATCH for 2.10 2/3] docker: fix dirty/stash detection on some systems

[Alex Bennée]

I reported this a while ago but evidently forgot to push the patch
upstream. Without this I wasn't seeing the dirty tree state submitted
to the docker build which is very confusing.

Reference:
  Subject: [PATCH 1/6] tests/docker/Makefile.include: fix diff-index call
  Date: Fri, 28 Oct 2016 17:33:34 +0100
  Message-Id: <20161028163339.31096-2-alex.bennee@linaro.org>

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index d7dafdbd27..ff580edfad 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -20,7 +20,7 @@ IMAGES ?= %
 # Make archive from git repo $1 to tar.gz $2
 make-archive-maybe = $(if $(wildcard $1/*), \
 	$(call quiet-command, \
-		(cd $1; if git diff-index --quiet HEAD -- &>/dev/null; then \
+		(cd $1; if git diff-index --quiet HEAD -- 2>/dev/null; then \
 			git archive -1 HEAD --format=tar.gz; \
 		else \
 			git archive -1 $$(git stash create) --format=tar.gz; \
-- 
2.13.0

Re: [Qemu-devel] [RFC PATCH for 2.10 2/3] docker: fix dirty/stash detection on some systems

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1399 bytes --]

On 07/20/2017 08:47 AM, Alex Bennée wrote:
> I reported this a while ago but evidently forgot to push the patch
> upstream. Without this I wasn't seeing the dirty tree state submitted
> to the docker build which is very confusing.
> 
> Reference:
>   Subject: [PATCH 1/6] tests/docker/Makefile.include: fix diff-index call
>   Date: Fri, 28 Oct 2016 17:33:34 +0100
>   Message-Id: <20161028163339.31096-2-alex.bennee@linaro.org>
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/docker/Makefile.include | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Reviewed-by: Eric Blake <eblake@redhat.com>

> 
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index d7dafdbd27..ff580edfad 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -20,7 +20,7 @@ IMAGES ?= %
>  # Make archive from git repo $1 to tar.gz $2
>  make-archive-maybe = $(if $(wildcard $1/*), \
>  	$(call quiet-command, \
> -		(cd $1; if git diff-index --quiet HEAD -- &>/dev/null; then \
> +		(cd $1; if git diff-index --quiet HEAD -- 2>/dev/null; then \
>  			git archive -1 HEAD --format=tar.gz; \
>  		else \
>  			git archive -1 $$(git stash create) --format=tar.gz; \
> 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: [Qemu-devel] [RFC PATCH for 2.10 2/3] docker: fix dirty/stash detection on some systems

[Philippe Mathieu-Daudé]

On 07/20/2017 10:47 AM, Alex Bennée wrote:
> I reported this a while ago but evidently forgot to push the patch
> upstream. Without this I wasn't seeing the dirty tree state submitted
> to the docker build which is very confusing.
> 
> Reference:
>    Subject: [PATCH 1/6] tests/docker/Makefile.include: fix diff-index call
>    Date: Fri, 28 Oct 2016 17:33:34 +0100
>    Message-Id: <20161028163339.31096-2-alex.bennee@linaro.org>
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

> ---
>   tests/docker/Makefile.include | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index d7dafdbd27..ff580edfad 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -20,7 +20,7 @@ IMAGES ?= %
>   # Make archive from git repo $1 to tar.gz $2
>   make-archive-maybe = $(if $(wildcard $1/*), \
>   	$(call quiet-command, \
> -		(cd $1; if git diff-index --quiet HEAD -- &>/dev/null; then \
> +		(cd $1; if git diff-index --quiet HEAD -- 2>/dev/null; then \
>   			git archive -1 HEAD --format=tar.gz; \
>   		else \
>   			git archive -1 $$(git stash create) --format=tar.gz; \
> 

[Qemu-devel] [RFC PATCH for 2.10 3/3] docker: ignore submodules when checking diff

[Alex Bennée]

The QEMU tree often has dirty submodules in it but this will confuse
things as git stash is not going to save their state directly. They
are likely just dirty from build files anyway. Ignore them in the
diff-index command.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index ff580edfad..2a560a5327 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -20,7 +20,7 @@ IMAGES ?= %
 # Make archive from git repo $1 to tar.gz $2
 make-archive-maybe = $(if $(wildcard $1/*), \
 	$(call quiet-command, \
-		(cd $1; if git diff-index --quiet HEAD -- 2>/dev/null; then \
+		(cd $1; if git diff-index --quiet --ignore-submodules HEAD -- 2>/dev/null; then \
 			git archive -1 HEAD --format=tar.gz; \
 		else \
 			git archive -1 $$(git stash create) --format=tar.gz; \
-- 
2.13.0

Re: [Qemu-devel] [RFC PATCH for 2.10 3/3] docker: ignore submodules when checking diff

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1544 bytes --]

On 07/20/2017 08:47 AM, Alex Bennée wrote:
> The QEMU tree often has dirty submodules in it but this will confuse
> things as git stash is not going to save their state directly. They
> are likely just dirty from build files anyway. Ignore them in the
> diff-index command.

It's annoying that our build dirties a submodule in the first place; it
would be nice to get rid of those bugs in our build process (ideally,
the build should treat submodule directories as read-only).  But that is
a bigger effort, so I'm okay with your hack in the meantime.

> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/docker/Makefile.include | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Reviewed-by: Eric Blake <eblake@redhat.com>

> 
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index ff580edfad..2a560a5327 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -20,7 +20,7 @@ IMAGES ?= %
>  # Make archive from git repo $1 to tar.gz $2
>  make-archive-maybe = $(if $(wildcard $1/*), \
>  	$(call quiet-command, \
> -		(cd $1; if git diff-index --quiet HEAD -- 2>/dev/null; then \
> +		(cd $1; if git diff-index --quiet --ignore-submodules HEAD -- 2>/dev/null; then \
>  			git archive -1 HEAD --format=tar.gz; \
>  		else \
>  			git archive -1 $$(git stash create) --format=tar.gz; \
> 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: [Qemu-devel] [RFC PATCH for 2.10 0/3] some docker fixes

[Alex Bennée]

Alex Bennée <alex.bennee@linaro.org> writes:

> Hi,
>
> I noticed some problems while running my local builds and while trying
> to reproduce Travis failures. They apply on top of the currently in
> flight pull-req pull-ci-updates-for-softfreeze-180717-2
>
> Alex Bennée (3):
>   docker: ensure NOUSER for travis images
>   docker: fix dirty/stash detection  on some systems
>   docker: ignore submodules when checking diff
>
>  tests/docker/Makefile.include | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)

Ping?

Any more reviews? Happy for me to roll a pull-req?

--
Alex Bennée