Re: [Qemu-devel] [PATCH v6 2/6] docker: don't rely on submodules existing in the main checkout

[Fam Zheng <famz@redhat.com>]

On Thu, 09/28 09:44, Daniel P. Berrange wrote:
> When building the tarball to pass into the docker/vm test image,
> the code relies on the git submodules being checked out in the
> main checkout.
> 
> ie if the developer has not run 'git submodule update --init dtc'
> most (all?) of the docker tests will fail due to the missing dtc
> package in the test images.

Not all, fail only in envs/configs where libfdt devel doesn't exist. But I'm not
sure about this patch (*).

> Patchew manually checks out the dtc
> submodule in the main git checkout, but this is a bad idea. The
> docker tests should never mess around with the developer's main
> GIT checkout.

Or in general, scripts should not mess around with developer's GIT checkouts,
which is the reason ./configure doesn't do it?

How about we clone/checkout dtc into a temporary dir from ./configure
automatically instead?

> 
> When running tests we want to have a predictable set of submodules
> included in the source that's tested. The build environment is
> completely independant of the developers host OS, so the submodules
> the developer has checked out should not be considered relevant for
> the tests.
> 
> This changes the archive-source.sh script so that it clones the
> current git checkout into a temporary directory, checks out a
> fixed set of submodules, builds the tarball and finally removes
> the temporary git clone.
> 
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
>  scripts/archive-source.sh | 23 ++++++++++++++++++++---
>  1 file changed, 20 insertions(+), 3 deletions(-)
> 
> diff --git a/scripts/archive-source.sh b/scripts/archive-source.sh
> index c4e7d98f4d..0d2046a80e 100755
> --- a/scripts/archive-source.sh
> +++ b/scripts/archive-source.sh
> @@ -18,9 +18,15 @@ if test $# -lt 1; then
>      error "Usage: $0 <output tarball>"
>  fi
>  
> -tar_file="$1"
> -list_file="$1.list"
> -submodules=$(git submodule foreach --recursive --quiet 'echo $name')
> +tar_file=`realpath "$1"`
> +list_file="${tar_file}.list"
> +vroot_dir="${tar_file}.vroot"
> +
> +# We want a predictable list of submodules for builds, that is
> +# independant of what the developer currently has initialized
> +# in their checkout, because the build environment is completely
> +# different to the host OS.
> +submodules="dtc"
>  
>  if test $? -ne 0; then
>      error "git submodule command failed"
> @@ -28,6 +34,14 @@ fi
>  
>  trap "status=$?; rm -f \"$list_file\"; exit \$status" 0 1 2 3 15
>  
> +git clone --shared . "$vroot_dir"

(*) This loses the feature that uncommitted changes are also reflected in the
tarball. Personally I rarely use it but since day one of docker tests we've
deliberately supported that.

Cc'ing Alex who originally implemented that with "git diff-index" and "git
archive".

Fam