[Qemu-devel] [PATCH] build: add make dist target (v2)

[Qemu-devel] [PATCH] build: add make dist target (v2)

[Anthony Liguori]

Let's stop screwing up releases by having a script do the work that Anthony's
fat fingers can't seem to get right.

Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
---
v1 -> v2
 - include the scripts for real this time
 - remove tar/tarbin from PHONY
---
 Makefile             |   19 ++++++++-----------
 scripts/make-release |   24 ++++++++++++++++++++++++
 2 files changed, 32 insertions(+), 11 deletions(-)
 create mode 100755 scripts/make-release

diff --git a/Makefile b/Makefile
index 9707fa0..abf825d 100644
--- a/Makefile
+++ b/Makefile
@@ -31,7 +31,7 @@ Makefile: ;
 configure: ;
 
 .PHONY: all clean cscope distclean dvi html info install install-doc \
-	pdf recurse-all speed tar tarbin test build-all
+	pdf recurse-all speed test build-all dist
 
 $(call set-vpath, $(SRC_PATH):$(SRC_PATH)/hw)
 
@@ -232,6 +232,13 @@ clean:
 	rm -f $$d/qemu-options.def; \
         done
 
+VERSION ?= $(shell cat VERSION)
+
+dist: qemu-$(VERSION).tar.bz2
+
+qemu-%.tar.bz2:
+	$(SRC_PATH)/scripts/make-release "$(SRC_PATH)" "$(patsubst qemu-%.tar.bz2,%,$@)"
+
 distclean: clean
 	rm -f config-host.mak config-host.h* config-host.ld $(DOCS) qemu-options.texi qemu-img-cmds.texi qemu-monitor.texi
 	rm -f config-all-devices.mak
@@ -390,15 +397,5 @@ qemu-doc.dvi qemu-doc.html qemu-doc.info qemu-doc.pdf: \
 	qemu-img.texi qemu-nbd.texi qemu-options.texi \
 	qemu-monitor.texi qemu-img-cmds.texi
 
-VERSION ?= $(shell cat VERSION)
-FILE = qemu-$(VERSION)
-
-# tar release (use 'make -k tar' on a checkouted tree)
-tar:
-	rm -rf /tmp/$(FILE)
-	cp -r . /tmp/$(FILE)
-	cd /tmp && tar zcvf ~/$(FILE).tar.gz $(FILE) --exclude CVS --exclude .git --exclude .svn
-	rm -rf /tmp/$(FILE)
-
 # Include automatically generated dependency files
 -include $(wildcard *.d audio/*.d slirp/*.d block/*.d net/*.d ui/*.d qapi/*.d qga/*.d)
diff --git a/scripts/make-release b/scripts/make-release
new file mode 100755
index 0000000..196c755
--- /dev/null
+++ b/scripts/make-release
@@ -0,0 +1,24 @@
+#!/bin/bash -e
+#
+# QEMU Release Script
+#
+# Copyright IBM, Corp. 2012
+#
+# Authors:
+#  Anthony Liguori <aliguori@us.ibm.com>
+#
+# This work is licensed under the terms of the GNU GPLv2 or later.
+# See the COPYING file in the top-level directory.
+
+src="$1"
+version="$2"
+destination=qemu-${version}
+
+git clone "${src}" ${destination}
+pushd ${destination}
+git checkout "v${version}"
+git submodule update --init
+rm -rf .git roms/*/.git
+popd
+tar cfj ${destination}.tar.bz2 ${destination}
+rm -rf ${destination}
-- 
1.7.5.4

Re: [Qemu-devel] [PATCH] build: add make dist target (v2)

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 2489 bytes --]

On 07/17/2012 12:33 PM, Anthony Liguori wrote:
> Let's stop screwing up releases by having a script do the work that Anthony's
> fat fingers can't seem to get right.
> 
> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
> Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
> ---
> +++ b/scripts/make-release
> @@ -0,0 +1,24 @@
> +#!/bin/bash -e

Is it worth tightening this up to avoid bashisms like pushd, and use
just POSIX sh?

> +# This work is licensed under the terms of the GNU GPLv2 or later.
> +# See the COPYING file in the top-level directory.
> +
> +src="$1"
> +version="$2"
> +destination=qemu-${version}

Inconsistent quoting.  The fact that you quoted $2 when assigning to
version makes me worry that you are trying to plan for someone calling
this script where $2 contains whitespace, but then destination would
contain whitespace.  This is equivalent with minimal typing (regardless
of whitespace, since variable assignment is not subject to word splitting):

src=$1
version=$2
destination=qemu-$version

or if you want to consistently use full quoting:

src="${1}"
version="${2}"
destination="qemu-${version}"

> +
> +git clone "${src}" ${destination}

But here is a line where it matters if $destination contains whitespace
because $2 contained whitespace.

> +pushd ${destination}
> +git checkout "v${version}"
> +git submodule update --init
> +rm -rf .git roms/*/.git
> +popd

The POSIX spelling to avoid pushd would be:

(
cd $destination
git checkout v$version
git submodule update --init
rm -rf .git roms/*/.git
)

[again, I did minimal typing; you may prefer the "v${version}" style
instead of minimalism]

> +tar cfj ${destination}.tar.bz2 ${destination}

'j' is a GNU tar extension.  Are you okay hard-coding this script to
only run on machines with GNU tar?  Or should you split this into 'tar c
... | bzip2 ...'?

> +rm -rf ${destination}
> 

For the record, I think releases are done so seldom, and on a
controlled-enough environment where extra tools can be relied on, that
this script probably does not have to be super-portable.  Therefore,
since what you have works for your environment, then even though I raked
it over the portability coals above I'm okay if you use it as-is rather
than posting a v3.  Hence, I give my:

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 620 bytes --]

Re: [Qemu-devel] [PATCH] build: add make dist target (v2)

[Michael Roth]

On Tue, Jul 17, 2012 at 01:33:32PM -0500, Anthony Liguori wrote:
> Let's stop screwing up releases by having a script do the work that Anthony's
> fat fingers can't seem to get right.
> 
> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
> Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>

Breaks if there's no tag corresponding with the contents of VERSION,
but that might be considered a feature (an alternative might be to
assume it's a development release, use current HEAD for master, and append the
short git hash to the version). Works well as far as I can tell though,
and I made a special point to confirm it did indeed output a bz2 :)

Tested-by: Michael Roth <mdroth@linux.vnet.ibm.com>

> ---
> v1 -> v2
>  - include the scripts for real this time
>  - remove tar/tarbin from PHONY
> ---
>  Makefile             |   19 ++++++++-----------
>  scripts/make-release |   24 ++++++++++++++++++++++++
>  2 files changed, 32 insertions(+), 11 deletions(-)
>  create mode 100755 scripts/make-release
> 
> diff --git a/Makefile b/Makefile
> index 9707fa0..abf825d 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -31,7 +31,7 @@ Makefile: ;
>  configure: ;
> 
>  .PHONY: all clean cscope distclean dvi html info install install-doc \
> -	pdf recurse-all speed tar tarbin test build-all
> +	pdf recurse-all speed test build-all dist
> 
>  $(call set-vpath, $(SRC_PATH):$(SRC_PATH)/hw)
> 
> @@ -232,6 +232,13 @@ clean:
>  	rm -f $$d/qemu-options.def; \
>          done
> 
> +VERSION ?= $(shell cat VERSION)
> +
> +dist: qemu-$(VERSION).tar.bz2
> +
> +qemu-%.tar.bz2:
> +	$(SRC_PATH)/scripts/make-release "$(SRC_PATH)" "$(patsubst qemu-%.tar.bz2,%,$@)"
> +
>  distclean: clean
>  	rm -f config-host.mak config-host.h* config-host.ld $(DOCS) qemu-options.texi qemu-img-cmds.texi qemu-monitor.texi
>  	rm -f config-all-devices.mak
> @@ -390,15 +397,5 @@ qemu-doc.dvi qemu-doc.html qemu-doc.info qemu-doc.pdf: \
>  	qemu-img.texi qemu-nbd.texi qemu-options.texi \
>  	qemu-monitor.texi qemu-img-cmds.texi
> 
> -VERSION ?= $(shell cat VERSION)
> -FILE = qemu-$(VERSION)
> -
> -# tar release (use 'make -k tar' on a checkouted tree)
> -tar:
> -	rm -rf /tmp/$(FILE)
> -	cp -r . /tmp/$(FILE)
> -	cd /tmp && tar zcvf ~/$(FILE).tar.gz $(FILE) --exclude CVS --exclude .git --exclude .svn
> -	rm -rf /tmp/$(FILE)
> -
>  # Include automatically generated dependency files
>  -include $(wildcard *.d audio/*.d slirp/*.d block/*.d net/*.d ui/*.d qapi/*.d qga/*.d)
> diff --git a/scripts/make-release b/scripts/make-release
> new file mode 100755
> index 0000000..196c755
> --- /dev/null
> +++ b/scripts/make-release
> @@ -0,0 +1,24 @@
> +#!/bin/bash -e
> +#
> +# QEMU Release Script
> +#
> +# Copyright IBM, Corp. 2012
> +#
> +# Authors:
> +#  Anthony Liguori <aliguori@us.ibm.com>
> +#
> +# This work is licensed under the terms of the GNU GPLv2 or later.
> +# See the COPYING file in the top-level directory.
> +
> +src="$1"
> +version="$2"
> +destination=qemu-${version}
> +
> +git clone "${src}" ${destination}
> +pushd ${destination}
> +git checkout "v${version}"
> +git submodule update --init
> +rm -rf .git roms/*/.git
> +popd
> +tar cfj ${destination}.tar.bz2 ${destination}
> +rm -rf ${destination}
> -- 
> 1.7.5.4
> 

Re: [Qemu-devel] [PATCH] build: add make dist target (v2)

[Gerd Hoffmann]

On 07/17/12 21:12, Michael Roth wrote:
> On Tue, Jul 17, 2012 at 01:33:32PM -0500, Anthony Liguori wrote:
>> Let's stop screwing up releases by having a script do the work that Anthony's
>> fat fingers can't seem to get right.
>>
>> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
>> Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
> 
> Breaks if there's no tag corresponding with the contents of VERSION,
> but that might be considered a feature (an alternative might be to
> assume it's a development release, use current HEAD for master, and append the
> short git hash to the version). Works well as far as I can tell though,
> and I made a special point to confirm it did indeed output a bz2 :)

Or just use 'git describe --long' to figure what the version is.  This
way you can easily build a tarball for any git commit, and a release
tarball is just 'git checkout v$version; make dist'.

cheers,
  Gerd

Re: [Qemu-devel] [PATCH] build: add make dist target (v2)

[Anthony Liguori]

Gerd Hoffmann <kraxel@redhat.com> writes:

> On 07/17/12 21:12, Michael Roth wrote:
>> On Tue, Jul 17, 2012 at 01:33:32PM -0500, Anthony Liguori wrote:
>>> Let's stop screwing up releases by having a script do the work that Anthony's
>>> fat fingers can't seem to get right.
>>>
>>> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
>>> Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
>> 
>> Breaks if there's no tag corresponding with the contents of VERSION,
>> but that might be considered a feature (an alternative might be to
>> assume it's a development release, use current HEAD for master, and append the
>> short git hash to the version). Works well as far as I can tell though,
>> and I made a special point to confirm it did indeed output a bz2 :)
>
> Or just use 'git describe --long' to figure what the version is.  This
> way you can easily build a tarball for any git commit, and a release
> tarball is just 'git checkout v$version; make dist'.

As long as it doesn't break release tarballs, I'm very open to patches
to make this more generally useful.

Regards,

Anthony Liguori

>
> cheers,
>   Gerd

Re: [Qemu-devel] [PATCH] build: add make dist target (v2)

[Daniel P. Berrange]

On Tue, Jul 17, 2012 at 01:33:32PM -0500, Anthony Liguori wrote:
> Let's stop screwing up releases by having a script do the work that Anthony's
> fat fingers can't seem to get right.
> 
> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
> Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
> ---
> v1 -> v2
>  - include the scripts for real this time
>  - remove tar/tarbin from PHONY
> ---
>  Makefile             |   19 ++++++++-----------
>  scripts/make-release |   24 ++++++++++++++++++++++++
>  2 files changed, 32 insertions(+), 11 deletions(-)
>  create mode 100755 scripts/make-release
> 
> diff --git a/Makefile b/Makefile
> index 9707fa0..abf825d 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -31,7 +31,7 @@ Makefile: ;
>  configure: ;
>  
>  .PHONY: all clean cscope distclean dvi html info install install-doc \
> -	pdf recurse-all speed tar tarbin test build-all
> +	pdf recurse-all speed test build-all dist
>  
>  $(call set-vpath, $(SRC_PATH):$(SRC_PATH)/hw)
>  
> @@ -232,6 +232,13 @@ clean:
>  	rm -f $$d/qemu-options.def; \
>          done
>  
> +VERSION ?= $(shell cat VERSION)
> +
> +dist: qemu-$(VERSION).tar.bz2
> +
> +qemu-%.tar.bz2:
> +	$(SRC_PATH)/scripts/make-release "$(SRC_PATH)" "$(patsubst qemu-%.tar.bz2,%,$@)"
> +
>  distclean: clean
>  	rm -f config-host.mak config-host.h* config-host.ld $(DOCS) qemu-options.texi qemu-img-cmds.texi qemu-monitor.texi
>  	rm -f config-all-devices.mak
> @@ -390,15 +397,5 @@ qemu-doc.dvi qemu-doc.html qemu-doc.info qemu-doc.pdf: \
>  	qemu-img.texi qemu-nbd.texi qemu-options.texi \
>  	qemu-monitor.texi qemu-img-cmds.texi
>  
> -VERSION ?= $(shell cat VERSION)
> -FILE = qemu-$(VERSION)
> -
> -# tar release (use 'make -k tar' on a checkouted tree)
> -tar:
> -	rm -rf /tmp/$(FILE)
> -	cp -r . /tmp/$(FILE)
> -	cd /tmp && tar zcvf ~/$(FILE).tar.gz $(FILE) --exclude CVS --exclude .git --exclude .svn
> -	rm -rf /tmp/$(FILE)
> -
>  # Include automatically generated dependency files
>  -include $(wildcard *.d audio/*.d slirp/*.d block/*.d net/*.d ui/*.d qapi/*.d qga/*.d)
> diff --git a/scripts/make-release b/scripts/make-release
> new file mode 100755
> index 0000000..196c755
> --- /dev/null
> +++ b/scripts/make-release
> @@ -0,0 +1,24 @@
> +#!/bin/bash -e
> +#
> +# QEMU Release Script
> +#
> +# Copyright IBM, Corp. 2012
> +#
> +# Authors:
> +#  Anthony Liguori <aliguori@us.ibm.com>
> +#
> +# This work is licensed under the terms of the GNU GPLv2 or later.
> +# See the COPYING file in the top-level directory.
> +
> +src="$1"
> +version="$2"
> +destination=qemu-${version}
> +
> +git clone "${src}" ${destination}
> +pushd ${destination}
> +git checkout "v${version}"
> +git submodule update --init
> +rm -rf .git roms/*/.git
> +popd
> +tar cfj ${destination}.tar.bz2 ${destination}
> +rm -rf ${destination}

Fancy providing an XZ compressed archive, in addition to the bz2 one?
It is almost 20% smaller with XZ with default compression levels...

$ ls -ahl qemu-1.1.1-1.tar*
-rw-rw-r--. 1 berrange berrange 9.2M Jul 17 19:20 qemu-1.1.1-1.tar.bz2
-rw-rw-r--. 1 berrange berrange 7.6M Jul 18 15:03 qemu-1.1.1-1.tar.xz

You can get it down to 7.3M if you use xz --best

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|