[PATCH v9 0/6] build upstream qemu and seabios by default

[PATCH v9 0/6] build upstream qemu and seabios by default

[Stefano Stabellini]

Hi all,
this is the nineth version of the patch series to introduce upstream qemu
and seabios in the xen-unstable build system.


Changes to v8:

- build upstream qemu out of tree;

- add a tools/qemu-xen-dir-force-update target;

- add a tools/firmware/seabios-dir-force-update target;

- call make install from subdir-all and subdir-install
qemu-xen-traditional and qemu-xen targets; 

- fix a typo in patch #5;


Changes to v7:

- call upstream qemu's configure script right before building qemu and
after building libxc and xenstore because it needs them;

- introduce a new patch to move the call to xen-setup after building
libxc and xenstore for consistency with upstream qemu;

- fix a typo in tools/Makefile (patch #4);


Changes to v6:

- add "set -e" to git-checkout.sh;

- add argument count check to git-checkout.sh;

- remove spurious semicolons in git-checkout.sh.


Changes to v5:

- use $GIT in git-checkout.sh;

- add an http mirror for seabios;

- use $(LIBEXEC) to configure upstream qemu;

- append a patch for libxenlight to find the upstream qemu binary under
$(LIBEXEC).


Changes to v4:

- remove an obsolete comment;

- use /bin/sh to execute git-checkout.sh rathen than /bin/bash.


Changes to v3:

- reduce the scope of git-checkout.sh, now it only does what the name
says; calling "configure" is responsibility of the caller. As a result
of this change, the build still works when the user specifies a local
directory in the CONFIG_QEMU environmental variable;

- use a more official qemu repository hosted on xenbits;

- use a changeset as QEMU_UPSTREAM_TAG rather than a branch name.



Changes to v2:

- move tools/git-checkout.sh to scripts/git-checkout.sh;

- use git-checkout.sh for seabios;

- improve seabios integration with tools/firmware make system;

- add qemu-xen-traditional, qemu-xen and seabios dir entries to
.hgignore.



Changes to v1:

- always build upstream qemu and seabios, rather than introducing them
as an option.


Cheers,

Stefano

[PATCH v9 1/6] Introduce git-checkout.sh

[stefano.stabellini]

From: stefano.stabellini@eu.citrix.com <stefano.stabellini@eu.citrix.com>

Introduce a script to perform git checkout on an external git tree; use
git-checkout.sh in ioemu-dir-find.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 .hgignore               |    2 +-
 scripts/git-checkout.sh |   27 +++++++++++++++++++++++++++
 tools/Makefile          |   20 ++++----------------
 3 files changed, 32 insertions(+), 17 deletions(-)
 create mode 100755 scripts/git-checkout.sh

diff --git a/.hgignore b/.hgignore
index e62fb2d..067d83d 100644
--- a/.hgignore
+++ b/.hgignore
@@ -295,7 +295,7 @@
 ^tools/xm-test/lib/XmTestLib/config.py$
 ^tools/xm-test/lib/XmTestReport/xmtest.py$
 ^tools/xm-test/tests/.*\.test$
-^tools/ioemu-remote
+^tools/ioemu-dir-remote
 ^tools/ioemu-dir$
 ^tools/ocaml/.*/.*\.annot$
 ^tools/ocaml/.*/.*\.cmx?a$
diff --git a/scripts/git-checkout.sh b/scripts/git-checkout.sh
new file mode 100755
index 0000000..15b3ce9
--- /dev/null
+++ b/scripts/git-checkout.sh
@@ -0,0 +1,27 @@
+#!/bin/sh
+
+if test $# -lt 3; then
+	echo "Usage: $0 <tree> <tag> <dir>"
+	exit 1
+fi
+
+TREE=$1
+TAG=$2
+DIR=$3
+
+set -e
+
+if test \! -d $DIR-remote; then
+	rm -rf $DIR-remote $DIR-remote.tmp
+	mkdir $DIR-remote.tmp; rmdir $DIR-remote.tmp
+	$GIT clone $TREE $DIR-remote.tmp
+	if test "$TAG" ; then
+		cd $DIR-remote.tmp
+		$GIT branch -D dummy >/dev/null 2>&1 ||:
+		$GIT checkout -b dummy $TAG
+		cd ..
+	fi
+	mv $DIR-remote.tmp $DIR-remote
+fi
+rm -f $DIR
+ln -sf $DIR-remote $DIR
diff --git a/tools/Makefile b/tools/Makefile
index 9389e1f..beccad2 100644
--- a/tools/Makefile
+++ b/tools/Makefile
@@ -71,7 +71,7 @@ clean: subdirs-clean
 
 .PHONY: distclean
 distclean: subdirs-distclean
-	rm -rf ioemu-dir ioemu-remote
+	rm -rf ioemu-dir ioemu-dir-remote
 
 ifneq ($(XEN_COMPILE_ARCH),$(XEN_TARGET_ARCH))
 IOEMU_CONFIGURE_CROSS ?= --cpu=$(XEN_TARGET_ARCH) \
@@ -89,20 +89,8 @@ ioemu-dir-find:
 	if test -d $(CONFIG_QEMU); then \
 		mkdir -p ioemu-dir; \
 	else \
-		if [ ! -d ioemu-remote ]; then \
-			rm -rf ioemu-remote ioemu-remote.tmp; \
-			mkdir ioemu-remote.tmp; rmdir ioemu-remote.tmp; \
-			$(GIT) clone $(CONFIG_QEMU) ioemu-remote.tmp; \
-			if [ "$(QEMU_TAG)" ]; then			\
-				cd ioemu-remote.tmp;			\
-				$(GIT) branch -D dummy >/dev/null 2>&1 ||:; \
-				$(GIT) checkout -b dummy $(QEMU_TAG);	\
-				cd ..;					\
-			fi;						\
-			mv ioemu-remote.tmp ioemu-remote; \
-		fi; \
-		rm -f ioemu-dir; \
-		ln -sf ioemu-remote ioemu-dir; \
+		export GIT=$(GIT); \
+		$(XEN_ROOT)/scripts/git-checkout.sh $(CONFIG_QEMU) $(QEMU_TAG) ioemu-dir; \
 	fi
 	set -e; \
 		$(buildmakevars2shellvars); \
@@ -113,7 +101,7 @@ ioemu-dir-find:
 ioemu-dir-force-update:
 	set -ex; \
 	if [ "$(QEMU_TAG)" ]; then \
-		cd ioemu-remote; \
+		cd ioemu-dir-remote; \
 		$(GIT) fetch origin; \
 		$(GIT) reset --hard $(QEMU_TAG); \
 	fi
-- 
1.7.2.5

Re: [PATCH v9 1/6] Introduce git-checkout.sh

[Christoph Egger]

On 11/23/11 14:51, stefano.stabellini@eu.citrix.com wrote:
> From: stefano.stabellini@eu.citrix.com<stefano.stabellini@eu.citrix.com>
>
> Introduce a script to perform git checkout on an external git tree; use
> git-checkout.sh in ioemu-dir-find.
>
> Signed-off-by: Stefano Stabellini<stefano.stabellini@eu.citrix.com>

[...]

> diff --git a/tools/Makefile b/tools/Makefile
> index 9389e1f..beccad2 100644
> --- a/tools/Makefile
> +++ b/tools/Makefile
> @@ -71,7 +71,7 @@ clean: subdirs-clean
>
>   .PHONY: distclean
>   distclean: subdirs-distclean
> -	rm -rf ioemu-dir ioemu-remote
> +	rm -rf ioemu-dir ioemu-dir-remote
>
>   ifneq ($(XEN_COMPILE_ARCH),$(XEN_TARGET_ARCH))
>   IOEMU_CONFIGURE_CROSS ?= --cpu=$(XEN_TARGET_ARCH) \
> @@ -89,20 +89,8 @@ ioemu-dir-find:
>   	if test -d $(CONFIG_QEMU); then \
>   		mkdir -p ioemu-dir; \
>   	else \
> -		if [ ! -d ioemu-remote ]; then \
> -			rm -rf ioemu-remote ioemu-remote.tmp; \
> -			mkdir ioemu-remote.tmp; rmdir ioemu-remote.tmp; \
> -			$(GIT) clone $(CONFIG_QEMU) ioemu-remote.tmp; \
> -			if [ "$(QEMU_TAG)" ]; then			\
> -				cd ioemu-remote.tmp;			\
> -				$(GIT) branch -D dummy>/dev/null 2>&1 ||:; \
> -				$(GIT) checkout -b dummy $(QEMU_TAG);	\
> -				cd ..;					\
> -			fi;						\
> -			mv ioemu-remote.tmp ioemu-remote; \
> -		fi; \
> -		rm -f ioemu-dir; \
> -		ln -sf ioemu-remote ioemu-dir; \
> +		export GIT=$(GIT); \
> +		$(XEN_ROOT)/scripts/git-checkout.sh $(CONFIG_QEMU) $(QEMU_TAG) ioemu-dir; \

When you use $(SHELL) then we do not need to care about the execute bit:

+		$(SHELL) $(XEN_ROOT)/scripts/git-checkout.sh $(CONFIG_QEMU) 
$(QEMU_TAG) ioemu-dir; \


Christoph


>   	fi
>   	set -e; \
>   		$(buildmakevars2shellvars); \
> @@ -113,7 +101,7 @@ ioemu-dir-find:
>   ioemu-dir-force-update:
>   	set -ex; \
>   	if [ "$(QEMU_TAG)" ]; then \
> -		cd ioemu-remote; \
> +		cd ioemu-dir-remote; \
>   		$(GIT) fetch origin; \
>   		$(GIT) reset --hard $(QEMU_TAG); \
>   	fi


-- 
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Einsteinring 24, 85689 Dornach b. Muenchen
Geschaeftsfuehrer: Alberto Bozzo, Andrew Bowd
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

Re: [PATCH v9 1/6] Introduce git-checkout.sh

[Ian Jackson]

Christoph Egger writes ("Re: [Xen-devel] [PATCH v9 1/6] Introduce git-checkout.sh"):
> On 11/23/11 14:51, stefano.stabellini@eu.citrix.com wrote:
> > +		$(XEN_ROOT)/scripts/git-checkout.sh $(CONFIG_QEMU) $(QEMU_TAG) ioemu-dir; \
> 
> When you use $(SHELL) then we do not need to care about the execute bit:
> 
> +		$(SHELL) $(XEN_ROOT)/scripts/git-checkout.sh $(CONFIG_QEMU) 
> $(QEMU_TAG) ioemu-dir; \

I'm afraid I don't agree with this.

Ian.

[PATCH v9 2/6] Rename ioemu-dir as qemu-xen-traditional-dir

[stefano.stabellini]

From: stefano.stabellini@eu.citrix.com <stefano.stabellini@eu.citrix.com>

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 .hgignore        |    4 ++--
 Makefile         |   14 +++++++-------
 stubdom/Makefile |    8 ++++----
 tools/Makefile   |   26 +++++++++++++-------------
 4 files changed, 26 insertions(+), 26 deletions(-)

diff --git a/.hgignore b/.hgignore
index 067d83d..c1486a6 100644
--- a/.hgignore
+++ b/.hgignore
@@ -295,8 +295,8 @@
 ^tools/xm-test/lib/XmTestLib/config.py$
 ^tools/xm-test/lib/XmTestReport/xmtest.py$
 ^tools/xm-test/tests/.*\.test$
-^tools/ioemu-dir-remote
-^tools/ioemu-dir$
+^tools/qemu-xen-traditional-dir-remote
+^tools/qemu-xen-traditional-dir$
 ^tools/ocaml/.*/.*\.annot$
 ^tools/ocaml/.*/.*\.cmx?a$
 ^tools/ocaml/.*/META$
diff --git a/Makefile b/Makefile
index 31e8795..c0197a5 100644
--- a/Makefile
+++ b/Makefile
@@ -70,7 +70,7 @@ install-tools:
 	$(MAKE) -C tools install
 
 ifeq ($(CONFIG_IOEMU),y)
-install-tools: tools/ioemu-dir
+install-tools: tools/qemu-xen-traditional-dir
 endif
 
 .PHONY: install-kernels
@@ -78,18 +78,18 @@ install-kernels:
 	for i in $(XKERNELS) ; do $(MAKE) $$i-install || exit 1; done
 
 .PHONY: install-stubdom
-install-stubdom: tools/ioemu-dir install-tools
+install-stubdom: tools/qemu-xen-traditional-dir install-tools
 	$(MAKE) -C stubdom install
 ifeq (x86_64,$(XEN_TARGET_ARCH))
 	XEN_TARGET_ARCH=x86_32 $(MAKE) -C stubdom install-grub
 endif
 
-tools/ioemu-dir:
-	$(MAKE) -C tools ioemu-dir-find
+tools/qemu-xen-traditional-dir:
+	$(MAKE) -C tools qemu-xen-traditional-dir-find
 
-.PHONY: tools/ioemu-dir-force-update
-tools/ioemu-dir-force-update:
-	$(MAKE) -C tools ioemu-dir-force-update
+.PHONY: tools/qemu-xen-traditional-dir-force-update
+tools/qemu-xen-traditional-dir-force-update:
+	$(MAKE) -C tools qemu-xen-traditional-dir-force-update
 
 .PHONY: install-docs
 install-docs:
diff --git a/stubdom/Makefile b/stubdom/Makefile
index 938fc0a..d03ccba 100644
--- a/stubdom/Makefile
+++ b/stubdom/Makefile
@@ -218,15 +218,15 @@ $(CROSS_ROOT): cross-newlib cross-zlib cross-libpci
 QEMU_ROOT := $(shell if [ -d "$(CONFIG_QEMU)" ]; then echo "$(CONFIG_QEMU)"; else echo .; fi)
 
 ifeq ($(QEMU_ROOT),.)
-$(XEN_ROOT)/tools/ioemu-dir:
-	$(CROSS_MAKE) -C $(XEN_ROOT)/tools ioemu-dir-find
+$(XEN_ROOT)/tools/qemu-xen-traditional-dir:
+	$(CROSS_MAKE) -C $(XEN_ROOT)/tools qemu-xen-traditional-dir-find
 
-ioemu/linkfarm.stamp: $(XEN_ROOT)/tools/ioemu-dir
+ioemu/linkfarm.stamp: $(XEN_ROOT)/tools/qemu-xen-traditional-dir
 	mkdir -p ioemu
 	set -e;									\
 	$(buildmakevars2shellvars);						\
 	cd ioemu;								\
-	src="$$XEN_ROOT/tools/ioemu-dir"; export src;				\
+	src="$$XEN_ROOT/tools/qemu-xen-traditional-dir"; export src;		\
 	(cd $$src && find * -type d -print) | xargs mkdir -p;			\
 	(cd $$src && find *	! -type l  -type f  $(addprefix ! -name ,	\
 			'*.[oda1]' 'config-*' config.mak qemu-dm qemu-img-xen	\
diff --git a/tools/Makefile b/tools/Makefile
index beccad2..c19d3c9 100644
--- a/tools/Makefile
+++ b/tools/Makefile
@@ -31,7 +31,7 @@ SUBDIRS-y += libvchan
 
 # do not recurse in to a dir we are about to delete
 ifneq "$(MAKECMDGOALS)" "distclean"
-SUBDIRS-$(CONFIG_IOEMU) += ioemu-dir
+SUBDIRS-$(CONFIG_IOEMU) += qemu-xen-traditional-dir
 endif
 
 SUBDIRS-y += xenpmd
@@ -71,7 +71,7 @@ clean: subdirs-clean
 
 .PHONY: distclean
 distclean: subdirs-distclean
-	rm -rf ioemu-dir ioemu-dir-remote
+	rm -rf qemu-xen-traditional-dir qemu-xen-traditional-dir-remote
 
 ifneq ($(XEN_COMPILE_ARCH),$(XEN_TARGET_ARCH))
 IOEMU_CONFIGURE_CROSS ?= --cpu=$(XEN_TARGET_ARCH) \
@@ -84,34 +84,34 @@ ifneq ($(QEMU_ROOT),.)
 export QEMU_ROOT
 endif
 
-ioemu-dir-find:
+qemu-xen-traditional-dir-find:
 	set -ex; \
 	if test -d $(CONFIG_QEMU); then \
-		mkdir -p ioemu-dir; \
+		mkdir -p qemu-xen-traditional-dir; \
 	else \
 		export GIT=$(GIT); \
-		$(XEN_ROOT)/scripts/git-checkout.sh $(CONFIG_QEMU) $(QEMU_TAG) ioemu-dir; \
+		$(XEN_ROOT)/scripts/git-checkout.sh $(CONFIG_QEMU) $(QEMU_TAG) qemu-xen-traditional-dir; \
 	fi
 	set -e; \
 		$(buildmakevars2shellvars); \
-		cd ioemu-dir; \
+		cd qemu-xen-traditional-dir; \
 		$(QEMU_ROOT)/xen-setup $(IOEMU_CONFIGURE_CROSS)
 
-.PHONY: ioemu-dir-force-update
-ioemu-dir-force-update:
+.PHONY: qemu-xen-traditional-dir-force-update
+qemu-xen-traditional-dir-force-update:
 	set -ex; \
 	if [ "$(QEMU_TAG)" ]; then \
-		cd ioemu-dir-remote; \
+		cd qemu-xen-traditional-dir-remote; \
 		$(GIT) fetch origin; \
 		$(GIT) reset --hard $(QEMU_TAG); \
 	fi
 
-subdir-all-ioemu-dir subdir-install-ioemu-dir: ioemu-dir-find
+subdir-all-qemu-xen-traditional-dir subdir-install-qemu-xen-traditional-dir: qemu-xen-traditional-dir-find
 
-subdir-clean-ioemu-dir:
-	set -e; if test -d ioemu-dir/.; then \
+subdir-clean-qemu-xen-traditional-dir:
+	set -e; if test -d qemu-xen-traditional-dir/.; then \
 		$(buildmakevars2shellvars); \
-		$(MAKE) -C ioemu-dir clean; \
+		$(MAKE) -C qemu-xen-traditional-dir clean; \
 	fi
 
 subdir-clean-debugger/gdbsx subdir-distclean-debugger/gdbsx: .phony
-- 
1.7.2.5

[PATCH v9 3/6] move the call to xen-setup after libxc and xenstore are built

[stefano.stabellini]

From: stefano.stabellini@eu.citrix.com <stefano.stabellini@eu.citrix.com>

Move the call to xen-setup, the wrapper script to configure
qemu-xen-traditional, right before building qemu-xen-traditional and
after libxc and xenstore are already built.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 tools/Makefile |    9 +++++----
 1 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/tools/Makefile b/tools/Makefile
index c19d3c9..20bee85 100644
--- a/tools/Makefile
+++ b/tools/Makefile
@@ -92,10 +92,6 @@ qemu-xen-traditional-dir-find:
 		export GIT=$(GIT); \
 		$(XEN_ROOT)/scripts/git-checkout.sh $(CONFIG_QEMU) $(QEMU_TAG) qemu-xen-traditional-dir; \
 	fi
-	set -e; \
-		$(buildmakevars2shellvars); \
-		cd qemu-xen-traditional-dir; \
-		$(QEMU_ROOT)/xen-setup $(IOEMU_CONFIGURE_CROSS)
 
 .PHONY: qemu-xen-traditional-dir-force-update
 qemu-xen-traditional-dir-force-update:
@@ -107,6 +103,11 @@ qemu-xen-traditional-dir-force-update:
 	fi
 
 subdir-all-qemu-xen-traditional-dir subdir-install-qemu-xen-traditional-dir: qemu-xen-traditional-dir-find
+	set -e; \
+		$(buildmakevars2shellvars); \
+		cd qemu-xen-traditional-dir; \
+		$(QEMU_ROOT)/xen-setup $(IOEMU_CONFIGURE_CROSS); \
+		$(MAKE) install
 
 subdir-clean-qemu-xen-traditional-dir:
 	set -e; if test -d qemu-xen-traditional-dir/.; then \
-- 
1.7.2.5

[PATCH v9 4/6] Clone and build upstream Qemu by default

[stefano.stabellini]

From: stefano.stabellini@eu.citrix.com <stefano.stabellini@eu.citrix.com>

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 .hgignore      |    2 ++
 Config.mk      |    7 +++++++
 Makefile       |    9 ++++++++-
 tools/Makefile |   44 ++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 61 insertions(+), 1 deletions(-)

diff --git a/.hgignore b/.hgignore
index c1486a6..efbb5ed 100644
--- a/.hgignore
+++ b/.hgignore
@@ -297,6 +297,8 @@
 ^tools/xm-test/tests/.*\.test$
 ^tools/qemu-xen-traditional-dir-remote
 ^tools/qemu-xen-traditional-dir$
+^tools/qemu-xen-dir-remote
+^tools/qemu-xen-dir$
 ^tools/ocaml/.*/.*\.annot$
 ^tools/ocaml/.*/.*\.cmx?a$
 ^tools/ocaml/.*/META$
diff --git a/Config.mk b/Config.mk
index b2c6ba2..e72324f 100644
--- a/Config.mk
+++ b/Config.mk
@@ -203,6 +203,13 @@ else
 QEMU_REMOTE=git://xenbits.xensource.com/qemu-xen-unstable.git
 endif
 
+ifeq ($(GIT_HTTP),y)
+QEMU_UPSTREAM_URL ?= http://xenbits.xensource.com/git-http/qemu-upstream-unstable.git
+else
+QEMU_UPSTREAM_URL ?= git://xenbits.xensource.com/qemu-upstream-unstable.git
+endif
+QEMU_UPSTREAM_TAG ?= 30f93db54875e1f2836e1f3d21672c33191f883f
+
 # Specify which qemu-dm to use. This may be `ioemu' to use the old
 # Mercurial in-tree version, or a local directory, or a git URL.
 # CONFIG_QEMU ?= `pwd`/$(XEN_ROOT)/../qemu-xen.git
diff --git a/Makefile b/Makefile
index c0197a5..edc5e3d 100644
--- a/Makefile
+++ b/Makefile
@@ -70,7 +70,7 @@ install-tools:
 	$(MAKE) -C tools install
 
 ifeq ($(CONFIG_IOEMU),y)
-install-tools: tools/qemu-xen-traditional-dir
+install-tools: tools/qemu-xen-traditional-dir tools/qemu-xen-dir
 endif
 
 .PHONY: install-kernels
@@ -91,6 +91,13 @@ tools/qemu-xen-traditional-dir:
 tools/qemu-xen-traditional-dir-force-update:
 	$(MAKE) -C tools qemu-xen-traditional-dir-force-update
 
+tools/qemu-xen-dir:
+	$(MAKE) -C tools qemu-xen-dir-find
+
+.PHONY: tools/qemu-xen-dir-force-update
+tools/qemu-xen-dir-force-update:
+	$(MAKE) -C tools qemu-xen-dir-force-update
+
 .PHONY: install-docs
 install-docs:
 	sh ./docs/check_pkgs && $(MAKE) -C docs install || true
diff --git a/tools/Makefile b/tools/Makefile
index 20bee85..f9a07d2 100644
--- a/tools/Makefile
+++ b/tools/Makefile
@@ -32,6 +32,7 @@ SUBDIRS-y += libvchan
 # do not recurse in to a dir we are about to delete
 ifneq "$(MAKECMDGOALS)" "distclean"
 SUBDIRS-$(CONFIG_IOEMU) += qemu-xen-traditional-dir
+SUBDIRS-$(CONFIG_IOEMU) += qemu-xen-dir
 endif
 
 SUBDIRS-y += xenpmd
@@ -72,6 +73,7 @@ clean: subdirs-clean
 .PHONY: distclean
 distclean: subdirs-distclean
 	rm -rf qemu-xen-traditional-dir qemu-xen-traditional-dir-remote
+	rm -rf qemu-xen-dir qemu-xen-dir-remote
 
 ifneq ($(XEN_COMPILE_ARCH),$(XEN_TARGET_ARCH))
 IOEMU_CONFIGURE_CROSS ?= --cpu=$(XEN_TARGET_ARCH) \
@@ -93,6 +95,14 @@ qemu-xen-traditional-dir-find:
 		$(XEN_ROOT)/scripts/git-checkout.sh $(CONFIG_QEMU) $(QEMU_TAG) qemu-xen-traditional-dir; \
 	fi
 
+qemu-xen-dir-find:
+	if test -d $(QEMU_UPSTREAM_URL) ; then \
+		mkdir -p qemu-xen-dir; \
+	else \
+		export GIT=$(GIT); \
+		$(XEN_ROOT)/scripts/git-checkout.sh $(QEMU_UPSTREAM_URL) $(QEMU_UPSTREAM_TAG) qemu-xen-dir ; \
+	fi
+
 .PHONY: qemu-xen-traditional-dir-force-update
 qemu-xen-traditional-dir-force-update:
 	set -ex; \
@@ -115,6 +125,40 @@ subdir-clean-qemu-xen-traditional-dir:
 		$(MAKE) -C qemu-xen-traditional-dir clean; \
 	fi
 
+.PHONY: qemu-xen-dir-force-update
+qemu-xen-dir-force-update:
+	set -ex; \
+	if [ "$(QEMU_UPSTREAM_TAG)" ]; then \
+		cd qemu-xen-dir-remote; \
+		$(GIT) fetch origin; \
+		$(GIT) reset --hard $(QEMU_UPSTREAM_TAG); \
+	fi
+
+subdir-all-qemu-xen-dir subdir-install-qemu-xen-dir: qemu-xen-dir-find
+	if test -d $(QEMU_UPSTREAM_URL) ; then \
+		source=$(QEMU_UPSTREAM_URL); \
+	else \
+		source=.; \
+	fi; \
+	cd qemu-xen-dir; \
+	$$source/configure --enable-xen --target-list=i386-softmmu \
+		--source-path=$$source \
+		--extra-cflags="-I$(XEN_ROOT)/tools/include \
+		-I$(XEN_ROOT)/tools/libxc \
+		-I$(XEN_ROOT)/tools/xenstore" \
+		--extra-ldflags="-L$(XEN_ROOT)/tools/libxc \
+		-L$(XEN_ROOT)/tools/xenstore" \
+		--bindir=$(LIBEXEC) \
+		--disable-kvm \
+		$(IOEMU_CONFIGURE_CROSS); \
+	$(MAKE) install
+
+subdir-clean-qemu-xen-dir:
+	set -e; if test -d qemu-xen-dir/.; then \
+		$(buildmakevars2shellvars); \
+		$(MAKE) -C qemu-xen-dir clean; \
+	fi
+
 subdir-clean-debugger/gdbsx subdir-distclean-debugger/gdbsx: .phony
 	$(MAKE) -C debugger/gdbsx clean
 
-- 
1.7.2.5

[PATCH v9 5/6] Clone and build Seabios by default

[stefano.stabellini]

From: stefano.stabellini@eu.citrix.com <stefano.stabellini@eu.citrix.com>

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 .hgignore                         |    2 +
 Config.mk                         |   12 ++-----
 Makefile                          |    4 ++
 tools/firmware/Makefile           |   21 ++++++++++-
 tools/firmware/hvmloader/Makefile |    1 +
 tools/firmware/seabios-config     |   73 +++++++++++++++++++++++++++++++++++++
 6 files changed, 102 insertions(+), 11 deletions(-)
 create mode 100644 tools/firmware/seabios-config

diff --git a/.hgignore b/.hgignore
index efbb5ed..c2023ec 100644
--- a/.hgignore
+++ b/.hgignore
@@ -299,6 +299,8 @@
 ^tools/qemu-xen-traditional-dir$
 ^tools/qemu-xen-dir-remote
 ^tools/qemu-xen-dir$
+^tools/firmware/seabios-dir-remote
+^tools/firmware/seabios-dir$
 ^tools/ocaml/.*/.*\.annot$
 ^tools/ocaml/.*/.*\.cmx?a$
 ^tools/ocaml/.*/META$
diff --git a/Config.mk b/Config.mk
index e72324f..30e0ab5 100644
--- a/Config.mk
+++ b/Config.mk
@@ -205,10 +205,13 @@ endif
 
 ifeq ($(GIT_HTTP),y)
 QEMU_UPSTREAM_URL ?= http://xenbits.xensource.com/git-http/qemu-upstream-unstable.git
+SEABIOS_UPSTREAM_URL ?= http://git.qemu.org/git/seabios.git
 else
 QEMU_UPSTREAM_URL ?= git://xenbits.xensource.com/qemu-upstream-unstable.git
+SEABIOS_UPSTREAM_URL ?= git://git.qemu.org/seabios.git
 endif
 QEMU_UPSTREAM_TAG ?= 30f93db54875e1f2836e1f3d21672c33191f883f
+SEABIOS_UPSTREAM_TAG ?= 7fc039e9c262b4199fab497f3e12f4e425c37560
 
 # Specify which qemu-dm to use. This may be `ioemu' to use the old
 # Mercurial in-tree version, or a local directory, or a git URL.
@@ -222,15 +225,6 @@ QEMU_TAG ?= 52834188eedfbbca5636fd869d4c86b3b3044439
 # Short answer -- do not enable this unless you know what you are
 # doing and are prepared for some pain.
 
-# SeaBIOS integration is a work in progress. Before enabling this
-# option you must clone git://git.qemu.org/seabios.git/, possibly add
-# some development patches and then build it yourself before pointing
-# this variable to it (using an absolute path).
-#
-# Note that using SeaBIOS requires the use the upstream qemu as the
-# device model.
-SEABIOS_DIR ?= 
-
 # Optional components
 XENSTAT_XENTOP     ?= y
 VTPM_TOOLS         ?= n
diff --git a/Makefile b/Makefile
index edc5e3d..8edea0d 100644
--- a/Makefile
+++ b/Makefile
@@ -98,6 +98,10 @@ tools/qemu-xen-dir:
 tools/qemu-xen-dir-force-update:
 	$(MAKE) -C tools qemu-xen-dir-force-update
 
+.PHONY: tools/firmware/seabios-dir-force-update
+tools/firmware/seabios-dir-force-update:
+	$(MAKE) -C tools/firmware seabios-dir-force-update
+
 .PHONY: install-docs
 install-docs:
 	sh ./docs/check_pkgs && $(MAKE) -C docs install || true
diff --git a/tools/firmware/Makefile b/tools/firmware/Makefile
index 4b6d144..c3ec9a0 100644
--- a/tools/firmware/Makefile
+++ b/tools/firmware/Makefile
@@ -6,13 +6,18 @@ TARGET      := hvmloader/hvmloader
 INST_DIR := $(DESTDIR)$(XENFIRMWAREDIR)
 
 SUBDIRS :=
+SUBDIRS += seabios-dir
 SUBDIRS += rombios
 SUBDIRS += vgabios
 SUBDIRS += etherboot
 SUBDIRS += hvmloader
 
+seabios-dir:
+	GIT=$(GIT) $(XEN_ROOT)/scripts/git-checkout.sh $(SEABIOS_UPSTREAM_URL) $(SEABIOS_UPSTREAM_TAG) seabios-dir
+	cp seabios-config seabios-dir/.config;
+
 .PHONY: all
-all:
+all: seabios-dir
 	@set -e; if [ $$((`( bcc -v 2>&1 | grep version || echo 0.0.0 ) | cut -d' ' -f 3 | awk -F. '{ printf "0x%02x%02x%02x", $$1, $$2, $$3}'`)) -lt $$((0x00100e)) ] ; then \
 	echo "==========================================================================="; \
 	echo "Require dev86 rpm or bin86 & bcc debs version >= 0.16.14 to build firmware!"; \
@@ -35,4 +40,16 @@ clean: subdirs-clean
 distclean: subdirs-distclean
 
 subdir-distclean-etherboot: .phony
-	$(MAKE) -C etherboot distclean
\ No newline at end of file
+	$(MAKE) -C etherboot distclean
+
+subdir-distclean-seabios-dir: .phony
+	rm -rf seabios-dir seabios-dir-remote
+
+.PHONY: seabios-dir-force-update
+seabios-dir-force-update:
+	set -ex; \
+	if [ "$(SEABIOS_UPSTREAM_TAG)" ]; then \
+		cd seabios-dir-remote; \
+		$(GIT) fetch origin; \
+		$(GIT) reset --hard $(SEABIOS_UPSTREAM_TAG); \
+	fi
diff --git a/tools/firmware/hvmloader/Makefile b/tools/firmware/hvmloader/Makefile
index 11e8f1c..ae8a1e2 100644
--- a/tools/firmware/hvmloader/Makefile
+++ b/tools/firmware/hvmloader/Makefile
@@ -44,6 +44,7 @@ CFLAGS += -DENABLE_ROMBIOS
 ROMBIOS_ROM := $(ROMBIOS_DIR)/BIOS-bochs-latest
 endif
 
+SEABIOS_DIR := ../seabios-dir
 ifneq ($(SEABIOS_DIR),)
 OBJS += seabios.o
 CFLAGS += -DENABLE_SEABIOS
diff --git a/tools/firmware/seabios-config b/tools/firmware/seabios-config
new file mode 100644
index 0000000..202d15f
--- /dev/null
+++ b/tools/firmware/seabios-config
@@ -0,0 +1,73 @@
+#
+# Automatically generated make config: don't edit
+# SeaBIOS Configuration
+# Wed Sep  7 13:03:21 2011
+#
+
+#
+# General Features
+#
+# CONFIG_COREBOOT is not set
+CONFIG_XEN=y
+CONFIG_THREADS=y
+# CONFIG_THREAD_OPTIONROMS is not set
+CONFIG_RELOCATE_INIT=y
+CONFIG_BOOTMENU=y
+# CONFIG_BOOTSPLASH is not set
+CONFIG_BOOTORDER=y
+
+#
+# Hardware support
+#
+CONFIG_ATA=y
+CONFIG_ATA_DMA=y
+CONFIG_ATA_PIO32=y
+CONFIG_AHCI=y
+CONFIG_VIRTIO_BLK=y
+CONFIG_FLOPPY=y
+CONFIG_PS2PORT=y
+CONFIG_USB=y
+CONFIG_USB_UHCI=y
+CONFIG_USB_OHCI=y
+CONFIG_USB_EHCI=y
+CONFIG_USB_MSC=y
+CONFIG_USB_HUB=y
+CONFIG_USB_KEYBOARD=y
+CONFIG_USB_MOUSE=y
+CONFIG_SERIAL=y
+CONFIG_LPT=y
+# CONFIG_USE_SMM is not set
+CONFIG_MTRR_INIT=y
+
+#
+# BIOS interfaces
+#
+CONFIG_DRIVES=y
+CONFIG_CDROM_BOOT=y
+CONFIG_CDROM_EMU=y
+CONFIG_PCIBIOS=y
+CONFIG_APMBIOS=y
+CONFIG_PNPBIOS=y
+CONFIG_OPTIONROMS=y
+# CONFIG_OPTIONROMS_DEPLOYED is not set
+CONFIG_PMM=y
+CONFIG_BOOT=y
+CONFIG_KEYBOARD=y
+CONFIG_KBD_CALL_INT15_4F=y
+CONFIG_MOUSE=y
+CONFIG_S3_RESUME=y
+# CONFIG_DISABLE_A20 is not set
+
+#
+# BIOS Tables
+#
+CONFIG_PIRTABLE=y
+CONFIG_MPTABLE=y
+CONFIG_SMBIOS=y
+CONFIG_ACPI=y
+
+#
+# Debugging
+#
+CONFIG_DEBUG_LEVEL=1
+# CONFIG_DEBUG_SERIAL is not set
-- 
1.7.2.5

Re: [PATCH v9 5/6] Clone and build Seabios by default

[Ian Jackson]

stefano.stabellini@eu.citrix.com writes ("[Xen-devel] [PATCH v9 5/6] Clone and build Seabios by default"):
>  QEMU_UPSTREAM_URL ?= http://xenbits.xensource.com/git-http/qemu-upstream-unstable.git
> +SEABIOS_UPSTREAM_URL ?= http://git.qemu.org/git/seabios.git

Do we wanted push-gated versions of these trees too ?

I see you have included a lockstep changeset number in xen-unstable
but are we really able to force these to update in lockstep ?

Ian.

Re: [PATCH v9 5/6] Clone and build Seabios by default

[Ian Campbell]

On Mon, 2011-12-12 at 17:39 +0000, Ian Jackson wrote:
> stefano.stabellini@eu.citrix.com writes ("[Xen-devel] [PATCH v9 5/6] Clone and build Seabios by default"):
> >  QEMU_UPSTREAM_URL ?= http://xenbits.xensource.com/git-http/qemu-upstream-unstable.git
> > +SEABIOS_UPSTREAM_URL ?= http://git.qemu.org/git/seabios.git
> 
> Do we wanted push-gated versions of these trees too ?
> 
> I see you have included a lockstep changeset number in xen-unstable
> but are we really able to force these to update in lockstep ?

We should have a mirror on xenbits, we generally avoid reliance on third
party resources in our builds. Not just source repositories but also for
things like tarball downloads.

Also the upstream for seabios is git://git.seabios.org/seabios.git I
don't think indirecting via qemu buys us anything.

Ian.

Re: [PATCH v9 5/6] Clone and build Seabios by default

[Stefano Stabellini]

On Mon, 12 Dec 2011, Ian Campbell wrote:
> On Mon, 2011-12-12 at 17:39 +0000, Ian Jackson wrote:
> > stefano.stabellini@eu.citrix.com writes ("[Xen-devel] [PATCH v9 5/6] Clone and build Seabios by default"):
> > >  QEMU_UPSTREAM_URL ?= http://xenbits.xensource.com/git-http/qemu-upstream-unstable.git
> > > +SEABIOS_UPSTREAM_URL ?= http://git.qemu.org/git/seabios.git
> > 
> > Do we wanted push-gated versions of these trees too ?
> > 
> > I see you have included a lockstep changeset number in xen-unstable
> > but are we really able to force these to update in lockstep ?
> 
> We should have a mirror on xenbits, we generally avoid reliance on third
> party resources in our builds. Not just source repositories but also for
> things like tarball downloads.
> 
> Also the upstream for seabios is git://git.seabios.org/seabios.git I
> don't think indirecting via qemu buys us anything.
 
I don't have a strong opinion on this, so if you'd like to change the
url to a xenbits tree, I am OK with that.
I guess you don't need me to send a patch for that, but let me know if
you want me to change something.

Re: [PATCH v9 5/6] Clone and build Seabios by default

[Ian Jackson]

Stefano Stabellini writes ("Re: [Xen-devel] [PATCH v9 5/6] Clone and build Seabios by default"):
> On Mon, 12 Dec 2011, Ian Campbell wrote:
> > On Mon, 2011-12-12 at 17:39 +0000, Ian Jackson wrote:
> > > I see you have included a lockstep changeset number in xen-unstable
> > > but are we really able to force these to update in lockstep ?

Did you have an answer for this ?

> > We should have a mirror on xenbits, we generally avoid reliance on third
> > party resources in our builds. Not just source repositories but also for
> > things like tarball downloads.
> > 
> > Also the upstream for seabios is git://git.seabios.org/seabios.git I
> > don't think indirecting via qemu buys us anything.
>  
> I don't have a strong opinion on this, so if you'd like to change the
> url to a xenbits tree, I am OK with that.
> I guess you don't need me to send a patch for that, but let me know if
> you want me to change something.

I agree with Ian's comments about using xenbits.

But before you set up any trees there, please let's answer the
question about our update paradigm.

Ian.

Re: [PATCH v9 5/6] Clone and build Seabios by default

[Stefano Stabellini]

On Tue, 13 Dec 2011, Ian Jackson wrote:
> Stefano Stabellini writes ("Re: [Xen-devel] [PATCH v9 5/6] Clone and build Seabios by default"):
> > On Mon, 12 Dec 2011, Ian Campbell wrote:
> > > On Mon, 2011-12-12 at 17:39 +0000, Ian Jackson wrote:
> > > > I see you have included a lockstep changeset number in xen-unstable
> > > > but are we really able to force these to update in lockstep ?
> 
> Did you have an answer for this ?

The answer is yes. Unless there is a bug that I didn't manage to find of
course.

Re: [PATCH v9 5/6] Clone and build Seabios by default

[Ian Jackson]

Stefano Stabellini writes ("Re: [Xen-devel] [PATCH v9 5/6] Clone and build Seabios by default"):
> On Tue, 13 Dec 2011, Ian Jackson wrote:
> > Stefano Stabellini writes ("Re: [Xen-devel] [PATCH v9 5/6] Clone and build Seabios by default"):
> > > On Mon, 12 Dec 2011, Ian Campbell wrote:
> > > > On Mon, 2011-12-12 at 17:39 +0000, Ian Jackson wrote:
> > > > > I see you have included a lockstep changeset number in xen-unstable
> > > > > but are we really able to force these to update in lockstep ?
> > 
> > Did you have an answer for this ?
> 
> The answer is yes. Unless there is a bug that I didn't manage to find of
> course.

That doesn't sound like an answer to my question.

To put it another way: are we intending to permanently maintain a
branch of seabios and a branch of qemu upstream ?  Are we going to
have one such branch for each branch of Xen ?  Are we expecting our
downstreams to use our specific version of qemu and seabios, rather
than building from upstream ?

If the answer to any of these questions is "no" then attempting to
nominate a specific changeset won't work.

Ian.

Re: [PATCH v9 5/6] Clone and build Seabios by default

[Stefano Stabellini]

On Tue, 13 Dec 2011, Ian Jackson wrote:
> Stefano Stabellini writes ("Re: [Xen-devel] [PATCH v9 5/6] Clone and build Seabios by default"):
> > On Tue, 13 Dec 2011, Ian Jackson wrote:
> > > Stefano Stabellini writes ("Re: [Xen-devel] [PATCH v9 5/6] Clone and build Seabios by default"):
> > > > On Mon, 12 Dec 2011, Ian Campbell wrote:
> > > > > On Mon, 2011-12-12 at 17:39 +0000, Ian Jackson wrote:
> > > > > > I see you have included a lockstep changeset number in xen-unstable
> > > > > > but are we really able to force these to update in lockstep ?
> > > 
> > > Did you have an answer for this ?
> > 
> > The answer is yes. Unless there is a bug that I didn't manage to find of
> > course.
> 
> That doesn't sound like an answer to my question.
> 
> To put it another way: are we intending to permanently maintain a
> branch of seabios and a branch of qemu upstream ?  Are we going to
> have one such branch for each branch of Xen ?  Are we expecting our
> downstreams to use our specific version of qemu and seabios, rather
> than building from upstream ?

I think "yes" in case of qemu, at the very least for the different
release times of the xen and qemu projects.
I don't have an opinion on seabios because I am nore sure how many
changes are we going to make to it. However I recall that you and/or
IanC were in favor of having or own branch of seabios as well.
BTW I think it is probably better if IanC maintains seabios if we decide
to have our own branch, given that he is the one that did most of the
work on it so far.

Re: [PATCH v9 5/6] Clone and build Seabios by default

[Ian Campbell]

On Tue, 2011-12-13 at 18:22 +0000, Stefano Stabellini wrote:
> On Tue, 13 Dec 2011, Ian Jackson wrote:
> > Stefano Stabellini writes ("Re: [Xen-devel] [PATCH v9 5/6] Clone and build Seabios by default"):
> > > On Tue, 13 Dec 2011, Ian Jackson wrote:
> > > > Stefano Stabellini writes ("Re: [Xen-devel] [PATCH v9 5/6] Clone and build Seabios by default"):
> > > > > On Mon, 12 Dec 2011, Ian Campbell wrote:
> > > > > > On Mon, 2011-12-12 at 17:39 +0000, Ian Jackson wrote:
> > > > > > > I see you have included a lockstep changeset number in xen-unstable
> > > > > > > but are we really able to force these to update in lockstep ?
> > > > 
> > > > Did you have an answer for this ?
> > > 
> > > The answer is yes. Unless there is a bug that I didn't manage to find of
> > > course.
> > 
> > That doesn't sound like an answer to my question.
> > 
> > To put it another way: are we intending to permanently maintain a
> > branch of seabios and a branch of qemu upstream ?  Are we going to
> > have one such branch for each branch of Xen ?  Are we expecting our
> > downstreams to use our specific version of qemu and seabios, rather
> > than building from upstream ?
> 
> I think "yes" in case of qemu, at the very least for the different
> release times of the xen and qemu projects.

Unfortunately I expect that distros will want to package qemu once (from
upstream) with support for all the various options, including Xen,
enabled and depend on that in their Xen packaging rather than packaging
a slightly different qemu a second time.

I'm not sure how we can address this without exploding our test matrix.
It seems like we need to test at a minimum the current qemu stable and
development branches :-/

> I don't have an opinion on seabios because I am nore sure how many
> changes are we going to make to it. However I recall that you and/or
> IanC were in favor of having or own branch of seabios as well.

I think that even if we never diverge from upstream and irrespective of
whether we stick in lockstep or not we should host a tree to point our
users (and default build) at. This gives us flexibility if we ever do
need to diverge and also isolates our users from issues with 3rd party
servers.

> BTW I think it is probably better if IanC maintains seabios if we decide
> to have our own branch, given that he is the one that did most of the
> work on it so far.

I'm happy to do that.

Ian.

Re: [PATCH v9 5/6] Clone and build Seabios by default

[Ian Jackson]

Ian Campbell writes ("Re: [Xen-devel] [PATCH v9 5/6] Clone and build Seabios by default"):
> Unfortunately I expect that distros will want to package qemu once (from
> upstream) with support for all the various options, including Xen,
> enabled and depend on that in their Xen packaging rather than packaging
> a slightly different qemu a second time.

Quite.

> I'm not sure how we can address this without exploding our test matrix.
> It seems like we need to test at a minimum the current qemu stable and
> development branches :-/

This is no different to the kernel, of course.  We don't do a very
good job of testing different upstream kernels but it's on my todo
list.

> > I don't have an opinion on seabios because I am nore sure how many
> > changes are we going to make to it. However I recall that you and/or
> > IanC were in favor of having or own branch of seabios as well.
> 
> I think that even if we never diverge from upstream and irrespective of
> whether we stick in lockstep or not we should host a tree to point our
> users (and default build) at. This gives us flexibility if we ever do
> need to diverge and also isolates our users from issues with 3rd party
> servers.

Indeed so.  But should it be one tree for each Xen branch, and does it
need a push gate ?

Ian.

Re: [PATCH v9 5/6] Clone and build Seabios by default

[Stefano Stabellini]

On Wed, 14 Dec 2011, Ian Jackson wrote:
> Ian Campbell writes ("Re: [Xen-devel] [PATCH v9 5/6] Clone and build Seabios by default"):
> > Unfortunately I expect that distros will want to package qemu once (from
> > upstream) with support for all the various options, including Xen,
> > enabled and depend on that in their Xen packaging rather than packaging
> > a slightly different qemu a second time.
> 
> Quite.

Considering that Qemu releases are not in sync with our releases, I
think that we should point xen-unstable to one of our own Qemu trees
rather than just a mirror of the Qemu stable branch on xenbits.
Otherwise we risk loosing important features, for example if there are
not going to be any new Qemu releases before Xen 4.2 is out, we are
going to miss pci passthrough and save/restore, while if we had our own
branch we would have them.
Of course it is still very important to keep Qemu stable branches stable
and functional, so that if distros decide to package a single upstream
Qemu, it still going to work fine, even though it is going to have fewer
features.


> > > I don't have an opinion on seabios because I am nore sure how many
> > > changes are we going to make to it. However I recall that you and/or
> > > IanC were in favor of having or own branch of seabios as well.
> > 
> > I think that even if we never diverge from upstream and irrespective of
> > whether we stick in lockstep or not we should host a tree to point our
> > users (and default build) at. This gives us flexibility if we ever do
> > need to diverge and also isolates our users from issues with 3rd party
> > servers.
> 
> Indeed so.  But should it be one tree for each Xen branch, and does it
> need a push gate ?
 
I think we might do without a tree for each Xen branch because we need
to support building upstream Qemu against Xen 4.2+ anyway.
Let's say that we have released Xen 4.2, we are now in the Xen 4.3
release cycle and we want to introduce an incompatible change in Qemu:
we would need to add a compatibility layer to allow it to build and run
on Xen 4.2 anyway.
The difficulty having a single tree would be keeping track of the
changes only present in our own tree, and possibly revert them before
pulling new commits from upstream that implement the same features in a
different way.

Regarding the push-gate, I am OK with it.

Re: [PATCH v9 5/6] Clone and build Seabios by default

[Ian Campbell]

On Wed, 2011-12-14 at 16:59 +0000, Stefano Stabellini wrote:
> On Wed, 14 Dec 2011, Ian Jackson wrote:
> > Ian Campbell writes ("Re: [Xen-devel] [PATCH v9 5/6] Clone and build Seabios by default"):
> > > Unfortunately I expect that distros will want to package qemu once (from
> > > upstream) with support for all the various options, including Xen,
> > > enabled and depend on that in their Xen packaging rather than packaging
> > > a slightly different qemu a second time.
> > 
> > Quite.
> 
> Considering that Qemu releases are not in sync with our releases, I
> think that we should point xen-unstable to one of our own Qemu trees
> rather than just a mirror of the Qemu stable branch on xenbits.
> Otherwise we risk loosing important features, for example if there are
> not going to be any new Qemu releases before Xen 4.2 is out, we are
> going to miss pci passthrough and save/restore, while if we had our own
> branch we would have them.
> Of course it is still very important to keep Qemu stable branches stable
> and functional, so that if distros decide to package a single upstream
> Qemu, it still going to work fine, even though it is going to have fewer
> features.

Agreed, all these trees are very important and therefore we must test
all of them, although perhaps not in the same way as we currently test
things.

How about this:

We host a qemu tree including a staging branch and associated push gate.
This tree tracks the current qemu stable branch and contains backports
of patches which have been applied to the current qemu development
branch (using a policy similar to the Linux stable series policy, in
order to prevent accidental forking). We point xen-devel at this and it
is used by the regular testing machinery and forms part of the Xen
releases.

Then as a separate independent test we periodically (e.g. weekly or
twice-weekly) test the upstream qemu development branch against some
baseline version of Xen (e.g. the most recent pass of either unstable or
current stable branch). This test doesn't gate anything but is just to
keep tabs on the development branch and spot regressions earlier etc.

Lastly we also test any releases (rc and final, perhaps beta) on both
the qemu stable and development branches against current latest unstable
and stable Xen branches. Again this doesn't gate anything but just keeps
tabs on things.

BTW I think periodic test + rc&releases is how we should handle Linux
upstream too and indeed any other external dependency which runs on a
disconnected schedule.

Ian.

> 
> 
> > > > I don't have an opinion on seabios because I am nore sure how many
> > > > changes are we going to make to it. However I recall that you and/or
> > > > IanC were in favor of having or own branch of seabios as well.
> > > 
> > > I think that even if we never diverge from upstream and irrespective of
> > > whether we stick in lockstep or not we should host a tree to point our
> > > users (and default build) at. This gives us flexibility if we ever do
> > > need to diverge and also isolates our users from issues with 3rd party
> > > servers.
> > 
> > Indeed so.  But should it be one tree for each Xen branch, and does it
> > need a push gate ?
>  
> I think we might do without a tree for each Xen branch because we need
> to support building upstream Qemu against Xen 4.2+ anyway.
> Let's say that we have released Xen 4.2, we are now in the Xen 4.3
> release cycle and we want to introduce an incompatible change in Qemu:
> we would need to add a compatibility layer to allow it to build and run
> on Xen 4.2 anyway.
> The difficulty having a single tree would be keeping track of the
> changes only present in our own tree, and possibly revert them before
> pulling new commits from upstream that implement the same features in a
> different way.
> 
> Regarding the push-gate, I am OK with it.

Re: [PATCH v9 5/6] Clone and build Seabios by default

[Stefano Stabellini]

On Thu, 15 Dec 2011, Ian Campbell wrote:
> > Considering that Qemu releases are not in sync with our releases, I
> > think that we should point xen-unstable to one of our own Qemu trees
> > rather than just a mirror of the Qemu stable branch on xenbits.
> > Otherwise we risk loosing important features, for example if there are
> > not going to be any new Qemu releases before Xen 4.2 is out, we are
> > going to miss pci passthrough and save/restore, while if we had our own
> > branch we would have them.
> > Of course it is still very important to keep Qemu stable branches stable
> > and functional, so that if distros decide to package a single upstream
> > Qemu, it still going to work fine, even though it is going to have fewer
> > features.
> 
> Agreed, all these trees are very important and therefore we must test
> all of them, although perhaps not in the same way as we currently test
> things.
> 
> How about this:
> 
> We host a qemu tree including a staging branch and associated push gate.
> This tree tracks the current qemu stable branch and contains backports
> of patches which have been applied to the current qemu development
> branch (using a policy similar to the Linux stable series policy, in
> order to prevent accidental forking). We point xen-devel at this and it
> is used by the regular testing machinery and forms part of the Xen
> releases.

I am OK with this, if everything goes well we shouldn't need anything
else. It might happen that because of time constraints we have to apply
patches to this tree before they are applied upstream, but it should
only be the exception, not the rule.


> Then as a separate independent test we periodically (e.g. weekly or
> twice-weekly) test the upstream qemu development branch against some
> baseline version of Xen (e.g. the most recent pass of either unstable or
> current stable branch). This test doesn't gate anything but is just to
> keep tabs on the development branch and spot regressions earlier etc.

twice-weekly would be good


> Lastly we also test any releases (rc and final, perhaps beta) on both
> the qemu stable and development branches against current latest unstable
> and stable Xen branches. Again this doesn't gate anything but just keeps
> tabs on things.
> 
> BTW I think periodic test + rc&releases is how we should handle Linux
> upstream too and indeed any other external dependency which runs on a
> disconnected schedule.

indeed

[PATCH v9 6/6] libxl: use new qemu at the location where xen-unstable installs it

[stefano.stabellini]

From: Ian Campbell <ian.campbell@citrix.com>

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
 tools/libxl/libxl_dm.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
index bf38877..633123f 100644
--- a/tools/libxl/libxl_dm.c
+++ b/tools/libxl/libxl_dm.c
@@ -55,7 +55,7 @@ const char *libxl__domain_device_model(libxl__gc *gc,
             dm = libxl__abs_path(gc, "qemu-dm", libxl_libexec_path());
             break;
         case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN:
-            dm = libxl__strdup(gc, "/usr/bin/qemu");
+            dm = libxl__abs_path(gc, "qemu", libxl_libexec_path());
             break;
         default:
             LIBXL__LOG(ctx, LIBXL__LOG_ERROR,
-- 
1.7.2.5