[PATCH 0/4] Reintroduce OVMF support

[PATCH 0/4] Reintroduce OVMF support

[Wei Liu]

This small series reintroduces OVMF support in Xen

You can fetch working OVMF tree on:
git://xenbits.xen.org/people/liuw/ovmf.git master

Working changeset that can be sticked in Config.mk is:
8833370303d3bf3153760ee42760ef1b9b5c562

Note that VNC doesn't work properly when using OVMF, but that's not OVMF's
problem. This issue should be addressed in Xen and I'm working on that.

Wei.

Ian Campbell (3):
  tools: clone ovmf to ovmf-dir directory
  tools: support system supplied ovmf binary
  tools: Enable OVMF build by default

Wei Liu (1):
  libxl: bump LIBXL_MAXMEM_CONSTANT to 2048

 Makefile                          |    4 ++++
 config/Tools.mk.in                |    1 +
 tools/configure                   |   20 +++++++++++++++-----
 tools/configure.ac                |   13 ++++++++++++-
 tools/firmware/Makefile           |   31 ++++++++++++-------------------
 tools/firmware/hvmloader/Makefile |    8 ++++++--
 tools/libxl/libxl_internal.h      |    2 +-
 7 files changed, 51 insertions(+), 28 deletions(-)

-- 
1.7.10.4

[PATCH 1/4] libxl: bump LIBXL_MAXMEM_CONSTANT to 2048

[Wei Liu]

When using OVMF we need to have 1MiB of memory in place for firmware.
Without this change we have:

(XEN) HVM128: Loading OVMF ...
(XEN) page_alloc.c:1460:d128 Over-allocation for domain 128: 33025 > 33024
(XEN) memory.c:132:d128 Could not allocate order=0 extent: id=128 memflags=0 (0 of 1)

This is not a fatal error as hvmloader will instead use low memory to
load OVMF, but it's better to eliminate such error.

Changing this constant doesn't necessary increase the total amount of
memory a guest uses because it's just a limit.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
---
 tools/libxl/libxl_internal.h |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 165dc00..908af20 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -87,7 +87,7 @@
 #define LIBXL_QEMU_BODGE_TIMEOUT 2
 #define LIBXL_XENCONSOLE_LIMIT 1048576
 #define LIBXL_XENCONSOLE_PROTOCOL "vt100"
-#define LIBXL_MAXMEM_CONSTANT 1024
+#define LIBXL_MAXMEM_CONSTANT 2048
 #define LIBXL_PV_EXTRA_MEMORY 1024
 #define LIBXL_HVM_EXTRA_MEMORY 2048
 #define LIBXL_MIN_DOM0_MEM (128*1024)
-- 
1.7.10.4

[PATCH 2/4] tools: clone ovmf to ovmf-dir directory

[Wei Liu]

From: Ian Campbell <ian.campbell@citrix.com>

for consistency with other foo-dir e.g. qemu, seabios.

Remove obsolete ovmf-find target.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 Makefile                          |    4 ++++
 tools/firmware/Makefile           |   29 ++++++++++-------------------
 tools/firmware/hvmloader/Makefile |    2 +-
 3 files changed, 15 insertions(+), 20 deletions(-)

diff --git a/Makefile b/Makefile
index d915660..4e48457 100644
--- a/Makefile
+++ b/Makefile
@@ -110,6 +110,10 @@ endif
 tools/firmware/seabios-dir-force-update:
 	$(MAKE) -C tools/firmware seabios-dir-force-update
 
+.PHONY: tools/firmware/ovmf-dir-force-update
+tools/firmware/ovmf-dir-force-update:
+	$(MAKE) -C tools/firmware ovmf-dir-force-update
+
 .PHONY: install-docs
 install-docs:
 	$(MAKE) -C docs install
diff --git a/tools/firmware/Makefile b/tools/firmware/Makefile
index f064765..8633748 100644
--- a/tools/firmware/Makefile
+++ b/tools/firmware/Makefile
@@ -6,7 +6,7 @@ TARGET      := hvmloader/hvmloader
 INST_DIR := $(DESTDIR)$(XENFIRMWAREDIR)
 
 SUBDIRS-y :=
-SUBDIRS-$(CONFIG_OVMF) += ovmf
+SUBDIRS-$(CONFIG_OVMF) += ovmf-dir
 ifeq ($(SEABIOS_PATH),)
 SUBDIRS-$(CONFIG_SEABIOS) += seabios-dir
 endif
@@ -15,9 +15,9 @@ SUBDIRS-$(CONFIG_ROMBIOS) += vgabios
 SUBDIRS-$(CONFIG_ROMBIOS) += etherboot
 SUBDIRS-y += hvmloader
 
-ovmf:
-	GIT=$(GIT) $(XEN_ROOT)/scripts/git-checkout.sh $(OVMF_UPSTREAM_URL) $(OVMF_UPSTREAM_REVISION) ovmf
-	cp ovmf-makefile ovmf/Makefile;
+ovmf-dir:
+	GIT=$(GIT) $(XEN_ROOT)/scripts/git-checkout.sh $(OVMF_UPSTREAM_URL) $(OVMF_UPSTREAM_REVISION) ovmf-dir
+	cp ovmf-makefile ovmf-dir/Makefile;
 
 seabios-dir:
 	GIT=$(GIT) $(XEN_ROOT)/scripts/git-checkout.sh $(SEABIOS_UPSTREAM_URL) $(SEABIOS_UPSTREAM_TAG) seabios-dir
@@ -52,32 +52,23 @@ subdir-distclean-etherboot: .phony
 	$(MAKE) -C etherboot distclean
 
 subdir-distclean-ovmf: .phony
-	rm -rf ovmf ovmf-remote
+	rm -rf ovmf-dir ovmf-dir-remote
 
 subdir-distclean-seabios-dir: .phony
 	rm -rf seabios-dir seabios-dir-remote
 
-.PHONY: ovmf-find
-ovmf-find:
-	if test -d $(OVMF_UPSTREAM_URL) ; then \
-	               mkdir -p ovmf; \
-        else \
-                export GIT=$(GIT); \
-                $(XEN_ROOT)/scripts/git-checkout.sh $(OVMF_UPSTREAM_URL) $(OVMF_UPSTREAM_REVISION) ovmf ; \
-        fi
-
-.PHONY: ovmf-force-update
-ovmf-force-update:
+.PHONY: ovmf-dir-force-update
+ovmf-dir-force-update:
 	set -ex; \
 	if [ "$(OVMF_UPSTREAM_REVISION)" ]; then \
-		cd ovmf-remote; \
+		cd ovmf-dir-remote; \
 		$(GIT) fetch origin; \
 		$(GIT) reset --hard $(OVMF_UPSTREAM_REVISION); \
 	fi
 
 subdir-clean-ovmf:
-	set -e; if test -d ovmf/.; then \
-		$(MAKE) -C ovmf clean; \
+	set -e; if test -d ovmf-dir/.; then \
+		$(MAKE) -C ovmf-dir clean; \
 	fi
 
 .PHONY: seabios-dir-force-update
diff --git a/tools/firmware/hvmloader/Makefile b/tools/firmware/hvmloader/Makefile
index 8ab1429..e27e457 100644
--- a/tools/firmware/hvmloader/Makefile
+++ b/tools/firmware/hvmloader/Makefile
@@ -37,7 +37,7 @@ endif
 
 CIRRUSVGA_DEBUG ?= n
 
-OVMF_DIR := ../ovmf
+OVMF_DIR := ../ovmf-dir
 ROMBIOS_DIR := ../rombios
 SEABIOS_DIR := ../seabios-dir
 
-- 
1.7.10.4

[PATCH 3/4] tools: support system supplied ovmf binary

[Wei Liu]

From: Ian Campbell <ian.campbell@citrix.com>

Debian Jessie at least contains an ovmf package that includes
/usr/share/ovmf/OVMF.fd. It's also possible that user may want to supply
his/her own ovmf binary.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 config/Tools.mk.in                |    1 +
 tools/configure                   |   16 +++++++++++++---
 tools/configure.ac                |   11 +++++++++++
 tools/firmware/Makefile           |    2 ++
 tools/firmware/hvmloader/Makefile |    6 +++++-
 5 files changed, 32 insertions(+), 4 deletions(-)

diff --git a/config/Tools.mk.in b/config/Tools.mk.in
index 67f5782..d9d3239 100644
--- a/config/Tools.mk.in
+++ b/config/Tools.mk.in
@@ -24,6 +24,7 @@ BCC                 := @BCC@
 IASL                := @IASL@
 FETCHER             := @FETCHER@
 SEABIOS_PATH        := @seabios_path@
+OVMF_PATH           := @ovmf_path@
 
 # Extra folder for libs/includes
 PREPEND_INCLUDES    := @PREPEND_INCLUDES@
diff --git a/tools/configure b/tools/configure
index 1da8652..22e74a7 100755
--- a/tools/configure
+++ b/tools/configure
@@ -682,6 +682,7 @@ APPEND_LIB
 APPEND_INCLUDES
 PREPEND_LIB
 PREPEND_INCLUDES
+ovmf_path
 seabios_path
 qemu_xen
 qemu_traditional
@@ -768,6 +769,7 @@ enable_blktap1
 enable_qemu_traditional
 with_system_qemu
 with_system_seabios
+with_system_ovmf
 '
       ac_precious_vars='build_alias
 host_alias
@@ -1440,6 +1442,9 @@ Optional Packages:
   --with-system-seabios[=PATH]
                           Use system supplied seabios PATH instead of building
                           and installing our own version
+  --with-system-ovmf[=PATH]
+                          Use system supplied OVMF PATH instead of building
+                          and installing our own version
 
 Some influential environment variables:
   CC          C compiler command
@@ -3771,9 +3776,14 @@ fi
 
 
 
-
-
-
+# Check whether --with-system-ovmf was given.
+if test "${with_system_ovmf+set}" = set; then :
+  withval=$with_system_ovmf;
+    case $withval in
+        no) ovmf_path= ;;
+        *)  ovmf_path=$withval ;;
+    esac
+fi
 
 
 for cppflag in $PREPEND_INCLUDES
diff --git a/tools/configure.ac b/tools/configure.ac
index 4f3c33a..b2941a4 100644
--- a/tools/configure.ac
+++ b/tools/configure.ac
@@ -108,6 +108,17 @@ AC_ARG_WITH([system-seabios],
 ],[])
 AC_SUBST(seabios_path)
 
+AC_ARG_WITH([system-ovmf],
+    AS_HELP_STRING([--with-system-ovmf@<:@=PATH@:>@],
+       [Use system supplied OVMF PATH instead of building and installing
+        our own version]),[
+    case $withval in
+        no) ovmf_path= ;;
+        *)  ovmf_path=$withval ;;
+    esac
+],[])
+AC_SUBST(ovmf_path)
+
 AC_ARG_VAR([PREPEND_INCLUDES],
     [List of include folders to prepend to CFLAGS (without -I)])
 AC_ARG_VAR([PREPEND_LIB],
diff --git a/tools/firmware/Makefile b/tools/firmware/Makefile
index 8633748..cb13212 100644
--- a/tools/firmware/Makefile
+++ b/tools/firmware/Makefile
@@ -6,7 +6,9 @@ TARGET      := hvmloader/hvmloader
 INST_DIR := $(DESTDIR)$(XENFIRMWAREDIR)
 
 SUBDIRS-y :=
+ifeq ($(OVMF_PATH),)
 SUBDIRS-$(CONFIG_OVMF) += ovmf-dir
+endif
 ifeq ($(SEABIOS_PATH),)
 SUBDIRS-$(CONFIG_SEABIOS) += seabios-dir
 endif
diff --git a/tools/firmware/hvmloader/Makefile b/tools/firmware/hvmloader/Makefile
index e27e457..00ee952 100644
--- a/tools/firmware/hvmloader/Makefile
+++ b/tools/firmware/hvmloader/Makefile
@@ -56,7 +56,11 @@ ROMS :=
 ifeq ($(CONFIG_OVMF),y)
 OBJS += ovmf.o
 CFLAGS += -DENABLE_OVMF
-OVMF_ROM := $(OVMF_DIR)/ovmf.bin
+ifeq ($(OVMF_PATH),)
+	OVMF_ROM := $(OVMF_DIR)/ovmf.bin
+else
+	OVMF_ROM := $(OVMF_PATH)
+endif
 ROMS += $(OVMF_ROM)
 endif
 
-- 
1.7.10.4

[PATCH 4/4] tools: Enable OVMF build by default

[Wei Liu]

From: Ian Campbell <ian.campbell@citrix.com>

The issues with non-GCC4.4 builds seem to have been resolved.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
 tools/configure    |    4 ++--
 tools/configure.ac |    2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/tools/configure b/tools/configure
index 22e74a7..eb58bcd 100755
--- a/tools/configure
+++ b/tools/configure
@@ -1422,7 +1422,7 @@ Optional Features:
   --enable-xenapi         Enable Xen API Bindings (default is DISABLED)
   --disable-ocamltools    Disable Ocaml tools (default is ENABLED)
   --disable-xsmpolicy     Disable XSM policy compilation (default is ENABLED)
-  --enable-ovmf           Enable OVMF (default is DISABLED)
+  --disable-ovmf          Enable OVMF (default is ENABLED)
   --disable-rombios       Disable ROM BIOS (default is ENABLED)
   --disable-seabios       Disable SeaBIOS (default is ENABLED)
   --disable-debug         Disable debug build of tools (default is ENABLED)
@@ -3584,7 +3584,7 @@ elif test "x$enable_ovmf" = "xyes"; then :
 
 elif test -z $ax_cv_ovmf; then :
 
-    ax_cv_ovmf="n"
+    ax_cv_ovmf="y"
 
 fi
 ovmf=$ax_cv_ovmf
diff --git a/tools/configure.ac b/tools/configure.ac
index b2941a4..86fcb28 100644
--- a/tools/configure.ac
+++ b/tools/configure.ac
@@ -53,7 +53,7 @@ AX_ARG_DEFAULT_ENABLE([monitors], [Disable xenstat and xentop monitoring tools])
 AX_ARG_DEFAULT_DISABLE([xenapi], [Enable Xen API Bindings])
 AX_ARG_DEFAULT_ENABLE([ocamltools], [Disable Ocaml tools])
 AX_ARG_DEFAULT_ENABLE([xsmpolicy], [Disable XSM policy compilation])
-AX_ARG_DEFAULT_DISABLE([ovmf], [Enable OVMF])
+AX_ARG_DEFAULT_ENABLE([ovmf], [Enable OVMF])
 AX_ARG_DEFAULT_ENABLE([rombios], [Disable ROM BIOS])
 AX_ARG_DEFAULT_ENABLE([seabios], [Disable SeaBIOS])
 AX_ARG_DEFAULT_ENABLE([debug], [Disable debug build of tools])
-- 
1.7.10.4

Re: [PATCH 2/4] tools: clone ovmf to ovmf-dir directory

[Jan Beulich]

>>> On 15.10.13 at 18:40, Wei Liu <wei.liu2@citrix.com> wrote:
> @@ -15,9 +15,9 @@ SUBDIRS-$(CONFIG_ROMBIOS) += vgabios
>  SUBDIRS-$(CONFIG_ROMBIOS) += etherboot
>  SUBDIRS-y += hvmloader
>  
> -ovmf:
> -	GIT=$(GIT) $(XEN_ROOT)/scripts/git-checkout.sh $(OVMF_UPSTREAM_URL) $(OVMF_UPSTREAM_REVISION) ovmf
> -	cp ovmf-makefile ovmf/Makefile;
> +ovmf-dir:
> +	GIT=$(GIT) $(XEN_ROOT)/scripts/git-checkout.sh $(OVMF_UPSTREAM_URL) $(OVMF_UPSTREAM_REVISION) ovmf-dir
> +	cp ovmf-makefile ovmf-dir/Makefile;

If $@ was used in rules like this, you wouldn't have to touch the
whole rule now (and others wouldn't have to in the future).

Jan

Re: [PATCH 4/4] tools: Enable OVMF build by default

[Jan Beulich]

>>> On 15.10.13 at 18:40, Wei Liu <wei.liu2@citrix.com> wrote:
> From: Ian Campbell <ian.campbell@citrix.com>
> 
> The issues with non-GCC4.4 builds seem to have been resolved.

So in patch 0/4 you say that you have a private working tree
(which makes me assume that xenbits.xen.org/ovmf.git isn't
suitable), and the series doesn't seem to do anything towards
updating xenbits.xen.org/ovmf.git. How is all this fitting
together? Will everyone but you now end up with a broken
build?

Jan

Re: [PATCH 0/4] Reintroduce OVMF support

[Fabio Fantoni]

Il 15/10/2013 18:40, Wei Liu ha scritto:
> This small series reintroduces OVMF support in Xen
>
> You can fetch working OVMF tree on:
> git://xenbits.xen.org/people/liuw/ovmf.git master
>
> Working changeset that can be sticked in Config.mk is:
> 8833370303d3bf3153760ee42760ef1b9b5c562
>
> Note that VNC doesn't work properly when using OVMF, but that's not OVMF's
> problem. This issue should be addressed in Xen and I'm working on that.
>
> Wei.

Thanks for work on it, when I tested it months ago there was lot of 
problems.
I think it would be good thing to give an eye to the links below to 
improve the hvm part:

- Integrate legacy bios support inside ovmf (I saw a draft on link above 
and I not know actual progress, there was also discussion about it on 
seabios mailing list)
http://git.infradead.org/users/dwmw2/edk2.git
http://git.infradead.org/users/dwmw2/seabios.git

- Since this is a new/experimental section, it would be a better path to 
move the actual 'static' data taken from hvmloader more 'dynamics' or at 
least have theme generated from qemu.
This would lead in turn to a better support for all the new qemu 
device/features and also to avoid eventual regressions with ovmf/qemu 
newer versions.
Actually even a single modification qemu-side, requires some 
modifications also on xen-side, lot of these tedious and long to apply/test.

Thanks for any reply.

>
> Ian Campbell (3):
>    tools: clone ovmf to ovmf-dir directory
>    tools: support system supplied ovmf binary
>    tools: Enable OVMF build by default
>
> Wei Liu (1):
>    libxl: bump LIBXL_MAXMEM_CONSTANT to 2048
>
>   Makefile                          |    4 ++++
>   config/Tools.mk.in                |    1 +
>   tools/configure                   |   20 +++++++++++++++-----
>   tools/configure.ac                |   13 ++++++++++++-
>   tools/firmware/Makefile           |   31 ++++++++++++-------------------
>   tools/firmware/hvmloader/Makefile |    8 ++++++--
>   tools/libxl/libxl_internal.h      |    2 +-
>   7 files changed, 51 insertions(+), 28 deletions(-)
>

Re: [PATCH 4/4] tools: Enable OVMF build by default

[Ian Campbell]

On Wed, 2013-10-16 at 11:02 +0100, Jan Beulich wrote:
> >>> On 15.10.13 at 18:40, Wei Liu <wei.liu2@citrix.com> wrote:
> > From: Ian Campbell <ian.campbell@citrix.com>
> > 
> > The issues with non-GCC4.4 builds seem to have been resolved.
> 
> So in patch 0/4 you say that you have a private working tree
> (which makes me assume that xenbits.xen.org/ovmf.git isn't
> suitable), and the series doesn't seem to do anything towards
> updating xenbits.xen.org/ovmf.git. How is all this fitting
> together? Will everyone but you now end up with a broken
> build?

I took
        Working changeset that can be sticked in Config.mk is:
        8833370303d3bf3153760ee42760ef1b9b5c562
from the 0th email to mean that the committer could/should make this
change at the same time as committing (to be fair, I also discussed this
with Wei IRL before he sent the series, so I had some context).

I can cope with that, although I'd have preferred that to be an actual
patch in the series.

Ian.

Re: [PATCH 4/4] tools: Enable OVMF build by default

[Wei Liu]

On Wed, Oct 16, 2013 at 11:37:14AM +0100, Ian Campbell wrote:
> On Wed, 2013-10-16 at 11:02 +0100, Jan Beulich wrote:
> > >>> On 15.10.13 at 18:40, Wei Liu <wei.liu2@citrix.com> wrote:
> > > From: Ian Campbell <ian.campbell@citrix.com>
> > > 
> > > The issues with non-GCC4.4 builds seem to have been resolved.
> > 
> > So in patch 0/4 you say that you have a private working tree
> > (which makes me assume that xenbits.xen.org/ovmf.git isn't
> > suitable), and the series doesn't seem to do anything towards
> > updating xenbits.xen.org/ovmf.git. How is all this fitting
> > together? Will everyone but you now end up with a broken
> > build?
> 
> I took
>         Working changeset that can be sticked in Config.mk is:
>         8833370303d3bf3153760ee42760ef1b9b5c562
> from the 0th email to mean that the committer could/should make this
> change at the same time as committing (to be fair, I also discussed this
> with Wei IRL before he sent the series, so I had some context).
> 
> I can cope with that, although I'd have preferred that to be an actual
> patch in the series.
> 

Sorry I misunderstood you. I can resend this series with change to
Config.mk.

Wei.

> Ian.

Re: [PATCH 2/4] tools: clone ovmf to ovmf-dir directory

[Wei Liu]

On Wed, Oct 16, 2013 at 10:58:59AM +0100, Jan Beulich wrote:
> >>> On 15.10.13 at 18:40, Wei Liu <wei.liu2@citrix.com> wrote:
> > @@ -15,9 +15,9 @@ SUBDIRS-$(CONFIG_ROMBIOS) += vgabios
> >  SUBDIRS-$(CONFIG_ROMBIOS) += etherboot
> >  SUBDIRS-y += hvmloader
> >  
> > -ovmf:
> > -	GIT=$(GIT) $(XEN_ROOT)/scripts/git-checkout.sh $(OVMF_UPSTREAM_URL) $(OVMF_UPSTREAM_REVISION) ovmf
> > -	cp ovmf-makefile ovmf/Makefile;
> > +ovmf-dir:
> > +	GIT=$(GIT) $(XEN_ROOT)/scripts/git-checkout.sh $(OVMF_UPSTREAM_URL) $(OVMF_UPSTREAM_REVISION) ovmf-dir
> > +	cp ovmf-makefile ovmf-dir/Makefile;
> 
> If $@ was used in rules like this, you wouldn't have to touch the
> whole rule now (and others wouldn't have to in the future).
> 

I will see what I can do.

Wei.

> Jan

Re: [PATCH 0/4] Reintroduce OVMF support

[Wei Liu]

On Wed, Oct 16, 2013 at 12:03:19PM +0200, Fabio Fantoni wrote:
> Il 15/10/2013 18:40, Wei Liu ha scritto:
> >This small series reintroduces OVMF support in Xen
> >
> >You can fetch working OVMF tree on:
> >git://xenbits.xen.org/people/liuw/ovmf.git master
> >
> >Working changeset that can be sticked in Config.mk is:
> >8833370303d3bf3153760ee42760ef1b9b5c562
> >
> >Note that VNC doesn't work properly when using OVMF, but that's not OVMF's
> >problem. This issue should be addressed in Xen and I'm working on that.
> >
> >Wei.
> 
> Thanks for work on it, when I tested it months ago there was lot of
> problems.

Could you elaborate on the problems you're seeing? I'm aware of the VNC
not refreshing problem but that's not to be fixed in OVMF.

> I think it would be good thing to give an eye to the links below to
> improve the hvm part:
> 
> - Integrate legacy bios support inside ovmf (I saw a draft on link
> above and I not know actual progress, there was also discussion
> about it on seabios mailing list)
> http://git.infradead.org/users/dwmw2/edk2.git
> http://git.infradead.org/users/dwmw2/seabios.git
> 

Hmm... I think this is orthogonal. We can get all those gravy new
features when we refresh our own tree.

Keep in mind that we're not forking EDK2. Actually we're trying to work
closely with upstream - I fixed a upstream bug to make OVMF work with
Xen before sending the series so in fact the tree I'm presenting is just
vanilla upstream tree.

The reason we have our own tree is that we need to have a central
location where users can pull from. Also we would test our branch to
avoid frustration.

> - Since this is a new/experimental section, it would be a better
> path to move the actual 'static' data taken from hvmloader more
> 'dynamics' or at least have theme generated from qemu.
> This would lead in turn to a better support for all the new qemu
> device/features and also to avoid eventual regressions with
> ovmf/qemu newer versions.

I don't see why OVMF would prevent you from using QEMU new device
features, but I sort of get the idea of separating hvmloader and other
firmwares. You could make an argument / request on xen-devel and ask
George to keep track of it, then we will see what we can do.

Wei.

Re: [PATCH 0/4] Reintroduce OVMF support

[David Vrabel]

On 15/10/13 17:40, Wei Liu wrote:
> This small series reintroduces OVMF support in Xen

I think somewhere in this series there needs to a brief description of
what OVMF is.

With the exception of the first patch this is just adding yet another
component to clone and build.  Why is this useful?  Why isn't
(effectively) --with-system-ovmf always enabled (and thus not actually
needed as an option)?

Isn't Xen's keenness to build it's own particular versions of stuff
something that causes significant problems for distributions?

David

Re: [PATCH 4/4] tools: Enable OVMF build by default

[Ian Campbell]

On Wed, 2013-10-16 at 12:38 +0100, Wei Liu wrote:
> On Wed, Oct 16, 2013 at 11:37:14AM +0100, Ian Campbell wrote:
> > On Wed, 2013-10-16 at 11:02 +0100, Jan Beulich wrote:
> > > >>> On 15.10.13 at 18:40, Wei Liu <wei.liu2@citrix.com> wrote:
> > > > From: Ian Campbell <ian.campbell@citrix.com>
> > > > 
> > > > The issues with non-GCC4.4 builds seem to have been resolved.
> > > 
> > > So in patch 0/4 you say that you have a private working tree
> > > (which makes me assume that xenbits.xen.org/ovmf.git isn't
> > > suitable), and the series doesn't seem to do anything towards
> > > updating xenbits.xen.org/ovmf.git. How is all this fitting
> > > together? Will everyone but you now end up with a broken
> > > build?
> > 
> > I took
> >         Working changeset that can be sticked in Config.mk is:
> >         8833370303d3bf3153760ee42760ef1b9b5c562
> > from the 0th email to mean that the committer could/should make this
> > change at the same time as committing (to be fair, I also discussed this
> > with Wei IRL before he sent the series, so I had some context).
> > 
> > I can cope with that, although I'd have preferred that to be an actual
> > patch in the series.
> > 
> 
> Sorry I misunderstood you. I can resend this series with change to
> Config.mk.

Unless other feedback has lead to other changes then appending a "5/4"
patch to this version is fine with me.

Re: [PATCH 0/4] Reintroduce OVMF support

[Ian Campbell]

On Wed, 2013-10-16 at 13:52 +0100, David Vrabel wrote:
> On 15/10/13 17:40, Wei Liu wrote:
> > This small series reintroduces OVMF support in Xen
> 
> I think somewhere in this series there needs to a brief description of
> what OVMF is.
> 
> With the exception of the first patch this is just adding yet another
> component to clone and build.

It's enabling an already existing component. The description you are
asking for was part of the original commit a few years ago. I suppose it
can't hurt to repeat it.

>   Why is this useful?  Why isn't
> (effectively) --with-system-ovmf always enabled (and thus not actually
> needed as an option)?
> 
> Isn't Xen's keenness to build it's own particular versions of stuff
> something that causes significant problems for distributions?

The lack of a --with-system style option does cause issues I think.

On the one hand building one automatically is convenient for direct
consumers of Xen and saves them having to learn about multiple project's
builds etc when all they want to do is install Xen from source (which
despite our recommendations to use distro packaging plenty of "end
users" to).

On the other hand I agree, Xen does pull in too much stuff at build
time.

We also have a legacy of actually pulling stuff into the tree (e.g.
libaio) which really would be better left to distros as build
dependencies, I've been trying to undo some of that sort of thing (and
fight against new instances) recently.

I guess it's a balancing act and for something like OVMF I think I would
come down on the side of including it but having the option to use some
existing binary.

FWIW qemu ships seabios and ovmf binaries and/or builds it at compile
time, for similar reasons I think.

Ian.

Re: [PATCH 4/4] tools: Enable OVMF build by default

[Wei Liu]

On Wed, Oct 16, 2013 at 01:53:39PM +0100, Ian Campbell wrote:
> On Wed, 2013-10-16 at 12:38 +0100, Wei Liu wrote:
> > On Wed, Oct 16, 2013 at 11:37:14AM +0100, Ian Campbell wrote:
> > > On Wed, 2013-10-16 at 11:02 +0100, Jan Beulich wrote:
> > > > >>> On 15.10.13 at 18:40, Wei Liu <wei.liu2@citrix.com> wrote:
> > > > > From: Ian Campbell <ian.campbell@citrix.com>
> > > > > 
> > > > > The issues with non-GCC4.4 builds seem to have been resolved.
> > > > 
> > > > So in patch 0/4 you say that you have a private working tree
> > > > (which makes me assume that xenbits.xen.org/ovmf.git isn't
> > > > suitable), and the series doesn't seem to do anything towards
> > > > updating xenbits.xen.org/ovmf.git. How is all this fitting
> > > > together? Will everyone but you now end up with a broken
> > > > build?
> > > 
> > > I took
> > >         Working changeset that can be sticked in Config.mk is:
> > >         8833370303d3bf3153760ee42760ef1b9b5c562
> > > from the 0th email to mean that the committer could/should make this
> > > change at the same time as committing (to be fair, I also discussed this
> > > with Wei IRL before he sent the series, so I had some context).
> > > 
> > > I can cope with that, although I'd have preferred that to be an actual
> > > patch in the series.
> > > 
> > 
> > Sorry I misunderstood you. I can resend this series with change to
> > Config.mk.
> 
> Unless other feedback has lead to other changes then appending a "5/4"
> patch to this version is fine with me.
> 

No, not really.

The only concerns I've got so far are:

* Jan on the change in Makefile
* David on the introductory email

I presume we can improve Makefile with a later patch. I will reply to
David directly. So there is no new code added.

Wei.

Re: [PATCH 0/4] Reintroduce OVMF support

[Wei Liu]

On Wed, Oct 16, 2013 at 01:52:43PM +0100, David Vrabel wrote:
> On 15/10/13 17:40, Wei Liu wrote:
> > This small series reintroduces OVMF support in Xen
> 
> I think somewhere in this series there needs to a brief description of
> what OVMF is.
> 

OVMF is a project to enable UEFI firmware for virtual machines. It's
part of the EDK2 kit. OVMF was first introduced to Xen about a year ago
but it was later disabled due to build process breakage.

Now that the build problem is resolved in upstream we would like to
re-enable it in Xen.

> With the exception of the first patch this is just adding yet another
> component to clone and build.  Why is this useful?  Why isn't
> (effectively) --with-system-ovmf always enabled (and thus not actually
> needed as an option)?
> 

Idealy we should always use the binary provided by distro, but
pratically their binaries are not always up to date / don't always work
with Xen.

> Isn't Xen's keenness to build it's own particular versions of stuff
> something that causes significant problems for distributions?
> 

We've been trying to work closely with upstream to fix bugs. But the
proecess for distros to pick up upstream bug fixes are different
stories which we have no control over.

That's why we provide several --with-system-FOO options in our build
system. The packager can choose in stock binaries if they want to. But
if the in stock binaries don't work they can use our trees.

Wei.

> David

Re: [PATCH 0/4] Reintroduce OVMF support

[Fabio Fantoni]

Il 16/10/2013 14:51, Wei Liu ha scritto:
> On Wed, Oct 16, 2013 at 12:03:19PM +0200, Fabio Fantoni wrote:
>> Il 15/10/2013 18:40, Wei Liu ha scritto:
>>> This small series reintroduces OVMF support in Xen
>>>
>>> You can fetch working OVMF tree on:
>>> git://xenbits.xen.org/people/liuw/ovmf.git master
>>>
>>> Working changeset that can be sticked in Config.mk is:
>>> 8833370303d3bf3153760ee42760ef1b9b5c562
>>>
>>> Note that VNC doesn't work properly when using OVMF, but that's not OVMF's
>>> problem. This issue should be addressed in Xen and I'm working on that.
>>>
>>> Wei.
>> Thanks for work on it, when I tested it months ago there was lot of
>> problems.
> Could you elaborate on the problems you're seeing? I'm aware of the VNC
> not refreshing problem but that's not to be fixed in OVMF.
>
>> I think it would be good thing to give an eye to the links below to
>> improve the hvm part:
>>
>> - Integrate legacy bios support inside ovmf (I saw a draft on link
>> above and I not know actual progress, there was also discussion
>> about it on seabios mailing list)
>> http://git.infradead.org/users/dwmw2/edk2.git
>> http://git.infradead.org/users/dwmw2/seabios.git
>>
> Hmm... I think this is orthogonal. We can get all those gravy new
> features when we refresh our own tree.
>
> Keep in mind that we're not forking EDK2. Actually we're trying to work
> closely with upstream - I fixed a upstream bug to make OVMF work with
> Xen before sending the series so in fact the tree I'm presenting is just
> vanilla upstream tree.
>
> The reason we have our own tree is that we need to have a central
> location where users can pull from. Also we would test our branch to
> avoid frustration.

Yes I don't want to fork ovmf, just check if there is already legacy 
bios support on upstream, otherwise keep an eye on it.
I think it is important to have both uefi and bios support in a single 
place in the future.

>
>> - Since this is a new/experimental section, it would be a better
>> path to move the actual 'static' data taken from hvmloader more
>> 'dynamics' or at least have theme generated from qemu.
>> This would lead in turn to a better support for all the new qemu
>> device/features and also to avoid eventual regressions with
>> ovmf/qemu newer versions.
> I don't see why OVMF would prevent you from using QEMU new device
> features, but I sort of get the idea of separating hvmloader and other
> firmwares. You could make an argument / request on xen-devel and ask
> George to keep track of it, then we will see what we can do.
>
> Wei.

Maybe I didn't explain myself well, I was talking about ACPI tables and 
all other static tables in hvmloader. I think it would be better to get 
these tables from OVMF/QEMU so that we don't need to update them in Xen 
every time something changes in new versions of OVMF/QEMU or particular 
devices (emulated or passthrough).
I don't know if what I have in mind is feasible and correct considering 
my lack of knowledge about it but if I understand correctly it would get 
rid of many problems with new versions of qemu or with optional devices 
affecting such part.

Re: [PATCH 0/4] Reintroduce OVMF support

[Wei Liu]

On Wed, Oct 16, 2013 at 04:13:37PM +0200, Fabio Fantoni wrote:
[...]
> >>- Integrate legacy bios support inside ovmf (I saw a draft on link
> >>above and I not know actual progress, there was also discussion
> >>about it on seabios mailing list)
> >>http://git.infradead.org/users/dwmw2/edk2.git
> >>http://git.infradead.org/users/dwmw2/seabios.git
> >>
> >Hmm... I think this is orthogonal. We can get all those gravy new
> >features when we refresh our own tree.
> >
> >Keep in mind that we're not forking EDK2. Actually we're trying to work
> >closely with upstream - I fixed a upstream bug to make OVMF work with
> >Xen before sending the series so in fact the tree I'm presenting is just
> >vanilla upstream tree.
> >
> >The reason we have our own tree is that we need to have a central
> >location where users can pull from. Also we would test our branch to
> >avoid frustration.
> 
> Yes I don't want to fork ovmf, just check if there is already legacy
> bios support on upstream, otherwise keep an eye on it.
> I think it is important to have both uefi and bios support in a
> single place in the future.
> 

What do you mean by "in a single place"? They are two types of firmwares
doing the same thing.

> >
> >>- Since this is a new/experimental section, it would be a better
> >>path to move the actual 'static' data taken from hvmloader more
> >>'dynamics' or at least have theme generated from qemu.
> >>This would lead in turn to a better support for all the new qemu
> >>device/features and also to avoid eventual regressions with
> >>ovmf/qemu newer versions.
> >I don't see why OVMF would prevent you from using QEMU new device
> >features, but I sort of get the idea of separating hvmloader and other
> >firmwares. You could make an argument / request on xen-devel and ask
> >George to keep track of it, then we will see what we can do.
> >
> >Wei.
> 
> Maybe I didn't explain myself well, I was talking about ACPI tables
> and all other static tables in hvmloader. I think it would be better
> to get these tables from OVMF/QEMU so that we don't need to update
> them in Xen every time something changes in new versions of
> OVMF/QEMU or particular devices (emulated or passthrough).
> I don't know if what I have in mind is feasible and correct
> considering my lack of knowledge about it but if I understand
> correctly it would get rid of many problems with new versions of
> qemu or with optional devices affecting such part.
> 

My shallow understanding is that, they (QEMU, hvmloader or any other
firmwares) need to agree upon things. The point is they need to reach
consesus, no matter which one is in charge. Even if QEMU / OVMF takes
charge we would still face the same problem?

On the other hand, it's critical for Xen to control those tables,
because obviously we trust the hypervisor more than external code. 

Wei.

Re: [PATCH 0/4] Reintroduce OVMF support

[Ian Campbell]

On Wed, 2013-10-16 at 16:00 +0100, Wei Liu wrote:
> On Wed, Oct 16, 2013 at 04:13:37PM +0200, Fabio Fantoni wrote:
> [...]
> > >>- Integrate legacy bios support inside ovmf (I saw a draft on link
> > >>above and I not know actual progress, there was also discussion
> > >>about it on seabios mailing list)
> > >>http://git.infradead.org/users/dwmw2/edk2.git
> > >>http://git.infradead.org/users/dwmw2/seabios.git
> > >>
> > >Hmm... I think this is orthogonal. We can get all those gravy new
> > >features when we refresh our own tree.
> > >
> > >Keep in mind that we're not forking EDK2. Actually we're trying to work
> > >closely with upstream - I fixed a upstream bug to make OVMF work with
> > >Xen before sending the series so in fact the tree I'm presenting is just
> > >vanilla upstream tree.
> > >
> > >The reason we have our own tree is that we need to have a central
> > >location where users can pull from. Also we would test our branch to
> > >avoid frustration.
> > 
> > Yes I don't want to fork ovmf, just check if there is already legacy
> > bios support on upstream, otherwise keep an eye on it.
> > I think it is important to have both uefi and bios support in a
> > single place in the future.
> > 
> 
> What do you mean by "in a single place"? They are two types of firmwares
> doing the same thing.
> 
> > >
> > >>- Since this is a new/experimental section, it would be a better
> > >>path to move the actual 'static' data taken from hvmloader more
> > >>'dynamics' or at least have theme generated from qemu.
> > >>This would lead in turn to a better support for all the new qemu
> > >>device/features and also to avoid eventual regressions with
> > >>ovmf/qemu newer versions.
> > >I don't see why OVMF would prevent you from using QEMU new device
> > >features, but I sort of get the idea of separating hvmloader and other
> > >firmwares. You could make an argument / request on xen-devel and ask
> > >George to keep track of it, then we will see what we can do.
> > >
> > >Wei.
> > 
> > Maybe I didn't explain myself well, I was talking about ACPI tables
> > and all other static tables in hvmloader. I think it would be better
> > to get these tables from OVMF/QEMU so that we don't need to update
> > them in Xen every time something changes in new versions of
> > OVMF/QEMU or particular devices (emulated or passthrough).
> > I don't know if what I have in mind is feasible and correct
> > considering my lack of knowledge about it but if I understand
> > correctly it would get rid of many problems with new versions of
> > qemu or with optional devices affecting such part.
> > 
> 
> My shallow understanding is that, they (QEMU, hvmloader or any other
> firmwares) need to agree upon things. The point is they need to reach
> consesus, no matter which one is in charge. Even if QEMU / OVMF takes
> charge we would still face the same problem?

QEMU / OVMF is a false choice. The alternative to OVMF is SeaBIOS.

Qemu is present in both configurations.

> On the other hand, it's critical for Xen to control those tables,
> because obviously we trust the hypervisor more than external code. 

It's not so much about trust as the fact that the ACPI tables are
supposed to describe the physical hardware, which is defined by Xen
(i.e. the interrupt routing is implemented by Xen and Xen tells qemu
what to provide). Therefore it is correct for Xen to provide the ACPI
tables (via hvmloader), in the same way that the vendor of your physical
computer provides ACPI tables for that hardware.

Ian.

Re: [PATCH 0/4] Reintroduce OVMF support

[Fabio Fantoni]

Il 16/10/2013 17:00, Wei Liu ha scritto:
> On Wed, Oct 16, 2013 at 04:13:37PM +0200, Fabio Fantoni wrote:
> [...]
>>>> - Integrate legacy bios support inside ovmf (I saw a draft on link
>>>> above and I not know actual progress, there was also discussion
>>>> about it on seabios mailing list)
>>>> http://git.infradead.org/users/dwmw2/edk2.git
>>>> http://git.infradead.org/users/dwmw2/seabios.git
>>>>
>>> Hmm... I think this is orthogonal. We can get all those gravy new
>>> features when we refresh our own tree.
>>>
>>> Keep in mind that we're not forking EDK2. Actually we're trying to work
>>> closely with upstream - I fixed a upstream bug to make OVMF work with
>>> Xen before sending the series so in fact the tree I'm presenting is just
>>> vanilla upstream tree.
>>>
>>> The reason we have our own tree is that we need to have a central
>>> location where users can pull from. Also we would test our branch to
>>> avoid frustration.
>> Yes I don't want to fork ovmf, just check if there is already legacy
>> bios support on upstream, otherwise keep an eye on it.
>> I think it is important to have both uefi and bios support in a
>> single place in the future.
>>
> What do you mean by "in a single place"? They are two types of firmwares
> doing the same thing.

I mean ovmf with seabios inside it.
This way we have a unique section xen side which would manage ovmf (with 
both uefi and bios support).
This could be a support to every o.s. including ones that not support uefi.
Take a look at these for example:
http://www.seabios.org/pipermail/seabios/2013-January/005344.html
http://www.seabios.org/pipermail/seabios/2013-February/005423.html
Now it seems already on upstream git.
Ian Campbell also seems to be in the thread about it and xen:
http://www.seabios.org/pipermail/seabios/2013-February/005431.html

>
>>>> - Since this is a new/experimental section, it would be a better
>>>> path to move the actual 'static' data taken from hvmloader more
>>>> 'dynamics' or at least have theme generated from qemu.
>>>> This would lead in turn to a better support for all the new qemu
>>>> device/features and also to avoid eventual regressions with
>>>> ovmf/qemu newer versions.
>>> I don't see why OVMF would prevent you from using QEMU new device
>>> features, but I sort of get the idea of separating hvmloader and other
>>> firmwares. You could make an argument / request on xen-devel and ask
>>> George to keep track of it, then we will see what we can do.
>>>
>>> Wei.
>> Maybe I didn't explain myself well, I was talking about ACPI tables
>> and all other static tables in hvmloader. I think it would be better
>> to get these tables from OVMF/QEMU so that we don't need to update
>> them in Xen every time something changes in new versions of
>> OVMF/QEMU or particular devices (emulated or passthrough).
>> I don't know if what I have in mind is feasible and correct
>> considering my lack of knowledge about it but if I understand
>> correctly it would get rid of many problems with new versions of
>> qemu or with optional devices affecting such part.
>>
> My shallow understanding is that, they (QEMU, hvmloader or any other
> firmwares) need to agree upon things. The point is they need to reach
> consesus, no matter which one is in charge. Even if QEMU / OVMF takes
> charge we would still face the same problem?
>
> On the other hand, it's critical for Xen to control those tables,
> because obviously we trust the hypervisor more than external code.
>
> Wei.

For the hvm domUs FWIK major part of devices are mainly managed by qemu.
I found this link that probably helps to clarify one of the actual 
problem and I think it is a smart idea to reconsider actual hvmloader 
section (even if only in some parts):
http://wiki.qemu.org/Features/ACPITableGeneration
> Once implemented, QEMU will be able to extend information passed to 
> Guest OS through ACPI tables without need for bios code changes. This 
> is widely desired as a way to avoid the churn and proliferation of 
> QEMU-specific interfaces associated with ACPI tables in bios code. 

Anyway if my idea is stupid and unuseful sorry for wasting your time.

Re: [PATCH 0/4] Reintroduce OVMF support

[Ian Campbell]

On Thu, 2013-10-17 at 11:13 +0200, Fabio Fantoni wrote:

> For the hvm domUs FWIK major part of devices are mainly managed by qemu.
> I found this link that probably helps to clarify one of the actual 
> problem and I think it is a smart idea to reconsider actual hvmloader 
> section (even if only in some parts):
> http://wiki.qemu.org/Features/ACPITableGeneration

What problem are you trying to solve here? Can you point to specific
problems with the existing setup?

The reason QEMU changed/are changing how they do things (as stated in
the link) is to avoid churn in SeaBIOS relating to qemu changes, we
don't have the problem since we don't generate our ACPI tables in
SeaBIOS anyway.

Ian.

Re: [PATCH 0/4] Reintroduce OVMF support

[Fabio Fantoni]

Il 16/10/2013 15:10, Wei Liu ha scritto:
> On Wed, Oct 16, 2013 at 01:52:43PM +0100, David Vrabel wrote:
>> On 15/10/13 17:40, Wei Liu wrote:
>>> This small series reintroduces OVMF support in Xen
>> I think somewhere in this series there needs to a brief description of
>> what OVMF is.
>>
> OVMF is a project to enable UEFI firmware for virtual machines. It's
> part of the EDK2 kit. OVMF was first introduced to Xen about a year ago
> but it was later disabled due to build process breakage.
>
> Now that the build problem is resolved in upstream we would like to
> re-enable it in Xen.

I will test ovmf shortly, there is a new patches version ready or near 
to be posted or I'll start to test with this version?
Thanks for any reply.

>
>> With the exception of the first patch this is just adding yet another
>> component to clone and build.  Why is this useful?  Why isn't
>> (effectively) --with-system-ovmf always enabled (and thus not actually
>> needed as an option)?
>>
> Idealy we should always use the binary provided by distro, but
> pratically their binaries are not always up to date / don't always work
> with Xen.
>
>> Isn't Xen's keenness to build it's own particular versions of stuff
>> something that causes significant problems for distributions?
>>
> We've been trying to work closely with upstream to fix bugs. But the
> proecess for distros to pick up upstream bug fixes are different
> stories which we have no control over.
>
> That's why we provide several --with-system-FOO options in our build
> system. The packager can choose in stock binaries if they want to. But
> if the in stock binaries don't work they can use our trees.
>
> Wei.
>
>> David
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH 0/4] Reintroduce OVMF support

[Wei Liu]

On Mon, Oct 28, 2013 at 11:53:39AM +0100, Fabio Fantoni wrote:
> Il 16/10/2013 15:10, Wei Liu ha scritto:
> >On Wed, Oct 16, 2013 at 01:52:43PM +0100, David Vrabel wrote:
> >>On 15/10/13 17:40, Wei Liu wrote:
> >>>This small series reintroduces OVMF support in Xen
> >>I think somewhere in this series there needs to a brief description of
> >>what OVMF is.
> >>
> >OVMF is a project to enable UEFI firmware for virtual machines. It's
> >part of the EDK2 kit. OVMF was first introduced to Xen about a year ago
> >but it was later disabled due to build process breakage.
> >
> >Now that the build problem is resolved in upstream we would like to
> >re-enable it in Xen.
> 
> I will test ovmf shortly, there is a new patches version ready or
> near to be posted or I'll start to test with this version?
> Thanks for any reply.
> 

This series has not been applied.

I think you can wait for this series to go through push gate before
conducting any test.

Wei.

> >
> >>With the exception of the first patch this is just adding yet another
> >>component to clone and build.  Why is this useful?  Why isn't
> >>(effectively) --with-system-ovmf always enabled (and thus not actually
> >>needed as an option)?
> >>
> >Idealy we should always use the binary provided by distro, but
> >pratically their binaries are not always up to date / don't always work
> >with Xen.
> >
> >>Isn't Xen's keenness to build it's own particular versions of stuff
> >>something that causes significant problems for distributions?
> >>
> >We've been trying to work closely with upstream to fix bugs. But the
> >proecess for distros to pick up upstream bug fixes are different
> >stories which we have no control over.
> >
> >That's why we provide several --with-system-FOO options in our build
> >system. The packager can choose in stock binaries if they want to. But
> >if the in stock binaries don't work they can use our trees.
> >
> >Wei.
> >
> >>David
> >_______________________________________________
> >Xen-devel mailing list
> >Xen-devel@lists.xen.org
> >http://lists.xen.org/xen-devel

Re: [PATCH 0/4] Reintroduce OVMF support

[Wei Liu]

I need to refresh the OVMF tree. I will resend the whole series after
that. Please drop this series.

Wei.

On Tue, Oct 15, 2013 at 05:40:01PM +0100, Wei Liu wrote:
> This small series reintroduces OVMF support in Xen
> 
> You can fetch working OVMF tree on:
> git://xenbits.xen.org/people/liuw/ovmf.git master
> 
> Working changeset that can be sticked in Config.mk is:
> 8833370303d3bf3153760ee42760ef1b9b5c562
> 
> Note that VNC doesn't work properly when using OVMF, but that's not OVMF's
> problem. This issue should be addressed in Xen and I'm working on that.
> 
> Wei.
> 
> Ian Campbell (3):
>   tools: clone ovmf to ovmf-dir directory
>   tools: support system supplied ovmf binary
>   tools: Enable OVMF build by default
> 
> Wei Liu (1):
>   libxl: bump LIBXL_MAXMEM_CONSTANT to 2048
> 
>  Makefile                          |    4 ++++
>  config/Tools.mk.in                |    1 +
>  tools/configure                   |   20 +++++++++++++++-----
>  tools/configure.ac                |   13 ++++++++++++-
>  tools/firmware/Makefile           |   31 ++++++++++++-------------------
>  tools/firmware/hvmloader/Makefile |    8 ++++++--
>  tools/libxl/libxl_internal.h      |    2 +-
>  7 files changed, 51 insertions(+), 28 deletions(-)
> 
> -- 
> 1.7.10.4

Re: [PATCH 1/4] libxl: bump LIBXL_MAXMEM_CONSTANT to 2048

[George Dunlap]

On Tue, Oct 15, 2013 at 5:40 PM, Wei Liu <wei.liu2@citrix.com> wrote:
> When using OVMF we need to have 1MiB of memory in place for firmware.
> Without this change we have:
>
> (XEN) HVM128: Loading OVMF ...
> (XEN) page_alloc.c:1460:d128 Over-allocation for domain 128: 33025 > 33024
> (XEN) memory.c:132:d128 Could not allocate order=0 extent: id=128 memflags=0 (0 of 1)
>
> This is not a fatal error as hvmloader will instead use low memory to
> load OVMF, but it's better to eliminate such error.

Wait -- hvmloader is actually allocating memory from Xen to load
firmware stuff like this?

That seems a bit weird... shouldn't it just use the memory it already
has, instead of asking for more?

 -George

Re: [PATCH 1/4] libxl: bump LIBXL_MAXMEM_CONSTANT to 2048

[Wei Liu]

On Tue, Oct 29, 2013 at 12:03:25PM +0000, George Dunlap wrote:
> On Tue, Oct 15, 2013 at 5:40 PM, Wei Liu <wei.liu2@citrix.com> wrote:
> > When using OVMF we need to have 1MiB of memory in place for firmware.
> > Without this change we have:
> >
> > (XEN) HVM128: Loading OVMF ...
> > (XEN) page_alloc.c:1460:d128 Over-allocation for domain 128: 33025 > 33024
> > (XEN) memory.c:132:d128 Could not allocate order=0 extent: id=128 memflags=0 (0 of 1)
> >
> > This is not a fatal error as hvmloader will instead use low memory to
> > load OVMF, but it's better to eliminate such error.
> 
> Wait -- hvmloader is actually allocating memory from Xen to load
> firmware stuff like this?
> 
> That seems a bit weird... shouldn't it just use the memory it already
> has, instead of asking for more?
> 

See hvmloader/util.c:mem_hole_populate_ram(). It's used in building ACPI
info area and allocating ram to load OVMF firmware volume.

Are you suggesting we change the behavior of hvmloader?

Wei.

>  -George

Re: [PATCH 1/4] libxl: bump LIBXL_MAXMEM_CONSTANT to 2048

[Wei Liu]

On Tue, Oct 29, 2013 at 12:15:13PM +0000, Wei Liu wrote:
> On Tue, Oct 29, 2013 at 12:03:25PM +0000, George Dunlap wrote:
> > On Tue, Oct 15, 2013 at 5:40 PM, Wei Liu <wei.liu2@citrix.com> wrote:
> > > When using OVMF we need to have 1MiB of memory in place for firmware.
> > > Without this change we have:
> > >
> > > (XEN) HVM128: Loading OVMF ...
> > > (XEN) page_alloc.c:1460:d128 Over-allocation for domain 128: 33025 > 33024
> > > (XEN) memory.c:132:d128 Could not allocate order=0 extent: id=128 memflags=0 (0 of 1)
> > >
> > > This is not a fatal error as hvmloader will instead use low memory to
> > > load OVMF, but it's better to eliminate such error.
> > 
> > Wait -- hvmloader is actually allocating memory from Xen to load
> > firmware stuff like this?
> > 
> > That seems a bit weird... shouldn't it just use the memory it already
> > has, instead of asking for more?
> > 
> 
> See hvmloader/util.c:mem_hole_populate_ram(). It's used in building ACPI
> info area and allocating ram to load OVMF firmware volume.
> 
> Are you suggesting we change the behavior of hvmloader?
> 

I think this boils down to the question whether firmware should be
considered extra overhead, like shadow ram.

Wei.

> Wei.
> 
> >  -George

Re: [PATCH 1/4] libxl: bump LIBXL_MAXMEM_CONSTANT to 2048

[George Dunlap]

On 10/29/2013 12:24 PM, Wei Liu wrote:
> On Tue, Oct 29, 2013 at 12:15:13PM +0000, Wei Liu wrote:
>> On Tue, Oct 29, 2013 at 12:03:25PM +0000, George Dunlap wrote:
>>> On Tue, Oct 15, 2013 at 5:40 PM, Wei Liu <wei.liu2@citrix.com> wrote:
>>>> When using OVMF we need to have 1MiB of memory in place for firmware.
>>>> Without this change we have:
>>>>
>>>> (XEN) HVM128: Loading OVMF ...
>>>> (XEN) page_alloc.c:1460:d128 Over-allocation for domain 128: 33025 > 33024
>>>> (XEN) memory.c:132:d128 Could not allocate order=0 extent: id=128 memflags=0 (0 of 1)
>>>>
>>>> This is not a fatal error as hvmloader will instead use low memory to
>>>> load OVMF, but it's better to eliminate such error.
>>>
>>> Wait -- hvmloader is actually allocating memory from Xen to load
>>> firmware stuff like this?
>>>
>>> That seems a bit weird... shouldn't it just use the memory it already
>>> has, instead of asking for more?
>>>
>>
>> See hvmloader/util.c:mem_hole_populate_ram(). It's used in building ACPI
>> info area and allocating ram to load OVMF firmware volume.
>>
>> Are you suggesting we change the behavior of hvmloader?
>>
>
> I think this boils down to the question whether firmware should be
> considered extra overhead, like shadow ram.

Hmm, I guess the firmware stays in memory even after the guest boots, 
doesn't it.

In any case, at the moment if that's how the firmware is treated, then 
it's probably best to just go along with it.  If we want to change that, 
we can consider doing that as a separate thing for the next release.

The name of this constant could use some change too -- it should be 
"LIBXL_MEMORY_SLACK" or something like that, perhaps with a comment to 
indicate what the slack is used for.  But that's not necessary for this 
patch series either, I don't think.

Acked-by: George Dunlap <george.dunlap@eu.citrix.com>

Re: [PATCH 1/4] libxl: bump LIBXL_MAXMEM_CONSTANT to 2048

[Wei Liu]

On Tue, Oct 29, 2013 at 12:29:02PM +0000, George Dunlap wrote:
> On 10/29/2013 12:24 PM, Wei Liu wrote:
> >On Tue, Oct 29, 2013 at 12:15:13PM +0000, Wei Liu wrote:
> >>On Tue, Oct 29, 2013 at 12:03:25PM +0000, George Dunlap wrote:
> >>>On Tue, Oct 15, 2013 at 5:40 PM, Wei Liu <wei.liu2@citrix.com> wrote:
> >>>>When using OVMF we need to have 1MiB of memory in place for firmware.
> >>>>Without this change we have:
> >>>>
> >>>>(XEN) HVM128: Loading OVMF ...
> >>>>(XEN) page_alloc.c:1460:d128 Over-allocation for domain 128: 33025 > 33024
> >>>>(XEN) memory.c:132:d128 Could not allocate order=0 extent: id=128 memflags=0 (0 of 1)
> >>>>
> >>>>This is not a fatal error as hvmloader will instead use low memory to
> >>>>load OVMF, but it's better to eliminate such error.
> >>>
> >>>Wait -- hvmloader is actually allocating memory from Xen to load
> >>>firmware stuff like this?
> >>>
> >>>That seems a bit weird... shouldn't it just use the memory it already
> >>>has, instead of asking for more?
> >>>
> >>
> >>See hvmloader/util.c:mem_hole_populate_ram(). It's used in building ACPI
> >>info area and allocating ram to load OVMF firmware volume.
> >>
> >>Are you suggesting we change the behavior of hvmloader?
> >>
> >
> >I think this boils down to the question whether firmware should be
> >considered extra overhead, like shadow ram.
> 
> Hmm, I guess the firmware stays in memory even after the guest
> boots, doesn't it.
> 
> In any case, at the moment if that's how the firmware is treated,
> then it's probably best to just go along with it.  If we want to
> change that, we can consider doing that as a separate thing for the
> next release.
> 
> The name of this constant could use some change too -- it should be
> "LIBXL_MEMORY_SLACK" or something like that, perhaps with a comment
> to indicate what the slack is used for.  But that's not necessary
> for this patch series either, I don't think.
> 
> Acked-by: George Dunlap <george.dunlap@eu.citrix.com>

Thanks, could you also ack V3 patch (the latest one) -- this is the first
version.

Wei.