[Qemu-devel] [RfC PATCH 0/3] edk2: add efi firmware builds

[Qemu-devel] [RfC PATCH 0/3] edk2: add efi firmware builds

[Gerd Hoffmann]

  Hi,

This patch series adds a edk2 submodule in roms, also a build script
and makefile rules to build ovmf and arm firmware.

The build script creates per-arch subdirs in pc-bios and places the
firmware files there.  They are not exactly small:

   kraxel@nilsson ~/projects/qemu (work/edk2)# du -s -h pc-bios/edk2-*
   2,8M    pc-bios/edk2-aarch64
   2,8M    pc-bios/edk2-arm
   2,1M    pc-bios/edk2-i386
   2,1M    pc-bios/edk2-x86_64

So, the question is how we wanna go forward with this.  Adding the
submodule and rules is a no-brainer I think.  But what about the
binaries, which sum up to roughly 10M?  Ship them all?  Ship the
64bit archs only, given that efi never really took off on i386
and arm?

Other comments?

cheers,
  Gerd

Gerd Hoffmann (3):
  edk2: add submodule
  edk2: add build script and rules
  edk2: use EfiRom utility from edk2 submodule

 .gitmodules          |  3 +++
 roms/Makefile        | 50 ++++++++++++++++++++++++++++++--------------
 roms/compile-edk2.sh | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 roms/edk2            |  1 +
 4 files changed, 97 insertions(+), 16 deletions(-)
 create mode 100755 roms/compile-edk2.sh
 create mode 160000 roms/edk2

-- 
1.8.3.1

[Qemu-devel] [RfC PATCH 1/3] edk2: add submodule

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 .gitmodules | 3 +++
 roms/edk2   | 1 +
 2 files changed, 4 insertions(+)
 create mode 160000 roms/edk2

diff --git a/.gitmodules b/.gitmodules
index ca323b4..3d4c85e 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -34,3 +34,6 @@
 [submodule "roms/skiboot"]
 	path = roms/skiboot
 	url = git://git.qemu.org/skiboot.git
+[submodule "roms/edk2"]
+	path = roms/edk2
+	url = https://github.com/tianocore/edk2
diff --git a/roms/edk2 b/roms/edk2
new file mode 160000
index 0000000..36e9e3e
--- /dev/null
+++ b/roms/edk2
@@ -0,0 +1 @@
+Subproject commit 36e9e3e8ea0ce477504b6d2e21603ea94847efae
-- 
1.8.3.1

[Qemu-devel] [RfC PATCH 2/3] edk2: add build script and rules

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 roms/Makefile        | 28 ++++++++++++++++++++++++-
 roms/compile-edk2.sh | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 86 insertions(+), 1 deletion(-)
 create mode 100755 roms/compile-edk2.sh

diff --git a/roms/Makefile b/roms/Makefile
index b5e5a69..be20fe4 100644
--- a/roms/Makefile
+++ b/roms/Makefile
@@ -37,6 +37,8 @@ find-cross-prefix = $(subst gcc,,$(notdir $(call find-cross-gcc,$(1))))
 powerpc64_cross_prefix := $(call find-cross-prefix,powerpc64)
 powerpc_cross_prefix := $(call find-cross-prefix,powerpc)
 x86_64_cross_prefix := $(call find-cross-prefix,x86_64)
+arm_cross_prefix := $(call find-cross-prefix,arm)
+aarch64_cross_prefix := $(call find-cross-prefix,aarch64)
 
 # tag our seabios builds
 SEABIOS_EXTRAVERSION="-prebuilt.qemu-project.org"
@@ -151,7 +153,31 @@ skiboot:
 	$(MAKE) -C skiboot CROSS=$(powerpc64_cross_prefix)
 	cp skiboot/skiboot.lid ../pc-bios/skiboot.lid
 
-clean:
+
+edk2: edk2-ovmf32 edk2-ovmf64 edk2-arm edk2-aarch64
+
+edk2-ovmf32:
+	sh ./compile-edk2.sh i386 $(x86_64_cross_prefix)
+
+edk2-ovmf64:
+	sh ./compile-edk2.sh x86_64 $(x86_64_cross_prefix)
+
+edk2-arm:
+	sh ./compile-edk2.sh arm $(arm_cross_prefix)
+
+edk2-aarch64:
+	sh ./compile-edk2.sh aarch64 $(aarch64_cross_prefix)
+
+edk2-clean:
+	make -C edk2/BaseTools clean
+	rm -rf edk2/Build
+	rm -rf edk2/Conf/BuildEnv.sh
+	rm -rf edk2/Conf/build_rule.txt
+	rm -rf edk2/Conf/target.txt
+	rm -rf edk2/Conf/tools_def.txt
+
+
+clean: edk2-clean
 	rm -rf seabios/.config seabios/out seabios/builds
 	$(MAKE) -C vgabios clean
 	rm -f vgabios/VGABIOS-lgpl-latest*
diff --git a/roms/compile-edk2.sh b/roms/compile-edk2.sh
new file mode 100755
index 0000000..1339028
--- /dev/null
+++ b/roms/compile-edk2.sh
@@ -0,0 +1,59 @@
+#!/bin/sh
+
+# args
+arch="$1"
+cross="$2"
+
+# figure gcc version, set cross build prefix vars
+gccver=$(${cross}gcc --version \
+        | awk '{ print $3; exit}' \
+        | cut -d. -f1,2 \
+        | tr -d ".")
+case "$gccver" in
+4*)     # nothing, keep "4x"
+        ;;
+5* | 6*)
+        gccver=5
+        ;;
+esac
+toolchain="GCC$gccver"
+eval "export GCC${gccver}_X64_PREFIX=${cross}"
+eval "export GCC${gccver}_ARM_PREFIX=${cross}"
+eval "export GCC${gccver}_AARCH64_PREFIX=${cross}"
+
+# what we are going to build?
+case "$arch" in
+i386)
+        barch="IA32"
+        project="OvmfPkg/OvmfPkgIa32.dsc"
+        match="OvmfIa32/*/FV/OVMF_*.fd"
+        ;;
+x86_64)
+        barch="X64"
+        project="OvmfPkg/OvmfPkgX64.dsc"
+        match="OvmfX64/*/FV/OVMF_*.fd"
+        ;;
+arm | aarch64)
+        barch="$(echo $arch | tr 'a-z' 'A-Z')"
+        project="ArmVirtPkg/ArmVirtQemu.dsc"
+        match="ArmVirtQemu-${barch}/*/FV/QEMU_*.fd"
+        ;;
+esac
+
+# setup edk2 build environment
+cd edk2
+source ./edksetup.sh --reconfig
+bopts=""
+bopts="$bopts -t $toolchain"
+bopts="$bopts -D HTTP_BOOT_ENABLE"
+bopts="$bopts -a $barch"
+bopts="$bopts -p $project"
+
+# go build everything
+make -C BaseTools || exit 1
+build $bopts      || exit 1
+
+# copy over results
+dest="../../pc-bios/edk2-${arch}"
+mkdir -p "$dest"
+cp -v Build/${match} "$dest"
-- 
1.8.3.1

[Qemu-devel] [RfC PATCH 3/3] edk2: use EfiRom utility from edk2 submodule

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 roms/Makefile | 22 +++++++---------------
 1 file changed, 7 insertions(+), 15 deletions(-)

diff --git a/roms/Makefile b/roms/Makefile
index be20fe4..569b450 100644
--- a/roms/Makefile
+++ b/roms/Makefile
@@ -43,17 +43,6 @@ aarch64_cross_prefix := $(call find-cross-prefix,aarch64)
 # tag our seabios builds
 SEABIOS_EXTRAVERSION="-prebuilt.qemu-project.org"
 
-#
-# EfiRom utility is shipped with edk2 / tianocore, in BaseTools/
-#
-# We need that to combine multiple images (legacy bios,
-# efi ia32, efi x64) into a single rom binary.
-#
-# We try to find it in the path.  You can also pass the location on
-# the command line, i.e. "make EFIROM=/path/to/EfiRom efirom"
-#
-EFIROM ?= $(shell which EfiRom 2>/dev/null)
-
 default:
 	@echo "nothing is build by default"
 	@echo "available build targets:"
@@ -62,8 +51,7 @@ default:
 	@echo "  lgplvgabios    -- update vgabios binaries (lgpl)"
 	@echo "  sgabios        -- update sgabios binaries"
 	@echo "  pxerom         -- update nic roms (bios only)"
-	@echo "  efirom         -- update nic roms (bios+efi, this needs"
-	@echo "                    the EfiRom utility from edk2 / tianocore)"
+	@echo "  efirom         -- update nic roms (bios+efi)"
 	@echo "  slof           -- update slof.bin"
 	@echo "  skiboot        -- update skiboot.lid"
 	@echo "  u-boot.e500    -- update u-boot.e500"
@@ -119,8 +107,9 @@ pxe-rom-%: build-pxe-roms
 
 efirom: $(patsubst %,efi-rom-%,$(pxerom_variants))
 
-efi-rom-%: build-pxe-roms build-efi-roms
-	$(EFIROM) -f "0x$(VID)" -i "0x$(DID)" -l 0x02 \
+efi-rom-%: build-pxe-roms build-efi-roms edk2-tools
+	edk2/BaseTools/Source/C/bin/EfiRom \
+		-f "0x$(VID)" -i "0x$(DID)" -l 0x02 \
 		-b ipxe/src/bin/$(VID)$(DID).rom \
 		-ec ipxe/src/bin-i386-efi/$(VID)$(DID).efidrv \
 		-ec ipxe/src/bin-x86_64-efi/$(VID)$(DID).efidrv \
@@ -168,6 +157,9 @@ edk2-arm:
 edk2-aarch64:
 	sh ./compile-edk2.sh aarch64 $(aarch64_cross_prefix)
 
+edk2-tools:
+	make -C edk2/BaseTools
+
 edk2-clean:
 	make -C edk2/BaseTools clean
 	rm -rf edk2/Build
-- 
1.8.3.1

Re: [Qemu-devel] [RfC PATCH 0/3] edk2: add efi firmware builds

[Daniel P. Berrange]

On Wed, Nov 23, 2016 at 02:24:41PM +0100, Gerd Hoffmann wrote:
>   Hi,
> 
> This patch series adds a edk2 submodule in roms, also a build script
> and makefile rules to build ovmf and arm firmware.
> 
> The build script creates per-arch subdirs in pc-bios and places the
> firmware files there.  They are not exactly small:
> 
>    kraxel@nilsson ~/projects/qemu (work/edk2)# du -s -h pc-bios/edk2-*
>    2,8M    pc-bios/edk2-aarch64
>    2,8M    pc-bios/edk2-arm
>    2,1M    pc-bios/edk2-i386
>    2,1M    pc-bios/edk2-x86_64
> 
> So, the question is how we wanna go forward with this.  Adding the
> submodule and rules is a no-brainer I think.  But what about the
> binaries, which sum up to roughly 10M?  Ship them all?  Ship the
> 64bit archs only, given that efi never really took off on i386
> and arm?
> 
> Other comments?

Should we think about our policy for distributing & shipping ROMS
more generally ?  Most distros will actively strip out the ROMs that
we ship in the QEMU tar.gz releases, and rebuild them from pristine
source in order to ensure they're fully complying with licensing
requirements wrt "full & corresponding source".

So should we consider actually shipping 2 tar.gz files for QEMU
releases. One minimal qemu-x.y.z.tar.gz that only contains pristine
QEMU source with no pre-built artifacts, and a second qemu-full-x.y.z.tar.gz
that contains the QEMU source, plus an arbitrary number of pre-built
blobs.

Alternatively, instead of a qemu-full-x.y.z.tar.gz, we could just
have a qemu-roms-x.y.z.tar.gz bundle which only contained the ROMs,
which users would use in combination with the other releae.

Vendors would use the minimal release, while users who just want to
build their own QEMU with least effort would  use the full release
tar.gz

If we had some split like this, then I think we would not have to
really worry about the fact that the ROMs will get quite large
as the size would only impact people wanting to download the full
release.

In terms of GIT, we could likewise make the binary ROMS live in
a submodule, so we don't bloat the main GIT repo, but I don't
think that's so critical

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://entangle-photo.org       -o-    http://search.cpan.org/~danberr/ :|

Re: [Qemu-devel] [RfC PATCH 0/3] edk2: add efi firmware builds

[Gerd Hoffmann]

  Hi,

> Should we think about our policy for distributing & shipping ROMS
> more generally ?  Most distros will actively strip out the ROMs that
> we ship in the QEMU tar.gz releases, and rebuild them from pristine
> source in order to ensure they're fully complying with licensing
> requirements wrt "full & corresponding source".

Or they don't even use the bundled rom sources and build from upstream
tarball instead.

> So should we consider actually shipping 2 tar.gz files for QEMU
> releases. One minimal qemu-x.y.z.tar.gz that only contains pristine
> QEMU source with no pre-built artifacts, and a second qemu-full-x.y.z.tar.gz
> that contains the QEMU source, plus an arbitrary number of pre-built
> blobs.

Makes sense.

> In terms of GIT, we could likewise make the binary ROMS live in
> a submodule, so we don't bloat the main GIT repo, but I don't
> think that's so critical

Adding some submodule (roms/prebuilt?) would also make the management
easier.

Basically the full tarball would be qemu plus submodules, and the
minimal tarball would be qemu without submodules.  You can simply use
"git archive" to create the later, which is a bit problematic today as
the tarball will include blobs without sources.

cheers,
  Gerd