[Qemu-devel] [RFC 0/3] qboot changes for PVH boot

[Qemu-devel] [RFC 0/3] qboot changes for PVH boot

[Liam Merwick]

For certain applications it is desirable to rapidly boot a KVM virtual
machine. In cases where legacy hardware and software support within the
guest is not needed, QEMU should be able to boot directly into the
uncompressed Linux kernel binary with minimal firmware involvement.

There already exists an ABI to allow this for Xen PVH guests and the ABI
is supported by Linux and FreeBSD:

   https://xenbits.xen.org/docs/unstable/misc/pvh.html

Details on the Linux changes: https://lkml.org/lkml/2018/4/16/1002

This patch series provides qboot support to populate the start_info struct
needed by the direct boot ABI and to configure the guest e820 tables to
enable QEMU to use that same entry point for booting KVM guests.

Usіng the methods/scripts documented by the NEMU team at

   https://github.com/intel/nemu/wiki/Measuring-Boot-Latency
   https://lists.gnu.org/archive/html/qemu-devel/2018-12/msg00200.html

below are some timings measured (vmlinux and bzImage from the same build).
Time to get to kernel start is almost halved (95ṁs -> 48ms)

QEMU + qboot + vmlinux (PVH + 4.20-rc4)
 qemu_init_end: 41.550521
 fw_start: 41.667139 (+0.116618)
 fw_do_boot: 47.448495 (+5.781356)
 linux_startup_64: 47.720785 (+0.27229)
 linux_start_kernel: 48.399541 (+0.678756)
 linux_start_user: 296.952056 (+248.552515)

QEMU + qboot + bzImage:
 qemu_init_end: 29.209276
 fw_start: 29.317342 (+0.108066)
 linux_start_boot: 36.679362 (+7.36202)
 linux_startup_64: 94.531349 (+57.851987)
 linux_start_kernel: 94.900913 (+0.369564)
 linux_start_user: 401.060971 (+306.160058)

QEMU + bzImage:
 qemu_init_end: 30.424430
 linux_startup_64: 893.770334 (+863.345904)
 linux_start_kernel: 894.17049 (+0.400156)
 linux_start_user: 1208.679768 (+314.509278)


Liam Merwick (3):
  pvh: Add x86/HVM direct boot ABI header file
  pvh: use x86/HVM direct boot ABI
  pvh: add benchmark exit point

 benchmark.h          |   3 +-
 fw_cfg.c             |  79 +++++++++++++++++++++++++++-
 include/start_info.h | 146 +++++++++++++++++++++++++++++++++++++++++++++++++++
 linuxboot.c          |   2 +-
 main.c               |   3 ++
 tables.c             |   9 ++++
 6 files changed, 239 insertions(+), 3 deletions(-)
 create mode 100644 include/start_info.h

-- 
1.8.3.1

[Qemu-devel] [RFC qboot 1/3] pvh: Add x86/HVM direct boot ABI header file

[Liam Merwick]

The x86/HVM direct boot ABI permits a guest to be able to boot directly
into the uncompressed Linux kernel binary.

https://xenbits.xen.org/docs/unstable/misc/pvh.html

This commit adds the header file that defines the start_info struct
that needs to be populated in order to use this ABI.

Signed-off-by: Maran Wilson <Maran.Wilson@oracle.com>
Signed-off-by: Liam Merwick <liam.merwick@oracle.com>
---
 include/start_info.h | 146 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 146 insertions(+)
 create mode 100644 include/start_info.h

diff --git a/include/start_info.h b/include/start_info.h
new file mode 100644
index 000000000000..348779eb10cd
--- /dev/null
+++ b/include/start_info.h
@@ -0,0 +1,146 @@
+/*
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * Copyright (c) 2016, Citrix Systems, Inc.
+ */
+
+#ifndef __XEN_PUBLIC_ARCH_X86_HVM_START_INFO_H__
+#define __XEN_PUBLIC_ARCH_X86_HVM_START_INFO_H__
+
+/*
+ * Start of day structure passed to PVH guests and to HVM guests in %ebx.
+ *
+ * NOTE: nothing will be loaded at physical address 0, so a 0 value in any
+ * of the address fields should be treated as not present.
+ *
+ *  0 +----------------+
+ *    | magic          | Contains the magic value XEN_HVM_START_MAGIC_VALUE
+ *    |                | ("xEn3" with the 0x80 bit of the "E" set).
+ *  4 +----------------+
+ *    | version        | Version of this structure. Current version is 1. New
+ *    |                | versions are guaranteed to be backwards-compatible.
+ *  8 +----------------+
+ *    | flags          | SIF_xxx flags.
+ * 12 +----------------+
+ *    | nr_modules     | Number of modules passed to the kernel.
+ * 16 +----------------+
+ *    | modlist_paddr  | Physical address of an array of modules
+ *    |                | (layout of the structure below).
+ * 24 +----------------+
+ *    | cmdline_paddr  | Physical address of the command line,
+ *    |                | a zero-terminated ASCII string.
+ * 32 +----------------+
+ *    | rsdp_paddr     | Physical address of the RSDP ACPI data structure.
+ * 40 +----------------+
+ *    | memmap_paddr   | Physical address of the (optional) memory map. Only
+ *    |                | present in version 1 and newer of the structure.
+ * 48 +----------------+
+ *    | memmap_entries | Number of entries in the memory map table. Only
+ *    |                | present in version 1 and newer of the structure.
+ *    |                | Zero if there is no memory map being provided.
+ * 52 +----------------+
+ *    | reserved       | Version 1 and newer only.
+ * 56 +----------------+
+ *
+ * The layout of each entry in the module structure is the following:
+ *
+ *  0 +----------------+
+ *    | paddr          | Physical address of the module.
+ *  8 +----------------+
+ *    | size           | Size of the module in bytes.
+ * 16 +----------------+
+ *    | cmdline_paddr  | Physical address of the command line,
+ *    |                | a zero-terminated ASCII string.
+ * 24 +----------------+
+ *    | reserved       |
+ * 32 +----------------+
+ *
+ * The layout of each entry in the memory map table is as follows:
+ *
+ *  0 +----------------+
+ *    | addr           | Base address
+ *  8 +----------------+
+ *    | size           | Size of mapping in bytes
+ * 16 +----------------+
+ *    | type           | Type of mapping as defined between the hypervisor
+ *    |                | and guest it's starting. E820_TYPE_xxx, for example.
+ * 20 +----------------|
+ *    | reserved       |
+ * 24 +----------------+
+ *
+ * The address and sizes are always a 64bit little endian unsigned integer.
+ *
+ * NB: Xen on x86 will always try to place all the data below the 4GiB
+ * boundary.
+ *
+ * Version numbers of the hvm_start_info structure have evolved like this:
+ *
+ * Version 0:
+ *
+ * Version 1:   Added the memmap_paddr/memmap_entries fields (plus 4 bytes of
+ *              padding) to the end of the hvm_start_info struct. These new
+ *              fields can be used to pass a memory map to the guest. The
+ *              memory map is optional and so guests that understand version 1
+ *              of the structure must check that memmap_entries is non-zero
+ *              before trying to read the memory map.
+ */
+#define XEN_HVM_START_MAGIC_VALUE 0x336ec578
+
+/*
+ * C representation of the x86/HVM start info layout.
+ *
+ * The canonical definition of this layout is above, this is just a way to
+ * represent the layout described there using C types.
+ */
+struct hvm_start_info {
+    uint32_t magic;             /* Contains the magic value 0x336ec578       */
+                                /* ("xEn3" with the 0x80 bit of the "E" set).*/
+    uint32_t version;           /* Version of this structure.                */
+    uint32_t flags;             /* SIF_xxx flags.                            */
+    uint32_t nr_modules;        /* Number of modules passed to the kernel.   */
+    uint64_t modlist_paddr;     /* Physical address of an array of           */
+                                /* hvm_modlist_entry.                        */
+    uint64_t cmdline_paddr;     /* Physical address of the command line.     */
+    uint64_t rsdp_paddr;        /* Physical address of the RSDP ACPI data    */
+                                /* structure.                                */
+    uint64_t memmap_paddr;      /* Physical address of an array of           */
+                                /* hvm_memmap_table_entry. Only present in   */
+                                /* version 1 and newer of the structure      */
+    uint32_t memmap_entries;    /* Number of entries in the memmap table.    */
+                                /* Only present in version 1 and newer of    */
+                                /* the structure. Value will be zero if      */
+                                /* there is no memory map being provided.    */
+    uint32_t reserved;
+};
+
+struct hvm_modlist_entry {
+    uint64_t paddr;             /* Physical address of the module.           */
+    uint64_t size;              /* Size of the module in bytes.              */
+    uint64_t cmdline_paddr;     /* Physical address of the command line.     */
+    uint64_t reserved;
+};
+
+struct hvm_memmap_table_entry {
+    uint64_t addr;              /* Base address of the memory region         */
+    uint64_t size;              /* Size of the memory region in bytes        */
+    uint32_t type;              /* Mapping type                              */
+    uint32_t reserved;
+};
+
+#endif /* __XEN_PUBLIC_ARCH_X86_HVM_START_INFO_H__ */
-- 
1.8.3.1

[Qemu-devel] [RFC qboot 2/3] pvh: use x86/HVM direct boot ABI

[Liam Merwick]

These changes (along with corresponding QEMU and Linux kernel changes)
enable a guest to be booted using the x86/HVM direct boot ABI.

QEMU parses the uncompressed kernel binary passed to it via -kernel
to read the ELF Note which contains the address to be loaded.  QEMU
then depends on qboot to populate the start_info struct needed by
the direct boot ABI and configure the guest e820 tables before
jumping to the loaded kernel entry.

Signed-off-by: George Kennedy <George.Kennedy@oracle.com>
Signed-off-by: Liam Merwick <Liam.Merwick@oracle.com>
---
 fw_cfg.c    | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 linuxboot.c |  2 +-
 main.c      |  3 +++
 tables.c    |  9 ++++++++
 4 files changed, 84 insertions(+), 2 deletions(-)

diff --git a/fw_cfg.c b/fw_cfg.c
index f5aac739b921..e13ec20d0e8b 100644
--- a/fw_cfg.c
+++ b/fw_cfg.c
@@ -8,6 +8,10 @@
 #include "linuxboot.h"
 #include "multiboot.h"
 #include "benchmark.h"
+#include "start_info.h"
+
+extern struct hvm_start_info start_info;
+extern inline uint32_t ldl_p(void *p);
 
 struct fw_cfg_file {
 	uint32_t size;
@@ -184,6 +188,67 @@ static void boot_multiboot_from_fw_cfg(void)
 	panic();
 }
 
+static void pvh_e820_setup()
+{
+	struct hvm_memmap_table_entry *pvh_e820p;
+	int i, pvh_e820_sz;
+
+	pvh_e820_sz = sizeof(struct hvm_memmap_table_entry) * e820->nr_map;
+
+	pvh_e820p = malloc(pvh_e820_sz);
+	memset(pvh_e820p, 0, pvh_e820_sz);
+
+	for (i = 0; i < e820->nr_map; i++) {
+		pvh_e820p[i].addr = e820->map[i].addr;
+		pvh_e820p[i].size = e820->map[i].size;
+		pvh_e820p[i].type = e820->map[i].type;
+	}
+	start_info.memmap_paddr = (uintptr_t)pvh_e820p;
+	start_info.memmap_entries = e820->nr_map;
+}
+
+void boot_pvh_from_fw_cfg(void)
+{
+	void *kernel_entry;
+	uint32_t sz;
+	struct linuxboot_args args;
+	struct hvm_modlist_entry ramdisk_mod;
+
+	start_info.magic = XEN_HVM_START_MAGIC_VALUE;
+	start_info.version = 1;
+	start_info.flags = 0;
+	start_info.nr_modules = 1;
+	start_info.reserved = 0;
+
+	fw_cfg_select(FW_CFG_CMDLINE_SIZE);
+	args.cmdline_size = fw_cfg_readl_le();
+	args.cmdline_addr = malloc(args.cmdline_size);
+	fw_cfg_read_entry(FW_CFG_CMDLINE_DATA, args.cmdline_addr,
+			  args.cmdline_size);
+	start_info.cmdline_paddr = (uintptr_t)args.cmdline_addr;
+
+	/* Use this field for pvhboot. Not used by pvhboot otherwise */
+	fw_cfg_read_entry(FW_CFG_KERNEL_DATA, &ramdisk_mod,
+			  sizeof(ramdisk_mod));
+	ramdisk_mod.cmdline_paddr = (uintptr_t)&ramdisk_mod;
+	start_info.modlist_paddr = (uintptr_t)&ramdisk_mod;
+
+	pvh_e820_setup();
+
+	fw_cfg_select(FW_CFG_KERNEL_SIZE);
+	sz = fw_cfg_readl_le();
+	if (!sz)
+		panic();
+
+	fw_cfg_select(FW_CFG_KERNEL_ENTRY);
+	kernel_entry = (void *) fw_cfg_readl_le();
+	asm volatile("movl %0, %%ebx" : : "r"(&start_info));
+
+	asm volatile("jmp *%2" : : "a" (0x2badb002),
+		     "b"(&start_info), "c"(kernel_entry));
+	panic();
+}
+
 void boot_from_fwcfg(void)
 {
 	struct linuxboot_args args;
@@ -208,8 +273,13 @@ void boot_from_fwcfg(void)
 	fw_cfg_select(FW_CFG_SETUP_DATA);
 	fw_cfg_read(args.header, sizeof(args.header));
 
-	if (!parse_bzimage(&args))
+	if (!parse_bzimage(&args)) {
+		uint8_t *header = args.header;
+
+		if (ldl_p(header) == 0x464c457f)  /* ELF magic */
+			boot_pvh_from_fw_cfg();
 		boot_multiboot_from_fw_cfg();
+	}
 
 	/* SETUP_DATA already selected */
 	if (args.setup_size > sizeof(args.header))
diff --git a/linuxboot.c b/linuxboot.c
index a5f1c4fa078d..573052cc0f78 100644
--- a/linuxboot.c
+++ b/linuxboot.c
@@ -12,7 +12,7 @@ static inline uint16_t lduw_p(void *p)
 	return val;
 }
 
-static inline uint32_t ldl_p(void *p)
+inline uint32_t ldl_p(void *p)
 {
 	uint32_t val;
 	memcpy(&val, p, 4);
diff --git a/main.c b/main.c
index 699cc1a8e6e7..725d50b94e1e 100644
--- a/main.c
+++ b/main.c
@@ -8,6 +8,9 @@
 #include "pflash.h"
 #include "pci.h"
 #include "benchmark.h"
+#include "start_info.h"
+
+struct hvm_start_info start_info = {0};
 
 static void set_realmode_int(int vec, void *p)
 {
diff --git a/tables.c b/tables.c
index 32b2406b9aed..47bef4125d9e 100644
--- a/tables.c
+++ b/tables.c
@@ -2,6 +2,9 @@
 #include "stdio.h"
 #include "fw_cfg.h"
 #include "string.h"
+#include "start_info.h"
+
+extern struct hvm_start_info start_info;
 
 struct loader_cmd {
 	uint32_t cmd;
@@ -122,6 +125,8 @@ static void do_checksum(char *file, uint32_t offset, uint32_t start, uint32_t le
 	p[offset] -= csum8(&p[start], len);
 }
 
+#define RSDP_FILE "etc/acpi/rsdp"
+
 void extract_acpi(void)
 {
 	int id = fw_cfg_file_id("etc/table-loader");
@@ -138,6 +143,10 @@ void extract_acpi(void)
 		struct loader_cmd *s = &script[i];
 		switch(script[i].cmd) {
 		case CMD_ALLOC:
+			if (strcmp(s->alloc.file, RSDP_FILE) == 0) {
+				start_info.rsdp_paddr =
+				    (uintptr_t)id_to_addr(id);
+			}
 			do_alloc(s->alloc.file, s->alloc.align, s->alloc.zone);
 			break;
 		case CMD_PTR:
-- 
1.8.3.1

[Qemu-devel] [RFC qboot 3/3] pvh: add benchmark exit point

[Liam Merwick]

This commit adds a PVH specific VM exit point for use in benchmarking
boot times using a QEMU specific device that terminates the QEMU process
and thus the VM itself when handling those VM exits. Since the VM
terminates right at those exit points, generic tools like time can
be used to measure the time spent between the QEMU startup
and termination moments.

The QEMU device used for those measurement is called isa-debug-exit
for the PC and Q35 machine types. These devices take 2 arguments:
iobase and iosize.  iobase specifies which IO port we need to write
into to have these devices eventually handle the corresponding VM exit.

If for example, QEMU is started with the following argument:

-device isa-debug-exit,iobase=0xf4

then any IO write to 0xf4 will terminate the QEMU process and the
corresponding VM.

Signed-off-by: Liam Merwick <liam.merwick@oracle.com>
---
 benchmark.h | 3 ++-
 fw_cfg.c    | 7 +++++++
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/benchmark.h b/benchmark.h
index 089c549b5803..2be08e7b2cec 100644
--- a/benchmark.h
+++ b/benchmark.h
@@ -9,5 +9,6 @@
 #define FW_START    1
 #define LINUX_START_FWCFG 2
 #define LINUX_START_BOOT  3
+#define LINUX_START_PVHBOOT  4
 
-#endif
+#endif /* BENCHMARK_H */
diff --git a/fw_cfg.c b/fw_cfg.c
index e13ec20d0e8b..690ff19e74a0 100644
--- a/fw_cfg.c
+++ b/fw_cfg.c
@@ -240,10 +240,17 @@ void boot_pvh_from_fw_cfg(void)
 	if (!sz)
 		panic();
 
+
 	fw_cfg_select(FW_CFG_KERNEL_ENTRY);
 	kernel_entry = (void *) fw_cfg_readl_le();
 	asm volatile("movl %0, %%ebx" : : "r"(&start_info));
 
+#ifdef BENCHMARK_HACK
+	/* Exit just before jumping to vmlinux, so that it is easy
+	 * to time/profile the firmware.
+	 */
+	outb(LINUX_EXIT_PORT, LINUX_START_PVHBOOT);
+#endif
 	asm volatile("jmp *%2" : : "a" (0x2badb002),
 		     "b"(&start_info), "c"(kernel_entry));
 	panic();
-- 
1.8.3.1

Re: [Qemu-devel] [RFC qboot 2/3] pvh: use x86/HVM direct boot ABI

[Paolo Bonzini]

On 05/12/18 23:31, Liam Merwick wrote:
> -static inline uint32_t ldl_p(void *p)
> +inline uint32_t ldl_p(void *p)
>  {
>  	uint32_t val;
>  	memcpy(&val, p, 4);
> diff --git a/main.c b/main.c

Can you make instead a header (memaccess.h?) with all of lduw_p, ldl_p,
stw_p, stl_p?

> 
> +
> +struct hvm_start_info start_info = {0};

Please put this declaration in linuxboot.c instead.

> 
> +#define RSDP_FILE "etc/acpi/rsdp"
> +
>  void extract_acpi(void)
>  {
>  	int id = fw_cfg_file_id("etc/table-loader");
> @@ -138,6 +143,10 @@ void extract_acpi(void)
>  		struct loader_cmd *s = &script[i];
>  		switch(script[i].cmd) {
>  		case CMD_ALLOC:
> +			if (strcmp(s->alloc.file, RSDP_FILE) == 0) {
> +				start_info.rsdp_paddr =
> +				    (uintptr_t)id_to_addr(id);
> +			}


Please instead do a memcmp with "RSD PTR" in do_alloc, and also only do
it if zone is ALLOC_FSEG.

Thanks,

Paolo

Re: [Qemu-devel] [RFC 0/3] qboot changes for PVH boot

[Paolo Bonzini]

On 05/12/18 23:31, Liam Merwick wrote:
> For certain applications it is desirable to rapidly boot a KVM virtual
> machine. In cases where legacy hardware and software support within the
> guest is not needed, QEMU should be able to boot directly into the
> uncompressed Linux kernel binary with minimal firmware involvement.
> 
> There already exists an ABI to allow this for Xen PVH guests and the ABI
> is supported by Linux and FreeBSD:
> 
>    https://xenbits.xen.org/docs/unstable/misc/pvh.html
> 
> Details on the Linux changes: https://lkml.org/lkml/2018/4/16/1002
> 
> This patch series provides qboot support to populate the start_info struct
> needed by the direct boot ABI and to configure the guest e820 tables to
> enable QEMU to use that same entry point for booting KVM guests.
> 
> Usіng the methods/scripts documented by the NEMU team at
> 
>    https://github.com/intel/nemu/wiki/Measuring-Boot-Latency
>    https://lists.gnu.org/archive/html/qemu-devel/2018-12/msg00200.html
> 
> below are some timings measured (vmlinux and bzImage from the same build).
> Time to get to kernel start is almost halved (95ṁs -> 48ms)

I had a few comments on patch 2, but overall it looks fine.  Can you
send a pull request on GitHub?

Thanks,

Paolo

> QEMU + qboot + vmlinux (PVH + 4.20-rc4)
>  qemu_init_end: 41.550521
>  fw_start: 41.667139 (+0.116618)
>  fw_do_boot: 47.448495 (+5.781356)
>  linux_startup_64: 47.720785 (+0.27229)
>  linux_start_kernel: 48.399541 (+0.678756)
>  linux_start_user: 296.952056 (+248.552515)
> 
> QEMU + qboot + bzImage:
>  qemu_init_end: 29.209276
>  fw_start: 29.317342 (+0.108066)
>  linux_start_boot: 36.679362 (+7.36202)
>  linux_startup_64: 94.531349 (+57.851987)
>  linux_start_kernel: 94.900913 (+0.369564)
>  linux_start_user: 401.060971 (+306.160058)
> 
> QEMU + bzImage:
>  qemu_init_end: 30.424430
>  linux_startup_64: 893.770334 (+863.345904)
>  linux_start_kernel: 894.17049 (+0.400156)
>  linux_start_user: 1208.679768 (+314.509278)
> 
> 
> Liam Merwick (3):
>   pvh: Add x86/HVM direct boot ABI header file
>   pvh: use x86/HVM direct boot ABI
>   pvh: add benchmark exit point
> 
>  benchmark.h          |   3 +-
>  fw_cfg.c             |  79 +++++++++++++++++++++++++++-
>  include/start_info.h | 146 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  linuxboot.c          |   2 +-
>  main.c               |   3 ++
>  tables.c             |   9 ++++
>  6 files changed, 239 insertions(+), 3 deletions(-)
>  create mode 100644 include/start_info.h
> 

Re: [Qemu-devel] [RFC 0/3] qboot changes for PVH boot

[Liam Merwick]

On 06/12/2018 20:13, Paolo Bonzini wrote:
> On 05/12/18 23:31, Liam Merwick wrote:
>> For certain applications it is desirable to rapidly boot a KVM virtual
>> machine. In cases where legacy hardware and software support within the
>> guest is not needed, QEMU should be able to boot directly into the
>> uncompressed Linux kernel binary with minimal firmware involvement.
>>
>> There already exists an ABI to allow this for Xen PVH guests and the ABI
>> is supported by Linux and FreeBSD:
>>
>>     https://xenbits.xen.org/docs/unstable/misc/pvh.html
>>
>> Details on the Linux changes: https://lkml.org/lkml/2018/4/16/1002
>>
>> This patch series provides qboot support to populate the start_info struct
>> needed by the direct boot ABI and to configure the guest e820 tables to
>> enable QEMU to use that same entry point for booting KVM guests.
>>
>> Usіng the methods/scripts documented by the NEMU team at
>>
>>     https://github.com/intel/nemu/wiki/Measuring-Boot-Latency
>>     https://lists.gnu.org/archive/html/qemu-devel/2018-12/msg00200.html
>>
>> below are some timings measured (vmlinux and bzImage from the same build).
>> Time to get to kernel start is almost halved (95ṁs -> 48ms)
> 
> I had a few comments on patch 2, but overall it looks fine.  Can you
> send a pull request on GitHub?
> 

Thanks Paolo.  I have made those changes and submitted a PR:

https://github.com/bonzini/qboot/pull/17

Regards,
Liam


> 
>> QEMU + qboot + vmlinux (PVH + 4.20-rc4)
>>   qemu_init_end: 41.550521
>>   fw_start: 41.667139 (+0.116618)
>>   fw_do_boot: 47.448495 (+5.781356)
>>   linux_startup_64: 47.720785 (+0.27229)
>>   linux_start_kernel: 48.399541 (+0.678756)
>>   linux_start_user: 296.952056 (+248.552515)
>>
>> QEMU + qboot + bzImage:
>>   qemu_init_end: 29.209276
>>   fw_start: 29.317342 (+0.108066)
>>   linux_start_boot: 36.679362 (+7.36202)
>>   linux_startup_64: 94.531349 (+57.851987)
>>   linux_start_kernel: 94.900913 (+0.369564)
>>   linux_start_user: 401.060971 (+306.160058)
>>
>> QEMU + bzImage:
>>   qemu_init_end: 30.424430
>>   linux_startup_64: 893.770334 (+863.345904)
>>   linux_start_kernel: 894.17049 (+0.400156)
>>   linux_start_user: 1208.679768 (+314.509278)
>>
>>
>> Liam Merwick (3):
>>    pvh: Add x86/HVM direct boot ABI header file
>>    pvh: use x86/HVM direct boot ABI
>>    pvh: add benchmark exit point
>>
>>   benchmark.h          |   3 +-
>>   fw_cfg.c             |  79 +++++++++++++++++++++++++++-
>>   include/start_info.h | 146 +++++++++++++++++++++++++++++++++++++++++++++++++++
>>   linuxboot.c          |   2 +-
>>   main.c               |   3 ++
>>   tables.c             |   9 ++++
>>   6 files changed, 239 insertions(+), 3 deletions(-)
>>   create mode 100644 include/start_info.h
>>
>