Xen 4.1 + Linux compiled with PVH == BOOM

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

Hey,

This is with Linux and

git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git stable/pvh.v11

I get Xen 4.1 (only) hypervisor to blow up with a Linux kernel that has been
compiled with PVH.

I think the same problem would show up if I tried to launch a PV guest 
compiled as PVH under Xen 4.1 as well - as the ELF parsing code is shared
with the toolstack.

The issue is that Xen 4.1 is missing git
commit 30832c06a8d1f9caff0987654ef9e24d59469d9a
Author: Mukesh Rathor <mukesh.rathor@oracle.com>
Date:   Tue Sep 10 16:38:43 2013 +0200

    libelf: add hvm callback vector feature
    
    Add XENFEAT_hvm_callback_vector to elf_xen_feature_names so we can
    ensure the kernel supports all features required for PVH mode when
    building a PVH domU here. Note, hvm callback is required for PVH.
    
    Signed-off-by: Mukesh Rathor <mukesh.rathor@oracle.com>
    Acked-by: Ian Campbell <ian.campbell@citrix.com>

where the message is a bit misleading. It also allows the hypervisor
to boot a PVH guest in PV mode.

Here is what the Xen serial log shows (with extra debugging sprinkled):

(XEN) elf_xen_parse_note: GUEST_OS = "linux"
(XEN) elf_xen_parse_note: GUEST_VERSION = "2.6"
(XEN) elf_xen_parse_note: XEN_VERSION = "xen-3.0"
(XEN) elf_xen_parse_note: VIRT_BASE = 0xffffffff80000000
(XEN) elf_xen_parse_note: ENTRY = 0xffffffff81cd51e0
(XEN) elf_xen_parse_note: HYPERCALL_PAGE = 0xffffffff81001000
(XEN) elf_xen_parse_note: FEATURES = "!writable_page_tables|pae_pgdir_above_4gb|writable_descriptor_tables|auto_translated_physmap|supervisor_mode_kernel|hvm_callback_vector"
(XEN) elf_xen_parse_features:70 r[writable_page_tables]
(XEN) elf_xen_parse_features:81 s[pae_pgdir_above_4gb]
(XEN) elf_xen_parse_features:81 s[writable_descriptor_tables]
(XEN) elf_xen_parse_features:81 s[auto_translated_physmap]
(XEN) elf_xen_parse_features:81 s[supervisor_mode_kernel]
(XEN) elf_xen_parse_features:88 5
(XEN) elf_xen_parse_note:206
(XEN) elf_xen_parse_notes:245 on Xen


We end up bailing out:

 87         if ( i == elf_xen_features ) {
 88             elf_msg(elf,"%s:%d %d\n", __func__, __LINE__, i);
 89             return -1;
 90         }

because the elf_xen_features (or rather elf_xen_feature_names)
does not have the "hvm_callback_vector parameter" and it can't parse
it - so it returns -1.

And that means:
203     case XEN_ELFNOTE_FEATURES:
204         if ( elf_xen_parse_features(elf, str, parms->f_supported,
205                                     parms->f_required) ) {
206             elf_msg(elf, "%s:%d\n", __func__, __LINE__);
207             return -1;
208         }

Which means we return at:
244         if ( elf_xen_parse_note(elf, parms, note) ) {
245             elf_msg(elf, "%s:%d on %s\n", __func__, __LINE__, note_name);
246             return ELF_NOTE_INVALID;
247         }

and never finish the construct_dom0.

Thought on how to fix it?

My thought was that we should return 0, that is in line with
some of the other code that deals with elf, such as elf_xen_parse_note
which returns 0:

130     if ( (type >= sizeof(note_desc) / sizeof(note_desc[0])) ||
131          (note_desc[type].name == NULL) )
132     {
133         elf_msg(elf, "%s: unknown xen elf note (0x%x)\n",
134                 __FUNCTION__, type);
135         return 0;
136     }


Like this:

diff --git a/xen/common/libelf/libelf-dominfo.c b/xen/common/libelf/libelf-dominfo.c
index fda19e7..c9ff61e 100644
--- a/xen/common/libelf/libelf-dominfo.c
+++ b/xen/common/libelf/libelf-dominfo.c
@@ -83,7 +83,9 @@ elf_errorstatus elf_xen_parse_features(const char *features,
             }
         }
         if ( i == elf_xen_features )
-            return -1;
+            return 0; /* We don't recognize this feature, and let the
+                       * caller figure out if it has all of the needed parts.
+                       */
     }
 
     return 0;



But perhaps that is not the way to do it and we should just cherry-pick
30832c06a8d1f9caff0987654ef9e24d59469d9a in Xen 4.1?

cThoughts?