Re: Bug#759018: [PATCH RFC] Provide prebuilt grub-xen binaries for host (dom0) use

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 04/09/14 15:25, Ian Campbell wrote:
> On Thu, 2014-09-04 at 15:15 +0100, Ian Jackson wrote:
>> Colin Watson writes ("Re: Bug#759018: [PATCH RFC] Provide prebuilt grub-xen binaries for host (dom0) use"):
>> ...
>>> There is also the question of whether the guest-side name should mention
>>> GRUB.  One might argue that it shouldn't because all that matters is
>>> that it uses the Multiboot protocol.  Then there is the question of who
>>> gets to own the architecture names ...
>> The general scheme seems sound.
>>
>>
>> Ian Campbell:
>>
>>>> I'm not sure what the best way to promulgate the spec is -- I
>>>> think a patch to add xen.git/docs/misc/pvgrub2.markdown would be
>>>> sufficient (it would end up under http://xenbits.xen.org/docs/).
>> Since this path is in /boot/xen and contains an executable which
>> expects to run in the Xen PV environment, it could also use Xen names
>> for the architectures.  I don't know whether a GNU config triplet arch
>> name as you suggest is easier or harder than that.
> About the same. Colin suggested the GNU config triplet names in his
> strawman and I couldn't think of a reason to change.
>
>> I have a question about the spec's wording about bitness:
>>
>> +It is not in general possible under Xen for a bootloader to boot a
>> +kernel of a different width from itself, and this extends to
>> +chainloading from a stage one. Therefore it is permissible to have
>> +both `/boot/xen/pvboot-i386.elf` and `/boot/xen/pvboot-x86\_64.elf`
>> +present in a guest to be used by the appropriate stage 1 (e.g. for
>> +systems with 32-bit userspace and an optional 64-bit kernel).
>>
>> Is it therefore expected that the host admin will be told out of band
>> what bitness the guest would prefer ?  And that then the host
>> toolstack will set up that bitness of guest, load its pvgrub for that
>> bitness, and hope that the guest has an appropriate-bitness core image
>> load in the canonical place ?
> Yes. Essentially you write kernel = /path/too/pvgrub-<32bit-name> or
> -<64bit-name> in your guest config. Yes, this sucks.
>
> AIUI in cloud interfaces etc you generally have to ask for a 32- or
> 64-bit domain explicitly too.
>
>> I wonder if we might, in the future, want this to be more automatic.
> I suppose that Would Be Nice(tm).
>
> I'm not sure but it might be that for a PVH guest (once grub is ported
> to that) we might be able to switch bitness at runtime and avoid this
> whole mess (which comes largely from the size of the p2m entries and the
> difficulties in switching, which is hidden from pvh guests)
>
>> I guess we could have a feature in the host's 64-bit pvgrub which
>> would look for and load 64-bit guest pvgrub if it exists, and
>> alternatively check for 32-bit guest pvgrub and if found exit
>> signalling somehow to the host toolstack to restart the domain with
>> the other bitness.
>>
>> But what if the host has both 64- and 32-bit pvgrub but in fact only
>> has one bitness of kernel ?  Signalling this back to the host by
>> somehow hiding or renaming one of the bitnesses of guest pvgrub seems
>> unpleasant.
> You mean if all guests happen to only use one bitness? You waste a
> roundtrip through a stunt domain which will do the exiting trick and
> restart, or you supply the right dom0 grub to start with I guess.

There is no real technical obstacle to prevent PV domains switching
width after starting.

In the past, the toolstack has directly loaded the target running
kernel, but that behaviour/assumption is no longer correct given these
chainloading plans.

As we no longer support 2-level PV guests, allowing a PV domain to
switch between 3 and 4 levels becomes easier to manage from Xens point
of view.

>From the top of my head, it would involve relaxing the restriction that
3 level PV guests can't pin L4 tables (but still enforce that a 3 level
PV guest can't load an L4 pagetable), and provide a new hypercall which
takes a desired with, new cr3 (referring to a pinned pagetable of the
appropriate new width) and a new eip to jump to.

~Andrew