Xen, ACPI and Linux

Xen, ACPI and Linux

[Stefano Stabellini]

Hi all,

Mark, Ard and I have just had a discussion on ACPI, EFI and booting
interfaces for Xen and kexec.

We all agree that the most important thing to do is to document
precisely what this interface looks like. Not just the device tree
nodes, but also who calls ExitBootServices, SetVirtualAddressMap, etc.
We need to make clear that it is an external interface and will be
maintained for backward compatibility going forward.  The proposed
renaming of the device tree nodes is OK, and can be part of it.



Regarding Runtime Services, the EFI spec doesn't allow a NULL pointer to
the Runtime Services table, so Mark would like to see a proper pointer
being passed there.  The function table could be populated with
hypercall wrappers in assembly, keeping the same interface to Xen that
we have today in drivers/xen/efi.c. It should be part of the initial
patch series.

If that turns out to be very hard to do (which is unlikely), or if that
approach has any unforeseen problems, we could specify in the external ABI
document that the Runtime Services table pointer can be NULL. Mark
would be less happy with this solution, let's try the other one first.


Cheers,

Stefano

Re: Xen, ACPI and Linux

[Jan Beulich]

>>> On 23.09.15 at 02:49, <stefano.stabellini@eu.citrix.com> wrote:
> Regarding Runtime Services, the EFI spec doesn't allow a NULL pointer to
> the Runtime Services table, so Mark would like to see a proper pointer
> being passed there.  The function table could be populated with
> hypercall wrappers in assembly, keeping the same interface to Xen that
> we have today in drivers/xen/efi.c. It should be part of the initial
> patch series.

I'm confused by the "interface to Xen" part: Aren't we talking about
what is being presented to Dom0?

In any event, the versioning question that I raised earlier remains:
Which version would you intend the Runtime Services table to carry
- the host one, or a Xen set one? In the latter case, won't you risk
wrong implications from the kernel looking at other version numbers
(yes, with proper coding it ought to be possible to avoid such, but
the multitude of version numbers in EFI doesn't exactly help to
avoid mistakes)? While in the former case you'd have to deal with
the table needing entries Xen may not know about.

Jan

Re: Xen, ACPI and Linux

[Ard Biesheuvel]

On 23 September 2015 at 01:12, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 23.09.15 at 02:49, <stefano.stabellini@eu.citrix.com> wrote:
>> Regarding Runtime Services, the EFI spec doesn't allow a NULL pointer to
>> the Runtime Services table, so Mark would like to see a proper pointer
>> being passed there.  The function table could be populated with
>> hypercall wrappers in assembly, keeping the same interface to Xen that
>> we have today in drivers/xen/efi.c. It should be part of the initial
>> patch series.
>
> I'm confused by the "interface to Xen" part: Aren't we talking about
> what is being presented to Dom0?
>

Yes we are.

> In any event, the versioning question that I raised earlier remains:
> Which version would you intend the Runtime Services table to carry
> - the host one, or a Xen set one? In the latter case, won't you risk
> wrong implications from the kernel looking at other version numbers
> (yes, with proper coding it ought to be possible to avoid such, but
> the multitude of version numbers in EFI doesn't exactly help to
> avoid mistakes)? While in the former case you'd have to deal with
> the table needing entries Xen may not know about.
>

This is simply addressed by populating the fake EFI system table
according to the UEFI spec version field that you put in the header.
No reason at all to base this on whatever the host provides, it should
simply be a version that is supported by arm64 (2.00 or greater)

-- 
Ard.

Re: Xen, ACPI and Linux

[Ian Campbell]

On Wed, 2015-09-23 at 01:18 -0700, Ard Biesheuvel wrote:
> On 23 September 2015 at 01:12, Jan Beulich <JBeulich@suse.com> wrote:
> > > > > On 23.09.15 at 02:49, <stefano.stabellini@eu.citrix.com> wrote:
> > > Regarding Runtime Services, the EFI spec doesn't allow a NULL pointer
> > > to
> > > the Runtime Services table, so Mark would like to see a proper
> > > pointer
> > > being passed there.  The function table could be populated with
> > > hypercall wrappers in assembly, keeping the same interface to Xen
> > > that
> > > we have today in drivers/xen/efi.c. It should be part of the initial
> > > patch series.
> > 
> > I'm confused by the "interface to Xen" part: Aren't we talking about
> > what is being presented to Dom0?
> > 
> 
> Yes we are.
> 
> > In any event, the versioning question that I raised earlier remains:
> > Which version would you intend the Runtime Services table to carry
> > - the host one, or a Xen set one? In the latter case, won't you risk
> > wrong implications from the kernel looking at other version numbers
> > (yes, with proper coding it ought to be possible to avoid such, but
> > the multitude of version numbers in EFI doesn't exactly help to
> > avoid mistakes)? While in the former case you'd have to deal with
> > the table needing entries Xen may not know about.
> > 
> 
> This is simply addressed by populating the fake EFI system table
> according to the UEFI spec version field that you put in the header.
> No reason at all to base this on whatever the host provides, it should
> simply be a version that is supported by arm64 (2.00 or greater)

This doesn't address Jan's concern wrt the multiple other places UEFI
exposes version numbers which may reflect the host and not this fake EFI
table.

Ian.

Re: Xen, ACPI and Linux

[Stefano Stabellini]

On Wed, 23 Sep 2015, Ian Campbell wrote:
> On Wed, 2015-09-23 at 01:18 -0700, Ard Biesheuvel wrote:
> > On 23 September 2015 at 01:12, Jan Beulich <JBeulich@suse.com> wrote:
> > > > > > On 23.09.15 at 02:49, <stefano.stabellini@eu.citrix.com> wrote:
> > > > Regarding Runtime Services, the EFI spec doesn't allow a NULL pointer
> > > > to
> > > > the Runtime Services table, so Mark would like to see a proper
> > > > pointer
> > > > being passed there.  The function table could be populated with
> > > > hypercall wrappers in assembly, keeping the same interface to Xen
> > > > that
> > > > we have today in drivers/xen/efi.c. It should be part of the initial
> > > > patch series.
> > > 
> > > I'm confused by the "interface to Xen" part: Aren't we talking about
> > > what is being presented to Dom0?
> > > 
> > 
> > Yes we are.
> > 
> > > In any event, the versioning question that I raised earlier remains:
> > > Which version would you intend the Runtime Services table to carry
> > > - the host one, or a Xen set one? In the latter case, won't you risk
> > > wrong implications from the kernel looking at other version numbers
> > > (yes, with proper coding it ought to be possible to avoid such, but
> > > the multitude of version numbers in EFI doesn't exactly help to
> > > avoid mistakes)? While in the former case you'd have to deal with
> > > the table needing entries Xen may not know about.
> > > 
> > 
> > This is simply addressed by populating the fake EFI system table
> > according to the UEFI spec version field that you put in the header.
> > No reason at all to base this on whatever the host provides, it should
> > simply be a version that is supported by arm64 (2.00 or greater)
> 
> This doesn't address Jan's concern wrt the multiple other places UEFI
> exposes version numbers which may reflect the host and not this fake EFI
> table.

The Runtime Services table has its own version field. I think that
theoretically it could be different from the other version fields, but I
don't know if it ever happens with real hardware.

If we don't want to support the case where Runtime Services have a
different revision version than the other tables, then I think that
going for the NULL Runtime Services table pointer approach might be
better.

Re: Xen, ACPI and Linux

[Stefano Stabellini]

On Wed, 23 Sep 2015, Stefano Stabellini wrote:
> On Wed, 23 Sep 2015, Ian Campbell wrote:
> > On Wed, 2015-09-23 at 01:18 -0700, Ard Biesheuvel wrote:
> > > On 23 September 2015 at 01:12, Jan Beulich <JBeulich@suse.com> wrote:
> > > > > > > On 23.09.15 at 02:49, <stefano.stabellini@eu.citrix.com> wrote:
> > > > > Regarding Runtime Services, the EFI spec doesn't allow a NULL pointer
> > > > > to
> > > > > the Runtime Services table, so Mark would like to see a proper
> > > > > pointer
> > > > > being passed there.  The function table could be populated with
> > > > > hypercall wrappers in assembly, keeping the same interface to Xen
> > > > > that
> > > > > we have today in drivers/xen/efi.c. It should be part of the initial
> > > > > patch series.
> > > > 
> > > > I'm confused by the "interface to Xen" part: Aren't we talking about
> > > > what is being presented to Dom0?
> > > > 
> > > 
> > > Yes we are.
> > > 
> > > > In any event, the versioning question that I raised earlier remains:
> > > > Which version would you intend the Runtime Services table to carry
> > > > - the host one, or a Xen set one? In the latter case, won't you risk
> > > > wrong implications from the kernel looking at other version numbers
> > > > (yes, with proper coding it ought to be possible to avoid such, but
> > > > the multitude of version numbers in EFI doesn't exactly help to
> > > > avoid mistakes)? While in the former case you'd have to deal with
> > > > the table needing entries Xen may not know about.
> > > > 
> > > 
> > > This is simply addressed by populating the fake EFI system table
> > > according to the UEFI spec version field that you put in the header.
> > > No reason at all to base this on whatever the host provides, it should
> > > simply be a version that is supported by arm64 (2.00 or greater)
> > 
> > This doesn't address Jan's concern wrt the multiple other places UEFI
> > exposes version numbers which may reflect the host and not this fake EFI
> > table.
> 
> The Runtime Services table has its own version field. I think that
> theoretically it could be different from the other version fields, but I
> don't know if it ever happens with real hardware.
> 
> If we don't want to support the case where Runtime Services have a
> different revision version than the other tables, then I think that
> going for the NULL Runtime Services table pointer approach might be
> better.

Looking again at the spec, it includes this line:

#define EFI_RUNTIME_SERVICES_REVISION EFI_SPECIFICATION_VERSION

the only way I can read it is that the EFI Runtime Services version
needs to match the EFI specification version.

Unfortunately it looks like that we cannot, according to spec, expose a
different version of Runtime Services.

Given that trying to emulate a set of Runtime Services which matches the
version of the ones on the host would be undesirable from Xen point of
view, would it be OK if we went ahead and added to the Xen-Dom0
interface on EFI, which we are about to introduce and document, that the
Runtime Services table pointer can be NULL?

Re: Xen, ACPI and Linux

[Stefano Stabellini]

On Thu, 24 Sep 2015, Stefano Stabellini wrote:
> On Wed, 23 Sep 2015, Stefano Stabellini wrote:
> > On Wed, 23 Sep 2015, Ian Campbell wrote:
> > > On Wed, 2015-09-23 at 01:18 -0700, Ard Biesheuvel wrote:
> > > > On 23 September 2015 at 01:12, Jan Beulich <JBeulich@suse.com> wrote:
> > > > > > > > On 23.09.15 at 02:49, <stefano.stabellini@eu.citrix.com> wrote:
> > > > > > Regarding Runtime Services, the EFI spec doesn't allow a NULL pointer
> > > > > > to
> > > > > > the Runtime Services table, so Mark would like to see a proper
> > > > > > pointer
> > > > > > being passed there.  The function table could be populated with
> > > > > > hypercall wrappers in assembly, keeping the same interface to Xen
> > > > > > that
> > > > > > we have today in drivers/xen/efi.c. It should be part of the initial
> > > > > > patch series.
> > > > > 
> > > > > I'm confused by the "interface to Xen" part: Aren't we talking about
> > > > > what is being presented to Dom0?
> > > > > 
> > > > 
> > > > Yes we are.
> > > > 
> > > > > In any event, the versioning question that I raised earlier remains:
> > > > > Which version would you intend the Runtime Services table to carry
> > > > > - the host one, or a Xen set one? In the latter case, won't you risk
> > > > > wrong implications from the kernel looking at other version numbers
> > > > > (yes, with proper coding it ought to be possible to avoid such, but
> > > > > the multitude of version numbers in EFI doesn't exactly help to
> > > > > avoid mistakes)? While in the former case you'd have to deal with
> > > > > the table needing entries Xen may not know about.
> > > > > 
> > > > 
> > > > This is simply addressed by populating the fake EFI system table
> > > > according to the UEFI spec version field that you put in the header.
> > > > No reason at all to base this on whatever the host provides, it should
> > > > simply be a version that is supported by arm64 (2.00 or greater)
> > > 
> > > This doesn't address Jan's concern wrt the multiple other places UEFI
> > > exposes version numbers which may reflect the host and not this fake EFI
> > > table.
> > 
> > The Runtime Services table has its own version field. I think that
> > theoretically it could be different from the other version fields, but I
> > don't know if it ever happens with real hardware.
> > 
> > If we don't want to support the case where Runtime Services have a
> > different revision version than the other tables, then I think that
> > going for the NULL Runtime Services table pointer approach might be
> > better.
> 
> Looking again at the spec, it includes this line:
> 
> #define EFI_RUNTIME_SERVICES_REVISION EFI_SPECIFICATION_VERSION
> 
> the only way I can read it is that the EFI Runtime Services version
> needs to match the EFI specification version.
> 
> Unfortunately it looks like that we cannot, according to spec, expose a
> different version of Runtime Services.
> 
> Given that trying to emulate a set of Runtime Services which matches the
> version of the ones on the host would be undesirable from Xen point of
> view, would it be OK if we went ahead and added to the Xen-Dom0
> interface on EFI, which we are about to introduce and document, that the
> Runtime Services table pointer can be NULL?

Mark, Christoffer and I had another chat.  Given that it is difficult to
provide a consistent set of tables to Dom0 (the Runtime Services version
could differ from the native tables version and that is not allowed by
the EFI spec), Mark would like to see EFI support being introduced in a
Xen specific manner in Dom0, so that it is very clear that it might
differ from the native EFI tables and spec.

This is what we need to do:

- xen_early_init() should be moved before efi_init() in
  arch/arm64/kernel/setup.c. This will cause the device tree parsing in
  xen_early_init to be changed because it will have to operate on the
  flatten device tree.
- xen_early_init should discover the EFI tables. It could do that via
  hypercalls or via device tree attributes under the Xen hypervisor node
  on device tree. The nodes under /chosen, described by
  Documentation/arm/uefi.txt, should not be reused.
- xen_early_init could call the efi initialization functions directly or
  could setup some data structures (to be defined) so that efi_init will
  later detect that we are running on Xen making the initialization
  slightly different.
- acpi will be initialized as usual


This is what Mark, Christoffer and I agreed, I hope that this plan works
for everybody else too.

Re: Xen, ACPI and Linux

[Ian Campbell]

On Fri, 2015-09-25 at 21:20 +0100, Stefano Stabellini wrote:
> On Thu, 24 Sep 2015, Stefano Stabellini wrote:
> > On Wed, 23 Sep 2015, Stefano Stabellini wrote:
> > > On Wed, 23 Sep 2015, Ian Campbell wrote:
> > > > On Wed, 2015-09-23 at 01:18 -0700, Ard Biesheuvel wrote:
> > > > > On 23 September 2015 at 01:12, Jan Beulich <JBeulich@suse.com>
> > > > > wrote:
> > > > > > > > > On 23.09.15 at 02:49, <stefano.stabellini@eu.citrix.com>
> > > > > > > > > wrote:
> > > > > > > Regarding Runtime Services, the EFI spec doesn't allow a NULL
> > > > > > > pointer
> > > > > > > to
> > > > > > > the Runtime Services table, so Mark would like to see a
> > > > > > > proper
> > > > > > > pointer
> > > > > > > being passed there.  The function table could be populated
> > > > > > > with
> > > > > > > hypercall wrappers in assembly, keeping the same interface to
> > > > > > > Xen
> > > > > > > that
> > > > > > > we have today in drivers/xen/efi.c. It should be part of the
> > > > > > > initial
> > > > > > > patch series.
> > > > > > 
> > > > > > I'm confused by the "interface to Xen" part: Aren't we talking
> > > > > > about
> > > > > > what is being presented to Dom0?
> > > > > > 
> > > > > 
> > > > > Yes we are.
> > > > > 
> > > > > > In any event, the versioning question that I raised earlier
> > > > > > remains:
> > > > > > Which version would you intend the Runtime Services table to
> > > > > > carry
> > > > > > - the host one, or a Xen set one? In the latter case, won't you
> > > > > > risk
> > > > > > wrong implications from the kernel looking at other version
> > > > > > numbers
> > > > > > (yes, with proper coding it ought to be possible to avoid such,
> > > > > > but
> > > > > > the multitude of version numbers in EFI doesn't exactly help to
> > > > > > avoid mistakes)? While in the former case you'd have to deal
> > > > > > with
> > > > > > the table needing entries Xen may not know about.
> > > > > > 
> > > > > 
> > > > > This is simply addressed by populating the fake EFI system table
> > > > > according to the UEFI spec version field that you put in the
> > > > > header.
> > > > > No reason at all to base this on whatever the host provides, it
> > > > > should
> > > > > simply be a version that is supported by arm64 (2.00 or greater)
> > > > 
> > > > This doesn't address Jan's concern wrt the multiple other places
> > > > UEFI
> > > > exposes version numbers which may reflect the host and not this
> > > > fake EFI
> > > > table.
> > > 
> > > The Runtime Services table has its own version field. I think that
> > > theoretically it could be different from the other version fields,
> > > but I
> > > don't know if it ever happens with real hardware.
> > > 
> > > If we don't want to support the case where Runtime Services have a
> > > different revision version than the other tables, then I think that
> > > going for the NULL Runtime Services table pointer approach might be
> > > better.
> > 
> > Looking again at the spec, it includes this line:
> > 
> > #define EFI_RUNTIME_SERVICES_REVISION EFI_SPECIFICATION_VERSION
> > 
> > the only way I can read it is that the EFI Runtime Services version
> > needs to match the EFI specification version.
> > 
> > Unfortunately it looks like that we cannot, according to spec, expose a
> > different version of Runtime Services.
> > 
> > Given that trying to emulate a set of Runtime Services which matches
> > the
> > version of the ones on the host would be undesirable from Xen point of
> > view, would it be OK if we went ahead and added to the Xen-Dom0
> > interface on EFI, which we are about to introduce and document, that
> > the
> > Runtime Services table pointer can be NULL?
> 
> Mark, Christoffer and I had another chat.  Given that it is difficult to
> provide a consistent set of tables to Dom0 (the Runtime Services version
> could differ from the native tables version and that is not allowed by
> the EFI spec), Mark would like to see EFI support being introduced in a
> Xen specific manner in Dom0, so that it is very clear that it might
> differ from the native EFI tables and spec.
> 
> This is what we need to do:
> 
> - xen_early_init() should be moved before efi_init() in
>   arch/arm64/kernel/setup.c. This will cause the device tree parsing in
>   xen_early_init to be changed because it will have to operate on the
>   flatten device tree.
> - xen_early_init should discover the EFI tables. It could do that via
>   hypercalls or via device tree attributes under the Xen hypervisor node
>   on device tree. The nodes under /chosen, described by
>   Documentation/arm/uefi.txt, should not be reused.
> - xen_early_init could call the efi initialization functions directly or
>   could setup some data structures (to be defined) so that efi_init will
>   later detect that we are running on Xen making the initialization
>   slightly different.
> - acpi will be initialized as usual
> 
> 
> This is what Mark, Christoffer and I agreed, I hope that this plan works
> for everybody else too.

My only question is whether, given this early init, we need to continue to
tie EFI and ACPI together in this way. Can't we detect ACPI directly
instead of EFI?

Ian.

Re: Xen, ACPI and Linux

[Stefano Stabellini]

On Mon, 28 Sep 2015, Ian Campbell wrote:
> On Fri, 2015-09-25 at 21:20 +0100, Stefano Stabellini wrote:
> > On Thu, 24 Sep 2015, Stefano Stabellini wrote:
> > > On Wed, 23 Sep 2015, Stefano Stabellini wrote:
> > > > On Wed, 23 Sep 2015, Ian Campbell wrote:
> > > > > On Wed, 2015-09-23 at 01:18 -0700, Ard Biesheuvel wrote:
> > > > > > On 23 September 2015 at 01:12, Jan Beulich <JBeulich@suse.com>
> > > > > > wrote:
> > > > > > > > > > On 23.09.15 at 02:49, <stefano.stabellini@eu.citrix.com>
> > > > > > > > > > wrote:
> > > > > > > > Regarding Runtime Services, the EFI spec doesn't allow a NULL
> > > > > > > > pointer
> > > > > > > > to
> > > > > > > > the Runtime Services table, so Mark would like to see a
> > > > > > > > proper
> > > > > > > > pointer
> > > > > > > > being passed there.  The function table could be populated
> > > > > > > > with
> > > > > > > > hypercall wrappers in assembly, keeping the same interface to
> > > > > > > > Xen
> > > > > > > > that
> > > > > > > > we have today in drivers/xen/efi.c. It should be part of the
> > > > > > > > initial
> > > > > > > > patch series.
> > > > > > > 
> > > > > > > I'm confused by the "interface to Xen" part: Aren't we talking
> > > > > > > about
> > > > > > > what is being presented to Dom0?
> > > > > > > 
> > > > > > 
> > > > > > Yes we are.
> > > > > > 
> > > > > > > In any event, the versioning question that I raised earlier
> > > > > > > remains:
> > > > > > > Which version would you intend the Runtime Services table to
> > > > > > > carry
> > > > > > > - the host one, or a Xen set one? In the latter case, won't you
> > > > > > > risk
> > > > > > > wrong implications from the kernel looking at other version
> > > > > > > numbers
> > > > > > > (yes, with proper coding it ought to be possible to avoid such,
> > > > > > > but
> > > > > > > the multitude of version numbers in EFI doesn't exactly help to
> > > > > > > avoid mistakes)? While in the former case you'd have to deal
> > > > > > > with
> > > > > > > the table needing entries Xen may not know about.
> > > > > > > 
> > > > > > 
> > > > > > This is simply addressed by populating the fake EFI system table
> > > > > > according to the UEFI spec version field that you put in the
> > > > > > header.
> > > > > > No reason at all to base this on whatever the host provides, it
> > > > > > should
> > > > > > simply be a version that is supported by arm64 (2.00 or greater)
> > > > > 
> > > > > This doesn't address Jan's concern wrt the multiple other places
> > > > > UEFI
> > > > > exposes version numbers which may reflect the host and not this
> > > > > fake EFI
> > > > > table.
> > > > 
> > > > The Runtime Services table has its own version field. I think that
> > > > theoretically it could be different from the other version fields,
> > > > but I
> > > > don't know if it ever happens with real hardware.
> > > > 
> > > > If we don't want to support the case where Runtime Services have a
> > > > different revision version than the other tables, then I think that
> > > > going for the NULL Runtime Services table pointer approach might be
> > > > better.
> > > 
> > > Looking again at the spec, it includes this line:
> > > 
> > > #define EFI_RUNTIME_SERVICES_REVISION EFI_SPECIFICATION_VERSION
> > > 
> > > the only way I can read it is that the EFI Runtime Services version
> > > needs to match the EFI specification version.
> > > 
> > > Unfortunately it looks like that we cannot, according to spec, expose a
> > > different version of Runtime Services.
> > > 
> > > Given that trying to emulate a set of Runtime Services which matches
> > > the
> > > version of the ones on the host would be undesirable from Xen point of
> > > view, would it be OK if we went ahead and added to the Xen-Dom0
> > > interface on EFI, which we are about to introduce and document, that
> > > the
> > > Runtime Services table pointer can be NULL?
> > 
> > Mark, Christoffer and I had another chat.  Given that it is difficult to
> > provide a consistent set of tables to Dom0 (the Runtime Services version
> > could differ from the native tables version and that is not allowed by
> > the EFI spec), Mark would like to see EFI support being introduced in a
> > Xen specific manner in Dom0, so that it is very clear that it might
> > differ from the native EFI tables and spec.
> > 
> > This is what we need to do:
> > 
> > - xen_early_init() should be moved before efi_init() in
> >   arch/arm64/kernel/setup.c. This will cause the device tree parsing in
> >   xen_early_init to be changed because it will have to operate on the
> >   flatten device tree.
> > - xen_early_init should discover the EFI tables. It could do that via
> >   hypercalls or via device tree attributes under the Xen hypervisor node
> >   on device tree. The nodes under /chosen, described by
> >   Documentation/arm/uefi.txt, should not be reused.
> > - xen_early_init could call the efi initialization functions directly or
> >   could setup some data structures (to be defined) so that efi_init will
> >   later detect that we are running on Xen making the initialization
> >   slightly different.
> > - acpi will be initialized as usual
> > 
> > 
> > This is what Mark, Christoffer and I agreed, I hope that this plan works
> > for everybody else too.
> 
> My only question is whether, given this early init, we need to continue to
> tie EFI and ACPI together in this way. Can't we detect ACPI directly
> instead of EFI?

It is still useful to go via EFI for two reasons:
- it is OK to detect ACPI from the EFI System Table, reusing the
  existing code
- we need to export EFI support to provide Runtime Services