Design doc of adding ACPI support for arm64 on Xen - version 3

Design doc of adding ACPI support for arm64 on Xen - version 3

[Shannon Zhao]

This document is going to explain the design details of Xen booting with
ACPI on ARM. Maybe parts of it may not be appropriate. Any comments are
welcome.

Changes v2->v3:
* remove the two HVM_PARAMs for grant table and let linux kernel use 
xlated_setup_gnttab_pages() to setup grant table.
* don't modify GTDT table
* add definition of event-channel interrupt flag
* state that route all Xen unused interrupt to Dom0
* state that reusing existing PCI bus_notifier for PCI devices MMIO mapping

To Xen itself booting with ACPI, this is similar to Linux kernel except
that Xen doesn't parse DSDT table. So I'll skip this part and focus on
how Xen prepares ACPI tables for Dom0 and how Xen passes them to Dom0.

1. Copy and change some EFI and ACPI tables
-------------------------------------------
a) Copy EFI_SYSTEM_TABLE and change the value of FirmwareVendor,
VendorGuid, VendorTable, ConfigurationTable. These changes are not very
special and it just assign values to these members.

b) Create EFI_MEMORY_DESCRIPTOR table. This will add memory start and
size information of Dom0. And Dom0 will get the memory information
through this EFI table.

c) Copy FADT table. Change the value of arm_boot_flags to enable PSCI
and HVC. Let the hypervisor_id be "XenVMM" in order to tell Dom0 that it
runs on Xen hypervisor, then Dom0 could through HVM_PARAM to get some
informations for booting necessity, such as event-channel interrupt.
Change header revison, length and checksum as well.

d) Copy MADT table. According to the value of dom0_max_vcpus to change
the number GICC entries.

e) Create STAO table. This table is a new added one that's used to
define a list of ACPI namespace names that are to be ignored by the OSPM
in Dom0. Currently we use it to tell OSPM should ignore UART defined in
SPCR table.

f) Copy XSDT table. Add a new table entry for STAO and change other
table's entries.

g) Change the value of xsdt_physical_address in RSDP table.

h) The rest of tables are not copied or changed. They are reused
including DSDT, SPCR, GTDT, etc.

All these tables will be copied to Dom0 memory except that the reused
tables(DSDT, SPCR, GTDT, etc) will be mapped to Dom0.

2. Create minimal DT to pass required information to Dom0
----------------------------------------------------------
The minimal DT mainly passes Dom0 bootargs, address and size of initrd
(if available), address and size of uefi system table, address and size
of uefi memory table, uefi-mmap-desc-size and uefi-mmap-desc-ver.

An example of the minimal DT:
/ {
     #address-cells = <2>;
     #size-cells = <1>;
     chosen {
         bootargs = "kernel=Image console=hvc0 earlycon=pl011,0x1c090000
root=/dev/vda2 rw rootfstype=ext4 init=/bin/sh acpi=force";
         linux,initrd-start = <0xXXXXXXXX>;
         linux,initrd-end = <0xXXXXXXXX>;
         linux,uefi-system-table = <0xXXXXXXXX>;
         linux,uefi-mmap-start = <0xXXXXXXXX>;
         linux,uefi-mmap-size = <0xXXXXXXXX>;
         linux,uefi-mmap-desc-size = <0xXXXXXXXX>;
         linux,uefi-mmap-desc-ver = <0xXXXXXXXX>;
     };
};

For details loook at
https://github.com/torvalds/linux/blob/master/Documentation/arm/uefi.txt

3. Dom0 gets grant table and event channel irq information
-----------------------------------------------------------
Make Linux call xlated_setup_gnttab_pages() to setup grant table. So it
doesn't need Xen pass grant table start and size information to Dom0.

To event channel interrupt, reuse HVM_PARAM_CALLBACK_IRQ and add a new
delivery type to get it.
val[63:56] == 3: val[15:8] is flag: val[7:0] is a PPI (ARM and ARM64
only)
The definition of flag reusing the definition of xenv table. Bit 0
stands interrupt mode and bit 1 stands interrupt polarity.

As said above, we assign the hypervisor_id be "XenVMM" to tell Dom0 that
it runs on Xen hypervisor. Then Dom0 could get it through hypercall
HVMOP_get_param.

4. Map MMIO regions
-------------------
Register a bus_notifier for platform and amba bus in Linux. Add a new
XENMAPSPACE "XENMAPSPACE_dev_mmio". Within the register, check if the
device is newly added, then call hypercall XENMEM_add_to_physmap to map
the mmio regions.

For PCI bus device, it could reuse the existing PCI bus_notifier like
X86.

5. Route device interrupts to Dom0
----------------------------------
Route all the SPI interrupts to Dom0 before Dom0 booting, except the
interrupts used by Xen. For uart, it could get the interrupt from SPCR
table and exclude it. For SMMU, it could get the interrupts from IORT
table and exclude them as well.

Thanks,
-- 
Shannon

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Julien Grall]

On 14/08/15 15:59, Shannon Zhao wrote:
> 2. Create minimal DT to pass required information to Dom0
> ----------------------------------------------------------
> The minimal DT mainly passes Dom0 bootargs, address and size of initrd
> (if available), address and size of uefi system table, address and size
> of uefi memory table, uefi-mmap-desc-size and uefi-mmap-desc-ver.
> 
> An example of the minimal DT:
> / {
>     #address-cells = <2>;
>     #size-cells = <1>;
>     chosen {
>         bootargs = "kernel=Image console=hvc0 earlycon=pl011,0x1c090000
> root=/dev/vda2 rw rootfstype=ext4 init=/bin/sh acpi=force";
>         linux,initrd-start = <0xXXXXXXXX>;
>         linux,initrd-end = <0xXXXXXXXX>;
>         linux,uefi-system-table = <0xXXXXXXXX>;
>         linux,uefi-mmap-start = <0xXXXXXXXX>;
>         linux,uefi-mmap-size = <0xXXXXXXXX>;
>         linux,uefi-mmap-desc-size = <0xXXXXXXXX>;
>         linux,uefi-mmap-desc-ver = <0xXXXXXXXX>;
>     };
> };
> 
> For details loook at
> https://github.com/torvalds/linux/blob/master/Documentation/arm/uefi.txt

I would have expect a summary on the discussion we had on the previous
thread [1].

Note that "linux,initrd-*" are well defined given that Xen, U-boot and
other bootloaders are using them. And IIRC, it's Linux specific.

Although, "linux,uefi-*" are not well defined (only used internally by
Linux betwen the EFI stub and the kernel) and we expect other OS to use
them in the future.

So I would prefer to the "linux," dropped for them.

> 
> 3. Dom0 gets grant table and event channel irq information
> -----------------------------------------------------------
> Make Linux call xlated_setup_gnttab_pages() to setup grant table. So it
> doesn't need Xen pass grant table start and size information to Dom0.

The design doc should be in general OS agnostic. I would say here:

The OS will have to find a place himself in the memory map for the grant
table region.

For instance, Linux can make usage of xlated_setup_gnttab_pages.

> To event channel interrupt, reuse HVM_PARAM_CALLBACK_IRQ and add a new
> delivery type to get it.
> val[63:56] == 3: val[15:8] is flag: val[7:0] is a PPI (ARM and ARM64
> only)
> The definition of flag reusing the definition of xenv table. Bit 0
> stands interrupt mode and bit 1 stands interrupt polarity.

Either give a link to the XENV table or explain what means the value in
each bit (i.e what 0 and 1 stands for?).

I would prefer the later.

> As said above, we assign the hypervisor_id be "XenVMM" to tell Dom0 that
> it runs on Xen hypervisor. Then Dom0 could get it through hypercall
> HVMOP_get_param.

Regards,

[1] http://lists.xen.org/archives/html/xen-devel/2015-08/msg01074.html

-- 
Julien Grall

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Konrad Rzeszutek Wilk]

On Fri, Aug 14, 2015 at 10:59:19PM +0800, Shannon Zhao wrote:
> This document is going to explain the design details of Xen booting with
> ACPI on ARM. Maybe parts of it may not be appropriate. Any comments are
> welcome.
> 
> Changes v2->v3:
> * remove the two HVM_PARAMs for grant table and let linux kernel use
> xlated_setup_gnttab_pages() to setup grant table.
> * don't modify GTDT table
> * add definition of event-channel interrupt flag
> * state that route all Xen unused interrupt to Dom0
> * state that reusing existing PCI bus_notifier for PCI devices MMIO mapping
> 
> To Xen itself booting with ACPI, this is similar to Linux kernel except
> that Xen doesn't parse DSDT table. So I'll skip this part and focus on
> how Xen prepares ACPI tables for Dom0 and how Xen passes them to Dom0.
> 
> 1. Copy and change some EFI and ACPI tables
> -------------------------------------------
> a) Copy EFI_SYSTEM_TABLE and change the value of FirmwareVendor,
> VendorGuid, VendorTable, ConfigurationTable. These changes are not very
> special and it just assign values to these members.
> 
> b) Create EFI_MEMORY_DESCRIPTOR table. This will add memory start and
> size information of Dom0. And Dom0 will get the memory information
> through this EFI table.
> 
> c) Copy FADT table. Change the value of arm_boot_flags to enable PSCI
> and HVC. Let the hypervisor_id be "XenVMM" in order to tell Dom0 that it
> runs on Xen hypervisor, then Dom0 could through HVM_PARAM to get some
> informations for booting necessity, such as event-channel interrupt.
> Change header revison, length and checksum as well.
> 
> d) Copy MADT table. According to the value of dom0_max_vcpus to change
> the number GICC entries.
> 
> e) Create STAO table. This table is a new added one that's used to
> define a list of ACPI namespace names that are to be ignored by the OSPM
> in Dom0. Currently we use it to tell OSPM should ignore UART defined in
> SPCR table.


Would it make sense to include this URL
http://wiki.xenproject.org/mediawiki/images/0/02/Status-override-table.pdf


> 
> f) Copy XSDT table. Add a new table entry for STAO and change other
> table's entries.
> 
> g) Change the value of xsdt_physical_address in RSDP table.
> 
> h) The rest of tables are not copied or changed. They are reused
> including DSDT, SPCR, GTDT, etc.
> 
> All these tables will be copied to Dom0 memory except that the reused
> tables(DSDT, SPCR, GTDT, etc) will be mapped to Dom0.
> 
> 2. Create minimal DT to pass required information to Dom0
> ----------------------------------------------------------
> The minimal DT mainly passes Dom0 bootargs, address and size of initrd
> (if available), address and size of uefi system table, address and size
> of uefi memory table, uefi-mmap-desc-size and uefi-mmap-desc-ver.
> 
> An example of the minimal DT:
> / {
>     #address-cells = <2>;
>     #size-cells = <1>;
>     chosen {
>         bootargs = "kernel=Image console=hvc0 earlycon=pl011,0x1c090000
> root=/dev/vda2 rw rootfstype=ext4 init=/bin/sh acpi=force";
>         linux,initrd-start = <0xXXXXXXXX>;
>         linux,initrd-end = <0xXXXXXXXX>;
>         linux,uefi-system-table = <0xXXXXXXXX>;
>         linux,uefi-mmap-start = <0xXXXXXXXX>;
>         linux,uefi-mmap-size = <0xXXXXXXXX>;
>         linux,uefi-mmap-desc-size = <0xXXXXXXXX>;
>         linux,uefi-mmap-desc-ver = <0xXXXXXXXX>;
>     };
> };
> 
> For details loook at
> https://github.com/torvalds/linux/blob/master/Documentation/arm/uefi.txt
> 
> 3. Dom0 gets grant table and event channel irq information
> -----------------------------------------------------------
> Make Linux call xlated_setup_gnttab_pages() to setup grant table. So it
> doesn't need Xen pass grant table start and size information to Dom0.
> 
> To event channel interrupt, reuse HVM_PARAM_CALLBACK_IRQ and add a new
> delivery type to get it.
> val[63:56] == 3: val[15:8] is flag: val[7:0] is a PPI (ARM and ARM64
> only)
> The definition of flag reusing the definition of xenv table. Bit 0
> stands interrupt mode and bit 1 stands interrupt polarity.
> 
> As said above, we assign the hypervisor_id be "XenVMM" to tell Dom0 that
> it runs on Xen hypervisor. Then Dom0 could get it through hypercall
> HVMOP_get_param.
> 
> 4. Map MMIO regions
> -------------------
> Register a bus_notifier for platform and amba bus in Linux. Add a new
> XENMAPSPACE "XENMAPSPACE_dev_mmio". Within the register, check if the
> device is newly added, then call hypercall XENMEM_add_to_physmap to map
> the mmio regions.
> 
> For PCI bus device, it could reuse the existing PCI bus_notifier like
> X86.
> 
> 5. Route device interrupts to Dom0
> ----------------------------------
> Route all the SPI interrupts to Dom0 before Dom0 booting, except the
> interrupts used by Xen. For uart, it could get the interrupt from SPCR
> table and exclude it. For SMMU, it could get the interrupts from IORT
> table and exclude them as well.
> 
> Thanks,
> -- 
> Shannon
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Shannon Zhao]

On 2015/8/14 23:17, Julien Grall wrote:
> On 14/08/15 15:59, Shannon Zhao wrote:
>> 2. Create minimal DT to pass required information to Dom0
>> ----------------------------------------------------------
>> The minimal DT mainly passes Dom0 bootargs, address and size of initrd
>> (if available), address and size of uefi system table, address and size
>> of uefi memory table, uefi-mmap-desc-size and uefi-mmap-desc-ver.
>>
>> An example of the minimal DT:
>> / {
>>      #address-cells = <2>;
>>      #size-cells = <1>;
>>      chosen {
>>          bootargs = "kernel=Image console=hvc0 earlycon=pl011,0x1c090000
>> root=/dev/vda2 rw rootfstype=ext4 init=/bin/sh acpi=force";
>>          linux,initrd-start = <0xXXXXXXXX>;
>>          linux,initrd-end = <0xXXXXXXXX>;
>>          linux,uefi-system-table = <0xXXXXXXXX>;
>>          linux,uefi-mmap-start = <0xXXXXXXXX>;
>>          linux,uefi-mmap-size = <0xXXXXXXXX>;
>>          linux,uefi-mmap-desc-size = <0xXXXXXXXX>;
>>          linux,uefi-mmap-desc-ver = <0xXXXXXXXX>;
>>      };
>> };
>>
>> For details loook at
>> https://github.com/torvalds/linux/blob/master/Documentation/arm/uefi.txt
>
> I would have expect a summary on the discussion we had on the previous
> thread [1].
>
> Note that "linux,initrd-*" are well defined given that Xen, U-boot and
> other bootloaders are using them. And IIRC, it's Linux specific.
>
> Although, "linux,uefi-*" are not well defined (only used internally by
> Linux betwen the EFI stub and the kernel) and we expect other OS to use
> them in the future.
>
> So I would prefer to the "linux," dropped for them.
>

Yes, I think it's good to drop the "linux," too. But if we drop the 
linux, would it impact the linux kernel booting with UEFI? And why we 
don't do it to Xen since Xen still uses "linux,"?

Thanks,
-- 
Shannon

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Jan Beulich]

>>> On 14.08.15 at 16:59, <shannon.zhao@linaro.org> wrote:
> 1. Copy and change some EFI and ACPI tables
> -------------------------------------------

While some explanation on the reasons for this was given in the past
discussion, I'm still missing a word on the "why" for each of these here.

> a) Copy EFI_SYSTEM_TABLE and change the value of FirmwareVendor,
> VendorGuid, VendorTable, ConfigurationTable. These changes are not very
> special and it just assign values to these members.

I continue to question the need for fiddling with this table, not the
least because I don't see how you intend to hand it to the Dom0
kernel: Are you expecting to call the kernel's ordinary EFI entry
point, despite it not itself running on EFI firmware?

> b) Create EFI_MEMORY_DESCRIPTOR table. This will add memory start and
> size information of Dom0. And Dom0 will get the memory information
> through this EFI table.

To some degree the same applies here: While I see that you have no
legacy vehicle like x86's E820, I also don't see how Dom0 - not being
able to make EFI boot or runtime services calls - would get hold of this
table. And if a non-EFI mechanism is to be used here, using the EFI
data structure would turn out to be just an arbitrary (or convenience)
decision, not something inherently required. Which I think should be
said explicitly if so, rather than leaving the reader guess.

> c) Copy FADT table. Change the value of arm_boot_flags to enable PSCI
> and HVC. Let the hypervisor_id be "XenVMM" in order to tell Dom0 that it
> runs on Xen hypervisor, then Dom0 could through HVM_PARAM to get some
> informations for booting necessity, such as event-channel interrupt.
> Change header revison, length and checksum as well.
> 
> d) Copy MADT table. According to the value of dom0_max_vcpus to change
> the number GICC entries.
> 
> e) Create STAO table. This table is a new added one that's used to
> define a list of ACPI namespace names that are to be ignored by the OSPM
> in Dom0. Currently we use it to tell OSPM should ignore UART defined in
> SPCR table.
> 
> f) Copy XSDT table. Add a new table entry for STAO and change other
> table's entries.
> 
> g) Change the value of xsdt_physical_address in RSDP table.

Which RSDP? Under EFI the table root is to be found from the
EFI Configuration Table.

Jan

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Julien Grall]

Hi,

On 17/08/2015 06:01, Shannon Zhao wrote:
>
>
> On 2015/8/14 23:17, Julien Grall wrote:
>> On 14/08/15 15:59, Shannon Zhao wrote:
>>> 2. Create minimal DT to pass required information to Dom0
>>> ----------------------------------------------------------
>>> The minimal DT mainly passes Dom0 bootargs, address and size of initrd
>>> (if available), address and size of uefi system table, address and size
>>> of uefi memory table, uefi-mmap-desc-size and uefi-mmap-desc-ver.
>>>
>>> An example of the minimal DT:
>>> / {
>>>      #address-cells = <2>;
>>>      #size-cells = <1>;
>>>      chosen {
>>>          bootargs = "kernel=Image console=hvc0 earlycon=pl011,0x1c090000
>>> root=/dev/vda2 rw rootfstype=ext4 init=/bin/sh acpi=force";
>>>          linux,initrd-start = <0xXXXXXXXX>;
>>>          linux,initrd-end = <0xXXXXXXXX>;
>>>          linux,uefi-system-table = <0xXXXXXXXX>;
>>>          linux,uefi-mmap-start = <0xXXXXXXXX>;
>>>          linux,uefi-mmap-size = <0xXXXXXXXX>;
>>>          linux,uefi-mmap-desc-size = <0xXXXXXXXX>;
>>>          linux,uefi-mmap-desc-ver = <0xXXXXXXXX>;
>>>      };
>>> };
>>>
>>> For details loook at
>>> https://github.com/torvalds/linux/blob/master/Documentation/arm/uefi.txt
>>
>> I would have expect a summary on the discussion we had on the previous
>> thread [1].
>>
>> Note that "linux,initrd-*" are well defined given that Xen, U-boot and
>> other bootloaders are using them. And IIRC, it's Linux specific.
>>
>> Although, "linux,uefi-*" are not well defined (only used internally by
>> Linux betwen the EFI stub and the kernel) and we expect other OS to use
>> them in the future.
>>
>> So I would prefer to the "linux," dropped for them.
>>
>
> Yes, I think it's good to drop the "linux," too. But if we drop the
> linux, would it impact the linux kernel booting with UEFI? And why we
> don't do it to Xen since Xen still uses "linux,"?

I don't understand your second question.

For the first question, as we discussed in several mail, the property 
"linux,uefi-*" are only used internally between the stub and Linux. The 
sub is compiled in the kernel so there is no issue to change the property.

Regards,

-- 
Julien Grall

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Julien Grall]

On 17/08/2015 08:33, Jan Beulich wrote:
>>>> On 14.08.15 at 16:59, <shannon.zhao@linaro.org> wrote:
>> 1. Copy and change some EFI and ACPI tables
>> -------------------------------------------
>
> While some explanation on the reasons for this was given in the past
> discussion, I'm still missing a word on the "why" for each of these here.
>
>> a) Copy EFI_SYSTEM_TABLE and change the value of FirmwareVendor,
>> VendorGuid, VendorTable, ConfigurationTable. These changes are not very
>> special and it just assign values to these members.
>
> I continue to question the need for fiddling with this table, not the
> least because I don't see how you intend to hand it to the Dom0
> kernel: Are you expecting to call the kernel's ordinary EFI entry
> point, despite it not itself running on EFI firmware?
>
>> b) Create EFI_MEMORY_DESCRIPTOR table. This will add memory start and
>> size information of Dom0. And Dom0 will get the memory information
>> through this EFI table.
>
> To some degree the same applies here: While I see that you have no
> legacy vehicle like x86's E820, I also don't see how Dom0 - not being
> able to make EFI boot or runtime services calls - would get hold of this
> table. And if a non-EFI mechanism is to be used here, using the EFI
> data structure would turn out to be just an arbitrary (or convenience)
> decision, not something inherently required. Which I think should be
> said explicitly if so, rather than leaving the reader guess.

It's not an arbitrary decision, when UEFI stub in Linux is using device 
tree properties to pass the UEFI table to the kernel ([1]).

When booting on Xen with ACPI, dom0 will use the non-EFI entry point. 
The easiest way to pass the memory information to Linux is using the 
UEFI DT properties.

Regards,

[1] https://github.com/torvalds/linux/blob/master/Documentation/arm/uefi.txt


-- 
Julien Grall

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Shannon Zhao]

Hi Julien,

On 2015/8/18 0:10, Julien Grall wrote:
> Hi,
> 
> On 17/08/2015 06:01, Shannon Zhao wrote:
>>
>>
>> On 2015/8/14 23:17, Julien Grall wrote:
>>> On 14/08/15 15:59, Shannon Zhao wrote:
>>>> 2. Create minimal DT to pass required information to Dom0
>>>> ----------------------------------------------------------
>>>> The minimal DT mainly passes Dom0 bootargs, address and size of initrd
>>>> (if available), address and size of uefi system table, address and size
>>>> of uefi memory table, uefi-mmap-desc-size and uefi-mmap-desc-ver.
>>>>
>>>> An example of the minimal DT:
>>>> / {
>>>>      #address-cells = <2>;
>>>>      #size-cells = <1>;
>>>>      chosen {
>>>>          bootargs = "kernel=Image console=hvc0
>>>> earlycon=pl011,0x1c090000
>>>> root=/dev/vda2 rw rootfstype=ext4 init=/bin/sh acpi=force";
>>>>          linux,initrd-start = <0xXXXXXXXX>;
>>>>          linux,initrd-end = <0xXXXXXXXX>;
>>>>          linux,uefi-system-table = <0xXXXXXXXX>;
>>>>          linux,uefi-mmap-start = <0xXXXXXXXX>;
>>>>          linux,uefi-mmap-size = <0xXXXXXXXX>;
>>>>          linux,uefi-mmap-desc-size = <0xXXXXXXXX>;
>>>>          linux,uefi-mmap-desc-ver = <0xXXXXXXXX>;
>>>>      };
>>>> };
>>>>
>>>> For details loook at
>>>> https://github.com/torvalds/linux/blob/master/Documentation/arm/uefi.txt
>>>>
>>>
>>> I would have expect a summary on the discussion we had on the previous
>>> thread [1].
>>>
>>> Note that "linux,initrd-*" are well defined given that Xen, U-boot and
>>> other bootloaders are using them. And IIRC, it's Linux specific.
>>>
>>> Although, "linux,uefi-*" are not well defined (only used internally by
>>> Linux betwen the EFI stub and the kernel) and we expect other OS to use
>>> them in the future.
>>>
>>> So I would prefer to the "linux," dropped for them.
>>>
>>
>> Yes, I think it's good to drop the "linux," too. But if we drop the
>> linux, would it impact the linux kernel booting with UEFI? And why we
>> don't do it to Xen since Xen still uses "linux,"?
> 
> I don't understand your second question.
> 
I mean that Xen is using the property "linux,uefi*" as well, and why we
don't drop that prefix for Xen?

> For the first question, as we discussed in several mail, the property
> "linux,uefi-*" are only used internally between the stub and Linux. The
> sub is compiled in the kernel so there is no issue to change the property.
> 
Since Linux defines the dt_params like below which is used to get EFI
info from DT, if we drop "linux," in Xen, does it need to drop the
"linux," in dt_params? If so, will this break the compatibility of
changed kernel with old UEFI? IIUC, there is not only Xen using these
properties, but also native host and QEMU guest.

static __initdata struct {
	const char name[32];
	const char propname[32];
	int offset;
	int size;
} dt_params[] = {
	UEFI_PARAM("System Table", "linux,uefi-system-table", system_table),
	UEFI_PARAM("MemMap Address", "linux,uefi-mmap-start", mmap),
	UEFI_PARAM("MemMap Size", "linux,uefi-mmap-size", mmap_size),
	UEFI_PARAM("MemMap Desc. Size", "linux,uefi-mmap-desc-size", desc_size),
	UEFI_PARAM("MemMap Desc. Version", "linux,uefi-mmap-desc-ver", desc_ver)
};

Thanks,
-- 
Shannon

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Shannon Zhao]

Hi Jan,

On 2015/8/17 23:33, Jan Beulich wrote:
>>>> On 14.08.15 at 16:59, <shannon.zhao@linaro.org> wrote:
>> 1. Copy and change some EFI and ACPI tables
>> -------------------------------------------
> 
> While some explanation on the reasons for this was given in the past
> discussion, I'm still missing a word on the "why" for each of these here.
> 
>> a) Copy EFI_SYSTEM_TABLE and change the value of FirmwareVendor,
>> VendorGuid, VendorTable, ConfigurationTable. These changes are not very
>> special and it just assign values to these members.
> 
> I continue to question the need for fiddling with this table, not the
> least because I don't see how you intend to hand it to the Dom0
> kernel: Are you expecting to call the kernel's ordinary EFI entry
> point, despite it not itself running on EFI firmware?
> 
Dom0 gets UEFI info from the minimal DT when booting with UEFI.
And the main purpose to create a EFI_SYSTEM_TABLE is to provide the ACPI
table root (RSDP) address to Dom0, so it could find the ACPI table.

Here since we don't support RUNTIME service fro Dom0 currently, we could
set the Attribute of EFI_SYSTEM_TABLE memory region to not be
EFI_MEMORY_RUNTIME or pass kernel command parameter "efi=noruntime" to
disable RUNTIME service.

>> b) Create EFI_MEMORY_DESCRIPTOR table. This will add memory start and
>> size information of Dom0. And Dom0 will get the memory information
>> through this EFI table.
> 
> To some degree the same applies here: While I see that you have no
> legacy vehicle like x86's E820, I also don't see how Dom0 - not being
> able to make EFI boot or runtime services calls - would get hold of this
> table. And if a non-EFI mechanism is to be used here, using the EFI
> data structure would turn out to be just an arbitrary (or convenience)
> decision, not something inherently required. Which I think should be
> said explicitly if so, rather than leaving the reader guess.
> 
>> c) Copy FADT table. Change the value of arm_boot_flags to enable PSCI
>> and HVC. Let the hypervisor_id be "XenVMM" in order to tell Dom0 that it
>> runs on Xen hypervisor, then Dom0 could through HVM_PARAM to get some
>> informations for booting necessity, such as event-channel interrupt.
>> Change header revison, length and checksum as well.
>>
>> d) Copy MADT table. According to the value of dom0_max_vcpus to change
>> the number GICC entries.
>>
>> e) Create STAO table. This table is a new added one that's used to
>> define a list of ACPI namespace names that are to be ignored by the OSPM
>> in Dom0. Currently we use it to tell OSPM should ignore UART defined in
>> SPCR table.
>>
>> f) Copy XSDT table. Add a new table entry for STAO and change other
>> table's entries.
>>
>> g) Change the value of xsdt_physical_address in RSDP table.
> 
> Which RSDP? Under EFI the table root is to be found from the
> EFI Configuration Table.
> 
Yes, the RSDP address is stored in EFI Configure Table. And RSDP table
has a field "xsdt_physical_address" that points to the XSDT table. As we
create a new XSDT and the address of XSDT is changed, so it needs to
update the value of "xsdt_physical_address" in RSDP. So Dom0 could get
the right XSDT table rather than the old one.

Thanks,
-- 
Shannon

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Jan Beulich]

>>> Julien Grall <julien.grall@citrix.com> 08/17/15 6:27 PM >>>
>On 17/08/2015 08:33, Jan Beulich wrote:
>>>>> On 14.08.15 at 16:59, <shannon.zhao@linaro.org> wrote:
>>> b) Create EFI_MEMORY_DESCRIPTOR table. This will add memory start and
>>> size information of Dom0. And Dom0 will get the memory information
>>> through this EFI table.
>>
>> To some degree the same applies here: While I see that you have no
>> legacy vehicle like x86's E820, I also don't see how Dom0 - not being
>> able to make EFI boot or runtime services calls - would get hold of this
>> table. And if a non-EFI mechanism is to be used here, using the EFI
>> data structure would turn out to be just an arbitrary (or convenience)
>> decision, not something inherently required. Which I think should be
>> said explicitly if so, rather than leaving the reader guess.
>
>It's not an arbitrary decision, when UEFI stub in Linux is using device 
>tree properties to pass the UEFI table to the kernel ([1]).
>
>When booting on Xen with ACPI, dom0 will use the non-EFI entry point. 
>The easiest way to pass the memory information to Linux is using the 
>UEFI DT properties.

In which case it is even more arbitrary to use the EFI data structure to
convey memory information (instead of expressing it in plain DT, which is
how I blindly assume non-EFI does it). Of course there's the small chance
that "UEFI DT properties" implies a certain binary format, but it's still odd
for a non-EFI entry point to assume EFI properties to be there...

Jan

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Jan Beulich]

>>> Shannon Zhao <zhaoshenglong@huawei.com> 08/18/15 5:46 AM >>>
>On 2015/8/17 23:33, Jan Beulich wrote:
>>>>> On 14.08.15 at 16:59, <shannon.zhao@linaro.org> wrote:
>>> 1. Copy and change some EFI and ACPI tables
>>> -------------------------------------------
>> 
>> While some explanation on the reasons for this was given in the past
>> discussion, I'm still missing a word on the "why" for each of these here.
>> 
>>> a) Copy EFI_SYSTEM_TABLE and change the value of FirmwareVendor,
>>> VendorGuid, VendorTable, ConfigurationTable. These changes are not very
>>> special and it just assign values to these members.
>> 
>> I continue to question the need for fiddling with this table, not the
>> least because I don't see how you intend to hand it to the Dom0
>> kernel: Are you expecting to call the kernel's ordinary EFI entry
>> point, despite it not itself running on EFI firmware?
>> 
>Dom0 gets UEFI info from the minimal DT when booting with UEFI.
>And the main purpose to create a EFI_SYSTEM_TABLE is to provide the ACPI
>table root (RSDP) address to Dom0, so it could find the ACPI table.

For that passing the configuration table would suffice. The more that Julien
in his reply said you're not even intending to use the kernel's EFI stub. I.e.
the question remains: How would you expect to hand the System Table
pointer to Dom0?

>Here since we don't support RUNTIME service fro Dom0 currently, we could
>set the Attribute of EFI_SYSTEM_TABLE memory region to not be
>EFI_MEMORY_RUNTIME or pass kernel command parameter "efi=noruntime" to
>disable RUNTIME service.

I don't see how this part of your reply is related.

>>> g) Change the value of xsdt_physical_address in RSDP table.
>> 
>> Which RSDP? Under EFI the table root is to be found from the
>> EFI Configuration Table.
>> 
>Yes, the RSDP address is stored in EFI Configure Table. And RSDP table
>has a field "xsdt_physical_address" that points to the XSDT table. As we
>create a new XSDT and the address of XSDT is changed, so it needs to
>update the value of "xsdt_physical_address" in RSDP. So Dom0 could get
>the right XSDT table rather than the old one.

Oh, sorry, I mixed things up.

Jan

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Julien Grall]

On 17/08/2015 20:19, Shannon Zhao wrote:
>>> Yes, I think it's good to drop the "linux," too. But if we drop the
>>> linux, would it impact the linux kernel booting with UEFI? And why we
>>> don't do it to Xen since Xen still uses "linux,"?
>>
>> I don't understand your second question.
>>
> I mean that Xen is using the property "linux,uefi*" as well, and why we
> don't drop that prefix for Xen?

As never say we shouldn't drop it in Xen... It's of course a nice clean 
up to have if we ever happen to standardize the properties with a 
different name.

>> For the first question, as we discussed in several mail, the property
>> "linux,uefi-*" are only used internally between the stub and Linux. The
>> sub is compiled in the kernel so there is no issue to change the property.
>>
> Since Linux defines the dt_params like below which is used to get EFI
> info from DT, if we drop "linux," in Xen, does it need to drop the
> "linux," in dt_params? If so, will this break the compatibility of
> changed kernel with old UEFI? IIUC, there is not only Xen using these
> properties, but also native host and QEMU guest.

I grepped "linux," and didn't spot any "linux,uefi-*" strings.

Anyway, why are you speaking about old UEFI? As said in different mail, 
the linux,uefi-* properties are only used internally between the EFI 
stub and the kernel. Both are living in the same binary so it's not 
exposed outside.

Those properties are not standardize so it would be wrong to use them to 
talk to the kernel.

Note that on Xen, we also used them internally. They were name 
"linux,uefi-*" because we re-use a part of the EFI stub from Linux. The 
names don't matter, so we can rename it without any issue

Regards,

-- 
Julien Grall

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Julien Grall]

Hi Jan,

On 17/08/2015 22:10, Jan Beulich wrote:
>>>> Julien Grall <julien.grall@citrix.com> 08/17/15 6:27 PM >>>
>> On 17/08/2015 08:33, Jan Beulich wrote:
>>>>>> On 14.08.15 at 16:59, <shannon.zhao@linaro.org> wrote:
>>>> b) Create EFI_MEMORY_DESCRIPTOR table. This will add memory start and
>>>> size information of Dom0. And Dom0 will get the memory information
>>>> through this EFI table.
>>>
>>> To some degree the same applies here: While I see that you have no
>>> legacy vehicle like x86's E820, I also don't see how Dom0 - not being
>>> able to make EFI boot or runtime services calls - would get hold of this
>>> table. And if a non-EFI mechanism is to be used here, using the EFI
>>> data structure would turn out to be just an arbitrary (or convenience)
>>> decision, not something inherently required. Which I think should be
>>> said explicitly if so, rather than leaving the reader guess.
>>
>> It's not an arbitrary decision, when UEFI stub in Linux is using device
>> tree properties to pass the UEFI table to the kernel ([1]).
>>
>> When booting on Xen with ACPI, dom0 will use the non-EFI entry point.
>> The easiest way to pass the memory information to Linux is using the
>> UEFI DT properties.
>
> In which case it is even more arbitrary to use the EFI data structure to
> convey memory information (instead of expressing it in plain DT, which is
> how I blindly assume non-EFI does it). Of course there's the small chance
> that "UEFI DT properties" implies a certain binary format, but it's still odd
> for a non-EFI entry point to assume EFI properties to be there...

Linux is able to boot either on ACPI or DT. When ACPI is used, the EFI 
stub (mandatory) will create a small DT in order to pass the command 
line and other informations (such as the EFI memory table) to the 
kernel. This is because the stub is self-contained and and never use 
variable living in the kernel.

In order to know whether you are booting using DT or ACPI, they check if 
the DT contains only the /chosen node.

Actually the EFI-stub will always jumped on the non-EFI path.

By passing the minimal DT as suggested on the design doc, we are 
avoiding to get a different code path for Xen in the kernel and we are 
finally acting as the EFI-stub was a nop for DOM0.

So I don't see how this would be arbitrary...

Regards,

-- 
Julien Grall

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Jan Beulich]

>>> On 18.08.15 at 08:43, <julien.grall@citrix.com> wrote:
> Hi Jan,
> 
> On 17/08/2015 22:10, Jan Beulich wrote:
>>>>> Julien Grall <julien.grall@citrix.com> 08/17/15 6:27 PM >>>
>>> On 17/08/2015 08:33, Jan Beulich wrote:
>>>>>>> On 14.08.15 at 16:59, <shannon.zhao@linaro.org> wrote:
>>>>> b) Create EFI_MEMORY_DESCRIPTOR table. This will add memory start and
>>>>> size information of Dom0. And Dom0 will get the memory information
>>>>> through this EFI table.
>>>>
>>>> To some degree the same applies here: While I see that you have no
>>>> legacy vehicle like x86's E820, I also don't see how Dom0 - not being
>>>> able to make EFI boot or runtime services calls - would get hold of this
>>>> table. And if a non-EFI mechanism is to be used here, using the EFI
>>>> data structure would turn out to be just an arbitrary (or convenience)
>>>> decision, not something inherently required. Which I think should be
>>>> said explicitly if so, rather than leaving the reader guess.
>>>
>>> It's not an arbitrary decision, when UEFI stub in Linux is using device
>>> tree properties to pass the UEFI table to the kernel ([1]).
>>>
>>> When booting on Xen with ACPI, dom0 will use the non-EFI entry point.
>>> The easiest way to pass the memory information to Linux is using the
>>> UEFI DT properties.
>>
>> In which case it is even more arbitrary to use the EFI data structure to
>> convey memory information (instead of expressing it in plain DT, which is
>> how I blindly assume non-EFI does it). Of course there's the small chance
>> that "UEFI DT properties" implies a certain binary format, but it's still 
> odd
>> for a non-EFI entry point to assume EFI properties to be there...
> 
> Linux is able to boot either on ACPI or DT. When ACPI is used, the EFI 
> stub (mandatory) will create a small DT in order to pass the command 
> line and other informations (such as the EFI memory table) to the 
> kernel. This is because the stub is self-contained and and never use 
> variable living in the kernel.
> 
> In order to know whether you are booting using DT or ACPI, they check if 
> the DT contains only the /chosen node.
> 
> Actually the EFI-stub will always jumped on the non-EFI path.
> 
> By passing the minimal DT as suggested on the design doc, we are 
> avoiding to get a different code path for Xen in the kernel and we are 
> finally acting as the EFI-stub was a nop for DOM0.
> 
> So I don't see how this would be arbitrary...

Indeed with your explanation this doesn't look arbitrary anymore.
But none of this was spelled out in the original document.

Jan

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Shannon Zhao]

On 2015/8/18 14:36, Julien Grall wrote:
> 
> 
> On 17/08/2015 20:19, Shannon Zhao wrote:
>>>> Yes, I think it's good to drop the "linux," too. But if we drop the
>>>> linux, would it impact the linux kernel booting with UEFI? And why we
>>>> don't do it to Xen since Xen still uses "linux,"?
>>>
>>> I don't understand your second question.
>>>
>> I mean that Xen is using the property "linux,uefi*" as well, and why we
>> don't drop that prefix for Xen?
> 
> As never say we shouldn't drop it in Xen... It's of course a nice clean
> up to have if we ever happen to standardize the properties with a
> different name.
> 
>>> For the first question, as we discussed in several mail, the property
>>> "linux,uefi-*" are only used internally between the stub and Linux. The
>>> sub is compiled in the kernel so there is no issue to change the
>>> property.
>>>
>> Since Linux defines the dt_params like below which is used to get EFI
>> info from DT, if we drop "linux," in Xen, does it need to drop the
>> "linux," in dt_params? If so, will this break the compatibility of
>> changed kernel with old UEFI? IIUC, there is not only Xen using these
>> properties, but also native host and QEMU guest.
> 
> I grepped "linux," and didn't spot any "linux,uefi-*" strings.
> 

In drivers/firmware/efi/efi.c line 478


static __initdata struct {
	const char name[32];
	const char propname[32];
	int offset;
	int size;
} dt_params[] = {
	UEFI_PARAM("System Table", "linux,uefi-system-table", system_table),
	UEFI_PARAM("MemMap Address", "linux,uefi-mmap-start", mmap),
	UEFI_PARAM("MemMap Size", "linux,uefi-mmap-size", mmap_size),
	UEFI_PARAM("MemMap Desc. Size", "linux,uefi-mmap-desc-size", desc_size),
	UEFI_PARAM("MemMap Desc. Version", "linux,uefi-mmap-desc-ver", desc_ver)
};

> Anyway, why are you speaking about old UEFI? As said in different mail,
> the linux,uefi-* properties are only used internally between the EFI
> stub and the kernel. Both are living in the same binary so it's not
> exposed outside.
> 

UEFI makes this minimal DT, right? To Dom0, Xen makes this minimal DT,
right? And EFI stub parses this DT through efi_get_fdt_params ==>
fdt_find_uefi_params in drivers/firmware/efi/efi.c. And
fdt_find_uefi_params uses dt_params[i].propname (e.g.
"linux,uefi-system-table") to get the matched property.

"prop = of_get_flat_dt_prop(node, dt_params[i].propname, &len);"

If we changed the property names in minimal DT but not change the
definition of dt_params[], it can't get the matched properties, right?
And if we both changed the property name in minimal DT and definition of
dt_params[], will this new kernel work well with the old UEFI which
still uses old property names to create the minimal DT?

> Those properties are not standardize so it would be wrong to use them to
> talk to the kernel.
> 
> Note that on Xen, we also used them internally. They were name
> "linux,uefi-*" because we re-use a part of the EFI stub from Linux. The
> names don't matter, so we can rename it without any issue
> 
> Regards,
> 

-- 
Shannon

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Shannon Zhao]

Hi Jan,

On 2015/8/18 13:14, Jan Beulich wrote:
>>>> Shannon Zhao <zhaoshenglong@huawei.com> 08/18/15 5:46 AM >>>
>> On 2015/8/17 23:33, Jan Beulich wrote:
>>>>>> On 14.08.15 at 16:59, <shannon.zhao@linaro.org> wrote:
>>>> 1. Copy and change some EFI and ACPI tables
>>>> -------------------------------------------
>>>
>>> While some explanation on the reasons for this was given in the past
>>> discussion, I'm still missing a word on the "why" for each of these here.
>>>
>>>> a) Copy EFI_SYSTEM_TABLE and change the value of FirmwareVendor,
>>>> VendorGuid, VendorTable, ConfigurationTable. These changes are not very
>>>> special and it just assign values to these members.
>>>
>>> I continue to question the need for fiddling with this table, not the
>>> least because I don't see how you intend to hand it to the Dom0
>>> kernel: Are you expecting to call the kernel's ordinary EFI entry
>>> point, despite it not itself running on EFI firmware?
>>>
>> Dom0 gets UEFI info from the minimal DT when booting with UEFI.
>> And the main purpose to create a EFI_SYSTEM_TABLE is to provide the ACPI
>> table root (RSDP) address to Dom0, so it could find the ACPI table.
> 
> For that passing the configuration table would suffice. The more that Julien
> in his reply said you're not even intending to use the kernel's EFI stub. I.e.
> the question remains: How would you expect to hand the System Table
> pointer to Dom0?
> 

Oh, I realize that there is some wrong expression about
EFI_SYSTEM_TABLE. We don't copy all the EFI System Table, but only the
EFI_SYSTEM_TABLE->Hdr. And assign the value of FirmwareVendor, set
NumberOfTableEntries to 1. Create only one EFI Configuration Table to
store Root table address. The other fields of EFI System Table stay
zero. So Dom0 only gets the ACPI table through System Table, others will
be invalid.

-- 
Shannon

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Shannon Zhao]

On 2015/8/18 15:01, Jan Beulich wrote:
>>>> On 18.08.15 at 08:43, <julien.grall@citrix.com> wrote:
>> > Hi Jan,
>> > 
>> > On 17/08/2015 22:10, Jan Beulich wrote:
>>>>>> >>>>> Julien Grall <julien.grall@citrix.com> 08/17/15 6:27 PM >>>
>>>> >>> On 17/08/2015 08:33, Jan Beulich wrote:
>>>>>>>> >>>>>>> On 14.08.15 at 16:59, <shannon.zhao@linaro.org> wrote:
>>>>>> >>>>> b) Create EFI_MEMORY_DESCRIPTOR table. This will add memory start and
>>>>>> >>>>> size information of Dom0. And Dom0 will get the memory information
>>>>>> >>>>> through this EFI table.
>>>>> >>>>
>>>>> >>>> To some degree the same applies here: While I see that you have no
>>>>> >>>> legacy vehicle like x86's E820, I also don't see how Dom0 - not being
>>>>> >>>> able to make EFI boot or runtime services calls - would get hold of this
>>>>> >>>> table. And if a non-EFI mechanism is to be used here, using the EFI
>>>>> >>>> data structure would turn out to be just an arbitrary (or convenience)
>>>>> >>>> decision, not something inherently required. Which I think should be
>>>>> >>>> said explicitly if so, rather than leaving the reader guess.
>>>> >>>
>>>> >>> It's not an arbitrary decision, when UEFI stub in Linux is using device
>>>> >>> tree properties to pass the UEFI table to the kernel ([1]).
>>>> >>>
>>>> >>> When booting on Xen with ACPI, dom0 will use the non-EFI entry point.
>>>> >>> The easiest way to pass the memory information to Linux is using the
>>>> >>> UEFI DT properties.
>>> >>
>>> >> In which case it is even more arbitrary to use the EFI data structure to
>>> >> convey memory information (instead of expressing it in plain DT, which is
>>> >> how I blindly assume non-EFI does it). Of course there's the small chance
>>> >> that "UEFI DT properties" implies a certain binary format, but it's still 
>> > odd
>>> >> for a non-EFI entry point to assume EFI properties to be there...
>> > 
>> > Linux is able to boot either on ACPI or DT. When ACPI is used, the EFI 
>> > stub (mandatory) will create a small DT in order to pass the command 
>> > line and other informations (such as the EFI memory table) to the 
>> > kernel. This is because the stub is self-contained and and never use 
>> > variable living in the kernel.
>> > 
>> > In order to know whether you are booting using DT or ACPI, they check if 
>> > the DT contains only the /chosen node.
>> > 
>> > Actually the EFI-stub will always jumped on the non-EFI path.
>> > 
>> > By passing the minimal DT as suggested on the design doc, we are 
>> > avoiding to get a different code path for Xen in the kernel and we are 
>> > finally acting as the EFI-stub was a nop for DOM0.
>> > 
>> > So I don't see how this would be arbitrary...
> Indeed with your explanation this doesn't look arbitrary anymore.
> But none of this was spelled out in the original document.

Oh, sorry. Will add more details explanation in next version.

-- 
Shannon

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Jan Beulich]

>>> On 18.08.15 at 09:35, <zhaoshenglong@huawei.com> wrote:
> Hi Jan,
> 
> On 2015/8/18 13:14, Jan Beulich wrote:
>>>>> Shannon Zhao <zhaoshenglong@huawei.com> 08/18/15 5:46 AM >>>
>>> On 2015/8/17 23:33, Jan Beulich wrote:
>>>>>>> On 14.08.15 at 16:59, <shannon.zhao@linaro.org> wrote:
>>>>> 1. Copy and change some EFI and ACPI tables
>>>>> -------------------------------------------
>>>>
>>>> While some explanation on the reasons for this was given in the past
>>>> discussion, I'm still missing a word on the "why" for each of these here.
>>>>
>>>>> a) Copy EFI_SYSTEM_TABLE and change the value of FirmwareVendor,
>>>>> VendorGuid, VendorTable, ConfigurationTable. These changes are not very
>>>>> special and it just assign values to these members.
>>>>
>>>> I continue to question the need for fiddling with this table, not the
>>>> least because I don't see how you intend to hand it to the Dom0
>>>> kernel: Are you expecting to call the kernel's ordinary EFI entry
>>>> point, despite it not itself running on EFI firmware?
>>>>
>>> Dom0 gets UEFI info from the minimal DT when booting with UEFI.
>>> And the main purpose to create a EFI_SYSTEM_TABLE is to provide the ACPI
>>> table root (RSDP) address to Dom0, so it could find the ACPI table.
>> 
>> For that passing the configuration table would suffice. The more that Julien
>> in his reply said you're not even intending to use the kernel's EFI stub. 
> I.e.
>> the question remains: How would you expect to hand the System Table
>> pointer to Dom0?
>> 
> 
> Oh, I realize that there is some wrong expression about
> EFI_SYSTEM_TABLE. We don't copy all the EFI System Table, but only the
> EFI_SYSTEM_TABLE->Hdr. And assign the value of FirmwareVendor, set
> NumberOfTableEntries to 1. Create only one EFI Configuration Table to
> store Root table address. The other fields of EFI System Table stay
> zero. So Dom0 only gets the ACPI table through System Table, others will
> be invalid.

While this makes it look even more hacky, it doesn't answer the
question.

Jan

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Shannon Zhao]

On 2015/8/18 16:15, Jan Beulich wrote:
>>>> On 18.08.15 at 09:35, <zhaoshenglong@huawei.com> wrote:
>> Hi Jan,
>>
>> On 2015/8/18 13:14, Jan Beulich wrote:
>>>>>> Shannon Zhao <zhaoshenglong@huawei.com> 08/18/15 5:46 AM >>>
>>>> On 2015/8/17 23:33, Jan Beulich wrote:
>>>>>>>> On 14.08.15 at 16:59, <shannon.zhao@linaro.org> wrote:
>>>>>> 1. Copy and change some EFI and ACPI tables
>>>>>> -------------------------------------------
>>>>>
>>>>> While some explanation on the reasons for this was given in the past
>>>>> discussion, I'm still missing a word on the "why" for each of these here.
>>>>>
>>>>>> a) Copy EFI_SYSTEM_TABLE and change the value of FirmwareVendor,
>>>>>> VendorGuid, VendorTable, ConfigurationTable. These changes are not very
>>>>>> special and it just assign values to these members.
>>>>>
>>>>> I continue to question the need for fiddling with this table, not the
>>>>> least because I don't see how you intend to hand it to the Dom0
>>>>> kernel: Are you expecting to call the kernel's ordinary EFI entry
>>>>> point, despite it not itself running on EFI firmware?
>>>>>
>>>> Dom0 gets UEFI info from the minimal DT when booting with UEFI.
>>>> And the main purpose to create a EFI_SYSTEM_TABLE is to provide the ACPI
>>>> table root (RSDP) address to Dom0, so it could find the ACPI table.
>>>
>>> For that passing the configuration table would suffice. The more that Julien
>>> in his reply said you're not even intending to use the kernel's EFI stub. 
>> I.e.
>>> the question remains: How would you expect to hand the System Table
>>> pointer to Dom0?
>>>

The System Table pointer will be passed to Dom0 through the minimal DT
property "linux,uefi-system-table".

>>
>> Oh, I realize that there is some wrong expression about
>> EFI_SYSTEM_TABLE. We don't copy all the EFI System Table, but only the
>> EFI_SYSTEM_TABLE->Hdr. And assign the value of FirmwareVendor, set
>> NumberOfTableEntries to 1. Create only one EFI Configuration Table to
>> store Root table address. The other fields of EFI System Table stay
>> zero. So Dom0 only gets the ACPI table through System Table, others will
>> be invalid.
> 
> While this makes it look even more hacky, it doesn't answer the
> question.
> 
> Jan
> 
> 
> .
> 

-- 
Shannon

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Jan Beulich]

>>> On 18.08.15 at 10:21, <zhaoshenglong@huawei.com> wrote:

> 
> On 2015/8/18 16:15, Jan Beulich wrote:
>>>>> On 18.08.15 at 09:35, <zhaoshenglong@huawei.com> wrote:
>>> Hi Jan,
>>>
>>> On 2015/8/18 13:14, Jan Beulich wrote:
>>>>>>> Shannon Zhao <zhaoshenglong@huawei.com> 08/18/15 5:46 AM >>>
>>>>> On 2015/8/17 23:33, Jan Beulich wrote:
>>>>>>>>> On 14.08.15 at 16:59, <shannon.zhao@linaro.org> wrote:
>>>>>>> 1. Copy and change some EFI and ACPI tables
>>>>>>> -------------------------------------------
>>>>>>
>>>>>> While some explanation on the reasons for this was given in the past
>>>>>> discussion, I'm still missing a word on the "why" for each of these here.
>>>>>>
>>>>>>> a) Copy EFI_SYSTEM_TABLE and change the value of FirmwareVendor,
>>>>>>> VendorGuid, VendorTable, ConfigurationTable. These changes are not very
>>>>>>> special and it just assign values to these members.
>>>>>>
>>>>>> I continue to question the need for fiddling with this table, not the
>>>>>> least because I don't see how you intend to hand it to the Dom0
>>>>>> kernel: Are you expecting to call the kernel's ordinary EFI entry
>>>>>> point, despite it not itself running on EFI firmware?
>>>>>>
>>>>> Dom0 gets UEFI info from the minimal DT when booting with UEFI.
>>>>> And the main purpose to create a EFI_SYSTEM_TABLE is to provide the ACPI
>>>>> table root (RSDP) address to Dom0, so it could find the ACPI table.
>>>>
>>>> For that passing the configuration table would suffice. The more that Julien
>>>> in his reply said you're not even intending to use the kernel's EFI stub. 
>>> I.e.
>>>> the question remains: How would you expect to hand the System Table
>>>> pointer to Dom0?
>>>>
> 
> The System Table pointer will be passed to Dom0 through the minimal DT
> property "linux,uefi-system-table".

And this is a requirement for Linux? I.e. it can't do with just the
Configuration Table (which architecturally would be more clean imo)?

Jan

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Shannon Zhao]

On 2015/8/18 17:11, Jan Beulich wrote:
>>>> On 18.08.15 at 10:21, <zhaoshenglong@huawei.com> wrote:
> 
>>
>> On 2015/8/18 16:15, Jan Beulich wrote:
>>>>>> On 18.08.15 at 09:35, <zhaoshenglong@huawei.com> wrote:
>>>> Hi Jan,
>>>>
>>>> On 2015/8/18 13:14, Jan Beulich wrote:
>>>>>>>> Shannon Zhao <zhaoshenglong@huawei.com> 08/18/15 5:46 AM >>>
>>>>>> On 2015/8/17 23:33, Jan Beulich wrote:
>>>>>>>>>> On 14.08.15 at 16:59, <shannon.zhao@linaro.org> wrote:
>>>>>>>> 1. Copy and change some EFI and ACPI tables
>>>>>>>> -------------------------------------------
>>>>>>>
>>>>>>> While some explanation on the reasons for this was given in the past
>>>>>>> discussion, I'm still missing a word on the "why" for each of these here.
>>>>>>>
>>>>>>>> a) Copy EFI_SYSTEM_TABLE and change the value of FirmwareVendor,
>>>>>>>> VendorGuid, VendorTable, ConfigurationTable. These changes are not very
>>>>>>>> special and it just assign values to these members.
>>>>>>>
>>>>>>> I continue to question the need for fiddling with this table, not the
>>>>>>> least because I don't see how you intend to hand it to the Dom0
>>>>>>> kernel: Are you expecting to call the kernel's ordinary EFI entry
>>>>>>> point, despite it not itself running on EFI firmware?
>>>>>>>
>>>>>> Dom0 gets UEFI info from the minimal DT when booting with UEFI.
>>>>>> And the main purpose to create a EFI_SYSTEM_TABLE is to provide the ACPI
>>>>>> table root (RSDP) address to Dom0, so it could find the ACPI table.
>>>>>
>>>>> For that passing the configuration table would suffice. The more that Julien
>>>>> in his reply said you're not even intending to use the kernel's EFI stub. 
>>>> I.e.
>>>>> the question remains: How would you expect to hand the System Table
>>>>> pointer to Dom0?
>>>>>
>>
>> The System Table pointer will be passed to Dom0 through the minimal DT
>> property "linux,uefi-system-table".
> 
> And this is a requirement for Linux? I.e. it can't do with just the
> Configuration Table (which architecturally would be more clean imo)?
> 

IIUC, this is a requirement for Linux. Because when Linux parses the
minimal DT, it uses below dt_params to match the DT properties. If it
doesn't match any of them, it will fial.
See efi_get_fdt_params in drivers/firmware/efi/efi.c.

static __initdata struct {
	const char name[32];
	const char propname[32];
	int offset;
	int size;
} dt_params[] = {
	UEFI_PARAM("System Table", "linux,uefi-system-table", system_table),
	UEFI_PARAM("MemMap Address", "linux,uefi-mmap-start", mmap),
	UEFI_PARAM("MemMap Size", "linux,uefi-mmap-size", mmap_size),
	UEFI_PARAM("MemMap Desc. Size", "linux,uefi-mmap-desc-size", desc_size),
	UEFI_PARAM("MemMap Desc. Version", "linux,uefi-mmap-desc-ver", desc_ver)
};

Another choice to pass ACPI root table address is to use the kernel
parameter "acpi_rsdp". But it still needs to supply the property
"linux,uefi-system-table" in DT to make the EFI driver works. While the
value of the "linux,uefi-system-table" could be something invalid, so
Dom0 would not use the EFI System table.

-- 
Shannon

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Julien Grall]

On 18/08/2015 00:23, Shannon Zhao wrote:
>
>
> On 2015/8/18 14:36, Julien Grall wrote:
>>
>>
>> On 17/08/2015 20:19, Shannon Zhao wrote:
>>>>> Yes, I think it's good to drop the "linux," too. But if we drop the
>>>>> linux, would it impact the linux kernel booting with UEFI? And why we
>>>>> don't do it to Xen since Xen still uses "linux,"?
>>>>
>>>> I don't understand your second question.
>>>>
>>> I mean that Xen is using the property "linux,uefi*" as well, and why we
>>> don't drop that prefix for Xen?
>>
>> As never say we shouldn't drop it in Xen... It's of course a nice clean
>> up to have if we ever happen to standardize the properties with a
>> different name.
>>
>>>> For the first question, as we discussed in several mail, the property
>>>> "linux,uefi-*" are only used internally between the stub and Linux. The
>>>> sub is compiled in the kernel so there is no issue to change the
>>>> property.
>>>>
>>> Since Linux defines the dt_params like below which is used to get EFI
>>> info from DT, if we drop "linux," in Xen, does it need to drop the
>>> "linux," in dt_params? If so, will this break the compatibility of
>>> changed kernel with old UEFI? IIUC, there is not only Xen using these
>>> properties, but also native host and QEMU guest.
>>
>> I grepped "linux," and didn't spot any "linux,uefi-*" strings.
>>
>
> In drivers/firmware/efi/efi.c line 478

Which is not QEMU but Linux...

>> Anyway, why are you speaking about old UEFI? As said in different mail,
>> the linux,uefi-* properties are only used internally between the EFI
>> stub and the kernel. Both are living in the same binary so it's not
>> exposed outside.
>>
>
> UEFI makes this minimal DT, right? To Dom0, Xen makes this minimal DT,
> right? And EFI stub parses this DT through efi_get_fdt_params ==>
> fdt_find_uefi_params in drivers/firmware/efi/efi.c. And
> fdt_find_uefi_params uses dt_params[i].propname (e.g.
> "linux,uefi-system-table") to get the matched property.
>
> "prop = of_get_flat_dt_prop(node, dt_params[i].propname, &len);"
>
> If we changed the property names in minimal DT but not change the
> definition of dt_params[], it can't get the matched properties, right?
> And if we both changed the property name in minimal DT and definition of
> dt_params[], will this new kernel work well with the old UEFI which
> still uses old property names to create the minimal DT?

Please read again my previous mail. The properties are created by the 
EFI stub (compiled within the Kernel) and neither UEFI firmware nor Xen.

Regards,

-- 
Julien Grall

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Julien Grall]

Hi,

On 18/08/2015 02:34, Shannon Zhao wrote:
> IIUC, this is a requirement for Linux. Because when Linux parses the
> minimal DT, it uses below dt_params to match the DT properties. If it
> doesn't match any of them, it will fial.
> See efi_get_fdt_params in drivers/firmware/efi/efi.c.

This is *not* a requirement for Linux. All those properties are internal 
to Linux so it's not exposed to anyone else. We have room to change it 
while we standardize the property.

Regards,

-- 
Julien Grall

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Julien Grall]

On 18/08/2015 00:01, Jan Beulich wrote:
>>>> On 18.08.15 at 08:43, <julien.grall@citrix.com> wrote:
>> Hi Jan,
>>
>> On 17/08/2015 22:10, Jan Beulich wrote:
>>>>>> Julien Grall <julien.grall@citrix.com> 08/17/15 6:27 PM >>>
>>>> On 17/08/2015 08:33, Jan Beulich wrote:
>>>>>>>> On 14.08.15 at 16:59, <shannon.zhao@linaro.org> wrote:
>>>>>> b) Create EFI_MEMORY_DESCRIPTOR table. This will add memory start and
>>>>>> size information of Dom0. And Dom0 will get the memory information
>>>>>> through this EFI table.
>>>>>
>>>>> To some degree the same applies here: While I see that you have no
>>>>> legacy vehicle like x86's E820, I also don't see how Dom0 - not being
>>>>> able to make EFI boot or runtime services calls - would get hold of this
>>>>> table. And if a non-EFI mechanism is to be used here, using the EFI
>>>>> data structure would turn out to be just an arbitrary (or convenience)
>>>>> decision, not something inherently required. Which I think should be
>>>>> said explicitly if so, rather than leaving the reader guess.
>>>>
>>>> It's not an arbitrary decision, when UEFI stub in Linux is using device
>>>> tree properties to pass the UEFI table to the kernel ([1]).
>>>>
>>>> When booting on Xen with ACPI, dom0 will use the non-EFI entry point.
>>>> The easiest way to pass the memory information to Linux is using the
>>>> UEFI DT properties.
>>>
>>> In which case it is even more arbitrary to use the EFI data structure to
>>> convey memory information (instead of expressing it in plain DT, which is
>>> how I blindly assume non-EFI does it). Of course there's the small chance
>>> that "UEFI DT properties" implies a certain binary format, but it's still
>> odd
>>> for a non-EFI entry point to assume EFI properties to be there...
>>
>> Linux is able to boot either on ACPI or DT. When ACPI is used, the EFI
>> stub (mandatory) will create a small DT in order to pass the command
>> line and other informations (such as the EFI memory table) to the
>> kernel. This is because the stub is self-contained and and never use
>> variable living in the kernel.
>>
>> In order to know whether you are booting using DT or ACPI, they check if
>> the DT contains only the /chosen node.
>>
>> Actually the EFI-stub will always jumped on the non-EFI path.
>>
>> By passing the minimal DT as suggested on the design doc, we are
>> avoiding to get a different code path for Xen in the kernel and we are
>> finally acting as the EFI-stub was a nop for DOM0.
>>
>> So I don't see how this would be arbitrary...
>
> Indeed with your explanation this doesn't look arbitrary anymore.
> But none of this was spelled out in the original document.

Sorry I'm forgetting every time that you may not be aware of all the 
justifications.

Regards,

-- 
Julien Grall

Re: Design doc of adding ACPI support for arm64 on Xen - version 3

[Ian Campbell]

On Tue, 2015-08-18 at 12:00 -0700, Julien Grall wrote:
> Hi,
> 
> On 18/08/2015 02:34, Shannon Zhao wrote:
> > IIUC, this is a requirement for Linux. Because when Linux parses the
> > minimal DT, it uses below dt_params to match the DT properties. If it
> > doesn't match any of them, it will fial.
> > See efi_get_fdt_params in drivers/firmware/efi/efi.c.
> 
> This is *not* a requirement for Linux. All those properties are internal 
> to Linux so it's not exposed to anyone else. We have room to change it 
> while we standardize the property.

This thread is not the correct forum for such standardisation, since we
(Xen) cannot do so in isolation. I've not yet seen any discussion in the
appropriate places and we really should be doing so before we
build/finalise a design based upon it (otherwise we get to the end and the
rest of the world says "no").

AFAICT the correct place to discuss would be the devicetree-spec ML[0], and
perhaps also some or all of:
      * Some appropriate UEFI forum?
      * Linux (~=devicetree-spec?) since we are building on their internal
        spec?
      * BSD, since we would like them to support it?

Is someone going to make such a proposal?

Ian.

[0] http://vger.kernel.org/vger-lists.html#devicetree-spec