* [PATCH v3 0/3] ACPI: i386: bump MADT to revision 5
@ 2023-04-21 21:48 Eric DeVolder
2023-04-21 21:48 ` [PATCH v3 1/3] ACPI: bios-tables-test.c step 2 (allowed-diff entries) Eric DeVolder
` (3 more replies)
0 siblings, 4 replies; 9+ messages in thread
From: Eric DeVolder @ 2023-04-21 21:48 UTC (permalink / raw)
To: shannon.zhaosl, mst, imammedo, ani, peter.maydell, qemu-arm,
qemu-devel, marcel.apfelbaum, pbonzini, richard.henderson,
eduardo
Cc: boris.ostrovsky, miguel.luis, eric.devolder
The following Linux kernel change broke CPU hotplug for MADT revision
less than 5.
e2869bd7af60 ("x86/acpi/boot: Do not register processors that cannot be onlined for x2APIC")
Discussion on this topic can be located here:
https://lore.kernel.org/linux-acpi/20230327191026.3454-1-eric.devolder@oracle.com/T/#t
which resulted in the following fixes Linux in 6.3-rc5:
a74fabfbd1b7: ("x86/ACPI/boot: Use FADT version to check support for online capable")
fed8d8773b8e: ("x86/acpi/boot: Correct acpi_is_processor_usable() check")
However, as part of the investigation into resolving this breakage, I
learned that i386 QEMU reports revision 1, while technically it
generates revision 3. Aarch64 generates and reports revision 4.
ACPI 6.3 bumps MADT revision to 5 as it introduces an Online Capable
flag that the above Linux patch utilizes to denote hot pluggable CPUs.
So in order to bump MADT to the current revision of 5, need to
validate that all MADT table changes between 1 and 5 are present
in QEMU.
Below is a table summarizing the changes to the MADT. This information
gleamed from the ACPI specs on uefi.org.
ACPI MADT What
Version Version
1.0 MADT not present
2.0 1 Section 5.2.10.4
3.0 2 Section 5.2.11.4
5.2.11.13 Local SAPIC Structure added two new fields:
ACPI Processor UID Value
ACPI Processor UID String
5.2.10.14 Platform Interrupt Sources Structure:
Reserved changed to Platform Interrupt Sources Flags
3.0b 2 Section 5.2.11.4
Added a section describing guidelines for the ordering of
processors in the MADT to support proper boot processor
and multi-threaded logical processor operation.
4.0 3 Section 5.2.12
Adds Processor Local x2APIC structure type 9
Adds Local x2APIC NMI structure type 0xA
5.0 3 Section 5.2.12
6.0 3 Section 5.2.12
6.0a 4 Section 5.2.12
Adds ARM GIC structure types 0xB-0xF
6.2a 45 Section 5.2.12 <--- version 45, is indeed accurate!
6.2b 5 Section 5.2.12
GIC ITS last Reserved offset changed to 16 from 20 (typo)
6.3 5 Section 5.2.12
Adds Local APIC Flags Online Capable!
Adds GICC SPE Overflow Interrupt field
6.4 5 Section 5.2.12
Adds Multiprocessor Wakeup Structure type 0x10
(change notes says structure previously misplaced?)
6.5 5 Section 5.2.12
For the MADT revision change 1 -> 2, the spec has a change to the
SAPIC structure. In general, QEMU does not generate/support SAPIC.
So the QEMU i386 MADT revision can safely be moved to 2.
For the MADT revision change 2 -> 3, the spec adds Local x2APIC
structures. QEMU has long supported x2apic ACPI structures. A simple
search of x2apic within QEMU source and hw/i386/acpi-common.c
specifically reveals this. So the QEMU i386 MADT revision can safely
be moved to 3.
For the MADT revision change 3 -> 4, the spec adds support for the ARM
GIC structures. QEMU ARM does in fact generate and report revision 4.
As these will not be used by i386 QEMU, so then the QEMU i386 MADT
revision can safely be moved to 4 as well.
Now for the MADT revision change 4 -> 5, the spec adds the Online
Capable flag to the Local APIC structure, and the ARM GICC SPE
Overflow Interrupt field.
For i386, the ARM SPE is not applicable.
For the i386 Local APIC flag Online Capable, the spec has certain rules
about this value. And in particuar setting this value now explicitly
indicates a hotpluggable CPU.
So this patch makes the needed changes to move i386 MADT to
revision 5.
Without these changes, the information below shows "how" CPU hotplug
breaks with the current upstream Linux kernel 6.3. For example, a Linux
guest started with:
qemu-system-x86_64 -smp 30,maxcpus=32 ...
and then attempting to hotplug a CPU:
(QEMU) device_add id=cpu30 driver=host-x86_64-cpu socket-id=0 core-id=30 thread-id=0
fails with the following:
APIC: NR_CPUS/possible_cpus limit of 30 reached. Processor 30/0x.
ACPI: Unable to map lapic to logical cpu number
acpi LNXCPU:1e: Enumeration failure
# dmesg | grep smpboot
smpboot: Allowing 30 CPUs, 0 hotplug CPUs
smpboot: CPU0: Intel(R) Xeon(R) CPU D-1533 @ 2.10GHz (family: 0x)
smpboot: Max logical packages: 1
smpboot: Total of 30 processors activated (125708.76 BogoMIPS)
# iasl -d /sys/firmware/tables/acpi/APIC
[000h 0000 4] Signature : "APIC" [Multiple APIC Descript
[004h 0004 4] Table Length : 00000170
[008h 0008 1] Revision : 01 <=====
[009h 0009 1] Checksum : 9C
[00Ah 0010 6] Oem ID : "BOCHS "
[010h 0016 8] Oem Table ID : "BXPC "
[018h 0024 4] Oem Revision : 00000001
[01Ch 0028 4] Asl Compiler ID : "BXPC"
[020h 0032 4] Asl Compiler Revision : 00000001
...
[114h 0276 1] Subtable Type : 00 [Processor Local APIC]
[115h 0277 1] Length : 08
[116h 0278 1] Processor ID : 1D
[117h 0279 1] Local Apic ID : 1D
[118h 0280 4] Flags (decoded below) : 00000001
Processor Enabled : 1 <=====
[11Ch 0284 1] Subtable Type : 00 [Processor Local APIC]
[11Dh 0285 1] Length : 08
[11Eh 0286 1] Processor ID : 1E
[11Fh 0287 1] Local Apic ID : 1E
[120h 0288 4] Flags (decoded below) : 00000000
Processor Enabled : 0 <=====
[124h 0292 1] Subtable Type : 00 [Processor Local APIC]
[125h 0293 1] Length : 08
[126h 0294 1] Processor ID : 1F
[127h 0295 1] Local Apic ID : 1F
[128h 0296 4] Flags (decoded below) : 00000000
Processor Enabled : 0 <=====
The (latest upstream) Linux kernel sees 30 Enabled processors, and
does not consider processors 31 and 32 to be hotpluggable.
With this patch series applied, by bumping MADT to revision 5, the
latest upstream Linux kernel correctly identifies 30 CPUs plus 2
hotpluggable CPUS.
CPU30 has been hot-added
smpboot: Booting Node 0 Processor 30 APIC 0x1e
Will online and init hotplugged CPU: 30
# dmesg | grep smpboot
smpboot: Allowing 32 CPUs, 2 hotplug CPUs
smpboot: CPU0: Intel(R) Xeon(R) CPU D-1533 @ 2.10GHz (family: 0x6, model: 0x56, stepping: 0x3)
smpboot: Max logical packages: 2
smpboot: Total of 30 processors activated (125708.76 BogoMIPS)
# iasl -d /sys/firmware/tables/acpi/APIC
[000h 0000 004h] Signature : "APIC" [Multiple APIC Descript
[004h 0004 004h] Table Length : 00000170
[008h 0008 001h] Revision : 05 <=====
[009h 0009 001h] Checksum : 94
[00Ah 0010 006h] Oem ID : "BOCHS "
[010h 0016 008h] Oem Table ID : "BXPC "
[018h 0024 004h] Oem Revision : 00000001
[01Ch 0028 004h] Asl Compiler ID : "BXPC"
[020h 0032 004h] Asl Compiler Revision : 00000001
...
[114h 0276 001h] Subtable Type : 00 [Processor Local APIC]
[115h 0277 001h] Length : 08
[116h 0278 001h] Processor ID : 1D
[117h 0279 001h] Local Apic ID : 1D
[118h 0280 004h] Flags (decoded below) : 00000001
Processor Enabled : 1 <=====
Runtime Online Capable : 0 <=====
[11Ch 0284 001h] Subtable Type : 00 [Processor Local APIC]
[11Dh 0285 001h] Length : 08
[11Eh 0286 001h] Processor ID : 1E
[11Fh 0287 001h] Local Apic ID : 1E
[120h 0288 004h] Flags (decoded below) : 00000002
Processor Enabled : 0 <=====
Runtime Online Capable : 1 <=====
[124h 0292 001h] Subtable Type : 00 [Processor Local APIC]
[125h 0293 001h] Length : 08
[126h 0294 001h] Processor ID : 1F
[127h 0295 001h] Local Apic ID : 1F
[128h 0296 004h] Flags (decoded below) : 00000002
Processor Enabled : 0 <=====
Runtime Online Capable : 1 <=====
In terms of testing older operating systems, I did:
- Windows Server 2008.
By opening Device Manager -> Processors, and watching/counting
the listed CPUs before and after hot un/plug operations.
When hot plugging a CPU, the new processor shows in the list;
one more processor is available after the hotplug.
When hot unplugging a CPU, Windows refuses, with the message:
The 'Intel(R) Xeon(R) CPU D-1533 @ 2.10GHz' device is not
removable and cannot be ejected or unplugged.
This is expected as hot unplug did not work in Windows 2008.
- RHEL 6.9
Kernel 2.6.32-696.el6.x86_64
Build Feb 21 2017
From dmesg:
ACPI: APIC 000000007ffe32f0 000F0 (v05 BOCHS BXPC 00000001 BXPC 00000001)
SMP: Allowing 16 CPUs, 8 hotplug CPUs
# cat /sys/devices/system/cpu/online
0-7
(QEMU) device_add id=cpu8 driver=host-x86_64-cpu socket-id=0 core-id=8 thread-id=0
CPU 8 got hotplugged
Booting Node 0 Processor 8 APIC 0x8
kvm-clock: cpu 8, msr 2830ed00
Will online and init hotplugged CPU: 8
microcode: CPU8 sig=0x50663, pf=0x1, revision=0x700001c
platform microcode: firmware: requesting intel-ucode/06-56-03
# cat /sys/devices/system/cpu/online
0-8
(QEMU) device_del id=cpu8
Broke affinity for irq 24
CPU 8 is now offline
# cat /sys/devices/system/cpu/online
0-7
RHEL 6.9
kernel 2.6.32-696.el6.x86_64
build Feb 21 2017
Both of the above 'legacy' operating systems behaved as expected.
Regards,
Eric
---
v3: 21apr2023
- Dropped ARM patch as there is no compelling need to move to
MADT revision 5, per Michael Tsirkin.
- Recoded the flags to be simpler, per Michael, Ani Sinha and
Miguel Luis.
- Regenerated the binary tables, as per bios-tables-test.c
instructions.
v2: 18apr2023
https://lists.gnu.org/archive/html/qemu-devel/2023-04/msg02576.html
- Code formatting changes per Igor Mammedov
- Included the steps in bios-tables-test.
v1: 28mar2023
https://lists.gnu.org/archive/html/qemu-devel/2023-03/msg06705.html
- Initial series for ARM and i386 MADT revision bump to 5.
Eric DeVolder (3):
ACPI: bios-tables-test.c step 2 (allowed-diff entries)
hw/acpi: i386: bump MADT to revision 5
ACPI: bios-tables-test.c step 5 (updated expected table binaries)
hw/i386/acpi-common.c | 4 ++--
tests/data/acpi/microvm/APIC | Bin 70 -> 70 bytes
tests/data/acpi/microvm/APIC.ioapic2 | Bin 82 -> 82 bytes
tests/data/acpi/microvm/APIC.pcie | Bin 110 -> 110 bytes
tests/data/acpi/pc/APIC | Bin 120 -> 120 bytes
tests/data/acpi/pc/APIC.acpihmat | Bin 128 -> 128 bytes
tests/data/acpi/pc/APIC.cphp | Bin 160 -> 160 bytes
tests/data/acpi/pc/APIC.dimmpxm | Bin 144 -> 144 bytes
tests/data/acpi/q35/APIC | Bin 120 -> 120 bytes
tests/data/acpi/q35/APIC.acpihmat | Bin 128 -> 128 bytes
tests/data/acpi/q35/APIC.acpihmat-noinitiator | Bin 144 -> 144 bytes
tests/data/acpi/q35/APIC.core-count2 | Bin 2478 -> 2478 bytes
tests/data/acpi/q35/APIC.cphp | Bin 160 -> 160 bytes
tests/data/acpi/q35/APIC.dimmpxm | Bin 144 -> 144 bytes
tests/data/acpi/q35/APIC.xapic | Bin 2686 -> 2686 bytes
15 files changed, 2 insertions(+), 2 deletions(-)
--
2.31.1
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH v3 1/3] ACPI: bios-tables-test.c step 2 (allowed-diff entries)
2023-04-21 21:48 [PATCH v3 0/3] ACPI: i386: bump MADT to revision 5 Eric DeVolder
@ 2023-04-21 21:48 ` Eric DeVolder
2023-04-21 21:49 ` [PATCH v3 2/3] ACPI: i386: bump MADT to revision 5 Eric DeVolder
` (2 subsequent siblings)
3 siblings, 0 replies; 9+ messages in thread
From: Eric DeVolder @ 2023-04-21 21:48 UTC (permalink / raw)
To: shannon.zhaosl, mst, imammedo, ani, peter.maydell, qemu-arm,
qemu-devel, marcel.apfelbaum, pbonzini, richard.henderson,
eduardo
Cc: boris.ostrovsky, miguel.luis, eric.devolder
Following the guidelines in tests/qtest/bios-tables-test.c, this
change sets-up bios-tables-test-allowed-diff.h to exclude the
imminent changes to the APIC tables, per step 2.
Signed-off-by: Eric DeVolder <eric.devolder@oracle.com>
---
tests/qtest/bios-tables-test-allowed-diff.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index dfb8523c8b..1e5e354ecf 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1 +1,5 @@
/* List of comma-separated changed AML files to ignore */
+"tests/data/acpi/pc/APIC",
+"tests/data/acpi/q35/APIC",
+"tests/data/acpi/microvm/APIC",
+"tests/data/acpi/virt/APIC",
--
2.31.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v3 2/3] ACPI: i386: bump MADT to revision 5
2023-04-21 21:48 [PATCH v3 0/3] ACPI: i386: bump MADT to revision 5 Eric DeVolder
2023-04-21 21:48 ` [PATCH v3 1/3] ACPI: bios-tables-test.c step 2 (allowed-diff entries) Eric DeVolder
@ 2023-04-21 21:49 ` Eric DeVolder
2023-04-21 21:49 ` [PATCH v3 3/3] ACPI: bios-tables-test.c step 5 (updated expected table binaries) Eric DeVolder
2023-05-05 21:53 ` [PATCH v3 0/3] ACPI: i386: bump MADT to revision 5 Eric DeVolder
3 siblings, 0 replies; 9+ messages in thread
From: Eric DeVolder @ 2023-04-21 21:49 UTC (permalink / raw)
To: shannon.zhaosl, mst, imammedo, ani, peter.maydell, qemu-arm,
qemu-devel, marcel.apfelbaum, pbonzini, richard.henderson,
eduardo
Cc: boris.ostrovsky, miguel.luis, eric.devolder
Currently i386 QEMU generates MADT revision 3, and reports
MADT revision 1. ACPI 6.3 introduces MADT revision 5.
For MADT revision 4, that introduces ARM GIC structures, which do
not apply to i386.
For MADT revision 5, the Local APIC flags introduces the Online
Capable bitfield.
Making MADT generate and report revision 5 will solve problems with
CPU hotplug (the Online Capable flag indicates hotpluggable CPUs).
Link: https://lore.kernel.org/linux-acpi/20230327191026.3454-1-eric.devolder@ora
cle.com/T/#t
Signed-off-by: Eric DeVolder <eric.devolder@oracle.com>
---
hw/i386/acpi-common.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/hw/i386/acpi-common.c b/hw/i386/acpi-common.c
index 52e5c1439a..5a5e73c399 100644
--- a/hw/i386/acpi-common.c
+++ b/hw/i386/acpi-common.c
@@ -39,7 +39,7 @@ void pc_madt_cpu_entry(int uid, const CPUArchIdList *apic_ids,
uint32_t apic_id = apic_ids->cpus[uid].arch_id;
/* Flags – Local APIC Flags */
uint32_t flags = apic_ids->cpus[uid].cpu != NULL || force_enabled ?
- 1 /* Enabled */ : 0;
+ 1 /* Enabled */ : 2 /* Online Capable */;
/* ACPI spec says that LAPIC entry for non present
* CPU may be omitted from MADT or it must be marked
@@ -102,7 +102,7 @@ void acpi_build_madt(GArray *table_data, BIOSLinker *linker,
MachineClass *mc = MACHINE_GET_CLASS(x86ms);
const CPUArchIdList *apic_ids = mc->possible_cpu_arch_ids(MACHINE(x86ms));
AcpiDeviceIfClass *adevc = ACPI_DEVICE_IF_GET_CLASS(adev);
- AcpiTable table = { .sig = "APIC", .rev = 1, .oem_id = oem_id,
+ AcpiTable table = { .sig = "APIC", .rev = 5, .oem_id = oem_id,
.oem_table_id = oem_table_id };
acpi_table_begin(&table, table_data);
--
2.31.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v3 3/3] ACPI: bios-tables-test.c step 5 (updated expected table binaries)
2023-04-21 21:48 [PATCH v3 0/3] ACPI: i386: bump MADT to revision 5 Eric DeVolder
2023-04-21 21:48 ` [PATCH v3 1/3] ACPI: bios-tables-test.c step 2 (allowed-diff entries) Eric DeVolder
2023-04-21 21:49 ` [PATCH v3 2/3] ACPI: i386: bump MADT to revision 5 Eric DeVolder
@ 2023-04-21 21:49 ` Eric DeVolder
2023-05-05 21:53 ` [PATCH v3 0/3] ACPI: i386: bump MADT to revision 5 Eric DeVolder
3 siblings, 0 replies; 9+ messages in thread
From: Eric DeVolder @ 2023-04-21 21:49 UTC (permalink / raw)
To: shannon.zhaosl, mst, imammedo, ani, peter.maydell, qemu-arm,
qemu-devel, marcel.apfelbaum, pbonzini, richard.henderson,
eduardo
Cc: boris.ostrovsky, miguel.luis, eric.devolder
Following the guidelines in tests/qtest/bios-tables-test.c, this
is step 6.
For the cpuhp test case, it is started with:
-smp 2,cores=3,sockets=2,maxcpus=6
So two of six CPUs are present, leaving 4 hot-pluggable CPUs. This
is what the disassembly diff below shows (two entries with Enabled=1
and the new Online Capable bit 0, and four entries with Enabled=0 and
Online Capable bit 1).
(NOTE: I'm only showing x86_64 .cphp as i386 is the same. And for
tests not involving hotplug, the diff shows just the corresponding
change to .Revision and .Checksum.)
/*
* Intel ACPI Component Architecture
* AML/ASL+ Disassembler version 20230331 (64-bit version)
* Copyright (c) 2000 - 2023 Intel Corporation
*
- * Disassembly of tests/data/acpi/pc/APIC.cphp, Fri Apr 21 16:50:07 2023
+ * Disassembly of /tmp/aml-9ON131, Fri Apr 21 16:50:07 2023
*
* ACPI Data Table [APIC]
*
* Format: [HexOffset DecimalOffset ByteLength] FieldName : FieldValue (in hex)
*/
[000h 0000 004h] Signature : "APIC" [Multiple APIC Description Table (MADT)]
[004h 0004 004h] Table Length : 000000A0
-[008h 0008 001h] Revision : 01
-[009h 0009 001h] Checksum : 18
+[008h 0008 001h] Revision : 05
+[009h 0009 001h] Checksum : 0C
[00Ah 0010 006h] Oem ID : "BOCHS "
[010h 0016 008h] Oem Table ID : "BXPC "
[018h 0024 004h] Oem Revision : 00000001
[01Ch 0028 004h] Asl Compiler ID : "BXPC"
[020h 0032 004h] Asl Compiler Revision : 00000001
[024h 0036 004h] Local Apic Address : FEE00000
[028h 0040 004h] Flags (decoded below) : 00000001
PC-AT Compatibility : 1
[02Ch 0044 001h] Subtable Type : 00 [Processor Local APIC]
[02Dh 0045 001h] Length : 08
[02Eh 0046 001h] Processor ID : 00
[02Fh 0047 001h] Local Apic ID : 00
[030h 0048 004h] Flags (decoded below) : 00000001
Processor Enabled : 1
Runtime Online Capable : 0
[034h 0052 001h] Subtable Type : 00 [Processor Local APIC]
[035h 0053 001h] Length : 08
[036h 0054 001h] Processor ID : 01
[037h 0055 001h] Local Apic ID : 01
[038h 0056 004h] Flags (decoded below) : 00000001
Processor Enabled : 1
Runtime Online Capable : 0
[03Ch 0060 001h] Subtable Type : 00 [Processor Local APIC]
[03Dh 0061 001h] Length : 08
[03Eh 0062 001h] Processor ID : 02
[03Fh 0063 001h] Local Apic ID : 02
-[040h 0064 004h] Flags (decoded below) : 00000000
+[040h 0064 004h] Flags (decoded below) : 00000002
Processor Enabled : 0
- Runtime Online Capable : 0
+ Runtime Online Capable : 1
[044h 0068 001h] Subtable Type : 00 [Processor Local APIC]
[045h 0069 001h] Length : 08
[046h 0070 001h] Processor ID : 03
[047h 0071 001h] Local Apic ID : 04
-[048h 0072 004h] Flags (decoded below) : 00000000
+[048h 0072 004h] Flags (decoded below) : 00000002
Processor Enabled : 0
- Runtime Online Capable : 0
+ Runtime Online Capable : 1
[04Ch 0076 001h] Subtable Type : 00 [Processor Local APIC]
[04Dh 0077 001h] Length : 08
[04Eh 0078 001h] Processor ID : 04
[04Fh 0079 001h] Local Apic ID : 05
-[050h 0080 004h] Flags (decoded below) : 00000000
+[050h 0080 004h] Flags (decoded below) : 00000002
Processor Enabled : 0
- Runtime Online Capable : 0
+ Runtime Online Capable : 1
[054h 0084 001h] Subtable Type : 00 [Processor Local APIC]
[055h 0085 001h] Length : 08
[056h 0086 001h] Processor ID : 05
[057h 0087 001h] Local Apic ID : 06
-[058h 0088 004h] Flags (decoded below) : 00000000
+[058h 0088 004h] Flags (decoded below) : 00000002
Processor Enabled : 0
- Runtime Online Capable : 0
+ Runtime Online Capable : 1
[05Ch 0092 001h] Subtable Type : 01 [I/O APIC]
[05Dh 0093 001h] Length : 0C
[05Eh 0094 001h] I/O Apic ID : 00
[05Fh 0095 001h] Reserved : 00
[060h 0096 004h] Address : FEC00000
[064h 0100 004h] Interrupt : 00000000
[068h 0104 001h] Subtable Type : 02 [Interrupt Source Override]
[069h 0105 001h] Length : 0A
[06Ah 0106 001h] Bus : 00
[06Bh 0107 001h] Source : 00
[06Ch 0108 004h] Interrupt : 00000002
[070h 0112 002h] Flags (decoded below) : 0000
Polarity : 0
Trigger Mode : 0
@@ -121,26 +121,26 @@
[093h 0147 001h] Source : 0B
[094h 0148 004h] Interrupt : 0000000B
[098h 0152 002h] Flags (decoded below) : 000D
Polarity : 1
Trigger Mode : 3
[09Ah 0154 001h] Subtable Type : 04 [Local APIC NMI]
[09Bh 0155 001h] Length : 06
[09Ch 0156 001h] Processor ID : FF
[09Dh 0157 002h] Flags (decoded below) : 0000
Polarity : 0
Trigger Mode : 0
[09Fh 0159 001h] Interrupt Input LINT : 01
Raw Table Data: Length 160 (0xA0)
- 0000: 41 50 49 43 A0 00 00 00 01 18 42 4F 43 48 53 20 // APIC......BOCHS
+ 0000: 41 50 49 43 A0 00 00 00 05 0C 42 4F 43 48 53 20 // APIC......BOCHS
0010: 42 58 50 43 20 20 20 20 01 00 00 00 42 58 50 43 // BXPC ....BXPC
0020: 01 00 00 00 00 00 E0 FE 01 00 00 00 00 08 00 00 // ................
0030: 01 00 00 00 00 08 01 01 01 00 00 00 00 08 02 02 // ................
- 0040: 00 00 00 00 00 08 03 04 00 00 00 00 00 08 04 05 // ................
- 0050: 00 00 00 00 00 08 05 06 00 00 00 00 01 0C 00 00 // ................
+ 0040: 02 00 00 00 00 08 03 04 02 00 00 00 00 08 04 05 // ................
+ 0050: 02 00 00 00 00 08 05 06 02 00 00 00 01 0C 00 00 // ................
0060: 00 00 C0 FE 00 00 00 00 02 0A 00 00 02 00 00 00 // ................
0070: 00 00 02 0A 00 05 05 00 00 00 0D 00 02 0A 00 09 // ................
0080: 09 00 00 00 0D 00 02 0A 00 0A 0A 00 00 00 0D 00 // ................
0090: 02 0A 00 0B 0B 00 00 00 0D 00 04 06 FF 00 00 01 // ................
**
Signed-off-by: Eric DeVolder <eric.devolder@oracle.com>
---
tests/data/acpi/microvm/APIC | Bin 70 -> 70 bytes
tests/data/acpi/microvm/APIC.ioapic2 | Bin 82 -> 82 bytes
tests/data/acpi/microvm/APIC.pcie | Bin 110 -> 110 bytes
tests/data/acpi/pc/APIC | Bin 120 -> 120 bytes
tests/data/acpi/pc/APIC.acpihmat | Bin 128 -> 128 bytes
tests/data/acpi/pc/APIC.cphp | Bin 160 -> 160 bytes
tests/data/acpi/pc/APIC.dimmpxm | Bin 144 -> 144 bytes
tests/data/acpi/q35/APIC | Bin 120 -> 120 bytes
tests/data/acpi/q35/APIC.acpihmat | Bin 128 -> 128 bytes
tests/data/acpi/q35/APIC.acpihmat-noinitiator | Bin 144 -> 144 bytes
tests/data/acpi/q35/APIC.core-count2 | Bin 2478 -> 2478 bytes
tests/data/acpi/q35/APIC.cphp | Bin 160 -> 160 bytes
tests/data/acpi/q35/APIC.dimmpxm | Bin 144 -> 144 bytes
tests/data/acpi/q35/APIC.xapic | Bin 2686 -> 2686 bytes
tests/data/acpi/virt/APIC | Bin 172 -> 172 bytes
tests/data/acpi/virt/APIC.acpihmatvirt | Bin 412 -> 412 bytes
tests/data/acpi/virt/APIC.topology | Bin 732 -> 732 bytes
tests/qtest/bios-tables-test-allowed-diff.h | 4 ----
18 files changed, 4 deletions(-)
diff --git a/tests/data/acpi/microvm/APIC b/tests/data/acpi/microvm/APIC
index 68dbd44a7e35a356083f086df60f70e424c4249f..e1c72bd3e60e46acc2c8b3de14f5992946cd55a3 100644
GIT binary patch
delta 16
XcmZ>B<8ln}barE4U|=nn$Ylcn95w?+
delta 16
XcmZ>B<8ln}barE4U|=kn$Ylcn95Mq&
diff --git a/tests/data/acpi/microvm/APIC.ioapic2 b/tests/data/acpi/microvm/APIC.ioapic2
index 3063c52cd3e9bbed29c06031b375900f4a49b9e0..7c4f1b61d6c37f893d0d46be44a00e3f54a27a75 100644
GIT binary patch
delta 16
XcmWFv;&Ke|bPi%*U|_AE$mIb59$o{5
delta 16
XcmWFv;&Ke|bPi%*U|?*X$mIb59$Ev1
diff --git a/tests/data/acpi/microvm/APIC.pcie b/tests/data/acpi/microvm/APIC.pcie
index 4e8f6ed8d6a866429fc17aecdeafc3fb5ef65fa3..23956151765cae221853d92448a619d5576119b5 100644
GIT binary patch
delta 16
Xcmd1H<8ln}bk1X7U|_A7$dv*BBEAFe
delta 16
Xcmd1H<8ln}bk1X7U|_77$dv*BBDw?a
diff --git a/tests/data/acpi/pc/APIC b/tests/data/acpi/pc/APIC
index 208331db53b7dd5c6205cce0e95427636b86dd64..48bdab41959d7895b01491c6136d7403df284f98 100644
GIT binary patch
delta 16
Xcmb=Z;BpM`bgp1vU|?;V$dv~GB#;Co
delta 16
Xcmb=Z;BpM`bgp1vU|{T;$dv~GB#Z<k
diff --git a/tests/data/acpi/pc/APIC.acpihmat b/tests/data/acpi/pc/APIC.acpihmat
index 812c4603f2701494f6bb761570323158a20d4043..57d15150f781504090c8be19fe063836a66ccdc1 100644
GIT binary patch
delta 18
ZcmZo*Y+&Sa4DfVrU|?WiEuP3#1^_261Tz2t
delta 18
ZcmZo*Y+&Sa4DfVrU|?WiET70#1^_221Tz2t
diff --git a/tests/data/acpi/pc/APIC.cphp b/tests/data/acpi/pc/APIC.cphp
index 65cc4f4a9aa2676140a6525cdac1e838274b1e07..8451b99c3065f76214023a88ef955de0efc75c60 100644
GIT binary patch
delta 47
scmZ3$xPXz%F~HM#0RsaAE6+qOGf5^Oi-Cig1wym1LTFYtriqd60Hq=YaR2}S
delta 47
scmZ3$xPXz%F~HM#0RsaAqr^lmGf5zWfq{dW1wym1LTFYthKZ5x0Hvk{aR2}S
diff --git a/tests/data/acpi/pc/APIC.dimmpxm b/tests/data/acpi/pc/APIC.dimmpxm
index d904d4a70ddecbb79a83a267af8e26f925e9f4c6..824a279eb7d4134554cb447808bab5a174657272 100644
GIT binary patch
delta 18
ZcmbQhIDwJNF~HM#0s{jBtNTQ*HUKF;1V;b>
delta 18
ZcmbQhIDwJNF~HM#0s{jBqxVFvHUKF)1V;b>
diff --git a/tests/data/acpi/q35/APIC b/tests/data/acpi/q35/APIC
index 208331db53b7dd5c6205cce0e95427636b86dd64..48bdab41959d7895b01491c6136d7403df284f98 100644
GIT binary patch
delta 16
Xcmb=Z;BpM`bgp1vU|?;V$dv~GB#;Co
delta 16
Xcmb=Z;BpM`bgp1vU|{T;$dv~GB#Z<k
diff --git a/tests/data/acpi/q35/APIC.acpihmat b/tests/data/acpi/q35/APIC.acpihmat
index 812c4603f2701494f6bb761570323158a20d4043..57d15150f781504090c8be19fe063836a66ccdc1 100644
GIT binary patch
delta 18
ZcmZo*Y+&Sa4DfVrU|?WiEuP3#1^_261Tz2t
delta 18
ZcmZo*Y+&Sa4DfVrU|?WiET70#1^_221Tz2t
diff --git a/tests/data/acpi/q35/APIC.acpihmat-noinitiator b/tests/data/acpi/q35/APIC.acpihmat-noinitiator
index d904d4a70ddecbb79a83a267af8e26f925e9f4c6..824a279eb7d4134554cb447808bab5a174657272 100644
GIT binary patch
delta 18
ZcmbQhIDwJNF~HM#0s{jBtNTQ*HUKF;1V;b>
delta 18
ZcmbQhIDwJNF~HM#0s{jBqxVFvHUKF)1V;b>
diff --git a/tests/data/acpi/q35/APIC.core-count2 b/tests/data/acpi/q35/APIC.core-count2
index a255082ef5bc39f0d92d3e372b91f09dd6d0d9a1..5b9093a60ee95608cca9560daa7fc07cf7703466 100644
GIT binary patch
delta 19
acmZ1{yiS<QF~HM#9VY_=>zR#QOE>{I`~{i-
delta 19
acmZ1{yiS<QF~HM#9VY_=<Ase}OE>{I_yw8(
diff --git a/tests/data/acpi/q35/APIC.cphp b/tests/data/acpi/q35/APIC.cphp
index 65cc4f4a9aa2676140a6525cdac1e838274b1e07..8451b99c3065f76214023a88ef955de0efc75c60 100644
GIT binary patch
delta 47
scmZ3$xPXz%F~HM#0RsaAE6+qOGf5^Oi-Cig1wym1LTFYtriqd60Hq=YaR2}S
delta 47
scmZ3$xPXz%F~HM#0RsaAqr^lmGf5zWfq{dW1wym1LTFYthKZ5x0Hvk{aR2}S
diff --git a/tests/data/acpi/q35/APIC.dimmpxm b/tests/data/acpi/q35/APIC.dimmpxm
index d904d4a70ddecbb79a83a267af8e26f925e9f4c6..824a279eb7d4134554cb447808bab5a174657272 100644
GIT binary patch
delta 18
ZcmbQhIDwJNF~HM#0s{jBtNTQ*HUKF;1V;b>
delta 18
ZcmbQhIDwJNF~HM#0s{jBqxVFvHUKF)1V;b>
diff --git a/tests/data/acpi/q35/APIC.xapic b/tests/data/acpi/q35/APIC.xapic
index c1969c35aa12b61d25e0134bbb8d2187ba42d663..9cc1db8e078fa3d23d1ab836fea70881a745bb07 100644
GIT binary patch
literal 2686
zcmXxmQ+ym<7=ZEFO*UD?c2e7RlP!`eX;Y-OZKt+v+qP2McB<62(r?~8=Q|hkf1mT4
znVXr56B!p{Pqb>9DJ3e_9vvSMl@Mo-(3e5eG?_(BOHNJGtZ8U47-idRG@^q8Iy$1s
zgicQA?2KkJS}bU_qRoabF6iotZf@xAjvgN9>4{!m=<SU@KIrR<etzihj{yM~7>GeZ
z7#xftAs8BpVPO~^ju8=<E*+*%j~Oyx#*COL6K2kgS+Zc(te7nuX3vf}a$wG!m@60N
z&W(BUVBWl#FCXU5j|B=~!Gc(*5Ed?sMT%h2qFAgL7B7w^N?^&7SgI73E{$c%VA--5
z8HrI*Xt!gza#+4RR;Yj#D`Ips#>8NyN?5rvR;hwjt75fk7#oXmaag@N)~JE;@mRAa
z)~bcIYh#@{n2>;V>telnSie3tXn+kHVxvabxG^?qf=!!Zvu4=5IksqlEn8x%R@k~V
zwrPWH+hV(R*uFh>=ztwNVy8~nxifa@f?c~}w{F<IJND>-J$quWUf8=g_UVIt`(nR-
z*uOsx7=Qx@Vqzi=8ia!f<B%aZbSMrRhQo*Bh!Hq)B#s(|qetVIF*tTCjvI&L$K!+v
zIB_CQnuL=l<CG~lbt+DqhSR6xj2Sp{CeE6LvuER+IXHJN&YOqx=i`C}xNspZT7-)i
z<B}z~bSW-dhRc`ZiWRtWC9Yb9t5@TiHMn*yu3Lxe*W-o_xN#$H+Ju`o<CZPBbt`V$
zhTFH}jvcsjC+^yXyLaQBJ-Bx-?%Rj^_v3*Bc<>+|I)q6{c=#|LIf6%z;<00R{5YOC
zfhSMmsZ)6RG@dzwXV2ofb9nwdUbui4FXE+3c=<A3xq?@(;<amd{W{*bfj4jBty_5e
zHr}~|ckkl8dwBmoK6ro+A7XMcK6-?YALElJ`1C10dxj|~`20D(c!4ip;;UEq`Zd0J
zgKyvByLb5hJ%0FrA3x%!Px$#We))o5zv8!V`29Qn_<=uv;;&!$`#1jigMa^GYN}}Q
z(6oQLGK<o35J<a6SK6Rjls2jsr5#j@(vGS{X_IPE+DWx2?W|gqHmeq;EviLnt7=i&
zrdpJCQ7uZlsurc)REyH?szqrJ)uObgYEjxtwJ7baT9o!tElT^U7Nz}Ei_-q8Md<+5
zqI95YQ94MqC>^X?lnzlXN{6ZzrNdN<(&4H_gNy$CwWL(J8uhbd+J7LE$)xYN%4D(p
WKUwt~*t6NvCaeDRKYi&h(*6VF!XOX;
literal 2686
zcmXZeQ+OOv7=Yo~aI?wAcAeU0vPDuQZHm;k?bNny+g57ZPHkIh=b!JKoA-S43@*-G
z{Lu+<wq%Q@nWAFiZLx`wF-ZxwNPU?!O_RN-X{l+3X8k@%vx&=QJ3C3uY;TVa4(RBJ
zPEP3TjAs1}`ZBw?phf?(w5uz+xuMmH?(XQ}fu5e|<%Qnf=;MRFzUb$N{{9#cfPsM+
z6okRS7!rb^p%@m1;o%q&fsv7zE*+*%j~Oyx#*COL6K2kgS+ZbM6lTqe*|K5w?3g15
z=FEw?a$)Y=m?sbB&5QZ+VgCGBpa2#uh=mGa;lfy?2o^1h#fo9^;#i^tmMn>-N@3~J
zSf&g{M`KJ3+H6?1ES4*W<;!D*3K$!UadB9&B37z|l`CVFDp<8D#>Znq0#>Vr)vIG-
zBG#yZHEUw6T3EX_CM98=I#{<Z)~kp0>tllk*svisYJ`m&W0NM>v?(@ghRvH}ix$|j
zCAMmXty^Q8HrTc;wrhv&+hd0g*s&va>V%y;W0x-2wJUb(hTXekj~>{wC-&-vy?bMy
zKG?S}_UniJ`{RHCn4F9Q2jZYXICwA)8G=KH;;>;jd^nC6fg?xas8Kk2G>#dAW5?pS
zaX5ZFPMClbC*q_@IC(NonSxWN;<RZveLBvVfiq{~tXVjFHqM!YbLZl`c{qPQE?9sI
z7viEtxOg!xS%OQK;<9D9d^xUIfh$+ys#UmpHLh8MYuDnsb+~>#ZrFeuH{zyExOp>f
z*@9cQ;<jzLeLL>hfjf8Nu3fl$H}2Vkd-vkLeYk%=9yovp4`NCR9y)}F595&|c=RY9
zJBG)P<B1b^@+6)*g{M#BnKO9yES@`u=g;GX3wZG&Ub=*rFXNRfc=ak?yN1`V<Bc15
z^CsT9g|~0xojZ8<F5bI`_wVC_2bh|Q4<F*ANBH<LK6!#qpW?G;`20D(c!4ip;;UEq
z`Zd0JgKyvByLb5hJ%0FrA3x%!Px$#We))o5zv8!V`29Qn_<=uv;;&!$`#1jigMa^G
zTAJwMscHX=3n<MXfYNFrFqF14@qj38uUeFLP%TP3surc4REyHiszqtDYEjxnwJ2>-
zElRtp7Ny-(i_%urqO`keQQAYbDDA0Ql=f0BN_(porF~S3(!Q!iX+PDXw7+UmIzY83
z9jIEA4pJ>j2dfsPLsW~>p{hmcFx8@TxN1>4LbYgele0}R{tY{ee%8iO>pNrYvugUz
Y81RgpG2$6JW5}~=7X9ge`jQv^2Net*5C8xG
diff --git a/tests/data/acpi/virt/APIC b/tests/data/acpi/virt/APIC
index 179d274770a23209b949c90a929525e22368568b..318cdea179b244639417efe4fe1acb2a79879e7a 100644
GIT binary patch
delta 18
ZcmZ3(xQ3C-F~HM#4FdxMtJ_4bMF1`e1hxPG
delta 18
ZcmZ3(xQ3C-F~HM#4FdxMi~B^bMF1`d1hxPG
diff --git a/tests/data/acpi/virt/APIC.acpihmatvirt b/tests/data/acpi/virt/APIC.acpihmatvirt
index 68200204c6f8f2706c9896dbbccc5ecbec130d26..a066df3651f9e832546d47b2a0bc8475f830bdc1 100644
GIT binary patch
delta 19
acmbQkJcpUfF~HM#4kH5tYu84uNsIt7;soFT
delta 19
acmbQkJcpUfF~HM#4kH5tOZP^uNsIt7;RN6S
diff --git a/tests/data/acpi/virt/APIC.topology b/tests/data/acpi/virt/APIC.topology
index 3a6ac525e7faeaec025fa6b3fc01dc67110e1296..7f10e34f6ba5af0a5c05acf0ac95f3d0b83e8339 100644
GIT binary patch
delta 19
acmcb^dWV(EF~HOL4if_d>+X$QmzV%QB?cD&
delta 19
acmcb^dWV(EF~HOL4if_d%btx~mzV%QBnB4%
diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index 1e5e354ecf..dfb8523c8b 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1,5 +1 @@
/* List of comma-separated changed AML files to ignore */
-"tests/data/acpi/pc/APIC",
-"tests/data/acpi/q35/APIC",
-"tests/data/acpi/microvm/APIC",
-"tests/data/acpi/virt/APIC",
--
2.31.1
---
tests/data/acpi/microvm/APIC | Bin 70 -> 70 bytes
tests/data/acpi/microvm/APIC.ioapic2 | Bin 82 -> 82 bytes
tests/data/acpi/microvm/APIC.pcie | Bin 110 -> 110 bytes
tests/data/acpi/pc/APIC | Bin 120 -> 120 bytes
tests/data/acpi/pc/APIC.acpihmat | Bin 128 -> 128 bytes
tests/data/acpi/pc/APIC.cphp | Bin 160 -> 160 bytes
tests/data/acpi/pc/APIC.dimmpxm | Bin 144 -> 144 bytes
tests/data/acpi/q35/APIC | Bin 120 -> 120 bytes
tests/data/acpi/q35/APIC.acpihmat | Bin 128 -> 128 bytes
tests/data/acpi/q35/APIC.acpihmat-noinitiator | Bin 144 -> 144 bytes
tests/data/acpi/q35/APIC.core-count2 | Bin 2478 -> 2478 bytes
tests/data/acpi/q35/APIC.cphp | Bin 160 -> 160 bytes
tests/data/acpi/q35/APIC.dimmpxm | Bin 144 -> 144 bytes
tests/data/acpi/q35/APIC.xapic | Bin 2686 -> 2686 bytes
tests/qtest/bios-tables-test-allowed-diff.h | 4 ----
15 files changed, 4 deletions(-)
diff --git a/tests/data/acpi/microvm/APIC b/tests/data/acpi/microvm/APIC
index 68dbd44a7e35a356083f086df60f70e424c4249f..e1c72bd3e60e46acc2c8b3de14f5992946cd55a3 100644
GIT binary patch
delta 16
XcmZ>B<8ln}barE4U|=nn$Ylcn95w?+
delta 16
XcmZ>B<8ln}barE4U|=kn$Ylcn95Mq&
diff --git a/tests/data/acpi/microvm/APIC.ioapic2 b/tests/data/acpi/microvm/APIC.ioapic2
index 3063c52cd3e9bbed29c06031b375900f4a49b9e0..7c4f1b61d6c37f893d0d46be44a00e3f54a27a75 100644
GIT binary patch
delta 16
XcmWFv;&Ke|bPi%*U|_AE$mIb59$o{5
delta 16
XcmWFv;&Ke|bPi%*U|?*X$mIb59$Ev1
diff --git a/tests/data/acpi/microvm/APIC.pcie b/tests/data/acpi/microvm/APIC.pcie
index 4e8f6ed8d6a866429fc17aecdeafc3fb5ef65fa3..23956151765cae221853d92448a619d5576119b5 100644
GIT binary patch
delta 16
Xcmd1H<8ln}bk1X7U|_A7$dv*BBEAFe
delta 16
Xcmd1H<8ln}bk1X7U|_77$dv*BBDw?a
diff --git a/tests/data/acpi/pc/APIC b/tests/data/acpi/pc/APIC
index 208331db53b7dd5c6205cce0e95427636b86dd64..48bdab41959d7895b01491c6136d7403df284f98 100644
GIT binary patch
delta 16
Xcmb=Z;BpM`bgp1vU|?;V$dv~GB#;Co
delta 16
Xcmb=Z;BpM`bgp1vU|{T;$dv~GB#Z<k
diff --git a/tests/data/acpi/pc/APIC.acpihmat b/tests/data/acpi/pc/APIC.acpihmat
index 812c4603f2701494f6bb761570323158a20d4043..57d15150f781504090c8be19fe063836a66ccdc1 100644
GIT binary patch
delta 18
ZcmZo*Y+&Sa4DfVrU|?WiEuP3#1^_261Tz2t
delta 18
ZcmZo*Y+&Sa4DfVrU|?WiET70#1^_221Tz2t
diff --git a/tests/data/acpi/pc/APIC.cphp b/tests/data/acpi/pc/APIC.cphp
index 65cc4f4a9aa2676140a6525cdac1e838274b1e07..8451b99c3065f76214023a88ef955de0efc75c60 100644
GIT binary patch
delta 47
scmZ3$xPXz%F~HM#0RsaAE6+qOGf5^Oi-Cig1wym1LTFYtriqd60Hq=YaR2}S
delta 47
scmZ3$xPXz%F~HM#0RsaAqr^lmGf5zWfq{dW1wym1LTFYthKZ5x0Hvk{aR2}S
diff --git a/tests/data/acpi/pc/APIC.dimmpxm b/tests/data/acpi/pc/APIC.dimmpxm
index d904d4a70ddecbb79a83a267af8e26f925e9f4c6..824a279eb7d4134554cb447808bab5a174657272 100644
GIT binary patch
delta 18
ZcmbQhIDwJNF~HM#0s{jBtNTQ*HUKF;1V;b>
delta 18
ZcmbQhIDwJNF~HM#0s{jBqxVFvHUKF)1V;b>
diff --git a/tests/data/acpi/q35/APIC b/tests/data/acpi/q35/APIC
index 208331db53b7dd5c6205cce0e95427636b86dd64..48bdab41959d7895b01491c6136d7403df284f98 100644
GIT binary patch
delta 16
Xcmb=Z;BpM`bgp1vU|?;V$dv~GB#;Co
delta 16
Xcmb=Z;BpM`bgp1vU|{T;$dv~GB#Z<k
diff --git a/tests/data/acpi/q35/APIC.acpihmat b/tests/data/acpi/q35/APIC.acpihmat
index 812c4603f2701494f6bb761570323158a20d4043..57d15150f781504090c8be19fe063836a66ccdc1 100644
GIT binary patch
delta 18
ZcmZo*Y+&Sa4DfVrU|?WiEuP3#1^_261Tz2t
delta 18
ZcmZo*Y+&Sa4DfVrU|?WiET70#1^_221Tz2t
diff --git a/tests/data/acpi/q35/APIC.acpihmat-noinitiator b/tests/data/acpi/q35/APIC.acpihmat-noinitiator
index d904d4a70ddecbb79a83a267af8e26f925e9f4c6..824a279eb7d4134554cb447808bab5a174657272 100644
GIT binary patch
delta 18
ZcmbQhIDwJNF~HM#0s{jBtNTQ*HUKF;1V;b>
delta 18
ZcmbQhIDwJNF~HM#0s{jBqxVFvHUKF)1V;b>
diff --git a/tests/data/acpi/q35/APIC.core-count2 b/tests/data/acpi/q35/APIC.core-count2
index a255082ef5bc39f0d92d3e372b91f09dd6d0d9a1..5b9093a60ee95608cca9560daa7fc07cf7703466 100644
GIT binary patch
delta 19
acmZ1{yiS<QF~HM#9VY_=>zR#QOE>{I`~{i-
delta 19
acmZ1{yiS<QF~HM#9VY_=<Ase}OE>{I_yw8(
diff --git a/tests/data/acpi/q35/APIC.cphp b/tests/data/acpi/q35/APIC.cphp
index 65cc4f4a9aa2676140a6525cdac1e838274b1e07..8451b99c3065f76214023a88ef955de0efc75c60 100644
GIT binary patch
delta 47
scmZ3$xPXz%F~HM#0RsaAE6+qOGf5^Oi-Cig1wym1LTFYtriqd60Hq=YaR2}S
delta 47
scmZ3$xPXz%F~HM#0RsaAqr^lmGf5zWfq{dW1wym1LTFYthKZ5x0Hvk{aR2}S
diff --git a/tests/data/acpi/q35/APIC.dimmpxm b/tests/data/acpi/q35/APIC.dimmpxm
index d904d4a70ddecbb79a83a267af8e26f925e9f4c6..824a279eb7d4134554cb447808bab5a174657272 100644
GIT binary patch
delta 18
ZcmbQhIDwJNF~HM#0s{jBtNTQ*HUKF;1V;b>
delta 18
ZcmbQhIDwJNF~HM#0s{jBqxVFvHUKF)1V;b>
diff --git a/tests/data/acpi/q35/APIC.xapic b/tests/data/acpi/q35/APIC.xapic
index c1969c35aa12b61d25e0134bbb8d2187ba42d663..9cc1db8e078fa3d23d1ab836fea70881a745bb07 100644
GIT binary patch
literal 2686
zcmXxmQ+ym<7=ZEFO*UD?c2e7RlP!`eX;Y-OZKt+v+qP2McB<62(r?~8=Q|hkf1mT4
znVXr56B!p{Pqb>9DJ3e_9vvSMl@Mo-(3e5eG?_(BOHNJGtZ8U47-idRG@^q8Iy$1s
zgicQA?2KkJS}bU_qRoabF6iotZf@xAjvgN9>4{!m=<SU@KIrR<etzihj{yM~7>GeZ
z7#xftAs8BpVPO~^ju8=<E*+*%j~Oyx#*COL6K2kgS+Zc(te7nuX3vf}a$wG!m@60N
z&W(BUVBWl#FCXU5j|B=~!Gc(*5Ed?sMT%h2qFAgL7B7w^N?^&7SgI73E{$c%VA--5
z8HrI*Xt!gza#+4RR;Yj#D`Ips#>8NyN?5rvR;hwjt75fk7#oXmaag@N)~JE;@mRAa
z)~bcIYh#@{n2>;V>telnSie3tXn+kHVxvabxG^?qf=!!Zvu4=5IksqlEn8x%R@k~V
zwrPWH+hV(R*uFh>=ztwNVy8~nxifa@f?c~}w{F<IJND>-J$quWUf8=g_UVIt`(nR-
z*uOsx7=Qx@Vqzi=8ia!f<B%aZbSMrRhQo*Bh!Hq)B#s(|qetVIF*tTCjvI&L$K!+v
zIB_CQnuL=l<CG~lbt+DqhSR6xj2Sp{CeE6LvuER+IXHJN&YOqx=i`C}xNspZT7-)i
z<B}z~bSW-dhRc`ZiWRtWC9Yb9t5@TiHMn*yu3Lxe*W-o_xN#$H+Ju`o<CZPBbt`V$
zhTFH}jvcsjC+^yXyLaQBJ-Bx-?%Rj^_v3*Bc<>+|I)q6{c=#|LIf6%z;<00R{5YOC
zfhSMmsZ)6RG@dzwXV2ofb9nwdUbui4FXE+3c=<A3xq?@(;<amd{W{*bfj4jBty_5e
zHr}~|ckkl8dwBmoK6ro+A7XMcK6-?YALElJ`1C10dxj|~`20D(c!4ip;;UEq`Zd0J
zgKyvByLb5hJ%0FrA3x%!Px$#We))o5zv8!V`29Qn_<=uv;;&!$`#1jigMa^GYN}}Q
z(6oQLGK<o35J<a6SK6Rjls2jsr5#j@(vGS{X_IPE+DWx2?W|gqHmeq;EviLnt7=i&
zrdpJCQ7uZlsurc)REyH?szqrJ)uObgYEjxtwJ7baT9o!tElT^U7Nz}Ei_-q8Md<+5
zqI95YQ94MqC>^X?lnzlXN{6ZzrNdN<(&4H_gNy$CwWL(J8uhbd+J7LE$)xYN%4D(p
WKUwt~*t6NvCaeDRKYi&h(*6VF!XOX;
literal 2686
zcmXZeQ+OOv7=Yo~aI?wAcAeU0vPDuQZHm;k?bNny+g57ZPHkIh=b!JKoA-S43@*-G
z{Lu+<wq%Q@nWAFiZLx`wF-ZxwNPU?!O_RN-X{l+3X8k@%vx&=QJ3C3uY;TVa4(RBJ
zPEP3TjAs1}`ZBw?phf?(w5uz+xuMmH?(XQ}fu5e|<%Qnf=;MRFzUb$N{{9#cfPsM+
z6okRS7!rb^p%@m1;o%q&fsv7zE*+*%j~Oyx#*COL6K2kgS+ZbM6lTqe*|K5w?3g15
z=FEw?a$)Y=m?sbB&5QZ+VgCGBpa2#uh=mGa;lfy?2o^1h#fo9^;#i^tmMn>-N@3~J
zSf&g{M`KJ3+H6?1ES4*W<;!D*3K$!UadB9&B37z|l`CVFDp<8D#>Znq0#>Vr)vIG-
zBG#yZHEUw6T3EX_CM98=I#{<Z)~kp0>tllk*svisYJ`m&W0NM>v?(@ghRvH}ix$|j
zCAMmXty^Q8HrTc;wrhv&+hd0g*s&va>V%y;W0x-2wJUb(hTXekj~>{wC-&-vy?bMy
zKG?S}_UniJ`{RHCn4F9Q2jZYXICwA)8G=KH;;>;jd^nC6fg?xas8Kk2G>#dAW5?pS
zaX5ZFPMClbC*q_@IC(NonSxWN;<RZveLBvVfiq{~tXVjFHqM!YbLZl`c{qPQE?9sI
z7viEtxOg!xS%OQK;<9D9d^xUIfh$+ys#UmpHLh8MYuDnsb+~>#ZrFeuH{zyExOp>f
z*@9cQ;<jzLeLL>hfjf8Nu3fl$H}2Vkd-vkLeYk%=9yovp4`NCR9y)}F595&|c=RY9
zJBG)P<B1b^@+6)*g{M#BnKO9yES@`u=g;GX3wZG&Ub=*rFXNRfc=ak?yN1`V<Bc15
z^CsT9g|~0xojZ8<F5bI`_wVC_2bh|Q4<F*ANBH<LK6!#qpW?G;`20D(c!4ip;;UEq
z`Zd0JgKyvByLb5hJ%0FrA3x%!Px$#We))o5zv8!V`29Qn_<=uv;;&!$`#1jigMa^G
zTAJwMscHX=3n<MXfYNFrFqF14@qj38uUeFLP%TP3surc4REyHiszqtDYEjxnwJ2>-
zElRtp7Ny-(i_%urqO`keQQAYbDDA0Ql=f0BN_(porF~S3(!Q!iX+PDXw7+UmIzY83
z9jIEA4pJ>j2dfsPLsW~>p{hmcFx8@TxN1>4LbYgele0}R{tY{ee%8iO>pNrYvugUz
Y81RgpG2$6JW5}~=7X9ge`jQv^2Net*5C8xG
diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index 1e5e354ecf..dfb8523c8b 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1,5 +1 @@
/* List of comma-separated changed AML files to ignore */
-"tests/data/acpi/pc/APIC",
-"tests/data/acpi/q35/APIC",
-"tests/data/acpi/microvm/APIC",
-"tests/data/acpi/virt/APIC",
--
2.31.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH v3 0/3] ACPI: i386: bump MADT to revision 5
2023-04-21 21:48 [PATCH v3 0/3] ACPI: i386: bump MADT to revision 5 Eric DeVolder
` (2 preceding siblings ...)
2023-04-21 21:49 ` [PATCH v3 3/3] ACPI: bios-tables-test.c step 5 (updated expected table binaries) Eric DeVolder
@ 2023-05-05 21:53 ` Eric DeVolder
2023-05-10 8:14 ` Igor Mammedov
3 siblings, 1 reply; 9+ messages in thread
From: Eric DeVolder @ 2023-05-05 21:53 UTC (permalink / raw)
To: shannon.zhaosl, mst, imammedo, ani, peter.maydell, qemu-arm,
qemu-devel, marcel.apfelbaum, pbonzini, richard.henderson,
eduardo
Cc: boris.ostrovsky, miguel.luis
Thoughts?
eric
On 4/21/23 16:48, Eric DeVolder wrote:
> The following Linux kernel change broke CPU hotplug for MADT revision
> less than 5.
>
> e2869bd7af60 ("x86/acpi/boot: Do not register processors that cannot be onlined for x2APIC")
>
> Discussion on this topic can be located here:
>
> https://lore.kernel.org/linux-acpi/20230327191026.3454-1-eric.devolder@oracle.com/T/#t
>
> which resulted in the following fixes Linux in 6.3-rc5:
>
> a74fabfbd1b7: ("x86/ACPI/boot: Use FADT version to check support for online capable")
> fed8d8773b8e: ("x86/acpi/boot: Correct acpi_is_processor_usable() check")
>
> However, as part of the investigation into resolving this breakage, I
> learned that i386 QEMU reports revision 1, while technically it
> generates revision 3. Aarch64 generates and reports revision 4.
>
> ACPI 6.3 bumps MADT revision to 5 as it introduces an Online Capable
> flag that the above Linux patch utilizes to denote hot pluggable CPUs.
>
> So in order to bump MADT to the current revision of 5, need to
> validate that all MADT table changes between 1 and 5 are present
> in QEMU.
>
> Below is a table summarizing the changes to the MADT. This information
> gleamed from the ACPI specs on uefi.org.
>
> ACPI MADT What
> Version Version
> 1.0 MADT not present
> 2.0 1 Section 5.2.10.4
> 3.0 2 Section 5.2.11.4
> 5.2.11.13 Local SAPIC Structure added two new fields:
> ACPI Processor UID Value
> ACPI Processor UID String
> 5.2.10.14 Platform Interrupt Sources Structure:
> Reserved changed to Platform Interrupt Sources Flags
> 3.0b 2 Section 5.2.11.4
> Added a section describing guidelines for the ordering of
> processors in the MADT to support proper boot processor
> and multi-threaded logical processor operation.
> 4.0 3 Section 5.2.12
> Adds Processor Local x2APIC structure type 9
> Adds Local x2APIC NMI structure type 0xA
> 5.0 3 Section 5.2.12
> 6.0 3 Section 5.2.12
> 6.0a 4 Section 5.2.12
> Adds ARM GIC structure types 0xB-0xF
> 6.2a 45 Section 5.2.12 <--- version 45, is indeed accurate!
> 6.2b 5 Section 5.2.12
> GIC ITS last Reserved offset changed to 16 from 20 (typo)
> 6.3 5 Section 5.2.12
> Adds Local APIC Flags Online Capable!
> Adds GICC SPE Overflow Interrupt field
> 6.4 5 Section 5.2.12
> Adds Multiprocessor Wakeup Structure type 0x10
> (change notes says structure previously misplaced?)
> 6.5 5 Section 5.2.12
>
> For the MADT revision change 1 -> 2, the spec has a change to the
> SAPIC structure. In general, QEMU does not generate/support SAPIC.
> So the QEMU i386 MADT revision can safely be moved to 2.
>
> For the MADT revision change 2 -> 3, the spec adds Local x2APIC
> structures. QEMU has long supported x2apic ACPI structures. A simple
> search of x2apic within QEMU source and hw/i386/acpi-common.c
> specifically reveals this. So the QEMU i386 MADT revision can safely
> be moved to 3.
>
> For the MADT revision change 3 -> 4, the spec adds support for the ARM
> GIC structures. QEMU ARM does in fact generate and report revision 4.
> As these will not be used by i386 QEMU, so then the QEMU i386 MADT
> revision can safely be moved to 4 as well.
>
> Now for the MADT revision change 4 -> 5, the spec adds the Online
> Capable flag to the Local APIC structure, and the ARM GICC SPE
> Overflow Interrupt field.
>
> For i386, the ARM SPE is not applicable.
>
> For the i386 Local APIC flag Online Capable, the spec has certain rules
> about this value. And in particuar setting this value now explicitly
> indicates a hotpluggable CPU.
>
> So this patch makes the needed changes to move i386 MADT to
> revision 5.
>
> Without these changes, the information below shows "how" CPU hotplug
> breaks with the current upstream Linux kernel 6.3. For example, a Linux
> guest started with:
>
> qemu-system-x86_64 -smp 30,maxcpus=32 ...
>
> and then attempting to hotplug a CPU:
>
> (QEMU) device_add id=cpu30 driver=host-x86_64-cpu socket-id=0 core-id=30 thread-id=0
>
> fails with the following:
>
> APIC: NR_CPUS/possible_cpus limit of 30 reached. Processor 30/0x.
> ACPI: Unable to map lapic to logical cpu number
> acpi LNXCPU:1e: Enumeration failure
>
> # dmesg | grep smpboot
> smpboot: Allowing 30 CPUs, 0 hotplug CPUs
> smpboot: CPU0: Intel(R) Xeon(R) CPU D-1533 @ 2.10GHz (family: 0x)
> smpboot: Max logical packages: 1
> smpboot: Total of 30 processors activated (125708.76 BogoMIPS)
>
> # iasl -d /sys/firmware/tables/acpi/APIC
> [000h 0000 4] Signature : "APIC" [Multiple APIC Descript
> [004h 0004 4] Table Length : 00000170
> [008h 0008 1] Revision : 01 <=====
> [009h 0009 1] Checksum : 9C
> [00Ah 0010 6] Oem ID : "BOCHS "
> [010h 0016 8] Oem Table ID : "BXPC "
> [018h 0024 4] Oem Revision : 00000001
> [01Ch 0028 4] Asl Compiler ID : "BXPC"
> [020h 0032 4] Asl Compiler Revision : 00000001
>
> ...
>
> [114h 0276 1] Subtable Type : 00 [Processor Local APIC]
> [115h 0277 1] Length : 08
> [116h 0278 1] Processor ID : 1D
> [117h 0279 1] Local Apic ID : 1D
> [118h 0280 4] Flags (decoded below) : 00000001
> Processor Enabled : 1 <=====
>
> [11Ch 0284 1] Subtable Type : 00 [Processor Local APIC]
> [11Dh 0285 1] Length : 08
> [11Eh 0286 1] Processor ID : 1E
> [11Fh 0287 1] Local Apic ID : 1E
> [120h 0288 4] Flags (decoded below) : 00000000
> Processor Enabled : 0 <=====
>
> [124h 0292 1] Subtable Type : 00 [Processor Local APIC]
> [125h 0293 1] Length : 08
> [126h 0294 1] Processor ID : 1F
> [127h 0295 1] Local Apic ID : 1F
> [128h 0296 4] Flags (decoded below) : 00000000
> Processor Enabled : 0 <=====
>
> The (latest upstream) Linux kernel sees 30 Enabled processors, and
> does not consider processors 31 and 32 to be hotpluggable.
>
> With this patch series applied, by bumping MADT to revision 5, the
> latest upstream Linux kernel correctly identifies 30 CPUs plus 2
> hotpluggable CPUS.
>
> CPU30 has been hot-added
> smpboot: Booting Node 0 Processor 30 APIC 0x1e
> Will online and init hotplugged CPU: 30
>
> # dmesg | grep smpboot
> smpboot: Allowing 32 CPUs, 2 hotplug CPUs
> smpboot: CPU0: Intel(R) Xeon(R) CPU D-1533 @ 2.10GHz (family: 0x6, model: 0x56, stepping: 0x3)
> smpboot: Max logical packages: 2
> smpboot: Total of 30 processors activated (125708.76 BogoMIPS)
>
> # iasl -d /sys/firmware/tables/acpi/APIC
> [000h 0000 004h] Signature : "APIC" [Multiple APIC Descript
> [004h 0004 004h] Table Length : 00000170
> [008h 0008 001h] Revision : 05 <=====
> [009h 0009 001h] Checksum : 94
> [00Ah 0010 006h] Oem ID : "BOCHS "
> [010h 0016 008h] Oem Table ID : "BXPC "
> [018h 0024 004h] Oem Revision : 00000001
> [01Ch 0028 004h] Asl Compiler ID : "BXPC"
> [020h 0032 004h] Asl Compiler Revision : 00000001
>
> ...
>
> [114h 0276 001h] Subtable Type : 00 [Processor Local APIC]
> [115h 0277 001h] Length : 08
> [116h 0278 001h] Processor ID : 1D
> [117h 0279 001h] Local Apic ID : 1D
> [118h 0280 004h] Flags (decoded below) : 00000001
> Processor Enabled : 1 <=====
> Runtime Online Capable : 0 <=====
>
> [11Ch 0284 001h] Subtable Type : 00 [Processor Local APIC]
> [11Dh 0285 001h] Length : 08
> [11Eh 0286 001h] Processor ID : 1E
> [11Fh 0287 001h] Local Apic ID : 1E
> [120h 0288 004h] Flags (decoded below) : 00000002
> Processor Enabled : 0 <=====
> Runtime Online Capable : 1 <=====
>
> [124h 0292 001h] Subtable Type : 00 [Processor Local APIC]
> [125h 0293 001h] Length : 08
> [126h 0294 001h] Processor ID : 1F
> [127h 0295 001h] Local Apic ID : 1F
> [128h 0296 004h] Flags (decoded below) : 00000002
> Processor Enabled : 0 <=====
> Runtime Online Capable : 1 <=====
>
> In terms of testing older operating systems, I did:
>
> - Windows Server 2008.
> By opening Device Manager -> Processors, and watching/counting
> the listed CPUs before and after hot un/plug operations.
> When hot plugging a CPU, the new processor shows in the list;
> one more processor is available after the hotplug.
> When hot unplugging a CPU, Windows refuses, with the message:
>
> The 'Intel(R) Xeon(R) CPU D-1533 @ 2.10GHz' device is not
> removable and cannot be ejected or unplugged.
>
> This is expected as hot unplug did not work in Windows 2008.
>
> - RHEL 6.9
> Kernel 2.6.32-696.el6.x86_64
> Build Feb 21 2017
> From dmesg:
> ACPI: APIC 000000007ffe32f0 000F0 (v05 BOCHS BXPC 00000001 BXPC 00000001)
> SMP: Allowing 16 CPUs, 8 hotplug CPUs
>
> # cat /sys/devices/system/cpu/online
> 0-7
>
> (QEMU) device_add id=cpu8 driver=host-x86_64-cpu socket-id=0 core-id=8 thread-id=0
>
> CPU 8 got hotplugged
> Booting Node 0 Processor 8 APIC 0x8
> kvm-clock: cpu 8, msr 2830ed00
> Will online and init hotplugged CPU: 8
> microcode: CPU8 sig=0x50663, pf=0x1, revision=0x700001c
> platform microcode: firmware: requesting intel-ucode/06-56-03
>
> # cat /sys/devices/system/cpu/online
> 0-8
>
> (QEMU) device_del id=cpu8
>
> Broke affinity for irq 24
> CPU 8 is now offline
>
> # cat /sys/devices/system/cpu/online
> 0-7
>
> RHEL 6.9
> kernel 2.6.32-696.el6.x86_64
> build Feb 21 2017
>
> Both of the above 'legacy' operating systems behaved as expected.
>
> Regards,
> Eric
>
> ---
> v3: 21apr2023
> - Dropped ARM patch as there is no compelling need to move to
> MADT revision 5, per Michael Tsirkin.
> - Recoded the flags to be simpler, per Michael, Ani Sinha and
> Miguel Luis.
> - Regenerated the binary tables, as per bios-tables-test.c
> instructions.
>
> v2: 18apr2023
> https://lists.gnu.org/archive/html/qemu-devel/2023-04/msg02576.html
> - Code formatting changes per Igor Mammedov
> - Included the steps in bios-tables-test.
>
> v1: 28mar2023
> https://lists.gnu.org/archive/html/qemu-devel/2023-03/msg06705.html
> - Initial series for ARM and i386 MADT revision bump to 5.
> Eric DeVolder (3):
> ACPI: bios-tables-test.c step 2 (allowed-diff entries)
> hw/acpi: i386: bump MADT to revision 5
> ACPI: bios-tables-test.c step 5 (updated expected table binaries)
>
> hw/i386/acpi-common.c | 4 ++--
> tests/data/acpi/microvm/APIC | Bin 70 -> 70 bytes
> tests/data/acpi/microvm/APIC.ioapic2 | Bin 82 -> 82 bytes
> tests/data/acpi/microvm/APIC.pcie | Bin 110 -> 110 bytes
> tests/data/acpi/pc/APIC | Bin 120 -> 120 bytes
> tests/data/acpi/pc/APIC.acpihmat | Bin 128 -> 128 bytes
> tests/data/acpi/pc/APIC.cphp | Bin 160 -> 160 bytes
> tests/data/acpi/pc/APIC.dimmpxm | Bin 144 -> 144 bytes
> tests/data/acpi/q35/APIC | Bin 120 -> 120 bytes
> tests/data/acpi/q35/APIC.acpihmat | Bin 128 -> 128 bytes
> tests/data/acpi/q35/APIC.acpihmat-noinitiator | Bin 144 -> 144 bytes
> tests/data/acpi/q35/APIC.core-count2 | Bin 2478 -> 2478 bytes
> tests/data/acpi/q35/APIC.cphp | Bin 160 -> 160 bytes
> tests/data/acpi/q35/APIC.dimmpxm | Bin 144 -> 144 bytes
> tests/data/acpi/q35/APIC.xapic | Bin 2686 -> 2686 bytes
> 15 files changed, 2 insertions(+), 2 deletions(-)
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v3 0/3] ACPI: i386: bump MADT to revision 5
2023-05-05 21:53 ` [PATCH v3 0/3] ACPI: i386: bump MADT to revision 5 Eric DeVolder
@ 2023-05-10 8:14 ` Igor Mammedov
2023-05-10 15:08 ` Eric DeVolder
0 siblings, 1 reply; 9+ messages in thread
From: Igor Mammedov @ 2023-05-10 8:14 UTC (permalink / raw)
To: Eric DeVolder
Cc: shannon.zhaosl, mst, ani, peter.maydell, qemu-arm, qemu-devel,
marcel.apfelbaum, pbonzini, richard.henderson, eduardo,
boris.ostrovsky, miguel.luis
On Fri, 5 May 2023 16:53:22 -0500
Eric DeVolder <eric.devolder@oracle.com> wrote:
> Thoughts?
I still don't think we need to bump x86 to rev 5 in QEMU.
> eric
>
> On 4/21/23 16:48, Eric DeVolder wrote:
> > The following Linux kernel change broke CPU hotplug for MADT revision
> > less than 5.
> >
> > e2869bd7af60 ("x86/acpi/boot: Do not register processors that cannot be onlined for x2APIC")
> >
> > Discussion on this topic can be located here:
> >
> > https://lore.kernel.org/linux-acpi/20230327191026.3454-1-eric.devolder@oracle.com/T/#t
have your kernel fix landed up upstream?
> >
> > which resulted in the following fixes Linux in 6.3-rc5:
> >
> > a74fabfbd1b7: ("x86/ACPI/boot: Use FADT version to check support for online capable")
> > fed8d8773b8e: ("x86/acpi/boot: Correct acpi_is_processor_usable() check")
> >
> > However, as part of the investigation into resolving this breakage, I
> > learned that i386 QEMU reports revision 1, while technically it
> > generates revision 3. Aarch64 generates and reports revision 4.
> >
> > ACPI 6.3 bumps MADT revision to 5 as it introduces an Online Capable
> > flag that the above Linux patch utilizes to denote hot pluggable CPUs.
> >
> > So in order to bump MADT to the current revision of 5, need to
> > validate that all MADT table changes between 1 and 5 are present
> > in QEMU.
> >
> > Below is a table summarizing the changes to the MADT. This information
> > gleamed from the ACPI specs on uefi.org.
> >
> > ACPI MADT What
> > Version Version
> > 1.0 MADT not present
> > 2.0 1 Section 5.2.10.4
> > 3.0 2 Section 5.2.11.4
> > 5.2.11.13 Local SAPIC Structure added two new fields:
> > ACPI Processor UID Value
> > ACPI Processor UID String
> > 5.2.10.14 Platform Interrupt Sources Structure:
> > Reserved changed to Platform Interrupt Sources Flags
> > 3.0b 2 Section 5.2.11.4
> > Added a section describing guidelines for the ordering of
> > processors in the MADT to support proper boot processor
> > and multi-threaded logical processor operation.
> > 4.0 3 Section 5.2.12
> > Adds Processor Local x2APIC structure type 9
> > Adds Local x2APIC NMI structure type 0xA
> > 5.0 3 Section 5.2.12
> > 6.0 3 Section 5.2.12
> > 6.0a 4 Section 5.2.12
> > Adds ARM GIC structure types 0xB-0xF
> > 6.2a 45 Section 5.2.12 <--- version 45, is indeed accurate!
> > 6.2b 5 Section 5.2.12
> > GIC ITS last Reserved offset changed to 16 from 20 (typo)
> > 6.3 5 Section 5.2.12
> > Adds Local APIC Flags Online Capable!
> > Adds GICC SPE Overflow Interrupt field
> > 6.4 5 Section 5.2.12
> > Adds Multiprocessor Wakeup Structure type 0x10
> > (change notes says structure previously misplaced?)
> > 6.5 5 Section 5.2.12
> >
> > For the MADT revision change 1 -> 2, the spec has a change to the
> > SAPIC structure. In general, QEMU does not generate/support SAPIC.
> > So the QEMU i386 MADT revision can safely be moved to 2.
> >
> > For the MADT revision change 2 -> 3, the spec adds Local x2APIC
> > structures. QEMU has long supported x2apic ACPI structures. A simple
> > search of x2apic within QEMU source and hw/i386/acpi-common.c
> > specifically reveals this. So the QEMU i386 MADT revision can safely
> > be moved to 3.
> >
> > For the MADT revision change 3 -> 4, the spec adds support for the ARM
> > GIC structures. QEMU ARM does in fact generate and report revision 4.
> > As these will not be used by i386 QEMU, so then the QEMU i386 MADT
> > revision can safely be moved to 4 as well.
> >
> > Now for the MADT revision change 4 -> 5, the spec adds the Online
> > Capable flag to the Local APIC structure, and the ARM GICC SPE
> > Overflow Interrupt field.
All ARM stuff is irrelevant in x86 patch
> > For i386, the ARM SPE is not applicable.
> >
> > For the i386 Local APIC flag Online Capable, the spec has certain rules
> > about this value. And in particuar setting this value now explicitly
> > indicates a hotpluggable CPU.
> >
> > So this patch makes the needed changes to move i386 MADT to
> > revision 5.
> >
> > Without these changes, the information below shows "how" CPU hotplug
> > breaks with the current upstream Linux kernel 6.3. For example, a Linux
> > guest started with:
what's the status wrt current upstream kernel?
> > qemu-system-x86_64 -smp 30,maxcpus=32 ...
> >
> > and then attempting to hotplug a CPU:
> >
> > (QEMU) device_add id=cpu30 driver=host-x86_64-cpu socket-id=0 core-id=30 thread-id=0
> >
> > fails with the following:
> >
> > APIC: NR_CPUS/possible_cpus limit of 30 reached. Processor 30/0x.
> > ACPI: Unable to map lapic to logical cpu number
> > acpi LNXCPU:1e: Enumeration failure
> >
> > # dmesg | grep smpboot
> > smpboot: Allowing 30 CPUs, 0 hotplug CPUs
> > smpboot: CPU0: Intel(R) Xeon(R) CPU D-1533 @ 2.10GHz (family: 0x)
> > smpboot: Max logical packages: 1
> > smpboot: Total of 30 processors activated (125708.76 BogoMIPS)
> >
> > # iasl -d /sys/firmware/tables/acpi/APIC
> > [000h 0000 4] Signature : "APIC" [Multiple APIC Descript
> > [004h 0004 4] Table Length : 00000170
> > [008h 0008 1] Revision : 01 <=====
> > [009h 0009 1] Checksum : 9C
> > [00Ah 0010 6] Oem ID : "BOCHS "
> > [010h 0016 8] Oem Table ID : "BXPC "
> > [018h 0024 4] Oem Revision : 00000001
> > [01Ch 0028 4] Asl Compiler ID : "BXPC"
> > [020h 0032 4] Asl Compiler Revision : 00000001
> >
> > ...
> >
> > [114h 0276 1] Subtable Type : 00 [Processor Local APIC]
> > [115h 0277 1] Length : 08
> > [116h 0278 1] Processor ID : 1D
> > [117h 0279 1] Local Apic ID : 1D
> > [118h 0280 4] Flags (decoded below) : 00000001
> > Processor Enabled : 1 <=====
> >
> > [11Ch 0284 1] Subtable Type : 00 [Processor Local APIC]
> > [11Dh 0285 1] Length : 08
> > [11Eh 0286 1] Processor ID : 1E
> > [11Fh 0287 1] Local Apic ID : 1E
> > [120h 0288 4] Flags (decoded below) : 00000000
> > Processor Enabled : 0 <=====
> >
> > [124h 0292 1] Subtable Type : 00 [Processor Local APIC]
> > [125h 0293 1] Length : 08
> > [126h 0294 1] Processor ID : 1F
> > [127h 0295 1] Local Apic ID : 1F
> > [128h 0296 4] Flags (decoded below) : 00000000
> > Processor Enabled : 0 <=====
> >
> > The (latest upstream) Linux kernel sees 30 Enabled processors, and
> > does not consider processors 31 and 32 to be hotpluggable.
> >
> > With this patch series applied, by bumping MADT to revision 5, the
> > latest upstream Linux kernel correctly identifies 30 CPUs plus 2
> > hotpluggable CPUS.
> >
> > CPU30 has been hot-added
> > smpboot: Booting Node 0 Processor 30 APIC 0x1e
> > Will online and init hotplugged CPU: 30
> >
> > # dmesg | grep smpboot
> > smpboot: Allowing 32 CPUs, 2 hotplug CPUs
> > smpboot: CPU0: Intel(R) Xeon(R) CPU D-1533 @ 2.10GHz (family: 0x6, model: 0x56, stepping: 0x3)
> > smpboot: Max logical packages: 2
> > smpboot: Total of 30 processors activated (125708.76 BogoMIPS)
> >
> > # iasl -d /sys/firmware/tables/acpi/APIC
> > [000h 0000 004h] Signature : "APIC" [Multiple APIC Descript
> > [004h 0004 004h] Table Length : 00000170
> > [008h 0008 001h] Revision : 05 <=====
> > [009h 0009 001h] Checksum : 94
> > [00Ah 0010 006h] Oem ID : "BOCHS "
> > [010h 0016 008h] Oem Table ID : "BXPC "
> > [018h 0024 004h] Oem Revision : 00000001
> > [01Ch 0028 004h] Asl Compiler ID : "BXPC"
> > [020h 0032 004h] Asl Compiler Revision : 00000001
> >
> > ...
> >
> > [114h 0276 001h] Subtable Type : 00 [Processor Local APIC]
> > [115h 0277 001h] Length : 08
> > [116h 0278 001h] Processor ID : 1D
> > [117h 0279 001h] Local Apic ID : 1D
> > [118h 0280 004h] Flags (decoded below) : 00000001
> > Processor Enabled : 1 <=====
> > Runtime Online Capable : 0 <=====
> >
> > [11Ch 0284 001h] Subtable Type : 00 [Processor Local APIC]
> > [11Dh 0285 001h] Length : 08
> > [11Eh 0286 001h] Processor ID : 1E
> > [11Fh 0287 001h] Local Apic ID : 1E
> > [120h 0288 004h] Flags (decoded below) : 00000002
> > Processor Enabled : 0 <=====
> > Runtime Online Capable : 1 <=====
> >
> > [124h 0292 001h] Subtable Type : 00 [Processor Local APIC]
> > [125h 0293 001h] Length : 08
> > [126h 0294 001h] Processor ID : 1F
> > [127h 0295 001h] Local Apic ID : 1F
> > [128h 0296 004h] Flags (decoded below) : 00000002
> > Processor Enabled : 0 <=====
> > Runtime Online Capable : 1 <=====
> >
> > In terms of testing older operating systems, I did:
> >
> > - Windows Server 2008.
> > By opening Device Manager -> Processors, and watching/counting
> > the listed CPUs before and after hot un/plug operations.
> > When hot plugging a CPU, the new processor shows in the list;
> > one more processor is available after the hotplug.
> > When hot unplugging a CPU, Windows refuses, with the message:
> >
> > The 'Intel(R) Xeon(R) CPU D-1533 @ 2.10GHz' device is not
> > removable and cannot be ejected or unplugged.
> >
> > This is expected as hot unplug did not work in Windows 2008.
> >
> > - RHEL 6.9
> > Kernel 2.6.32-696.el6.x86_64
> > Build Feb 21 2017
> > From dmesg:
> > ACPI: APIC 000000007ffe32f0 000F0 (v05 BOCHS BXPC 00000001 BXPC 00000001)
> > SMP: Allowing 16 CPUs, 8 hotplug CPUs
> >
> > # cat /sys/devices/system/cpu/online
> > 0-7
> >
> > (QEMU) device_add id=cpu8 driver=host-x86_64-cpu socket-id=0 core-id=8 thread-id=0
> >
> > CPU 8 got hotplugged
> > Booting Node 0 Processor 8 APIC 0x8
> > kvm-clock: cpu 8, msr 2830ed00
> > Will online and init hotplugged CPU: 8
> > microcode: CPU8 sig=0x50663, pf=0x1, revision=0x700001c
> > platform microcode: firmware: requesting intel-ucode/06-56-03
> >
> > # cat /sys/devices/system/cpu/online
> > 0-8
> >
> > (QEMU) device_del id=cpu8
> >
> > Broke affinity for irq 24
> > CPU 8 is now offline
> >
> > # cat /sys/devices/system/cpu/online
> > 0-7
> >
> > RHEL 6.9
> > kernel 2.6.32-696.el6.x86_64
> > build Feb 21 2017
> >
> > Both of the above 'legacy' operating systems behaved as expected.
> >
> > Regards,
> > Eric
> >
> > ---
> > v3: 21apr2023
> > - Dropped ARM patch as there is no compelling need to move to
> > MADT revision 5, per Michael Tsirkin.
> > - Recoded the flags to be simpler, per Michael, Ani Sinha and
> > Miguel Luis.
> > - Regenerated the binary tables, as per bios-tables-test.c
> > instructions.
> >
> > v2: 18apr2023
> > https://lists.gnu.org/archive/html/qemu-devel/2023-04/msg02576.html
> > - Code formatting changes per Igor Mammedov
> > - Included the steps in bios-tables-test.
> >
> > v1: 28mar2023
> > https://lists.gnu.org/archive/html/qemu-devel/2023-03/msg06705.html
> > - Initial series for ARM and i386 MADT revision bump to 5.
> > Eric DeVolder (3):
> > ACPI: bios-tables-test.c step 2 (allowed-diff entries)
> > hw/acpi: i386: bump MADT to revision 5
> > ACPI: bios-tables-test.c step 5 (updated expected table binaries)
> >
> > hw/i386/acpi-common.c | 4 ++--
> > tests/data/acpi/microvm/APIC | Bin 70 -> 70 bytes
> > tests/data/acpi/microvm/APIC.ioapic2 | Bin 82 -> 82 bytes
> > tests/data/acpi/microvm/APIC.pcie | Bin 110 -> 110 bytes
> > tests/data/acpi/pc/APIC | Bin 120 -> 120 bytes
> > tests/data/acpi/pc/APIC.acpihmat | Bin 128 -> 128 bytes
> > tests/data/acpi/pc/APIC.cphp | Bin 160 -> 160 bytes
> > tests/data/acpi/pc/APIC.dimmpxm | Bin 144 -> 144 bytes
> > tests/data/acpi/q35/APIC | Bin 120 -> 120 bytes
> > tests/data/acpi/q35/APIC.acpihmat | Bin 128 -> 128 bytes
> > tests/data/acpi/q35/APIC.acpihmat-noinitiator | Bin 144 -> 144 bytes
> > tests/data/acpi/q35/APIC.core-count2 | Bin 2478 -> 2478 bytes
> > tests/data/acpi/q35/APIC.cphp | Bin 160 -> 160 bytes
> > tests/data/acpi/q35/APIC.dimmpxm | Bin 144 -> 144 bytes
> > tests/data/acpi/q35/APIC.xapic | Bin 2686 -> 2686 bytes
> > 15 files changed, 2 insertions(+), 2 deletions(-)
> >
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v3 0/3] ACPI: i386: bump MADT to revision 5
2023-05-10 8:14 ` Igor Mammedov
@ 2023-05-10 15:08 ` Eric DeVolder
2023-05-10 15:45 ` Igor Mammedov
0 siblings, 1 reply; 9+ messages in thread
From: Eric DeVolder @ 2023-05-10 15:08 UTC (permalink / raw)
To: Igor Mammedov
Cc: shannon.zhaosl, mst, ani, peter.maydell, qemu-arm, qemu-devel,
marcel.apfelbaum, pbonzini, richard.henderson, eduardo,
boris.ostrovsky, miguel.luis
On 5/10/23 03:14, Igor Mammedov wrote:
> On Fri, 5 May 2023 16:53:22 -0500
> Eric DeVolder <eric.devolder@oracle.com> wrote:
>
>> Thoughts?
>
> I still don't think we need to bump x86 to rev 5 in QEMU.
Linux v6.3 has the fix merged (so crisis averted 8).
The investigation allowed me to opportunistically provide this patch.
I think this should receive serious consideration for merging,
more so because generating MADT .revision 3 and reporting .revision 1
seems wrong to me.
This patch seems really straight forward, and low risk, now.
>
>> eric
>>
>> On 4/21/23 16:48, Eric DeVolder wrote:
>>> The following Linux kernel change broke CPU hotplug for MADT revision
>>> less than 5.
>>>
>>> e2869bd7af60 ("x86/acpi/boot: Do not register processors that cannot be onlined for x2APIC")
>>>
>>> Discussion on this topic can be located here:
>>>
>>> https://lore.kernel.org/linux-acpi/20230327191026.3454-1-eric.devolder@oracle.com/T/#t
>
> have your kernel fix landed up upstream?
>
Yes, merged and available in v6.3.
>>>
>>> which resulted in the following fixes Linux in 6.3-rc5:
>>>
>>> a74fabfbd1b7: ("x86/ACPI/boot: Use FADT version to check support for online capable")
>>> fed8d8773b8e: ("x86/acpi/boot: Correct acpi_is_processor_usable() check")
>>>
>>> However, as part of the investigation into resolving this breakage, I
>>> learned that i386 QEMU reports revision 1, while technically it
>>> generates revision 3. Aarch64 generates and reports revision 4.
>>>
>>> ACPI 6.3 bumps MADT revision to 5 as it introduces an Online Capable
>>> flag that the above Linux patch utilizes to denote hot pluggable CPUs.
>>>
>>> So in order to bump MADT to the current revision of 5, need to
>>> validate that all MADT table changes between 1 and 5 are present
>>> in QEMU.
>>>
>>> Below is a table summarizing the changes to the MADT. This information
>>> gleamed from the ACPI specs on uefi.org.
>>>
>>> ACPI MADT What
>>> Version Version
>>> 1.0 MADT not present
>>> 2.0 1 Section 5.2.10.4
>>> 3.0 2 Section 5.2.11.4
>>> 5.2.11.13 Local SAPIC Structure added two new fields:
>>> ACPI Processor UID Value
>>> ACPI Processor UID String
>>> 5.2.10.14 Platform Interrupt Sources Structure:
>>> Reserved changed to Platform Interrupt Sources Flags
>>> 3.0b 2 Section 5.2.11.4
>>> Added a section describing guidelines for the ordering of
>>> processors in the MADT to support proper boot processor
>>> and multi-threaded logical processor operation.
>>> 4.0 3 Section 5.2.12
>>> Adds Processor Local x2APIC structure type 9
>>> Adds Local x2APIC NMI structure type 0xA
>>> 5.0 3 Section 5.2.12
>>> 6.0 3 Section 5.2.12
>>> 6.0a 4 Section 5.2.12
>>> Adds ARM GIC structure types 0xB-0xF
>>> 6.2a 45 Section 5.2.12 <--- version 45, is indeed accurate!
>>> 6.2b 5 Section 5.2.12
>>> GIC ITS last Reserved offset changed to 16 from 20 (typo)
>>> 6.3 5 Section 5.2.12
>>> Adds Local APIC Flags Online Capable!
>>> Adds GICC SPE Overflow Interrupt field
>>> 6.4 5 Section 5.2.12
>>> Adds Multiprocessor Wakeup Structure type 0x10
>>> (change notes says structure previously misplaced?)
>>> 6.5 5 Section 5.2.12
>>>
>>> For the MADT revision change 1 -> 2, the spec has a change to the
>>> SAPIC structure. In general, QEMU does not generate/support SAPIC.
>>> So the QEMU i386 MADT revision can safely be moved to 2.
>>>
>>> For the MADT revision change 2 -> 3, the spec adds Local x2APIC
>>> structures. QEMU has long supported x2apic ACPI structures. A simple
>>> search of x2apic within QEMU source and hw/i386/acpi-common.c
>>> specifically reveals this. So the QEMU i386 MADT revision can safely
>>> be moved to 3.
>>>
>>> For the MADT revision change 3 -> 4, the spec adds support for the ARM
>>> GIC structures. QEMU ARM does in fact generate and report revision 4.
>>> As these will not be used by i386 QEMU, so then the QEMU i386 MADT
>>> revision can safely be moved to 4 as well.
>>>
>>> Now for the MADT revision change 4 -> 5, the spec adds the Online
>>> Capable flag to the Local APIC structure, and the ARM GICC SPE
>>> Overflow Interrupt field.
>
> All ARM stuff is irrelevant in x86 patch
>
sure
>>> For i386, the ARM SPE is not applicable.
>>>
>>> For the i386 Local APIC flag Online Capable, the spec has certain rules
>>> about this value. And in particuar setting this value now explicitly
>>> indicates a hotpluggable CPU.
>>>
>>> So this patch makes the needed changes to move i386 MADT to
>>> revision 5.
>>>
>>> Without these changes, the information below shows "how" CPU hotplug
>>> breaks with the current upstream Linux kernel 6.3. For example, a Linux
>>> guest started with:
>
> what's the status wrt current upstream kernel?
>
qemu cpu hotplug works with upstream kernel ( >= v6.3) but only because the
kernel fix allows the legacy fallback assumption that the non Enabled
cpus specified in MADT are online capable.
with this patch applied to qemu, the kernel honors the Online Capable bit
for cpus not Enabled at boot.
>>> qemu-system-x86_64 -smp 30,maxcpus=32 ...
>>>
>>> and then attempting to hotplug a CPU:
>>>
>>> (QEMU) device_add id=cpu30 driver=host-x86_64-cpu socket-id=0 core-id=30 thread-id=0
>>>
>>> fails with the following:
>>>
>>> APIC: NR_CPUS/possible_cpus limit of 30 reached. Processor 30/0x.
>>> ACPI: Unable to map lapic to logical cpu number
>>> acpi LNXCPU:1e: Enumeration failure
>>>
>>> # dmesg | grep smpboot
>>> smpboot: Allowing 30 CPUs, 0 hotplug CPUs
>>> smpboot: CPU0: Intel(R) Xeon(R) CPU D-1533 @ 2.10GHz (family: 0x)
>>> smpboot: Max logical packages: 1
>>> smpboot: Total of 30 processors activated (125708.76 BogoMIPS)
>>>
>>> # iasl -d /sys/firmware/tables/acpi/APIC
>>> [000h 0000 4] Signature : "APIC" [Multiple APIC Descript
>>> [004h 0004 4] Table Length : 00000170
>>> [008h 0008 1] Revision : 01 <=====
>>> [009h 0009 1] Checksum : 9C
>>> [00Ah 0010 6] Oem ID : "BOCHS "
>>> [010h 0016 8] Oem Table ID : "BXPC "
>>> [018h 0024 4] Oem Revision : 00000001
>>> [01Ch 0028 4] Asl Compiler ID : "BXPC"
>>> [020h 0032 4] Asl Compiler Revision : 00000001
>>>
>>> ...
>>>
>>> [114h 0276 1] Subtable Type : 00 [Processor Local APIC]
>>> [115h 0277 1] Length : 08
>>> [116h 0278 1] Processor ID : 1D
>>> [117h 0279 1] Local Apic ID : 1D
>>> [118h 0280 4] Flags (decoded below) : 00000001
>>> Processor Enabled : 1 <=====
>>>
>>> [11Ch 0284 1] Subtable Type : 00 [Processor Local APIC]
>>> [11Dh 0285 1] Length : 08
>>> [11Eh 0286 1] Processor ID : 1E
>>> [11Fh 0287 1] Local Apic ID : 1E
>>> [120h 0288 4] Flags (decoded below) : 00000000
>>> Processor Enabled : 0 <=====
>>>
>>> [124h 0292 1] Subtable Type : 00 [Processor Local APIC]
>>> [125h 0293 1] Length : 08
>>> [126h 0294 1] Processor ID : 1F
>>> [127h 0295 1] Local Apic ID : 1F
>>> [128h 0296 4] Flags (decoded below) : 00000000
>>> Processor Enabled : 0 <=====
>>>
>>> The (latest upstream) Linux kernel sees 30 Enabled processors, and
>>> does not consider processors 31 and 32 to be hotpluggable.
>>>
>>> With this patch series applied, by bumping MADT to revision 5, the
>>> latest upstream Linux kernel correctly identifies 30 CPUs plus 2
>>> hotpluggable CPUS.
>>>
>>> CPU30 has been hot-added
>>> smpboot: Booting Node 0 Processor 30 APIC 0x1e
>>> Will online and init hotplugged CPU: 30
>>>
>>> # dmesg | grep smpboot
>>> smpboot: Allowing 32 CPUs, 2 hotplug CPUs
>>> smpboot: CPU0: Intel(R) Xeon(R) CPU D-1533 @ 2.10GHz (family: 0x6, model: 0x56, stepping: 0x3)
>>> smpboot: Max logical packages: 2
>>> smpboot: Total of 30 processors activated (125708.76 BogoMIPS)
>>>
>>> # iasl -d /sys/firmware/tables/acpi/APIC
>>> [000h 0000 004h] Signature : "APIC" [Multiple APIC Descript
>>> [004h 0004 004h] Table Length : 00000170
>>> [008h 0008 001h] Revision : 05 <=====
>>> [009h 0009 001h] Checksum : 94
>>> [00Ah 0010 006h] Oem ID : "BOCHS "
>>> [010h 0016 008h] Oem Table ID : "BXPC "
>>> [018h 0024 004h] Oem Revision : 00000001
>>> [01Ch 0028 004h] Asl Compiler ID : "BXPC"
>>> [020h 0032 004h] Asl Compiler Revision : 00000001
>>>
>>> ...
>>>
>>> [114h 0276 001h] Subtable Type : 00 [Processor Local APIC]
>>> [115h 0277 001h] Length : 08
>>> [116h 0278 001h] Processor ID : 1D
>>> [117h 0279 001h] Local Apic ID : 1D
>>> [118h 0280 004h] Flags (decoded below) : 00000001
>>> Processor Enabled : 1 <=====
>>> Runtime Online Capable : 0 <=====
>>>
>>> [11Ch 0284 001h] Subtable Type : 00 [Processor Local APIC]
>>> [11Dh 0285 001h] Length : 08
>>> [11Eh 0286 001h] Processor ID : 1E
>>> [11Fh 0287 001h] Local Apic ID : 1E
>>> [120h 0288 004h] Flags (decoded below) : 00000002
>>> Processor Enabled : 0 <=====
>>> Runtime Online Capable : 1 <=====
>>>
>>> [124h 0292 001h] Subtable Type : 00 [Processor Local APIC]
>>> [125h 0293 001h] Length : 08
>>> [126h 0294 001h] Processor ID : 1F
>>> [127h 0295 001h] Local Apic ID : 1F
>>> [128h 0296 004h] Flags (decoded below) : 00000002
>>> Processor Enabled : 0 <=====
>>> Runtime Online Capable : 1 <=====
>>>
>>> In terms of testing older operating systems, I did:
>>>
>>> - Windows Server 2008.
>>> By opening Device Manager -> Processors, and watching/counting
>>> the listed CPUs before and after hot un/plug operations.
>>> When hot plugging a CPU, the new processor shows in the list;
>>> one more processor is available after the hotplug.
>>> When hot unplugging a CPU, Windows refuses, with the message:
>>>
>>> The 'Intel(R) Xeon(R) CPU D-1533 @ 2.10GHz' device is not
>>> removable and cannot be ejected or unplugged.
>>>
>>> This is expected as hot unplug did not work in Windows 2008.
>>>
>>> - RHEL 6.9
>>> Kernel 2.6.32-696.el6.x86_64
>>> Build Feb 21 2017
>>> From dmesg:
>>> ACPI: APIC 000000007ffe32f0 000F0 (v05 BOCHS BXPC 00000001 BXPC 00000001)
>>> SMP: Allowing 16 CPUs, 8 hotplug CPUs
>>>
>>> # cat /sys/devices/system/cpu/online
>>> 0-7
>>>
>>> (QEMU) device_add id=cpu8 driver=host-x86_64-cpu socket-id=0 core-id=8 thread-id=0
>>>
>>> CPU 8 got hotplugged
>>> Booting Node 0 Processor 8 APIC 0x8
>>> kvm-clock: cpu 8, msr 2830ed00
>>> Will online and init hotplugged CPU: 8
>>> microcode: CPU8 sig=0x50663, pf=0x1, revision=0x700001c
>>> platform microcode: firmware: requesting intel-ucode/06-56-03
>>>
>>> # cat /sys/devices/system/cpu/online
>>> 0-8
>>>
>>> (QEMU) device_del id=cpu8
>>>
>>> Broke affinity for irq 24
>>> CPU 8 is now offline
>>>
>>> # cat /sys/devices/system/cpu/online
>>> 0-7
>>>
>>> RHEL 6.9
>>> kernel 2.6.32-696.el6.x86_64
>>> build Feb 21 2017
>>>
>>> Both of the above 'legacy' operating systems behaved as expected.
>>>
>>> Regards,
>>> Eric
>>>
>>> ---
>>> v3: 21apr2023
>>> - Dropped ARM patch as there is no compelling need to move to
>>> MADT revision 5, per Michael Tsirkin.
>>> - Recoded the flags to be simpler, per Michael, Ani Sinha and
>>> Miguel Luis.
>>> - Regenerated the binary tables, as per bios-tables-test.c
>>> instructions.
>>>
>>> v2: 18apr2023
>>> https://lists.gnu.org/archive/html/qemu-devel/2023-04/msg02576.html
>>> - Code formatting changes per Igor Mammedov
>>> - Included the steps in bios-tables-test.
>>>
>>> v1: 28mar2023
>>> https://lists.gnu.org/archive/html/qemu-devel/2023-03/msg06705.html
>>> - Initial series for ARM and i386 MADT revision bump to 5.
>>> Eric DeVolder (3):
>>> ACPI: bios-tables-test.c step 2 (allowed-diff entries)
>>> hw/acpi: i386: bump MADT to revision 5
>>> ACPI: bios-tables-test.c step 5 (updated expected table binaries)
>>>
>>> hw/i386/acpi-common.c | 4 ++--
>>> tests/data/acpi/microvm/APIC | Bin 70 -> 70 bytes
>>> tests/data/acpi/microvm/APIC.ioapic2 | Bin 82 -> 82 bytes
>>> tests/data/acpi/microvm/APIC.pcie | Bin 110 -> 110 bytes
>>> tests/data/acpi/pc/APIC | Bin 120 -> 120 bytes
>>> tests/data/acpi/pc/APIC.acpihmat | Bin 128 -> 128 bytes
>>> tests/data/acpi/pc/APIC.cphp | Bin 160 -> 160 bytes
>>> tests/data/acpi/pc/APIC.dimmpxm | Bin 144 -> 144 bytes
>>> tests/data/acpi/q35/APIC | Bin 120 -> 120 bytes
>>> tests/data/acpi/q35/APIC.acpihmat | Bin 128 -> 128 bytes
>>> tests/data/acpi/q35/APIC.acpihmat-noinitiator | Bin 144 -> 144 bytes
>>> tests/data/acpi/q35/APIC.core-count2 | Bin 2478 -> 2478 bytes
>>> tests/data/acpi/q35/APIC.cphp | Bin 160 -> 160 bytes
>>> tests/data/acpi/q35/APIC.dimmpxm | Bin 144 -> 144 bytes
>>> tests/data/acpi/q35/APIC.xapic | Bin 2686 -> 2686 bytes
>>> 15 files changed, 2 insertions(+), 2 deletions(-)
>>>
>>
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v3 0/3] ACPI: i386: bump MADT to revision 5
2023-05-10 15:08 ` Eric DeVolder
@ 2023-05-10 15:45 ` Igor Mammedov
2023-05-10 16:28 ` Eric DeVolder
0 siblings, 1 reply; 9+ messages in thread
From: Igor Mammedov @ 2023-05-10 15:45 UTC (permalink / raw)
To: Eric DeVolder
Cc: shannon.zhaosl, mst, ani, peter.maydell, qemu-arm, qemu-devel,
marcel.apfelbaum, pbonzini, richard.henderson, eduardo,
boris.ostrovsky, miguel.luis
On Wed, 10 May 2023 10:08:50 -0500
Eric DeVolder <eric.devolder@oracle.com> wrote:
> On 5/10/23 03:14, Igor Mammedov wrote:
> > On Fri, 5 May 2023 16:53:22 -0500
> > Eric DeVolder <eric.devolder@oracle.com> wrote:
> >
> >> Thoughts?
> >
> > I still don't think we need to bump x86 to rev 5 in QEMU.
>
> Linux v6.3 has the fix merged (so crisis averted 8).
> The investigation allowed me to opportunistically provide this patch.
> I think this should receive serious consideration for merging,
> more so because generating MADT .revision 3 and reporting .revision 1
> seems wrong to me.
It's a way much simpler to bump revision to 3 without introducing
OnlineCapable handling. So if you post rev3 patch, I'll gladly
ack it.
(+include kernel commit ids of kernel side fix, so if someone
stumbles upon it, one can easily find what to backport)
> This patch seems really straight forward, and low risk, now.
>
> >
> >> eric
> >>
> >> On 4/21/23 16:48, Eric DeVolder wrote:
> >>> The following Linux kernel change broke CPU hotplug for MADT revision
> >>> less than 5.
> >>>
> >>> e2869bd7af60 ("x86/acpi/boot: Do not register processors that cannot be onlined for x2APIC")
> >>>
> >>> Discussion on this topic can be located here:
> >>>
> >>> https://lore.kernel.org/linux-acpi/20230327191026.3454-1-eric.devolder@oracle.com/T/#t
> >
> > have your kernel fix landed up upstream?
> >
> Yes, merged and available in v6.3.
>
> >>>
> >>> which resulted in the following fixes Linux in 6.3-rc5:
> >>>
> >>> a74fabfbd1b7: ("x86/ACPI/boot: Use FADT version to check support for online capable")
> >>> fed8d8773b8e: ("x86/acpi/boot: Correct acpi_is_processor_usable() check")
> >>>
> >>> However, as part of the investigation into resolving this breakage, I
> >>> learned that i386 QEMU reports revision 1, while technically it
> >>> generates revision 3. Aarch64 generates and reports revision 4.
> >>>
> >>> ACPI 6.3 bumps MADT revision to 5 as it introduces an Online Capable
> >>> flag that the above Linux patch utilizes to denote hot pluggable CPUs.
> >>>
> >>> So in order to bump MADT to the current revision of 5, need to
> >>> validate that all MADT table changes between 1 and 5 are present
> >>> in QEMU.
> >>>
> >>> Below is a table summarizing the changes to the MADT. This information
> >>> gleamed from the ACPI specs on uefi.org.
> >>>
> >>> ACPI MADT What
> >>> Version Version
> >>> 1.0 MADT not present
> >>> 2.0 1 Section 5.2.10.4
> >>> 3.0 2 Section 5.2.11.4
> >>> 5.2.11.13 Local SAPIC Structure added two new fields:
> >>> ACPI Processor UID Value
> >>> ACPI Processor UID String
> >>> 5.2.10.14 Platform Interrupt Sources Structure:
> >>> Reserved changed to Platform Interrupt Sources Flags
> >>> 3.0b 2 Section 5.2.11.4
> >>> Added a section describing guidelines for the ordering of
> >>> processors in the MADT to support proper boot processor
> >>> and multi-threaded logical processor operation.
> >>> 4.0 3 Section 5.2.12
> >>> Adds Processor Local x2APIC structure type 9
> >>> Adds Local x2APIC NMI structure type 0xA
> >>> 5.0 3 Section 5.2.12
> >>> 6.0 3 Section 5.2.12
> >>> 6.0a 4 Section 5.2.12
> >>> Adds ARM GIC structure types 0xB-0xF
> >>> 6.2a 45 Section 5.2.12 <--- version 45, is indeed accurate!
> >>> 6.2b 5 Section 5.2.12
> >>> GIC ITS last Reserved offset changed to 16 from 20 (typo)
> >>> 6.3 5 Section 5.2.12
> >>> Adds Local APIC Flags Online Capable!
> >>> Adds GICC SPE Overflow Interrupt field
> >>> 6.4 5 Section 5.2.12
> >>> Adds Multiprocessor Wakeup Structure type 0x10
> >>> (change notes says structure previously misplaced?)
> >>> 6.5 5 Section 5.2.12
> >>>
> >>> For the MADT revision change 1 -> 2, the spec has a change to the
> >>> SAPIC structure. In general, QEMU does not generate/support SAPIC.
> >>> So the QEMU i386 MADT revision can safely be moved to 2.
> >>>
> >>> For the MADT revision change 2 -> 3, the spec adds Local x2APIC
> >>> structures. QEMU has long supported x2apic ACPI structures. A simple
> >>> search of x2apic within QEMU source and hw/i386/acpi-common.c
> >>> specifically reveals this. So the QEMU i386 MADT revision can safely
> >>> be moved to 3.
> >>>
> >>> For the MADT revision change 3 -> 4, the spec adds support for the ARM
> >>> GIC structures. QEMU ARM does in fact generate and report revision 4.
> >>> As these will not be used by i386 QEMU, so then the QEMU i386 MADT
> >>> revision can safely be moved to 4 as well.
> >>>
> >>> Now for the MADT revision change 4 -> 5, the spec adds the Online
> >>> Capable flag to the Local APIC structure, and the ARM GICC SPE
> >>> Overflow Interrupt field.
> >
> > All ARM stuff is irrelevant in x86 patch
> >
> sure
>
> >>> For i386, the ARM SPE is not applicable.
> >>>
> >>> For the i386 Local APIC flag Online Capable, the spec has certain rules
> >>> about this value. And in particuar setting this value now explicitly
> >>> indicates a hotpluggable CPU.
> >>>
> >>> So this patch makes the needed changes to move i386 MADT to
> >>> revision 5.
> >>>
> >>> Without these changes, the information below shows "how" CPU hotplug
> >>> breaks with the current upstream Linux kernel 6.3. For example, a Linux
> >>> guest started with:
> >
> > what's the status wrt current upstream kernel?
> >
> qemu cpu hotplug works with upstream kernel ( >= v6.3) but only because the
> kernel fix allows the legacy fallback assumption that the non Enabled
> cpus specified in MADT are online capable.
Kernel should be able to handle MADT revisions below 5 correctly as well,
so frankly speaking it's a kernel regression that's been fixed.
(affected distros should pull in your kernel patch[es] if
they are interested in cpu hotplug)
It's great that you dug through all the mess they made in
spec and found/fixed a bug in kernel.
But that doesn't justify complicating QEMU [1] just to make
recently broken kernel work or just because patch is already
written.
And no, all effort you put into figuring it out is not wasted,
since the bug was fixed at the place of regression (kernel)
and all involved have learned from it and grateful to you
fixing it up.
> with this patch applied to qemu, the kernel honors the Online Capable bit
> for cpus not Enabled at boot.
This is not really applicable/interesting to QEMU,
I read this bit in spec as a workaround for hardware vendors
that might simplify their life.
1) But it is useless for QEMU (see my reply in v2 or v1 review thread
with analysis as to why).
So I'd rather avoid adding 'Online Capable bit' to QEMU unless
we have to do so (i.e. we would need MADTr5 for some other feature).
>
>
> >>> qemu-system-x86_64 -smp 30,maxcpus=32 ...
> >>>
> >>> and then attempting to hotplug a CPU:
> >>>
> >>> (QEMU) device_add id=cpu30 driver=host-x86_64-cpu socket-id=0 core-id=30 thread-id=0
> >>>
> >>> fails with the following:
> >>>
> >>> APIC: NR_CPUS/possible_cpus limit of 30 reached. Processor 30/0x.
> >>> ACPI: Unable to map lapic to logical cpu number
> >>> acpi LNXCPU:1e: Enumeration failure
> >>>
> >>> # dmesg | grep smpboot
> >>> smpboot: Allowing 30 CPUs, 0 hotplug CPUs
> >>> smpboot: CPU0: Intel(R) Xeon(R) CPU D-1533 @ 2.10GHz (family: 0x)
> >>> smpboot: Max logical packages: 1
> >>> smpboot: Total of 30 processors activated (125708.76 BogoMIPS)
> >>>
> >>> # iasl -d /sys/firmware/tables/acpi/APIC
> >>> [000h 0000 4] Signature : "APIC" [Multiple APIC Descript
> >>> [004h 0004 4] Table Length : 00000170
> >>> [008h 0008 1] Revision : 01 <=====
> >>> [009h 0009 1] Checksum : 9C
> >>> [00Ah 0010 6] Oem ID : "BOCHS "
> >>> [010h 0016 8] Oem Table ID : "BXPC "
> >>> [018h 0024 4] Oem Revision : 00000001
> >>> [01Ch 0028 4] Asl Compiler ID : "BXPC"
> >>> [020h 0032 4] Asl Compiler Revision : 00000001
> >>>
> >>> ...
> >>>
> >>> [114h 0276 1] Subtable Type : 00 [Processor Local APIC]
> >>> [115h 0277 1] Length : 08
> >>> [116h 0278 1] Processor ID : 1D
> >>> [117h 0279 1] Local Apic ID : 1D
> >>> [118h 0280 4] Flags (decoded below) : 00000001
> >>> Processor Enabled : 1 <=====
> >>>
> >>> [11Ch 0284 1] Subtable Type : 00 [Processor Local APIC]
> >>> [11Dh 0285 1] Length : 08
> >>> [11Eh 0286 1] Processor ID : 1E
> >>> [11Fh 0287 1] Local Apic ID : 1E
> >>> [120h 0288 4] Flags (decoded below) : 00000000
> >>> Processor Enabled : 0 <=====
> >>>
> >>> [124h 0292 1] Subtable Type : 00 [Processor Local APIC]
> >>> [125h 0293 1] Length : 08
> >>> [126h 0294 1] Processor ID : 1F
> >>> [127h 0295 1] Local Apic ID : 1F
> >>> [128h 0296 4] Flags (decoded below) : 00000000
> >>> Processor Enabled : 0 <=====
> >>>
> >>> The (latest upstream) Linux kernel sees 30 Enabled processors, and
> >>> does not consider processors 31 and 32 to be hotpluggable.
> >>>
> >>> With this patch series applied, by bumping MADT to revision 5, the
> >>> latest upstream Linux kernel correctly identifies 30 CPUs plus 2
> >>> hotpluggable CPUS.
> >>>
> >>> CPU30 has been hot-added
> >>> smpboot: Booting Node 0 Processor 30 APIC 0x1e
> >>> Will online and init hotplugged CPU: 30
> >>>
> >>> # dmesg | grep smpboot
> >>> smpboot: Allowing 32 CPUs, 2 hotplug CPUs
> >>> smpboot: CPU0: Intel(R) Xeon(R) CPU D-1533 @ 2.10GHz (family: 0x6, model: 0x56, stepping: 0x3)
> >>> smpboot: Max logical packages: 2
> >>> smpboot: Total of 30 processors activated (125708.76 BogoMIPS)
> >>>
> >>> # iasl -d /sys/firmware/tables/acpi/APIC
> >>> [000h 0000 004h] Signature : "APIC" [Multiple APIC Descript
> >>> [004h 0004 004h] Table Length : 00000170
> >>> [008h 0008 001h] Revision : 05 <=====
> >>> [009h 0009 001h] Checksum : 94
> >>> [00Ah 0010 006h] Oem ID : "BOCHS "
> >>> [010h 0016 008h] Oem Table ID : "BXPC "
> >>> [018h 0024 004h] Oem Revision : 00000001
> >>> [01Ch 0028 004h] Asl Compiler ID : "BXPC"
> >>> [020h 0032 004h] Asl Compiler Revision : 00000001
> >>>
> >>> ...
> >>>
> >>> [114h 0276 001h] Subtable Type : 00 [Processor Local APIC]
> >>> [115h 0277 001h] Length : 08
> >>> [116h 0278 001h] Processor ID : 1D
> >>> [117h 0279 001h] Local Apic ID : 1D
> >>> [118h 0280 004h] Flags (decoded below) : 00000001
> >>> Processor Enabled : 1 <=====
> >>> Runtime Online Capable : 0 <=====
> >>>
> >>> [11Ch 0284 001h] Subtable Type : 00 [Processor Local APIC]
> >>> [11Dh 0285 001h] Length : 08
> >>> [11Eh 0286 001h] Processor ID : 1E
> >>> [11Fh 0287 001h] Local Apic ID : 1E
> >>> [120h 0288 004h] Flags (decoded below) : 00000002
> >>> Processor Enabled : 0 <=====
> >>> Runtime Online Capable : 1 <=====
> >>>
> >>> [124h 0292 001h] Subtable Type : 00 [Processor Local APIC]
> >>> [125h 0293 001h] Length : 08
> >>> [126h 0294 001h] Processor ID : 1F
> >>> [127h 0295 001h] Local Apic ID : 1F
> >>> [128h 0296 004h] Flags (decoded below) : 00000002
> >>> Processor Enabled : 0 <=====
> >>> Runtime Online Capable : 1 <=====
> >>>
> >>> In terms of testing older operating systems, I did:
> >>>
> >>> - Windows Server 2008.
> >>> By opening Device Manager -> Processors, and watching/counting
> >>> the listed CPUs before and after hot un/plug operations.
> >>> When hot plugging a CPU, the new processor shows in the list;
> >>> one more processor is available after the hotplug.
> >>> When hot unplugging a CPU, Windows refuses, with the message:
> >>>
> >>> The 'Intel(R) Xeon(R) CPU D-1533 @ 2.10GHz' device is not
> >>> removable and cannot be ejected or unplugged.
> >>>
> >>> This is expected as hot unplug did not work in Windows 2008.
> >>>
> >>> - RHEL 6.9
> >>> Kernel 2.6.32-696.el6.x86_64
> >>> Build Feb 21 2017
> >>> From dmesg:
> >>> ACPI: APIC 000000007ffe32f0 000F0 (v05 BOCHS BXPC 00000001 BXPC 00000001)
> >>> SMP: Allowing 16 CPUs, 8 hotplug CPUs
> >>>
> >>> # cat /sys/devices/system/cpu/online
> >>> 0-7
> >>>
> >>> (QEMU) device_add id=cpu8 driver=host-x86_64-cpu socket-id=0 core-id=8 thread-id=0
> >>>
> >>> CPU 8 got hotplugged
> >>> Booting Node 0 Processor 8 APIC 0x8
> >>> kvm-clock: cpu 8, msr 2830ed00
> >>> Will online and init hotplugged CPU: 8
> >>> microcode: CPU8 sig=0x50663, pf=0x1, revision=0x700001c
> >>> platform microcode: firmware: requesting intel-ucode/06-56-03
> >>>
> >>> # cat /sys/devices/system/cpu/online
> >>> 0-8
> >>>
> >>> (QEMU) device_del id=cpu8
> >>>
> >>> Broke affinity for irq 24
> >>> CPU 8 is now offline
> >>>
> >>> # cat /sys/devices/system/cpu/online
> >>> 0-7
> >>>
> >>> RHEL 6.9
> >>> kernel 2.6.32-696.el6.x86_64
> >>> build Feb 21 2017
> >>>
> >>> Both of the above 'legacy' operating systems behaved as expected.
> >>>
> >>> Regards,
> >>> Eric
> >>>
> >>> ---
> >>> v3: 21apr2023
> >>> - Dropped ARM patch as there is no compelling need to move to
> >>> MADT revision 5, per Michael Tsirkin.
> >>> - Recoded the flags to be simpler, per Michael, Ani Sinha and
> >>> Miguel Luis.
> >>> - Regenerated the binary tables, as per bios-tables-test.c
> >>> instructions.
> >>>
> >>> v2: 18apr2023
> >>> https://lists.gnu.org/archive/html/qemu-devel/2023-04/msg02576.html
> >>> - Code formatting changes per Igor Mammedov
> >>> - Included the steps in bios-tables-test.
> >>>
> >>> v1: 28mar2023
> >>> https://lists.gnu.org/archive/html/qemu-devel/2023-03/msg06705.html
> >>> - Initial series for ARM and i386 MADT revision bump to 5.
> >>> Eric DeVolder (3):
> >>> ACPI: bios-tables-test.c step 2 (allowed-diff entries)
> >>> hw/acpi: i386: bump MADT to revision 5
> >>> ACPI: bios-tables-test.c step 5 (updated expected table binaries)
> >>>
> >>> hw/i386/acpi-common.c | 4 ++--
> >>> tests/data/acpi/microvm/APIC | Bin 70 -> 70 bytes
> >>> tests/data/acpi/microvm/APIC.ioapic2 | Bin 82 -> 82 bytes
> >>> tests/data/acpi/microvm/APIC.pcie | Bin 110 -> 110 bytes
> >>> tests/data/acpi/pc/APIC | Bin 120 -> 120 bytes
> >>> tests/data/acpi/pc/APIC.acpihmat | Bin 128 -> 128 bytes
> >>> tests/data/acpi/pc/APIC.cphp | Bin 160 -> 160 bytes
> >>> tests/data/acpi/pc/APIC.dimmpxm | Bin 144 -> 144 bytes
> >>> tests/data/acpi/q35/APIC | Bin 120 -> 120 bytes
> >>> tests/data/acpi/q35/APIC.acpihmat | Bin 128 -> 128 bytes
> >>> tests/data/acpi/q35/APIC.acpihmat-noinitiator | Bin 144 -> 144 bytes
> >>> tests/data/acpi/q35/APIC.core-count2 | Bin 2478 -> 2478 bytes
> >>> tests/data/acpi/q35/APIC.cphp | Bin 160 -> 160 bytes
> >>> tests/data/acpi/q35/APIC.dimmpxm | Bin 144 -> 144 bytes
> >>> tests/data/acpi/q35/APIC.xapic | Bin 2686 -> 2686 bytes
> >>> 15 files changed, 2 insertions(+), 2 deletions(-)
> >>>
> >>
> >
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v3 0/3] ACPI: i386: bump MADT to revision 5
2023-05-10 15:45 ` Igor Mammedov
@ 2023-05-10 16:28 ` Eric DeVolder
0 siblings, 0 replies; 9+ messages in thread
From: Eric DeVolder @ 2023-05-10 16:28 UTC (permalink / raw)
To: Igor Mammedov
Cc: shannon.zhaosl, mst, ani, peter.maydell, qemu-arm, qemu-devel,
marcel.apfelbaum, pbonzini, richard.henderson, eduardo,
boris.ostrovsky, miguel.luis
On 5/10/23 10:45, Igor Mammedov wrote:
> On Wed, 10 May 2023 10:08:50 -0500
> Eric DeVolder <eric.devolder@oracle.com> wrote:
>
>> On 5/10/23 03:14, Igor Mammedov wrote:
>>> On Fri, 5 May 2023 16:53:22 -0500
>>> Eric DeVolder <eric.devolder@oracle.com> wrote:
>>>
>>>> Thoughts?
>>>
>>> I still don't think we need to bump x86 to rev 5 in QEMU.
>>
>> Linux v6.3 has the fix merged (so crisis averted 8).
>> The investigation allowed me to opportunistically provide this patch.
>> I think this should receive serious consideration for merging,
>> more so because generating MADT .revision 3 and reporting .revision 1
>> seems wrong to me.
>
> It's a way much simpler to bump revision to 3 without introducing
> OnlineCapable handling. So if you post rev3 patch, I'll gladly
> ack it.
> (+include kernel commit ids of kernel side fix, so if someone
> stumbles upon it, one can easily find what to backport)
>
OK, I'll do that. It'll be next week as I'm on a short week this week.
>> This patch seems really straight forward, and low risk, now.
>>
>>>
>>>> eric
>>>>
>>>> On 4/21/23 16:48, Eric DeVolder wrote:
>>>>> The following Linux kernel change broke CPU hotplug for MADT revision
>>>>> less than 5.
>>>>>
>>>>> e2869bd7af60 ("x86/acpi/boot: Do not register processors that cannot be onlined for x2APIC")
>>>>>
>>>>> Discussion on this topic can be located here:
>>>>>
>>>>> https://lore.kernel.org/linux-acpi/20230327191026.3454-1-eric.devolder@oracle.com/T/#t
>>>
>>> have your kernel fix landed up upstream?
>>>
>> Yes, merged and available in v6.3.
>>
>>>>>
>>>>> which resulted in the following fixes Linux in 6.3-rc5:
>>>>>
>>>>> a74fabfbd1b7: ("x86/ACPI/boot: Use FADT version to check support for online capable")
>>>>> fed8d8773b8e: ("x86/acpi/boot: Correct acpi_is_processor_usable() check")
>>>>>
>>>>> However, as part of the investigation into resolving this breakage, I
>>>>> learned that i386 QEMU reports revision 1, while technically it
>>>>> generates revision 3. Aarch64 generates and reports revision 4.
>>>>>
>>>>> ACPI 6.3 bumps MADT revision to 5 as it introduces an Online Capable
>>>>> flag that the above Linux patch utilizes to denote hot pluggable CPUs.
>>>>>
>>>>> So in order to bump MADT to the current revision of 5, need to
>>>>> validate that all MADT table changes between 1 and 5 are present
>>>>> in QEMU.
>>>>>
>>>>> Below is a table summarizing the changes to the MADT. This information
>>>>> gleamed from the ACPI specs on uefi.org.
>>>>>
>>>>> ACPI MADT What
>>>>> Version Version
>>>>> 1.0 MADT not present
>>>>> 2.0 1 Section 5.2.10.4
>>>>> 3.0 2 Section 5.2.11.4
>>>>> 5.2.11.13 Local SAPIC Structure added two new fields:
>>>>> ACPI Processor UID Value
>>>>> ACPI Processor UID String
>>>>> 5.2.10.14 Platform Interrupt Sources Structure:
>>>>> Reserved changed to Platform Interrupt Sources Flags
>>>>> 3.0b 2 Section 5.2.11.4
>>>>> Added a section describing guidelines for the ordering of
>>>>> processors in the MADT to support proper boot processor
>>>>> and multi-threaded logical processor operation.
>>>>> 4.0 3 Section 5.2.12
>>>>> Adds Processor Local x2APIC structure type 9
>>>>> Adds Local x2APIC NMI structure type 0xA
>>>>> 5.0 3 Section 5.2.12
>>>>> 6.0 3 Section 5.2.12
>>>>> 6.0a 4 Section 5.2.12
>>>>> Adds ARM GIC structure types 0xB-0xF
>>>>> 6.2a 45 Section 5.2.12 <--- version 45, is indeed accurate!
>>>>> 6.2b 5 Section 5.2.12
>>>>> GIC ITS last Reserved offset changed to 16 from 20 (typo)
>>>>> 6.3 5 Section 5.2.12
>>>>> Adds Local APIC Flags Online Capable!
>>>>> Adds GICC SPE Overflow Interrupt field
>>>>> 6.4 5 Section 5.2.12
>>>>> Adds Multiprocessor Wakeup Structure type 0x10
>>>>> (change notes says structure previously misplaced?)
>>>>> 6.5 5 Section 5.2.12
>>>>>
>>>>> For the MADT revision change 1 -> 2, the spec has a change to the
>>>>> SAPIC structure. In general, QEMU does not generate/support SAPIC.
>>>>> So the QEMU i386 MADT revision can safely be moved to 2.
>>>>>
>>>>> For the MADT revision change 2 -> 3, the spec adds Local x2APIC
>>>>> structures. QEMU has long supported x2apic ACPI structures. A simple
>>>>> search of x2apic within QEMU source and hw/i386/acpi-common.c
>>>>> specifically reveals this. So the QEMU i386 MADT revision can safely
>>>>> be moved to 3.
>>>>>
>>>>> For the MADT revision change 3 -> 4, the spec adds support for the ARM
>>>>> GIC structures. QEMU ARM does in fact generate and report revision 4.
>>>>> As these will not be used by i386 QEMU, so then the QEMU i386 MADT
>>>>> revision can safely be moved to 4 as well.
>>>>>
>>>>> Now for the MADT revision change 4 -> 5, the spec adds the Online
>>>>> Capable flag to the Local APIC structure, and the ARM GICC SPE
>>>>> Overflow Interrupt field.
>>>
>>> All ARM stuff is irrelevant in x86 patch
>>>
>> sure
>>
>>>>> For i386, the ARM SPE is not applicable.
>>>>>
>>>>> For the i386 Local APIC flag Online Capable, the spec has certain rules
>>>>> about this value. And in particuar setting this value now explicitly
>>>>> indicates a hotpluggable CPU.
>>>>>
>>>>> So this patch makes the needed changes to move i386 MADT to
>>>>> revision 5.
>>>>>
>>>>> Without these changes, the information below shows "how" CPU hotplug
>>>>> breaks with the current upstream Linux kernel 6.3. For example, a Linux
>>>>> guest started with:
>>>
>>> what's the status wrt current upstream kernel?
>>>
>> qemu cpu hotplug works with upstream kernel ( >= v6.3) but only because the
>> kernel fix allows the legacy fallback assumption that the non Enabled
>> cpus specified in MADT are online capable.
>
> Kernel should be able to handle MADT revisions below 5 correctly as well,
> so frankly speaking it's a kernel regression that's been fixed.
> (affected distros should pull in your kernel patch[es] if
> they are interested in cpu hotplug)
>
> It's great that you dug through all the mess they made in
> spec and found/fixed a bug in kernel.
> But that doesn't justify complicating QEMU [1] just to make
> recently broken kernel work or just because patch is already
> written.
>
> And no, all effort you put into figuring it out is not wasted,
> since the bug was fixed at the place of regression (kernel)
> and all involved have learned from it and grateful to you
> fixing it up.
>
Of course!
>> with this patch applied to qemu, the kernel honors the Online Capable bit
>> for cpus not Enabled at boot.
>
> This is not really applicable/interesting to QEMU,
> I read this bit in spec as a workaround for hardware vendors
> that might simplify their life.
True, but it is in the spec. And not complicated.
>
> 1) But it is useless for QEMU (see my reply in v2 or v1 review thread
> with analysis as to why).
> So I'd rather avoid adding 'Online Capable bit' to QEMU unless
> we have to do so (i.e. we would need MADTr5 for some other feature).
Afaik, OnlineCapable is the only feature added in .revision 5 (other than
that ARM feature).
eric
>
>>
>>
>>>>> qemu-system-x86_64 -smp 30,maxcpus=32 ...
>>>>>
>>>>> and then attempting to hotplug a CPU:
>>>>>
>>>>> (QEMU) device_add id=cpu30 driver=host-x86_64-cpu socket-id=0 core-id=30 thread-id=0
>>>>>
>>>>> fails with the following:
>>>>>
>>>>> APIC: NR_CPUS/possible_cpus limit of 30 reached. Processor 30/0x.
>>>>> ACPI: Unable to map lapic to logical cpu number
>>>>> acpi LNXCPU:1e: Enumeration failure
>>>>>
>>>>> # dmesg | grep smpboot
>>>>> smpboot: Allowing 30 CPUs, 0 hotplug CPUs
>>>>> smpboot: CPU0: Intel(R) Xeon(R) CPU D-1533 @ 2.10GHz (family: 0x)
>>>>> smpboot: Max logical packages: 1
>>>>> smpboot: Total of 30 processors activated (125708.76 BogoMIPS)
>>>>>
>>>>> # iasl -d /sys/firmware/tables/acpi/APIC
>>>>> [000h 0000 4] Signature : "APIC" [Multiple APIC Descript
>>>>> [004h 0004 4] Table Length : 00000170
>>>>> [008h 0008 1] Revision : 01 <=====
>>>>> [009h 0009 1] Checksum : 9C
>>>>> [00Ah 0010 6] Oem ID : "BOCHS "
>>>>> [010h 0016 8] Oem Table ID : "BXPC "
>>>>> [018h 0024 4] Oem Revision : 00000001
>>>>> [01Ch 0028 4] Asl Compiler ID : "BXPC"
>>>>> [020h 0032 4] Asl Compiler Revision : 00000001
>>>>>
>>>>> ...
>>>>>
>>>>> [114h 0276 1] Subtable Type : 00 [Processor Local APIC]
>>>>> [115h 0277 1] Length : 08
>>>>> [116h 0278 1] Processor ID : 1D
>>>>> [117h 0279 1] Local Apic ID : 1D
>>>>> [118h 0280 4] Flags (decoded below) : 00000001
>>>>> Processor Enabled : 1 <=====
>>>>>
>>>>> [11Ch 0284 1] Subtable Type : 00 [Processor Local APIC]
>>>>> [11Dh 0285 1] Length : 08
>>>>> [11Eh 0286 1] Processor ID : 1E
>>>>> [11Fh 0287 1] Local Apic ID : 1E
>>>>> [120h 0288 4] Flags (decoded below) : 00000000
>>>>> Processor Enabled : 0 <=====
>>>>>
>>>>> [124h 0292 1] Subtable Type : 00 [Processor Local APIC]
>>>>> [125h 0293 1] Length : 08
>>>>> [126h 0294 1] Processor ID : 1F
>>>>> [127h 0295 1] Local Apic ID : 1F
>>>>> [128h 0296 4] Flags (decoded below) : 00000000
>>>>> Processor Enabled : 0 <=====
>>>>>
>>>>> The (latest upstream) Linux kernel sees 30 Enabled processors, and
>>>>> does not consider processors 31 and 32 to be hotpluggable.
>>>>>
>>>>> With this patch series applied, by bumping MADT to revision 5, the
>>>>> latest upstream Linux kernel correctly identifies 30 CPUs plus 2
>>>>> hotpluggable CPUS.
>>>>>
>>>>> CPU30 has been hot-added
>>>>> smpboot: Booting Node 0 Processor 30 APIC 0x1e
>>>>> Will online and init hotplugged CPU: 30
>>>>>
>>>>> # dmesg | grep smpboot
>>>>> smpboot: Allowing 32 CPUs, 2 hotplug CPUs
>>>>> smpboot: CPU0: Intel(R) Xeon(R) CPU D-1533 @ 2.10GHz (family: 0x6, model: 0x56, stepping: 0x3)
>>>>> smpboot: Max logical packages: 2
>>>>> smpboot: Total of 30 processors activated (125708.76 BogoMIPS)
>>>>>
>>>>> # iasl -d /sys/firmware/tables/acpi/APIC
>>>>> [000h 0000 004h] Signature : "APIC" [Multiple APIC Descript
>>>>> [004h 0004 004h] Table Length : 00000170
>>>>> [008h 0008 001h] Revision : 05 <=====
>>>>> [009h 0009 001h] Checksum : 94
>>>>> [00Ah 0010 006h] Oem ID : "BOCHS "
>>>>> [010h 0016 008h] Oem Table ID : "BXPC "
>>>>> [018h 0024 004h] Oem Revision : 00000001
>>>>> [01Ch 0028 004h] Asl Compiler ID : "BXPC"
>>>>> [020h 0032 004h] Asl Compiler Revision : 00000001
>>>>>
>>>>> ...
>>>>>
>>>>> [114h 0276 001h] Subtable Type : 00 [Processor Local APIC]
>>>>> [115h 0277 001h] Length : 08
>>>>> [116h 0278 001h] Processor ID : 1D
>>>>> [117h 0279 001h] Local Apic ID : 1D
>>>>> [118h 0280 004h] Flags (decoded below) : 00000001
>>>>> Processor Enabled : 1 <=====
>>>>> Runtime Online Capable : 0 <=====
>>>>>
>>>>> [11Ch 0284 001h] Subtable Type : 00 [Processor Local APIC]
>>>>> [11Dh 0285 001h] Length : 08
>>>>> [11Eh 0286 001h] Processor ID : 1E
>>>>> [11Fh 0287 001h] Local Apic ID : 1E
>>>>> [120h 0288 004h] Flags (decoded below) : 00000002
>>>>> Processor Enabled : 0 <=====
>>>>> Runtime Online Capable : 1 <=====
>>>>>
>>>>> [124h 0292 001h] Subtable Type : 00 [Processor Local APIC]
>>>>> [125h 0293 001h] Length : 08
>>>>> [126h 0294 001h] Processor ID : 1F
>>>>> [127h 0295 001h] Local Apic ID : 1F
>>>>> [128h 0296 004h] Flags (decoded below) : 00000002
>>>>> Processor Enabled : 0 <=====
>>>>> Runtime Online Capable : 1 <=====
>>>>>
>>>>> In terms of testing older operating systems, I did:
>>>>>
>>>>> - Windows Server 2008.
>>>>> By opening Device Manager -> Processors, and watching/counting
>>>>> the listed CPUs before and after hot un/plug operations.
>>>>> When hot plugging a CPU, the new processor shows in the list;
>>>>> one more processor is available after the hotplug.
>>>>> When hot unplugging a CPU, Windows refuses, with the message:
>>>>>
>>>>> The 'Intel(R) Xeon(R) CPU D-1533 @ 2.10GHz' device is not
>>>>> removable and cannot be ejected or unplugged.
>>>>>
>>>>> This is expected as hot unplug did not work in Windows 2008.
>>>>>
>>>>> - RHEL 6.9
>>>>> Kernel 2.6.32-696.el6.x86_64
>>>>> Build Feb 21 2017
>>>>> From dmesg:
>>>>> ACPI: APIC 000000007ffe32f0 000F0 (v05 BOCHS BXPC 00000001 BXPC 00000001)
>>>>> SMP: Allowing 16 CPUs, 8 hotplug CPUs
>>>>>
>>>>> # cat /sys/devices/system/cpu/online
>>>>> 0-7
>>>>>
>>>>> (QEMU) device_add id=cpu8 driver=host-x86_64-cpu socket-id=0 core-id=8 thread-id=0
>>>>>
>>>>> CPU 8 got hotplugged
>>>>> Booting Node 0 Processor 8 APIC 0x8
>>>>> kvm-clock: cpu 8, msr 2830ed00
>>>>> Will online and init hotplugged CPU: 8
>>>>> microcode: CPU8 sig=0x50663, pf=0x1, revision=0x700001c
>>>>> platform microcode: firmware: requesting intel-ucode/06-56-03
>>>>>
>>>>> # cat /sys/devices/system/cpu/online
>>>>> 0-8
>>>>>
>>>>> (QEMU) device_del id=cpu8
>>>>>
>>>>> Broke affinity for irq 24
>>>>> CPU 8 is now offline
>>>>>
>>>>> # cat /sys/devices/system/cpu/online
>>>>> 0-7
>>>>>
>>>>> RHEL 6.9
>>>>> kernel 2.6.32-696.el6.x86_64
>>>>> build Feb 21 2017
>>>>>
>>>>> Both of the above 'legacy' operating systems behaved as expected.
>>>>>
>>>>> Regards,
>>>>> Eric
>>>>>
>>>>> ---
>>>>> v3: 21apr2023
>>>>> - Dropped ARM patch as there is no compelling need to move to
>>>>> MADT revision 5, per Michael Tsirkin.
>>>>> - Recoded the flags to be simpler, per Michael, Ani Sinha and
>>>>> Miguel Luis.
>>>>> - Regenerated the binary tables, as per bios-tables-test.c
>>>>> instructions.
>>>>>
>>>>> v2: 18apr2023
>>>>> https://lists.gnu.org/archive/html/qemu-devel/2023-04/msg02576.html
>>>>> - Code formatting changes per Igor Mammedov
>>>>> - Included the steps in bios-tables-test.
>>>>>
>>>>> v1: 28mar2023
>>>>> https://lists.gnu.org/archive/html/qemu-devel/2023-03/msg06705.html
>>>>> - Initial series for ARM and i386 MADT revision bump to 5.
>>>>> Eric DeVolder (3):
>>>>> ACPI: bios-tables-test.c step 2 (allowed-diff entries)
>>>>> hw/acpi: i386: bump MADT to revision 5
>>>>> ACPI: bios-tables-test.c step 5 (updated expected table binaries)
>>>>>
>>>>> hw/i386/acpi-common.c | 4 ++--
>>>>> tests/data/acpi/microvm/APIC | Bin 70 -> 70 bytes
>>>>> tests/data/acpi/microvm/APIC.ioapic2 | Bin 82 -> 82 bytes
>>>>> tests/data/acpi/microvm/APIC.pcie | Bin 110 -> 110 bytes
>>>>> tests/data/acpi/pc/APIC | Bin 120 -> 120 bytes
>>>>> tests/data/acpi/pc/APIC.acpihmat | Bin 128 -> 128 bytes
>>>>> tests/data/acpi/pc/APIC.cphp | Bin 160 -> 160 bytes
>>>>> tests/data/acpi/pc/APIC.dimmpxm | Bin 144 -> 144 bytes
>>>>> tests/data/acpi/q35/APIC | Bin 120 -> 120 bytes
>>>>> tests/data/acpi/q35/APIC.acpihmat | Bin 128 -> 128 bytes
>>>>> tests/data/acpi/q35/APIC.acpihmat-noinitiator | Bin 144 -> 144 bytes
>>>>> tests/data/acpi/q35/APIC.core-count2 | Bin 2478 -> 2478 bytes
>>>>> tests/data/acpi/q35/APIC.cphp | Bin 160 -> 160 bytes
>>>>> tests/data/acpi/q35/APIC.dimmpxm | Bin 144 -> 144 bytes
>>>>> tests/data/acpi/q35/APIC.xapic | Bin 2686 -> 2686 bytes
>>>>> 15 files changed, 2 insertions(+), 2 deletions(-)
>>>>>
>>>>
>>>
>>
>
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2023-05-10 16:29 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-04-21 21:48 [PATCH v3 0/3] ACPI: i386: bump MADT to revision 5 Eric DeVolder
2023-04-21 21:48 ` [PATCH v3 1/3] ACPI: bios-tables-test.c step 2 (allowed-diff entries) Eric DeVolder
2023-04-21 21:49 ` [PATCH v3 2/3] ACPI: i386: bump MADT to revision 5 Eric DeVolder
2023-04-21 21:49 ` [PATCH v3 3/3] ACPI: bios-tables-test.c step 5 (updated expected table binaries) Eric DeVolder
2023-05-05 21:53 ` [PATCH v3 0/3] ACPI: i386: bump MADT to revision 5 Eric DeVolder
2023-05-10 8:14 ` Igor Mammedov
2023-05-10 15:08 ` Eric DeVolder
2023-05-10 15:45 ` Igor Mammedov
2023-05-10 16:28 ` Eric DeVolder
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).