* [PATCH v7 1/7] hw/arm/virt: Introduce virt_set_high_memmap() helper
2022-10-29 22:43 [PATCH v7 0/7] hw/arm/virt: Improve address assignment for high memory regions Gavin Shan
@ 2022-10-29 22:43 ` Gavin Shan
2022-10-29 22:43 ` [PATCH v7 2/7] hw/arm/virt: Rename variable size to region_size in virt_set_high_memmap() Gavin Shan
` (6 subsequent siblings)
7 siblings, 0 replies; 11+ messages in thread
From: Gavin Shan @ 2022-10-29 22:43 UTC (permalink / raw)
To: qemu-arm
Cc: qemu-devel, maz, eric.auger, cohuck, richard.henderson,
peter.maydell, zhenyzha, shan.gavin
This introduces virt_set_high_memmap() helper. The logic of high
memory region address assignment is moved to the helper. The intention
is to make the subsequent optimization for high memory region address
assignment easier.
No functional change intended.
Signed-off-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
---
hw/arm/virt.c | 74 ++++++++++++++++++++++++++++-----------------------
1 file changed, 41 insertions(+), 33 deletions(-)
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index cda9defe8f..7572c44bda 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1689,6 +1689,46 @@ static uint64_t virt_cpu_mp_affinity(VirtMachineState *vms, int idx)
return arm_cpu_mp_affinity(idx, clustersz);
}
+static void virt_set_high_memmap(VirtMachineState *vms,
+ hwaddr base, int pa_bits)
+{
+ int i;
+
+ for (i = VIRT_LOWMEMMAP_LAST; i < ARRAY_SIZE(extended_memmap); i++) {
+ hwaddr size = extended_memmap[i].size;
+ bool fits;
+
+ base = ROUND_UP(base, size);
+ vms->memmap[i].base = base;
+ vms->memmap[i].size = size;
+
+ /*
+ * Check each device to see if they fit in the PA space,
+ * moving highest_gpa as we go.
+ *
+ * For each device that doesn't fit, disable it.
+ */
+ fits = (base + size) <= BIT_ULL(pa_bits);
+ if (fits) {
+ vms->highest_gpa = base + size - 1;
+ }
+
+ switch (i) {
+ case VIRT_HIGH_GIC_REDIST2:
+ vms->highmem_redists &= fits;
+ break;
+ case VIRT_HIGH_PCIE_ECAM:
+ vms->highmem_ecam &= fits;
+ break;
+ case VIRT_HIGH_PCIE_MMIO:
+ vms->highmem_mmio &= fits;
+ break;
+ }
+
+ base += size;
+ }
+}
+
static void virt_set_memmap(VirtMachineState *vms, int pa_bits)
{
MachineState *ms = MACHINE(vms);
@@ -1744,39 +1784,7 @@ static void virt_set_memmap(VirtMachineState *vms, int pa_bits)
/* We know for sure that at least the memory fits in the PA space */
vms->highest_gpa = memtop - 1;
- for (i = VIRT_LOWMEMMAP_LAST; i < ARRAY_SIZE(extended_memmap); i++) {
- hwaddr size = extended_memmap[i].size;
- bool fits;
-
- base = ROUND_UP(base, size);
- vms->memmap[i].base = base;
- vms->memmap[i].size = size;
-
- /*
- * Check each device to see if they fit in the PA space,
- * moving highest_gpa as we go.
- *
- * For each device that doesn't fit, disable it.
- */
- fits = (base + size) <= BIT_ULL(pa_bits);
- if (fits) {
- vms->highest_gpa = base + size - 1;
- }
-
- switch (i) {
- case VIRT_HIGH_GIC_REDIST2:
- vms->highmem_redists &= fits;
- break;
- case VIRT_HIGH_PCIE_ECAM:
- vms->highmem_ecam &= fits;
- break;
- case VIRT_HIGH_PCIE_MMIO:
- vms->highmem_mmio &= fits;
- break;
- }
-
- base += size;
- }
+ virt_set_high_memmap(vms, base, pa_bits);
if (device_memory_size > 0) {
ms->device_memory = g_malloc0(sizeof(*ms->device_memory));
--
2.23.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v7 2/7] hw/arm/virt: Rename variable size to region_size in virt_set_high_memmap()
2022-10-29 22:43 [PATCH v7 0/7] hw/arm/virt: Improve address assignment for high memory regions Gavin Shan
2022-10-29 22:43 ` [PATCH v7 1/7] hw/arm/virt: Introduce virt_set_high_memmap() helper Gavin Shan
@ 2022-10-29 22:43 ` Gavin Shan
2022-10-29 22:43 ` [PATCH v7 3/7] hw/arm/virt: Introduce variable region_base " Gavin Shan
` (5 subsequent siblings)
7 siblings, 0 replies; 11+ messages in thread
From: Gavin Shan @ 2022-10-29 22:43 UTC (permalink / raw)
To: qemu-arm
Cc: qemu-devel, maz, eric.auger, cohuck, richard.henderson,
peter.maydell, zhenyzha, shan.gavin
This renames variable 'size' to 'region_size' in virt_set_high_memmap().
Its counterpart ('region_base') will be introduced in next patch.
No functional change intended.
Signed-off-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
---
hw/arm/virt.c | 15 ++++++++-------
1 file changed, 8 insertions(+), 7 deletions(-)
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 7572c44bda..e2ae88cf8b 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1692,15 +1692,16 @@ static uint64_t virt_cpu_mp_affinity(VirtMachineState *vms, int idx)
static void virt_set_high_memmap(VirtMachineState *vms,
hwaddr base, int pa_bits)
{
+ hwaddr region_size;
+ bool fits;
int i;
for (i = VIRT_LOWMEMMAP_LAST; i < ARRAY_SIZE(extended_memmap); i++) {
- hwaddr size = extended_memmap[i].size;
- bool fits;
+ region_size = extended_memmap[i].size;
- base = ROUND_UP(base, size);
+ base = ROUND_UP(base, region_size);
vms->memmap[i].base = base;
- vms->memmap[i].size = size;
+ vms->memmap[i].size = region_size;
/*
* Check each device to see if they fit in the PA space,
@@ -1708,9 +1709,9 @@ static void virt_set_high_memmap(VirtMachineState *vms,
*
* For each device that doesn't fit, disable it.
*/
- fits = (base + size) <= BIT_ULL(pa_bits);
+ fits = (base + region_size) <= BIT_ULL(pa_bits);
if (fits) {
- vms->highest_gpa = base + size - 1;
+ vms->highest_gpa = base + region_size - 1;
}
switch (i) {
@@ -1725,7 +1726,7 @@ static void virt_set_high_memmap(VirtMachineState *vms,
break;
}
- base += size;
+ base += region_size;
}
}
--
2.23.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v7 3/7] hw/arm/virt: Introduce variable region_base in virt_set_high_memmap()
2022-10-29 22:43 [PATCH v7 0/7] hw/arm/virt: Improve address assignment for high memory regions Gavin Shan
2022-10-29 22:43 ` [PATCH v7 1/7] hw/arm/virt: Introduce virt_set_high_memmap() helper Gavin Shan
2022-10-29 22:43 ` [PATCH v7 2/7] hw/arm/virt: Rename variable size to region_size in virt_set_high_memmap() Gavin Shan
@ 2022-10-29 22:43 ` Gavin Shan
2022-10-29 22:43 ` [PATCH v7 4/7] hw/arm/virt: Introduce virt_get_high_memmap_enabled() helper Gavin Shan
` (4 subsequent siblings)
7 siblings, 0 replies; 11+ messages in thread
From: Gavin Shan @ 2022-10-29 22:43 UTC (permalink / raw)
To: qemu-arm
Cc: qemu-devel, maz, eric.auger, cohuck, richard.henderson,
peter.maydell, zhenyzha, shan.gavin
This introduces variable 'region_base' for the base address of the
specific high memory region. It's the preparatory work to optimize
high memory region address assignment.
No functional change intended.
Signed-off-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
---
hw/arm/virt.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index e2ae88cf8b..0bf3cb7057 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1692,15 +1692,15 @@ static uint64_t virt_cpu_mp_affinity(VirtMachineState *vms, int idx)
static void virt_set_high_memmap(VirtMachineState *vms,
hwaddr base, int pa_bits)
{
- hwaddr region_size;
+ hwaddr region_base, region_size;
bool fits;
int i;
for (i = VIRT_LOWMEMMAP_LAST; i < ARRAY_SIZE(extended_memmap); i++) {
+ region_base = ROUND_UP(base, extended_memmap[i].size);
region_size = extended_memmap[i].size;
- base = ROUND_UP(base, region_size);
- vms->memmap[i].base = base;
+ vms->memmap[i].base = region_base;
vms->memmap[i].size = region_size;
/*
@@ -1709,9 +1709,9 @@ static void virt_set_high_memmap(VirtMachineState *vms,
*
* For each device that doesn't fit, disable it.
*/
- fits = (base + region_size) <= BIT_ULL(pa_bits);
+ fits = (region_base + region_size) <= BIT_ULL(pa_bits);
if (fits) {
- vms->highest_gpa = base + region_size - 1;
+ vms->highest_gpa = region_base + region_size - 1;
}
switch (i) {
@@ -1726,7 +1726,7 @@ static void virt_set_high_memmap(VirtMachineState *vms,
break;
}
- base += region_size;
+ base = region_base + region_size;
}
}
--
2.23.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v7 4/7] hw/arm/virt: Introduce virt_get_high_memmap_enabled() helper
2022-10-29 22:43 [PATCH v7 0/7] hw/arm/virt: Improve address assignment for high memory regions Gavin Shan
` (2 preceding siblings ...)
2022-10-29 22:43 ` [PATCH v7 3/7] hw/arm/virt: Introduce variable region_base " Gavin Shan
@ 2022-10-29 22:43 ` Gavin Shan
2022-10-29 22:43 ` [PATCH v7 5/7] hw/arm/virt: Improve high memory region address assignment Gavin Shan
` (3 subsequent siblings)
7 siblings, 0 replies; 11+ messages in thread
From: Gavin Shan @ 2022-10-29 22:43 UTC (permalink / raw)
To: qemu-arm
Cc: qemu-devel, maz, eric.auger, cohuck, richard.henderson,
peter.maydell, zhenyzha, shan.gavin
This introduces virt_get_high_memmap_enabled() helper, which returns
the pointer to vms->highmem_{redists, ecam, mmio}. The pointer will
be used in the subsequent patches.
No functional change intended.
Signed-off-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
---
hw/arm/virt.c | 32 +++++++++++++++++++-------------
1 file changed, 19 insertions(+), 13 deletions(-)
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 0bf3cb7057..ee98a8a3b6 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1689,14 +1689,31 @@ static uint64_t virt_cpu_mp_affinity(VirtMachineState *vms, int idx)
return arm_cpu_mp_affinity(idx, clustersz);
}
+static inline bool *virt_get_high_memmap_enabled(VirtMachineState *vms,
+ int index)
+{
+ bool *enabled_array[] = {
+ &vms->highmem_redists,
+ &vms->highmem_ecam,
+ &vms->highmem_mmio,
+ };
+
+ assert(ARRAY_SIZE(extended_memmap) - VIRT_LOWMEMMAP_LAST ==
+ ARRAY_SIZE(enabled_array));
+ assert(index - VIRT_LOWMEMMAP_LAST < ARRAY_SIZE(enabled_array));
+
+ return enabled_array[index - VIRT_LOWMEMMAP_LAST];
+}
+
static void virt_set_high_memmap(VirtMachineState *vms,
hwaddr base, int pa_bits)
{
hwaddr region_base, region_size;
- bool fits;
+ bool *region_enabled, fits;
int i;
for (i = VIRT_LOWMEMMAP_LAST; i < ARRAY_SIZE(extended_memmap); i++) {
+ region_enabled = virt_get_high_memmap_enabled(vms, i);
region_base = ROUND_UP(base, extended_memmap[i].size);
region_size = extended_memmap[i].size;
@@ -1714,18 +1731,7 @@ static void virt_set_high_memmap(VirtMachineState *vms,
vms->highest_gpa = region_base + region_size - 1;
}
- switch (i) {
- case VIRT_HIGH_GIC_REDIST2:
- vms->highmem_redists &= fits;
- break;
- case VIRT_HIGH_PCIE_ECAM:
- vms->highmem_ecam &= fits;
- break;
- case VIRT_HIGH_PCIE_MMIO:
- vms->highmem_mmio &= fits;
- break;
- }
-
+ *region_enabled &= fits;
base = region_base + region_size;
}
}
--
2.23.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v7 5/7] hw/arm/virt: Improve high memory region address assignment
2022-10-29 22:43 [PATCH v7 0/7] hw/arm/virt: Improve address assignment for high memory regions Gavin Shan
` (3 preceding siblings ...)
2022-10-29 22:43 ` [PATCH v7 4/7] hw/arm/virt: Introduce virt_get_high_memmap_enabled() helper Gavin Shan
@ 2022-10-29 22:43 ` Gavin Shan
2022-10-29 22:43 ` [PATCH v7 6/7] hw/arm/virt: Add 'compact-highmem' property Gavin Shan
` (2 subsequent siblings)
7 siblings, 0 replies; 11+ messages in thread
From: Gavin Shan @ 2022-10-29 22:43 UTC (permalink / raw)
To: qemu-arm
Cc: qemu-devel, maz, eric.auger, cohuck, richard.henderson,
peter.maydell, zhenyzha, shan.gavin
There are three high memory regions, which are VIRT_HIGH_REDIST2,
VIRT_HIGH_PCIE_ECAM and VIRT_HIGH_PCIE_MMIO. Their base addresses
are floating on highest RAM address. However, they can be disabled
in several cases.
(1) One specific high memory region is likely to be disabled by
code by toggling vms->highmem_{redists, ecam, mmio}.
(2) VIRT_HIGH_PCIE_ECAM region is disabled on machine, which is
'virt-2.12' or ealier than it.
(3) VIRT_HIGH_PCIE_ECAM region is disabled when firmware is loaded
on 32-bits system.
(4) One specific high memory region is disabled when it breaks the
PA space limit.
The current implementation of virt_set_{memmap, high_memmap}() isn't
optimized because the high memory region's PA space is always reserved,
regardless of whatever the actual state in the corresponding
vms->highmem_{redists, ecam, mmio} flag. In the code, 'base' and
'vms->highest_gpa' are always increased for case (1), (2) and (3).
It's unnecessary since the assigned PA space for the disabled high
memory region won't be used afterwards.
Improve the address assignment for those three high memory region by
skipping the address assignment for one specific high memory region if
it has been disabled in case (1), (2) and (3). The memory layout may
be changed after the improvement is applied, which leads to potential
migration breakage. So 'vms->highmem_compact' is added to control if
the improvement should be applied. For now, 'vms->highmem_compact' is
set to false, meaning that we don't have memory layout change until it
becomes configurable through property 'compact-highmem' in next patch.
Signed-off-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
---
hw/arm/virt.c | 15 ++++++++++-----
include/hw/arm/virt.h | 1 +
2 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index ee98a8a3b6..3b4c995f80 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1721,18 +1721,23 @@ static void virt_set_high_memmap(VirtMachineState *vms,
vms->memmap[i].size = region_size;
/*
- * Check each device to see if they fit in the PA space,
- * moving highest_gpa as we go.
+ * Check each device to see if it fits in the PA space,
+ * moving highest_gpa as we go. For compatibility, move
+ * highest_gpa for disabled fitting devices as well, if
+ * the compact layout has been disabled.
*
* For each device that doesn't fit, disable it.
*/
fits = (region_base + region_size) <= BIT_ULL(pa_bits);
- if (fits) {
- vms->highest_gpa = region_base + region_size - 1;
+ *region_enabled &= fits;
+ if (vms->highmem_compact && !*region_enabled) {
+ continue;
}
- *region_enabled &= fits;
base = region_base + region_size;
+ if (fits) {
+ vms->highest_gpa = base - 1;
+ }
}
}
diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
index 6ec479ca2b..709f623741 100644
--- a/include/hw/arm/virt.h
+++ b/include/hw/arm/virt.h
@@ -144,6 +144,7 @@ struct VirtMachineState {
PFlashCFI01 *flash[2];
bool secure;
bool highmem;
+ bool highmem_compact;
bool highmem_ecam;
bool highmem_mmio;
bool highmem_redists;
--
2.23.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v7 6/7] hw/arm/virt: Add 'compact-highmem' property
2022-10-29 22:43 [PATCH v7 0/7] hw/arm/virt: Improve address assignment for high memory regions Gavin Shan
` (4 preceding siblings ...)
2022-10-29 22:43 ` [PATCH v7 5/7] hw/arm/virt: Improve high memory region address assignment Gavin Shan
@ 2022-10-29 22:43 ` Gavin Shan
2022-10-29 22:43 ` [PATCH v7 7/7] hw/arm/virt: Add properties to disable high memory regions Gavin Shan
2022-11-21 17:16 ` [PATCH v7 0/7] hw/arm/virt: Improve address assignment for " Peter Maydell
7 siblings, 0 replies; 11+ messages in thread
From: Gavin Shan @ 2022-10-29 22:43 UTC (permalink / raw)
To: qemu-arm
Cc: qemu-devel, maz, eric.auger, cohuck, richard.henderson,
peter.maydell, zhenyzha, shan.gavin
After the improvement to high memory region address assignment is
applied, the memory layout can be changed, introducing possible
migration breakage. For example, VIRT_HIGH_PCIE_MMIO memory region
is disabled or enabled when the optimization is applied or not, with
the following configuration. The configuration is only achievable by
modifying the source code until more properties are added to allow
users selectively disable those high memory regions.
pa_bits = 40;
vms->highmem_redists = false;
vms->highmem_ecam = false;
vms->highmem_mmio = true;
# qemu-system-aarch64 -accel kvm -cpu host \
-machine virt-7.2,compact-highmem={on, off} \
-m 4G,maxmem=511G -monitor stdio
Region compact-highmem=off compact-highmem=on
----------------------------------------------------------------
MEM [1GB 512GB] [1GB 512GB]
HIGH_GIC_REDISTS2 [512GB 512GB+64MB] [disabled]
HIGH_PCIE_ECAM [512GB+256MB 512GB+512MB] [disabled]
HIGH_PCIE_MMIO [disabled] [512GB 1TB]
In order to keep backwords compatibility, we need to disable the
optimization on machine, which is virt-7.1 or ealier than it. It
means the optimization is enabled by default from virt-7.2. Besides,
'compact-highmem' property is added so that the optimization can be
explicitly enabled or disabled on all machine types by users.
Signed-off-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
---
docs/system/arm/virt.rst | 4 ++++
hw/arm/virt.c | 32 ++++++++++++++++++++++++++++++++
include/hw/arm/virt.h | 1 +
3 files changed, 37 insertions(+)
diff --git a/docs/system/arm/virt.rst b/docs/system/arm/virt.rst
index 20442ea2c1..4454706392 100644
--- a/docs/system/arm/virt.rst
+++ b/docs/system/arm/virt.rst
@@ -94,6 +94,10 @@ highmem
address space above 32 bits. The default is ``on`` for machine types
later than ``virt-2.12``.
+compact-highmem
+ Set ``on``/``off`` to enable/disable the compact layout for high memory regions.
+ The default is ``on`` for machine types later than ``virt-7.2``.
+
gic-version
Specify the version of the Generic Interrupt Controller (GIC) to provide.
Valid values are:
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 3b4c995f80..020a95cfa2 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -174,6 +174,12 @@ static const MemMapEntry base_memmap[] = {
* Note the extended_memmap is sized so that it eventually also includes the
* base_memmap entries (VIRT_HIGH_GIC_REDIST2 index is greater than the last
* index of base_memmap).
+ *
+ * The memory map for these Highmem IO Regions can be in legacy or compact
+ * layout, depending on 'compact-highmem' property. With legacy layout, the
+ * PA space for one specific region is always reserved, even if the region
+ * has been disabled or doesn't fit into the PA space. However, the PA space
+ * for the region won't be reserved in these circumstances with compact layout.
*/
static MemMapEntry extended_memmap[] = {
/* Additional 64 MB redist region (can contain up to 512 redistributors) */
@@ -2351,6 +2357,20 @@ static void virt_set_highmem(Object *obj, bool value, Error **errp)
vms->highmem = value;
}
+static bool virt_get_compact_highmem(Object *obj, Error **errp)
+{
+ VirtMachineState *vms = VIRT_MACHINE(obj);
+
+ return vms->highmem_compact;
+}
+
+static void virt_set_compact_highmem(Object *obj, bool value, Error **errp)
+{
+ VirtMachineState *vms = VIRT_MACHINE(obj);
+
+ vms->highmem_compact = value;
+}
+
static bool virt_get_its(Object *obj, Error **errp)
{
VirtMachineState *vms = VIRT_MACHINE(obj);
@@ -2969,6 +2989,13 @@ static void virt_machine_class_init(ObjectClass *oc, void *data)
"Set on/off to enable/disable using "
"physical address space above 32 bits");
+ object_class_property_add_bool(oc, "compact-highmem",
+ virt_get_compact_highmem,
+ virt_set_compact_highmem);
+ object_class_property_set_description(oc, "compact-highmem",
+ "Set on/off to enable/disable compact "
+ "layout for high memory regions");
+
object_class_property_add_str(oc, "gic-version", virt_get_gic_version,
virt_set_gic_version);
object_class_property_set_description(oc, "gic-version",
@@ -3053,6 +3080,7 @@ static void virt_instance_init(Object *obj)
/* High memory is enabled by default */
vms->highmem = true;
+ vms->highmem_compact = !vmc->no_highmem_compact;
vms->gic_version = VIRT_GIC_VERSION_NOSEL;
vms->highmem_ecam = !vmc->no_highmem_ecam;
@@ -3122,8 +3150,12 @@ DEFINE_VIRT_MACHINE_AS_LATEST(7, 2)
static void virt_machine_7_1_options(MachineClass *mc)
{
+ VirtMachineClass *vmc = VIRT_MACHINE_CLASS(OBJECT_CLASS(mc));
+
virt_machine_7_2_options(mc);
compat_props_add(mc->compat_props, hw_compat_7_1, hw_compat_7_1_len);
+ /* Compact layout for high memory regions was introduced with 7.2 */
+ vmc->no_highmem_compact = true;
}
DEFINE_VIRT_MACHINE(7, 1)
diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
index 709f623741..c7dd59d7f1 100644
--- a/include/hw/arm/virt.h
+++ b/include/hw/arm/virt.h
@@ -125,6 +125,7 @@ struct VirtMachineClass {
bool no_pmu;
bool claim_edge_triggered_timers;
bool smbios_old_sys_ver;
+ bool no_highmem_compact;
bool no_highmem_ecam;
bool no_ged; /* Machines < 4.2 have no support for ACPI GED device */
bool kvm_no_adjvtime;
--
2.23.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH v7 7/7] hw/arm/virt: Add properties to disable high memory regions
2022-10-29 22:43 [PATCH v7 0/7] hw/arm/virt: Improve address assignment for high memory regions Gavin Shan
` (5 preceding siblings ...)
2022-10-29 22:43 ` [PATCH v7 6/7] hw/arm/virt: Add 'compact-highmem' property Gavin Shan
@ 2022-10-29 22:43 ` Gavin Shan
2022-11-02 8:57 ` Cornelia Huck
2022-11-03 10:12 ` Eric Auger
2022-11-21 17:16 ` [PATCH v7 0/7] hw/arm/virt: Improve address assignment for " Peter Maydell
7 siblings, 2 replies; 11+ messages in thread
From: Gavin Shan @ 2022-10-29 22:43 UTC (permalink / raw)
To: qemu-arm
Cc: qemu-devel, maz, eric.auger, cohuck, richard.henderson,
peter.maydell, zhenyzha, shan.gavin
The 3 high memory regions are usually enabled by default, but they may
be not used. For example, VIRT_HIGH_GIC_REDIST2 isn't needed by GICv2.
This leads to waste in the PA space.
Add properties ("highmem-redists", "highmem-ecam", "highmem-mmio") to
allow users selectively disable them if needed. After that, the high
memory region for GICv3 or GICv4 redistributor can be disabled by user,
the number of maximal supported CPUs needs to be calculated based on
'vms->highmem_redists'. The follow-up error message is also improved
to indicate if the high memory region for GICv3 and GICv4 has been
enabled or not.
Suggested-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
---
docs/system/arm/virt.rst | 13 +++++++
hw/arm/virt.c | 75 ++++++++++++++++++++++++++++++++++++++--
2 files changed, 86 insertions(+), 2 deletions(-)
diff --git a/docs/system/arm/virt.rst b/docs/system/arm/virt.rst
index 4454706392..188a4f211f 100644
--- a/docs/system/arm/virt.rst
+++ b/docs/system/arm/virt.rst
@@ -98,6 +98,19 @@ compact-highmem
Set ``on``/``off`` to enable/disable the compact layout for high memory regions.
The default is ``on`` for machine types later than ``virt-7.2``.
+highmem-redists
+ Set ``on``/``off`` to enable/disable the high memory region for GICv3 or
+ GICv4 redistributor. The default is ``on``. Setting this to ``off`` will
+ limit the maximum number of CPUs when GICv3 or GICv4 is used.
+
+highmem-ecam
+ Set ``on``/``off`` to enable/disable the high memory region for PCI ECAM.
+ The default is ``on`` for machine types later than ``virt-3.0``.
+
+highmem-mmio
+ Set ``on``/``off`` to enable/disable the high memory region for PCI MMIO.
+ The default is ``on``.
+
gic-version
Specify the version of the Generic Interrupt Controller (GIC) to provide.
Valid values are:
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 020a95cfa2..65cff7add1 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -2095,14 +2095,20 @@ static void machvirt_init(MachineState *machine)
if (vms->gic_version == VIRT_GIC_VERSION_2) {
virt_max_cpus = GIC_NCPU;
} else {
- virt_max_cpus = virt_redist_capacity(vms, VIRT_GIC_REDIST) +
- virt_redist_capacity(vms, VIRT_HIGH_GIC_REDIST2);
+ virt_max_cpus = virt_redist_capacity(vms, VIRT_GIC_REDIST);
+ if (vms->highmem_redists) {
+ virt_max_cpus += virt_redist_capacity(vms, VIRT_HIGH_GIC_REDIST2);
+ }
}
if (max_cpus > virt_max_cpus) {
error_report("Number of SMP CPUs requested (%d) exceeds max CPUs "
"supported by machine 'mach-virt' (%d)",
max_cpus, virt_max_cpus);
+ if (vms->gic_version != VIRT_GIC_VERSION_2 && !vms->highmem_redists) {
+ error_printf("Try 'highmem-redists=on' for more CPUs\n");
+ }
+
exit(1);
}
@@ -2371,6 +2377,49 @@ static void virt_set_compact_highmem(Object *obj, bool value, Error **errp)
vms->highmem_compact = value;
}
+static bool virt_get_highmem_redists(Object *obj, Error **errp)
+{
+ VirtMachineState *vms = VIRT_MACHINE(obj);
+
+ return vms->highmem_redists;
+}
+
+static void virt_set_highmem_redists(Object *obj, bool value, Error **errp)
+{
+ VirtMachineState *vms = VIRT_MACHINE(obj);
+
+ vms->highmem_redists = value;
+}
+
+static bool virt_get_highmem_ecam(Object *obj, Error **errp)
+{
+ VirtMachineState *vms = VIRT_MACHINE(obj);
+
+ return vms->highmem_ecam;
+}
+
+static void virt_set_highmem_ecam(Object *obj, bool value, Error **errp)
+{
+ VirtMachineState *vms = VIRT_MACHINE(obj);
+
+ vms->highmem_ecam = value;
+}
+
+static bool virt_get_highmem_mmio(Object *obj, Error **errp)
+{
+ VirtMachineState *vms = VIRT_MACHINE(obj);
+
+ return vms->highmem_mmio;
+}
+
+static void virt_set_highmem_mmio(Object *obj, bool value, Error **errp)
+{
+ VirtMachineState *vms = VIRT_MACHINE(obj);
+
+ vms->highmem_mmio = value;
+}
+
+
static bool virt_get_its(Object *obj, Error **errp)
{
VirtMachineState *vms = VIRT_MACHINE(obj);
@@ -2996,6 +3045,28 @@ static void virt_machine_class_init(ObjectClass *oc, void *data)
"Set on/off to enable/disable compact "
"layout for high memory regions");
+ object_class_property_add_bool(oc, "highmem-redists",
+ virt_get_highmem_redists,
+ virt_set_highmem_redists);
+ object_class_property_set_description(oc, "highmem-redists",
+ "Set on/off to enable/disable high "
+ "memory region for GICv3 or GICv4 "
+ "redistributor");
+
+ object_class_property_add_bool(oc, "highmem-ecam",
+ virt_get_highmem_ecam,
+ virt_set_highmem_ecam);
+ object_class_property_set_description(oc, "highmem-ecam",
+ "Set on/off to enable/disable high "
+ "memory region for PCI ECAM");
+
+ object_class_property_add_bool(oc, "highmem-mmio",
+ virt_get_highmem_mmio,
+ virt_set_highmem_mmio);
+ object_class_property_set_description(oc, "highmem-mmio",
+ "Set on/off to enable/disable high "
+ "memory region for PCI MMIO");
+
object_class_property_add_str(oc, "gic-version", virt_get_gic_version,
virt_set_gic_version);
object_class_property_set_description(oc, "gic-version",
--
2.23.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH v7 7/7] hw/arm/virt: Add properties to disable high memory regions
2022-10-29 22:43 ` [PATCH v7 7/7] hw/arm/virt: Add properties to disable high memory regions Gavin Shan
@ 2022-11-02 8:57 ` Cornelia Huck
2022-11-03 10:12 ` Eric Auger
1 sibling, 0 replies; 11+ messages in thread
From: Cornelia Huck @ 2022-11-02 8:57 UTC (permalink / raw)
To: Gavin Shan, qemu-arm
Cc: qemu-devel, maz, eric.auger, richard.henderson, peter.maydell,
zhenyzha, shan.gavin
On Sun, Oct 30 2022, Gavin Shan <gshan@redhat.com> wrote:
> The 3 high memory regions are usually enabled by default, but they may
> be not used. For example, VIRT_HIGH_GIC_REDIST2 isn't needed by GICv2.
> This leads to waste in the PA space.
>
> Add properties ("highmem-redists", "highmem-ecam", "highmem-mmio") to
> allow users selectively disable them if needed. After that, the high
> memory region for GICv3 or GICv4 redistributor can be disabled by user,
> the number of maximal supported CPUs needs to be calculated based on
> 'vms->highmem_redists'. The follow-up error message is also improved
> to indicate if the high memory region for GICv3 and GICv4 has been
> enabled or not.
>
> Suggested-by: Marc Zyngier <maz@kernel.org>
> Signed-off-by: Gavin Shan <gshan@redhat.com>
> Reviewed-by: Marc Zyngier <maz@kernel.org>
> ---
> docs/system/arm/virt.rst | 13 +++++++
> hw/arm/virt.c | 75 ++++++++++++++++++++++++++++++++++++++--
> 2 files changed, 86 insertions(+), 2 deletions(-)
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [PATCH v7 7/7] hw/arm/virt: Add properties to disable high memory regions
2022-10-29 22:43 ` [PATCH v7 7/7] hw/arm/virt: Add properties to disable high memory regions Gavin Shan
2022-11-02 8:57 ` Cornelia Huck
@ 2022-11-03 10:12 ` Eric Auger
1 sibling, 0 replies; 11+ messages in thread
From: Eric Auger @ 2022-11-03 10:12 UTC (permalink / raw)
To: Gavin Shan, qemu-arm
Cc: qemu-devel, maz, cohuck, richard.henderson, peter.maydell,
zhenyzha, shan.gavin
Hi Gavin,
On 10/30/22 00:43, Gavin Shan wrote:
> The 3 high memory regions are usually enabled by default, but they may
> be not used. For example, VIRT_HIGH_GIC_REDIST2 isn't needed by GICv2.
> This leads to waste in the PA space.
>
> Add properties ("highmem-redists", "highmem-ecam", "highmem-mmio") to
> allow users selectively disable them if needed. After that, the high
> memory region for GICv3 or GICv4 redistributor can be disabled by user,
> the number of maximal supported CPUs needs to be calculated based on
> 'vms->highmem_redists'. The follow-up error message is also improved
> to indicate if the high memory region for GICv3 and GICv4 has been
> enabled or not.
>
> Suggested-by: Marc Zyngier <maz@kernel.org>
> Signed-off-by: Gavin Shan <gshan@redhat.com>
> Reviewed-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Thanks
Eric
> ---
> docs/system/arm/virt.rst | 13 +++++++
> hw/arm/virt.c | 75 ++++++++++++++++++++++++++++++++++++++--
> 2 files changed, 86 insertions(+), 2 deletions(-)
>
> diff --git a/docs/system/arm/virt.rst b/docs/system/arm/virt.rst
> index 4454706392..188a4f211f 100644
> --- a/docs/system/arm/virt.rst
> +++ b/docs/system/arm/virt.rst
> @@ -98,6 +98,19 @@ compact-highmem
> Set ``on``/``off`` to enable/disable the compact layout for high memory regions.
> The default is ``on`` for machine types later than ``virt-7.2``.
>
> +highmem-redists
> + Set ``on``/``off`` to enable/disable the high memory region for GICv3 or
> + GICv4 redistributor. The default is ``on``. Setting this to ``off`` will
> + limit the maximum number of CPUs when GICv3 or GICv4 is used.
> +
> +highmem-ecam
> + Set ``on``/``off`` to enable/disable the high memory region for PCI ECAM.
> + The default is ``on`` for machine types later than ``virt-3.0``.
> +
> +highmem-mmio
> + Set ``on``/``off`` to enable/disable the high memory region for PCI MMIO.
> + The default is ``on``.
> +
> gic-version
> Specify the version of the Generic Interrupt Controller (GIC) to provide.
> Valid values are:
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index 020a95cfa2..65cff7add1 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -2095,14 +2095,20 @@ static void machvirt_init(MachineState *machine)
> if (vms->gic_version == VIRT_GIC_VERSION_2) {
> virt_max_cpus = GIC_NCPU;
> } else {
> - virt_max_cpus = virt_redist_capacity(vms, VIRT_GIC_REDIST) +
> - virt_redist_capacity(vms, VIRT_HIGH_GIC_REDIST2);
> + virt_max_cpus = virt_redist_capacity(vms, VIRT_GIC_REDIST);
> + if (vms->highmem_redists) {
> + virt_max_cpus += virt_redist_capacity(vms, VIRT_HIGH_GIC_REDIST2);
> + }
> }
>
> if (max_cpus > virt_max_cpus) {
> error_report("Number of SMP CPUs requested (%d) exceeds max CPUs "
> "supported by machine 'mach-virt' (%d)",
> max_cpus, virt_max_cpus);
> + if (vms->gic_version != VIRT_GIC_VERSION_2 && !vms->highmem_redists) {
> + error_printf("Try 'highmem-redists=on' for more CPUs\n");
> + }
> +
> exit(1);
> }
>
> @@ -2371,6 +2377,49 @@ static void virt_set_compact_highmem(Object *obj, bool value, Error **errp)
> vms->highmem_compact = value;
> }
>
> +static bool virt_get_highmem_redists(Object *obj, Error **errp)
> +{
> + VirtMachineState *vms = VIRT_MACHINE(obj);
> +
> + return vms->highmem_redists;
> +}
> +
> +static void virt_set_highmem_redists(Object *obj, bool value, Error **errp)
> +{
> + VirtMachineState *vms = VIRT_MACHINE(obj);
> +
> + vms->highmem_redists = value;
> +}
> +
> +static bool virt_get_highmem_ecam(Object *obj, Error **errp)
> +{
> + VirtMachineState *vms = VIRT_MACHINE(obj);
> +
> + return vms->highmem_ecam;
> +}
> +
> +static void virt_set_highmem_ecam(Object *obj, bool value, Error **errp)
> +{
> + VirtMachineState *vms = VIRT_MACHINE(obj);
> +
> + vms->highmem_ecam = value;
> +}
> +
> +static bool virt_get_highmem_mmio(Object *obj, Error **errp)
> +{
> + VirtMachineState *vms = VIRT_MACHINE(obj);
> +
> + return vms->highmem_mmio;
> +}
> +
> +static void virt_set_highmem_mmio(Object *obj, bool value, Error **errp)
> +{
> + VirtMachineState *vms = VIRT_MACHINE(obj);
> +
> + vms->highmem_mmio = value;
> +}
> +
> +
> static bool virt_get_its(Object *obj, Error **errp)
> {
> VirtMachineState *vms = VIRT_MACHINE(obj);
> @@ -2996,6 +3045,28 @@ static void virt_machine_class_init(ObjectClass *oc, void *data)
> "Set on/off to enable/disable compact "
> "layout for high memory regions");
>
> + object_class_property_add_bool(oc, "highmem-redists",
> + virt_get_highmem_redists,
> + virt_set_highmem_redists);
> + object_class_property_set_description(oc, "highmem-redists",
> + "Set on/off to enable/disable high "
> + "memory region for GICv3 or GICv4 "
> + "redistributor");
> +
> + object_class_property_add_bool(oc, "highmem-ecam",
> + virt_get_highmem_ecam,
> + virt_set_highmem_ecam);
> + object_class_property_set_description(oc, "highmem-ecam",
> + "Set on/off to enable/disable high "
> + "memory region for PCI ECAM");
> +
> + object_class_property_add_bool(oc, "highmem-mmio",
> + virt_get_highmem_mmio,
> + virt_set_highmem_mmio);
> + object_class_property_set_description(oc, "highmem-mmio",
> + "Set on/off to enable/disable high "
> + "memory region for PCI MMIO");
> +
> object_class_property_add_str(oc, "gic-version", virt_get_gic_version,
> virt_set_gic_version);
> object_class_property_set_description(oc, "gic-version",
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v7 0/7] hw/arm/virt: Improve address assignment for high memory regions
2022-10-29 22:43 [PATCH v7 0/7] hw/arm/virt: Improve address assignment for high memory regions Gavin Shan
` (6 preceding siblings ...)
2022-10-29 22:43 ` [PATCH v7 7/7] hw/arm/virt: Add properties to disable high memory regions Gavin Shan
@ 2022-11-21 17:16 ` Peter Maydell
7 siblings, 0 replies; 11+ messages in thread
From: Peter Maydell @ 2022-11-21 17:16 UTC (permalink / raw)
To: Gavin Shan
Cc: qemu-arm, qemu-devel, maz, eric.auger, cohuck, richard.henderson,
zhenyzha, shan.gavin
On Sat, 29 Oct 2022 at 23:43, Gavin Shan <gshan@redhat.com> wrote:
>
> There are three high memory regions, which are VIRT_HIGH_REDIST2,
> VIRT_HIGH_PCIE_ECAM and VIRT_HIGH_PCIE_MMIO. Their base addresses
> are floating on highest RAM address. However, they can be disabled
> in several cases.
>
> (1) One specific high memory region is disabled by developer by
> toggling vms->highmem_{redists, ecam, mmio}.
>
> (2) VIRT_HIGH_PCIE_ECAM region is disabled on machine, which is
> 'virt-2.12' or ealier than it.
>
> (3) VIRT_HIGH_PCIE_ECAM region is disabled when firmware is loaded
> on 32-bits system.
>
> (4) One specific high memory region is disabled when it breaks the
> PA space limit.
>
> The current implementation of virt_set_memmap() isn't comprehensive
> because the space for one specific high memory region is always
> reserved from the PA space for case (1), (2) and (3). In the code,
> 'base' and 'vms->highest_gpa' are always increased for those three
> cases. It's unnecessary since the assigned space of the disabled
> high memory region won't be used afterwards.
>
> The series intends to improve the address assignment for these
> high memory regions and introduces new properties for user to
> selectively disable those 3 high memory regions.
I've applied this patchset to a target-arm-for-8.0 branch
that will end up being the target-arm queue once 7.2 is out.
thanks
-- PMM
^ permalink raw reply [flat|nested] 11+ messages in thread