* [PATCH v5 1/6] hw/arm/virt: Introduce virt_set_high_memmap() helper
2022-10-11 23:18 [PATCH v5 0/6] hw/arm/virt: Improve address assignment for high memory regions Gavin Shan
@ 2022-10-11 23:18 ` Gavin Shan
2022-10-11 23:18 ` [PATCH v5 2/6] hw/arm/virt: Rename variable size to region_size in virt_set_high_memmap() Gavin Shan
` (4 subsequent siblings)
5 siblings, 0 replies; 18+ messages in thread
From: Gavin Shan @ 2022-10-11 23:18 UTC (permalink / raw)
To: qemu-arm
Cc: qemu-devel, maz, eric.auger, cohuck, zhenyzha, richard.henderson,
peter.maydell, shan.gavin
This introduces virt_set_high_memmap() helper. The logic of high
memory region address assignment is moved to the helper. The intention
is to make the subsequent optimization for high memory region address
assignment easier.
No functional change intended.
Signed-off-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
---
hw/arm/virt.c | 74 ++++++++++++++++++++++++++++-----------------------
1 file changed, 41 insertions(+), 33 deletions(-)
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index cda9defe8f..7572c44bda 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1689,6 +1689,46 @@ static uint64_t virt_cpu_mp_affinity(VirtMachineState *vms, int idx)
return arm_cpu_mp_affinity(idx, clustersz);
}
+static void virt_set_high_memmap(VirtMachineState *vms,
+ hwaddr base, int pa_bits)
+{
+ int i;
+
+ for (i = VIRT_LOWMEMMAP_LAST; i < ARRAY_SIZE(extended_memmap); i++) {
+ hwaddr size = extended_memmap[i].size;
+ bool fits;
+
+ base = ROUND_UP(base, size);
+ vms->memmap[i].base = base;
+ vms->memmap[i].size = size;
+
+ /*
+ * Check each device to see if they fit in the PA space,
+ * moving highest_gpa as we go.
+ *
+ * For each device that doesn't fit, disable it.
+ */
+ fits = (base + size) <= BIT_ULL(pa_bits);
+ if (fits) {
+ vms->highest_gpa = base + size - 1;
+ }
+
+ switch (i) {
+ case VIRT_HIGH_GIC_REDIST2:
+ vms->highmem_redists &= fits;
+ break;
+ case VIRT_HIGH_PCIE_ECAM:
+ vms->highmem_ecam &= fits;
+ break;
+ case VIRT_HIGH_PCIE_MMIO:
+ vms->highmem_mmio &= fits;
+ break;
+ }
+
+ base += size;
+ }
+}
+
static void virt_set_memmap(VirtMachineState *vms, int pa_bits)
{
MachineState *ms = MACHINE(vms);
@@ -1744,39 +1784,7 @@ static void virt_set_memmap(VirtMachineState *vms, int pa_bits)
/* We know for sure that at least the memory fits in the PA space */
vms->highest_gpa = memtop - 1;
- for (i = VIRT_LOWMEMMAP_LAST; i < ARRAY_SIZE(extended_memmap); i++) {
- hwaddr size = extended_memmap[i].size;
- bool fits;
-
- base = ROUND_UP(base, size);
- vms->memmap[i].base = base;
- vms->memmap[i].size = size;
-
- /*
- * Check each device to see if they fit in the PA space,
- * moving highest_gpa as we go.
- *
- * For each device that doesn't fit, disable it.
- */
- fits = (base + size) <= BIT_ULL(pa_bits);
- if (fits) {
- vms->highest_gpa = base + size - 1;
- }
-
- switch (i) {
- case VIRT_HIGH_GIC_REDIST2:
- vms->highmem_redists &= fits;
- break;
- case VIRT_HIGH_PCIE_ECAM:
- vms->highmem_ecam &= fits;
- break;
- case VIRT_HIGH_PCIE_MMIO:
- vms->highmem_mmio &= fits;
- break;
- }
-
- base += size;
- }
+ virt_set_high_memmap(vms, base, pa_bits);
if (device_memory_size > 0) {
ms->device_memory = g_malloc0(sizeof(*ms->device_memory));
--
2.23.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH v5 2/6] hw/arm/virt: Rename variable size to region_size in virt_set_high_memmap()
2022-10-11 23:18 [PATCH v5 0/6] hw/arm/virt: Improve address assignment for high memory regions Gavin Shan
2022-10-11 23:18 ` [PATCH v5 1/6] hw/arm/virt: Introduce virt_set_high_memmap() helper Gavin Shan
@ 2022-10-11 23:18 ` Gavin Shan
2022-10-11 23:18 ` [PATCH v5 3/6] hw/arm/virt: Introduce variable region_base " Gavin Shan
` (3 subsequent siblings)
5 siblings, 0 replies; 18+ messages in thread
From: Gavin Shan @ 2022-10-11 23:18 UTC (permalink / raw)
To: qemu-arm
Cc: qemu-devel, maz, eric.auger, cohuck, zhenyzha, richard.henderson,
peter.maydell, shan.gavin
This renames variable 'size' to 'region_size' in virt_set_high_memmap().
Its counterpart ('region_base') will be introduced in next patch.
No functional change intended.
Signed-off-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
---
hw/arm/virt.c | 15 ++++++++-------
1 file changed, 8 insertions(+), 7 deletions(-)
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 7572c44bda..e2ae88cf8b 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1692,15 +1692,16 @@ static uint64_t virt_cpu_mp_affinity(VirtMachineState *vms, int idx)
static void virt_set_high_memmap(VirtMachineState *vms,
hwaddr base, int pa_bits)
{
+ hwaddr region_size;
+ bool fits;
int i;
for (i = VIRT_LOWMEMMAP_LAST; i < ARRAY_SIZE(extended_memmap); i++) {
- hwaddr size = extended_memmap[i].size;
- bool fits;
+ region_size = extended_memmap[i].size;
- base = ROUND_UP(base, size);
+ base = ROUND_UP(base, region_size);
vms->memmap[i].base = base;
- vms->memmap[i].size = size;
+ vms->memmap[i].size = region_size;
/*
* Check each device to see if they fit in the PA space,
@@ -1708,9 +1709,9 @@ static void virt_set_high_memmap(VirtMachineState *vms,
*
* For each device that doesn't fit, disable it.
*/
- fits = (base + size) <= BIT_ULL(pa_bits);
+ fits = (base + region_size) <= BIT_ULL(pa_bits);
if (fits) {
- vms->highest_gpa = base + size - 1;
+ vms->highest_gpa = base + region_size - 1;
}
switch (i) {
@@ -1725,7 +1726,7 @@ static void virt_set_high_memmap(VirtMachineState *vms,
break;
}
- base += size;
+ base += region_size;
}
}
--
2.23.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH v5 3/6] hw/arm/virt: Introduce variable region_base in virt_set_high_memmap()
2022-10-11 23:18 [PATCH v5 0/6] hw/arm/virt: Improve address assignment for high memory regions Gavin Shan
2022-10-11 23:18 ` [PATCH v5 1/6] hw/arm/virt: Introduce virt_set_high_memmap() helper Gavin Shan
2022-10-11 23:18 ` [PATCH v5 2/6] hw/arm/virt: Rename variable size to region_size in virt_set_high_memmap() Gavin Shan
@ 2022-10-11 23:18 ` Gavin Shan
2022-10-11 23:18 ` [PATCH v5 4/6] hw/arm/virt: Introduce virt_get_high_memmap_enabled() helper Gavin Shan
` (2 subsequent siblings)
5 siblings, 0 replies; 18+ messages in thread
From: Gavin Shan @ 2022-10-11 23:18 UTC (permalink / raw)
To: qemu-arm
Cc: qemu-devel, maz, eric.auger, cohuck, zhenyzha, richard.henderson,
peter.maydell, shan.gavin
This introduces variable 'region_base' for the base address of the
specific high memory region. It's the preparatory work to optimize
high memory region address assignment.
No functional change intended.
Signed-off-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
---
hw/arm/virt.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index e2ae88cf8b..0bf3cb7057 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1692,15 +1692,15 @@ static uint64_t virt_cpu_mp_affinity(VirtMachineState *vms, int idx)
static void virt_set_high_memmap(VirtMachineState *vms,
hwaddr base, int pa_bits)
{
- hwaddr region_size;
+ hwaddr region_base, region_size;
bool fits;
int i;
for (i = VIRT_LOWMEMMAP_LAST; i < ARRAY_SIZE(extended_memmap); i++) {
+ region_base = ROUND_UP(base, extended_memmap[i].size);
region_size = extended_memmap[i].size;
- base = ROUND_UP(base, region_size);
- vms->memmap[i].base = base;
+ vms->memmap[i].base = region_base;
vms->memmap[i].size = region_size;
/*
@@ -1709,9 +1709,9 @@ static void virt_set_high_memmap(VirtMachineState *vms,
*
* For each device that doesn't fit, disable it.
*/
- fits = (base + region_size) <= BIT_ULL(pa_bits);
+ fits = (region_base + region_size) <= BIT_ULL(pa_bits);
if (fits) {
- vms->highest_gpa = base + region_size - 1;
+ vms->highest_gpa = region_base + region_size - 1;
}
switch (i) {
@@ -1726,7 +1726,7 @@ static void virt_set_high_memmap(VirtMachineState *vms,
break;
}
- base += region_size;
+ base = region_base + region_size;
}
}
--
2.23.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH v5 4/6] hw/arm/virt: Introduce virt_get_high_memmap_enabled() helper
2022-10-11 23:18 [PATCH v5 0/6] hw/arm/virt: Improve address assignment for high memory regions Gavin Shan
` (2 preceding siblings ...)
2022-10-11 23:18 ` [PATCH v5 3/6] hw/arm/virt: Introduce variable region_base " Gavin Shan
@ 2022-10-11 23:18 ` Gavin Shan
2022-10-19 13:50 ` Cornelia Huck
2022-10-20 4:57 ` Eric Auger
2022-10-11 23:18 ` [PATCH v5 5/6] hw/arm/virt: Improve high memory region address assignment Gavin Shan
2022-10-11 23:18 ` [PATCH v5 6/6] hw/arm/virt: Add 'compact-highmem' property Gavin Shan
5 siblings, 2 replies; 18+ messages in thread
From: Gavin Shan @ 2022-10-11 23:18 UTC (permalink / raw)
To: qemu-arm
Cc: qemu-devel, maz, eric.auger, cohuck, zhenyzha, richard.henderson,
peter.maydell, shan.gavin
This introduces virt_get_high_memmap_enabled() helper, which returns
the pointer to vms->highmem_{redists, ecam, mmio}. The pointer will
be used in the subsequent patches.
No functional change intended.
Signed-off-by: Gavin Shan <gshan@redhat.com>
Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
---
hw/arm/virt.c | 32 +++++++++++++++++++-------------
1 file changed, 19 insertions(+), 13 deletions(-)
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 0bf3cb7057..ee98a8a3b6 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1689,14 +1689,31 @@ static uint64_t virt_cpu_mp_affinity(VirtMachineState *vms, int idx)
return arm_cpu_mp_affinity(idx, clustersz);
}
+static inline bool *virt_get_high_memmap_enabled(VirtMachineState *vms,
+ int index)
+{
+ bool *enabled_array[] = {
+ &vms->highmem_redists,
+ &vms->highmem_ecam,
+ &vms->highmem_mmio,
+ };
+
+ assert(ARRAY_SIZE(extended_memmap) - VIRT_LOWMEMMAP_LAST ==
+ ARRAY_SIZE(enabled_array));
+ assert(index - VIRT_LOWMEMMAP_LAST < ARRAY_SIZE(enabled_array));
+
+ return enabled_array[index - VIRT_LOWMEMMAP_LAST];
+}
+
static void virt_set_high_memmap(VirtMachineState *vms,
hwaddr base, int pa_bits)
{
hwaddr region_base, region_size;
- bool fits;
+ bool *region_enabled, fits;
int i;
for (i = VIRT_LOWMEMMAP_LAST; i < ARRAY_SIZE(extended_memmap); i++) {
+ region_enabled = virt_get_high_memmap_enabled(vms, i);
region_base = ROUND_UP(base, extended_memmap[i].size);
region_size = extended_memmap[i].size;
@@ -1714,18 +1731,7 @@ static void virt_set_high_memmap(VirtMachineState *vms,
vms->highest_gpa = region_base + region_size - 1;
}
- switch (i) {
- case VIRT_HIGH_GIC_REDIST2:
- vms->highmem_redists &= fits;
- break;
- case VIRT_HIGH_PCIE_ECAM:
- vms->highmem_ecam &= fits;
- break;
- case VIRT_HIGH_PCIE_MMIO:
- vms->highmem_mmio &= fits;
- break;
- }
-
+ *region_enabled &= fits;
base = region_base + region_size;
}
}
--
2.23.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* Re: [PATCH v5 4/6] hw/arm/virt: Introduce virt_get_high_memmap_enabled() helper
2022-10-11 23:18 ` [PATCH v5 4/6] hw/arm/virt: Introduce virt_get_high_memmap_enabled() helper Gavin Shan
@ 2022-10-19 13:50 ` Cornelia Huck
2022-10-20 4:57 ` Eric Auger
1 sibling, 0 replies; 18+ messages in thread
From: Cornelia Huck @ 2022-10-19 13:50 UTC (permalink / raw)
To: Gavin Shan, qemu-arm
Cc: qemu-devel, maz, eric.auger, zhenyzha, richard.henderson,
peter.maydell, shan.gavin
On Wed, Oct 12 2022, Gavin Shan <gshan@redhat.com> wrote:
> This introduces virt_get_high_memmap_enabled() helper, which returns
> the pointer to vms->highmem_{redists, ecam, mmio}. The pointer will
> be used in the subsequent patches.
>
> No functional change intended.
>
> Signed-off-by: Gavin Shan <gshan@redhat.com>
> Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
> ---
> hw/arm/virt.c | 32 +++++++++++++++++++-------------
> 1 file changed, 19 insertions(+), 13 deletions(-)
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [PATCH v5 4/6] hw/arm/virt: Introduce virt_get_high_memmap_enabled() helper
2022-10-11 23:18 ` [PATCH v5 4/6] hw/arm/virt: Introduce virt_get_high_memmap_enabled() helper Gavin Shan
2022-10-19 13:50 ` Cornelia Huck
@ 2022-10-20 4:57 ` Eric Auger
1 sibling, 0 replies; 18+ messages in thread
From: Eric Auger @ 2022-10-20 4:57 UTC (permalink / raw)
To: Gavin Shan, qemu-arm
Cc: qemu-devel, maz, cohuck, zhenyzha, richard.henderson,
peter.maydell, shan.gavin
On 10/12/22 01:18, Gavin Shan wrote:
> This introduces virt_get_high_memmap_enabled() helper, which returns
> the pointer to vms->highmem_{redists, ecam, mmio}. The pointer will
> be used in the subsequent patches.
>
> No functional change intended.
>
> Signed-off-by: Gavin Shan <gshan@redhat.com>
> Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Eric
> ---
> hw/arm/virt.c | 32 +++++++++++++++++++-------------
> 1 file changed, 19 insertions(+), 13 deletions(-)
>
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index 0bf3cb7057..ee98a8a3b6 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -1689,14 +1689,31 @@ static uint64_t virt_cpu_mp_affinity(VirtMachineState *vms, int idx)
> return arm_cpu_mp_affinity(idx, clustersz);
> }
>
> +static inline bool *virt_get_high_memmap_enabled(VirtMachineState *vms,
> + int index)
> +{
> + bool *enabled_array[] = {
> + &vms->highmem_redists,
> + &vms->highmem_ecam,
> + &vms->highmem_mmio,
> + };
> +
> + assert(ARRAY_SIZE(extended_memmap) - VIRT_LOWMEMMAP_LAST ==
> + ARRAY_SIZE(enabled_array));
> + assert(index - VIRT_LOWMEMMAP_LAST < ARRAY_SIZE(enabled_array));
> +
> + return enabled_array[index - VIRT_LOWMEMMAP_LAST];
> +}
> +
> static void virt_set_high_memmap(VirtMachineState *vms,
> hwaddr base, int pa_bits)
> {
> hwaddr region_base, region_size;
> - bool fits;
> + bool *region_enabled, fits;
> int i;
>
> for (i = VIRT_LOWMEMMAP_LAST; i < ARRAY_SIZE(extended_memmap); i++) {
> + region_enabled = virt_get_high_memmap_enabled(vms, i);
> region_base = ROUND_UP(base, extended_memmap[i].size);
> region_size = extended_memmap[i].size;
>
> @@ -1714,18 +1731,7 @@ static void virt_set_high_memmap(VirtMachineState *vms,
> vms->highest_gpa = region_base + region_size - 1;
> }
>
> - switch (i) {
> - case VIRT_HIGH_GIC_REDIST2:
> - vms->highmem_redists &= fits;
> - break;
> - case VIRT_HIGH_PCIE_ECAM:
> - vms->highmem_ecam &= fits;
> - break;
> - case VIRT_HIGH_PCIE_MMIO:
> - vms->highmem_mmio &= fits;
> - break;
> - }
> -
> + *region_enabled &= fits;
> base = region_base + region_size;
> }
> }
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH v5 5/6] hw/arm/virt: Improve high memory region address assignment
2022-10-11 23:18 [PATCH v5 0/6] hw/arm/virt: Improve address assignment for high memory regions Gavin Shan
` (3 preceding siblings ...)
2022-10-11 23:18 ` [PATCH v5 4/6] hw/arm/virt: Introduce virt_get_high_memmap_enabled() helper Gavin Shan
@ 2022-10-11 23:18 ` Gavin Shan
2022-10-19 13:54 ` Cornelia Huck
2022-10-19 20:07 ` Eric Auger
2022-10-11 23:18 ` [PATCH v5 6/6] hw/arm/virt: Add 'compact-highmem' property Gavin Shan
5 siblings, 2 replies; 18+ messages in thread
From: Gavin Shan @ 2022-10-11 23:18 UTC (permalink / raw)
To: qemu-arm
Cc: qemu-devel, maz, eric.auger, cohuck, zhenyzha, richard.henderson,
peter.maydell, shan.gavin
There are three high memory regions, which are VIRT_HIGH_REDIST2,
VIRT_HIGH_PCIE_ECAM and VIRT_HIGH_PCIE_MMIO. Their base addresses
are floating on highest RAM address. However, they can be disabled
in several cases.
(1) One specific high memory region is disabled by developer by
toggling vms->highmem_{redists, ecam, mmio}.
(2) VIRT_HIGH_PCIE_ECAM region is disabled on machine, which is
'virt-2.12' or ealier than it.
(3) VIRT_HIGH_PCIE_ECAM region is disabled when firmware is loaded
on 32-bits system.
(4) One specific high memory region is disabled when it breaks the
PA space limit.
The current implementation of virt_set_memmap() isn't comprehensive
because the space for one specific high memory region is always
reserved from the PA space for case (1), (2) and (3). In the code,
'base' and 'vms->highest_gpa' are always increased for those three
cases. It's unnecessary since the assigned space of the disabled
high memory region won't be used afterwards.
This improves the address assignment for those three high memory
region by skipping the address assignment for one specific high
memory region if it has been disabled in case (1), (2) and (3).
'vms->high_compact' is false for now, meaning that we don't have
any behavior changes until it becomes configurable through property
'compact-highmem' in next patch.
Signed-off-by: Gavin Shan <gshan@redhat.com>
Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
---
hw/arm/virt.c | 23 +++++++++++++++--------
include/hw/arm/virt.h | 1 +
2 files changed, 16 insertions(+), 8 deletions(-)
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index ee98a8a3b6..c05cfb5314 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1717,22 +1717,29 @@ static void virt_set_high_memmap(VirtMachineState *vms,
region_base = ROUND_UP(base, extended_memmap[i].size);
region_size = extended_memmap[i].size;
- vms->memmap[i].base = region_base;
- vms->memmap[i].size = region_size;
-
/*
* Check each device to see if they fit in the PA space,
- * moving highest_gpa as we go.
+ * moving highest_gpa as we go. For compatibility, move
+ * highest_gpa for disabled fitting devices as well, if
+ * the compact layout has been disabled.
*
* For each device that doesn't fit, disable it.
*/
fits = (region_base + region_size) <= BIT_ULL(pa_bits);
- if (fits) {
+ if (*region_enabled && fits) {
+ vms->memmap[i].base = region_base;
+ vms->memmap[i].size = region_size;
vms->highest_gpa = region_base + region_size - 1;
+ base = region_base + region_size;
+ } else {
+ *region_enabled = false;
+ if (!vms->highmem_compact) {
+ base = region_base + region_size;
+ if (fits) {
+ vms->highest_gpa = region_base + region_size - 1;
+ }
+ }
}
-
- *region_enabled &= fits;
- base = region_base + region_size;
}
}
diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
index 6ec479ca2b..709f623741 100644
--- a/include/hw/arm/virt.h
+++ b/include/hw/arm/virt.h
@@ -144,6 +144,7 @@ struct VirtMachineState {
PFlashCFI01 *flash[2];
bool secure;
bool highmem;
+ bool highmem_compact;
bool highmem_ecam;
bool highmem_mmio;
bool highmem_redists;
--
2.23.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* Re: [PATCH v5 5/6] hw/arm/virt: Improve high memory region address assignment
2022-10-11 23:18 ` [PATCH v5 5/6] hw/arm/virt: Improve high memory region address assignment Gavin Shan
@ 2022-10-19 13:54 ` Cornelia Huck
2022-10-19 20:07 ` Eric Auger
1 sibling, 0 replies; 18+ messages in thread
From: Cornelia Huck @ 2022-10-19 13:54 UTC (permalink / raw)
To: Gavin Shan, qemu-arm
Cc: qemu-devel, maz, eric.auger, zhenyzha, richard.henderson,
peter.maydell, shan.gavin
On Wed, Oct 12 2022, Gavin Shan <gshan@redhat.com> wrote:
> There are three high memory regions, which are VIRT_HIGH_REDIST2,
> VIRT_HIGH_PCIE_ECAM and VIRT_HIGH_PCIE_MMIO. Their base addresses
> are floating on highest RAM address. However, they can be disabled
> in several cases.
>
> (1) One specific high memory region is disabled by developer by
> toggling vms->highmem_{redists, ecam, mmio}.
>
> (2) VIRT_HIGH_PCIE_ECAM region is disabled on machine, which is
> 'virt-2.12' or ealier than it.
>
> (3) VIRT_HIGH_PCIE_ECAM region is disabled when firmware is loaded
> on 32-bits system.
>
> (4) One specific high memory region is disabled when it breaks the
> PA space limit.
>
> The current implementation of virt_set_memmap() isn't comprehensive
> because the space for one specific high memory region is always
> reserved from the PA space for case (1), (2) and (3). In the code,
> 'base' and 'vms->highest_gpa' are always increased for those three
> cases. It's unnecessary since the assigned space of the disabled
> high memory region won't be used afterwards.
>
> This improves the address assignment for those three high memory
> region by skipping the address assignment for one specific high
> memory region if it has been disabled in case (1), (2) and (3).
> 'vms->high_compact' is false for now, meaning that we don't have
> any behavior changes until it becomes configurable through property
> 'compact-highmem' in next patch.
>
> Signed-off-by: Gavin Shan <gshan@redhat.com>
> Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
> ---
> hw/arm/virt.c | 23 +++++++++++++++--------
> include/hw/arm/virt.h | 1 +
> 2 files changed, 16 insertions(+), 8 deletions(-)
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [PATCH v5 5/6] hw/arm/virt: Improve high memory region address assignment
2022-10-11 23:18 ` [PATCH v5 5/6] hw/arm/virt: Improve high memory region address assignment Gavin Shan
2022-10-19 13:54 ` Cornelia Huck
@ 2022-10-19 20:07 ` Eric Auger
2022-10-19 22:58 ` Gavin Shan
1 sibling, 1 reply; 18+ messages in thread
From: Eric Auger @ 2022-10-19 20:07 UTC (permalink / raw)
To: Gavin Shan, qemu-arm
Cc: qemu-devel, maz, cohuck, zhenyzha, richard.henderson,
peter.maydell, shan.gavin
Hi Gavin
On 10/12/22 01:18, Gavin Shan wrote:
> There are three high memory regions, which are VIRT_HIGH_REDIST2,
> VIRT_HIGH_PCIE_ECAM and VIRT_HIGH_PCIE_MMIO. Their base addresses
> are floating on highest RAM address. However, they can be disabled
> in several cases.
>
> (1) One specific high memory region is disabled by developer by
> toggling vms->highmem_{redists, ecam, mmio}.
I would replace the above sentence by
One specific high memory region is likely to be disabled by the code by toggling vms->highmem_{redists, ecam, mmio}:
>
> (2) VIRT_HIGH_PCIE_ECAM region is disabled on machine, which is
> 'virt-2.12' or ealier than it.
>
> (3) VIRT_HIGH_PCIE_ECAM region is disabled when firmware is loaded
> on 32-bits system.
>
> (4) One specific high memory region is disabled when it breaks the
> PA space limit.
>
> The current implementation of virt_set_memmap() isn't comprehensive
> because the space for one specific high memory region is always
> reserved from the PA space for case (1), (2) and (3).
I would suggest:
isn't optimized because the high memory region PA range is always
reserved whatever the actual state of the corresponding vms->highmem_
* flag.
> In the code,
> 'base' and 'vms->highest_gpa' are always increased for those three
> cases. It's unnecessary since the assigned space of the disabled
> high memory region won't be used afterwards.
>
> This improves the address assignment for those three high memory
s/This improves/Improve
> region by skipping the address assignment for one specific high
> memory region if it has been disabled in case (1), (2) and (3).
> 'vms->high_compact' is false for now, meaning that we don't have
s/hight_compat/highmem_compact
You also may justify the introduction of this new field.
> any behavior changes until it becomes configurable through property
> 'compact-highmem' in next patch.
>
> Signed-off-by: Gavin Shan <gshan@redhat.com>
> Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
> ---
> hw/arm/virt.c | 23 +++++++++++++++--------
> include/hw/arm/virt.h | 1 +
> 2 files changed, 16 insertions(+), 8 deletions(-)
>
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index ee98a8a3b6..c05cfb5314 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -1717,22 +1717,29 @@ static void virt_set_high_memmap(VirtMachineState *vms,
> region_base = ROUND_UP(base, extended_memmap[i].size);
> region_size = extended_memmap[i].size;
>
> - vms->memmap[i].base = region_base;
> - vms->memmap[i].size = region_size;
> -
> /*
> * Check each device to see if they fit in the PA space,
while we are at it, you can change s/they fit/it fits
> - * moving highest_gpa as we go.
> + * moving highest_gpa as we go. For compatibility, move
> + * highest_gpa for disabled fitting devices as well, if
> + * the compact layout has been disabled.
> *
> * For each device that doesn't fit, disable it.
> */
> fits = (region_base + region_size) <= BIT_ULL(pa_bits);
> - if (fits) {
> + if (*region_enabled && fits) {
> + vms->memmap[i].base = region_base;
> + vms->memmap[i].size = region_size;
> vms->highest_gpa = region_base + region_size - 1;
> + base = region_base + region_size;
> + } else {
> + *region_enabled = false;
> + if (!vms->highmem_compact) {
> + base = region_base + region_size;
> + if (fits) {
> + vms->highest_gpa = region_base + region_size - 1;
> + }
> + }
> }
> -
> - *region_enabled &= fits;
> - base = region_base + region_size;
> }
> }
This looks quite complicated to me. It is not obvious for instance we
have the same code as before when highmem_compact is not set. Typically
vms->memmap[i].base/size are not always set as they were to be and impact on the rest of the code must be double checked.
Could this be rewritten in that way (pseudocode totally untested).
static void fit_highmem_slot(vms, *base, i, pa_bits)
{
region_enabled = virt_get_high_memmap_enabled(vms, i);
region_base = ROUND_UP(*base, extended_memmap[i].size);
region_size = extended_memmap[i].size;
fits = (region_base + region_size) <= BIT_ULL(pa_bits);
*region_enabled &= fits;
vms->memmap[i].base = region_base;
vms->memmap[i].size = region_size;
/* compact layout only allocates space for the region if this latter
is enabled & fits*/
if (vms->highmem_compact && !region_enabled) {
return;
}
/* account for the region and update the base address/highest_gpa if
needed*/
*base = region_base + region_size;
if (fits) {
vms->highest_gpa = *base - 1;
}
}
static void virt_set_high_memmap(VirtMachineState *vms,
hwaddr base, int pa_bits)
{
hwaddr region_base, region_size;
bool *region_enabled, fits;
int i;
for (i = VIRT_LOWMEMMAP_LAST; i < ARRAY_SIZE(extended_memmap); i++) {
/* we do not break in case the region does not fit since
fit_highmem_slot also updates the enabled status of the region */
fit_highmem_slot(vms, &base, i, pa_bits);
}
}
>
> diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
> index 6ec479ca2b..709f623741 100644
> --- a/include/hw/arm/virt.h
> +++ b/include/hw/arm/virt.h
> @@ -144,6 +144,7 @@ struct VirtMachineState {
> PFlashCFI01 *flash[2];
> bool secure;
> bool highmem;
> + bool highmem_compact;
> bool highmem_ecam;
> bool highmem_mmio;
> bool highmem_redists;
Thanks
Eric
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [PATCH v5 5/6] hw/arm/virt: Improve high memory region address assignment
2022-10-19 20:07 ` Eric Auger
@ 2022-10-19 22:58 ` Gavin Shan
0 siblings, 0 replies; 18+ messages in thread
From: Gavin Shan @ 2022-10-19 22:58 UTC (permalink / raw)
To: eric.auger, qemu-arm
Cc: qemu-devel, maz, cohuck, zhenyzha, richard.henderson,
peter.maydell, shan.gavin
Hi Eric,
On 10/20/22 4:07 AM, Eric Auger wrote:
> On 10/12/22 01:18, Gavin Shan wrote:
>> There are three high memory regions, which are VIRT_HIGH_REDIST2,
>> VIRT_HIGH_PCIE_ECAM and VIRT_HIGH_PCIE_MMIO. Their base addresses
>> are floating on highest RAM address. However, they can be disabled
>> in several cases.
>>
>> (1) One specific high memory region is disabled by developer by
>> toggling vms->highmem_{redists, ecam, mmio}.
> I would replace the above sentence by
>
> One specific high memory region is likely to be disabled by the code by toggling vms->highmem_{redists, ecam, mmio}:
>
Ok.
>>
>> (2) VIRT_HIGH_PCIE_ECAM region is disabled on machine, which is
>> 'virt-2.12' or ealier than it.
>>
>> (3) VIRT_HIGH_PCIE_ECAM region is disabled when firmware is loaded
>> on 32-bits system.
>>
>> (4) One specific high memory region is disabled when it breaks the
>> PA space limit.
>>
>> The current implementation of virt_set_memmap() isn't comprehensive
>> because the space for one specific high memory region is always
>> reserved from the PA space for case (1), (2) and (3).
> I would suggest:
> isn't optimized because the high memory region PA range is always
>
> reserved whatever the actual state of the corresponding vms->highmem_
> * flag.
>
Ok. I will have something like below in next revision.
The current implementation of virt_set_{memmap, high_memmap}() isn't
optimized because the high memory region's PA space is always reserved,
regardless of whatever the actual state in the corresponding
vms->highmem_{redists, ecam, mmio} flag. In the code, ....
>> In the code,
>> 'base' and 'vms->highest_gpa' are always increased for those three
>> cases. It's unnecessary since the assigned space of the disabled
>> high memory region won't be used afterwards.
>>
>> This improves the address assignment for those three high memory
> s/This improves/Improve
Ok.
>> region by skipping the address assignment for one specific high
>> memory region if it has been disabled in case (1), (2) and (3).
>> 'vms->high_compact' is false for now, meaning that we don't have
> s/hight_compat/highmem_compact
>
> You also may justify the introduction of this new field.
Thanks. It should be 'highmem_compact'. Yes, it makes sense to
justify the addition of 'vms->highmem_compact'. I will have something
like below in next revision.
The memory layout may be changed after the improvement is applied, which
leads to potential migration breakage. So 'vms->highmem_compact' is added
to control if the improvement should be applied. For now, 'vms->highmem_compact'
is set to false, meaning that we don't have memory layout change until it
becomes configurable through property 'compact-highmem' in next patch.
>> any behavior changes until it becomes configurable through property
>> 'compact-highmem' in next patch.
>>>> Signed-off-by: Gavin Shan <gshan@redhat.com>
>> Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
>> ---
>> hw/arm/virt.c | 23 +++++++++++++++--------
>> include/hw/arm/virt.h | 1 +
>> 2 files changed, 16 insertions(+), 8 deletions(-)
>>
>> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
>> index ee98a8a3b6..c05cfb5314 100644
>> --- a/hw/arm/virt.c
>> +++ b/hw/arm/virt.c
>> @@ -1717,22 +1717,29 @@ static void virt_set_high_memmap(VirtMachineState *vms,
>> region_base = ROUND_UP(base, extended_memmap[i].size);
>> region_size = extended_memmap[i].size;
>>
>> - vms->memmap[i].base = region_base;
>> - vms->memmap[i].size = region_size;
>> -
>> /*
>> * Check each device to see if they fit in the PA space,
> while we are at it, you can change s/they fit/it fits
Agreed.
>> - * moving highest_gpa as we go.
>> + * moving highest_gpa as we go. For compatibility, move
>> + * highest_gpa for disabled fitting devices as well, if
>> + * the compact layout has been disabled.
>> *
>> * For each device that doesn't fit, disable it.
>> */
>> fits = (region_base + region_size) <= BIT_ULL(pa_bits);
>> - if (fits) {
>> + if (*region_enabled && fits) {
>> + vms->memmap[i].base = region_base;
>> + vms->memmap[i].size = region_size;
>> vms->highest_gpa = region_base + region_size - 1;
>> + base = region_base + region_size;
>> + } else {
>> + *region_enabled = false;
>> + if (!vms->highmem_compact) {
>> + base = region_base + region_size;
>> + if (fits) {
>> + vms->highest_gpa = region_base + region_size - 1;
>> + }
>> + }
>> }
>> -
>> - *region_enabled &= fits;
>> - base = region_base + region_size;
>> }
>> }
> This looks quite complicated to me. It is not obvious for instance we
> have the same code as before when highmem_compact is not set. Typically
>
> vms->memmap[i].base/size are not always set as they were to be and impact on the rest of the code must be double checked.
>
> Could this be rewritten in that way (pseudocode totally untested).
>
>
> static void fit_highmem_slot(vms, *base, i, pa_bits)
> {
> region_enabled = virt_get_high_memmap_enabled(vms, i);
> region_base = ROUND_UP(*base, extended_memmap[i].size);
> region_size = extended_memmap[i].size;
> fits = (region_base + region_size) <= BIT_ULL(pa_bits);
> *region_enabled &= fits;
> vms->memmap[i].base = region_base;
> vms->memmap[i].size = region_size;
>
> /* compact layout only allocates space for the region if this latter
> is enabled & fits*/
> if (vms->highmem_compact && !region_enabled) {
> return;
> }
>
> /* account for the region and update the base address/highest_gpa if
> needed*/
> *base = region_base + region_size;
> if (fits) {
> vms->highest_gpa = *base - 1;
> }
> }
>
> static void virt_set_high_memmap(VirtMachineState *vms,
> hwaddr base, int pa_bits)
> {
> hwaddr region_base, region_size;
> bool *region_enabled, fits;
> int i;
>
> for (i = VIRT_LOWMEMMAP_LAST; i < ARRAY_SIZE(extended_memmap); i++) {
> /* we do not break in case the region does not fit since
> fit_highmem_slot also updates the enabled status of the region */
> fit_highmem_slot(vms, &base, i, pa_bits);
> }
> }
>
>>
>> diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
>> index 6ec479ca2b..709f623741 100644
>> --- a/include/hw/arm/virt.h
>> +++ b/include/hw/arm/virt.h
>> @@ -144,6 +144,7 @@ struct VirtMachineState {
>> PFlashCFI01 *flash[2];
>> bool secure;
>> bool highmem;
>> + bool highmem_compact;
>> bool highmem_ecam;
>> bool highmem_mmio;
>> bool highmem_redists;
I checked it again. When the corresponding high memory region is disabled, skipping to
setting vms->memmap[i].base/size won't affect the left code.
I would avoid introducing another helper here, meaning I want to squeeze everything
into existing virt_set_high_memmap(). With additional helper, there are too many
function calls in the stack to accomplish this sort of simple task (address assignment
for high memory regions). Without the additional helper, the changes can be simply as
below.
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index ee98a8a3b6..da9e23a8ad 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1721,18 +1721,23 @@ static void virt_set_high_memmap(VirtMachineState *vms,
vms->memmap[i].size = region_size;
/*
- * Check each device to see if they fit in the PA space,
- * moving highest_gpa as we go.
+ * Check each device to see if it fits in the PA space,
+ * moving highest_gpa as we go. For compatibility, move
+ * highest_gpa for disabled fitting devices as well, if
+ * the compact layout has been disabled.
*
* For each device that doesn't fit, disable it.
*/
fits = (region_base + region_size) <= BIT_ULL(pa_bits);
- if (fits) {
- vms->highest_gpa = region_base + region_size - 1;
+ *region_enabled &= fits;
+ if (vms->highmem_compact && !*region_enabled) {
+ continue;
}
- *region_enabled &= fits;
base = region_base + region_size;
+ if (fits) {
+ vms->highest_gpa = region_base + region_size - 1;
+ }
}
}
Thanks,
Gavin
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v5 6/6] hw/arm/virt: Add 'compact-highmem' property
2022-10-11 23:18 [PATCH v5 0/6] hw/arm/virt: Improve address assignment for high memory regions Gavin Shan
` (4 preceding siblings ...)
2022-10-11 23:18 ` [PATCH v5 5/6] hw/arm/virt: Improve high memory region address assignment Gavin Shan
@ 2022-10-11 23:18 ` Gavin Shan
2022-10-19 14:00 ` Cornelia Huck
2022-10-19 20:18 ` Eric Auger
5 siblings, 2 replies; 18+ messages in thread
From: Gavin Shan @ 2022-10-11 23:18 UTC (permalink / raw)
To: qemu-arm
Cc: qemu-devel, maz, eric.auger, cohuck, zhenyzha, richard.henderson,
peter.maydell, shan.gavin
After the improvement to high memory region address assignment is
applied, the memory layout can be changed, introducing possible
migration breakage. For example, VIRT_HIGH_PCIE_MMIO memory region
is disabled or enabled when the optimization is applied or not, with
the following configuration.
pa_bits = 40;
vms->highmem_redists = false;
vms->highmem_ecam = false;
vms->highmem_mmio = true;
# qemu-system-aarch64 -accel kvm -cpu host \
-machine virt-7.2,compact-highmem={on, off} \
-m 4G,maxmem=511G -monitor stdio
Region compact-highmem=off compact-highmem=on
----------------------------------------------------------------
RAM [1GB 512GB] [1GB 512GB]
HIGH_GIC_REDISTS [512GB 512GB+64MB] [disabled]
HIGH_PCIE_ECAM [512GB+256MB 512GB+512MB] [disabled]
HIGH_PCIE_MMIO [disabled] [512GB 1TB]
In order to keep backwords compatibility, we need to disable the
optimization on machines, which is virt-7.1 or ealier than it. It
means the optimization is enabled by default from virt-7.2. Besides,
'compact-highmem' property is added so that the optimization can be
explicitly enabled or disabled on all machine types by users.
Signed-off-by: Gavin Shan <gshan@redhat.com>
Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
---
docs/system/arm/virt.rst | 4 ++++
hw/arm/virt.c | 47 ++++++++++++++++++++++++++++++++++++++++
include/hw/arm/virt.h | 1 +
3 files changed, 52 insertions(+)
diff --git a/docs/system/arm/virt.rst b/docs/system/arm/virt.rst
index 20442ea2c1..75bf5a4994 100644
--- a/docs/system/arm/virt.rst
+++ b/docs/system/arm/virt.rst
@@ -94,6 +94,10 @@ highmem
address space above 32 bits. The default is ``on`` for machine types
later than ``virt-2.12``.
+compact-highmem
+ Set ``on``/``off`` to enable/disable compact space for high memory regions.
+ The default is ``on`` for machine types later than ``virt-7.2``
+
gic-version
Specify the version of the Generic Interrupt Controller (GIC) to provide.
Valid values are:
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index c05cfb5314..8f1dba0ece 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -174,6 +174,27 @@ static const MemMapEntry base_memmap[] = {
* Note the extended_memmap is sized so that it eventually also includes the
* base_memmap entries (VIRT_HIGH_GIC_REDIST2 index is greater than the last
* index of base_memmap).
+ *
+ * The addresses assigned to these regions are affected by 'compact-highmem'
+ * property, which is to enable or disable the compact space in the Highmem
+ * IO regions. For example, VIRT_HIGH_PCIE_MMIO can be disabled or enabled
+ * depending on the property in the following scenario.
+ *
+ * pa_bits = 40;
+ * vms->highmem_redists = false;
+ * vms->highmem_ecam = false;
+ * vms->highmem_mmio = true;
+ *
+ * # qemu-system-aarch64 -accel kvm -cpu host \
+ * -machine virt-7.2,compact-highmem={on, off} \
+ * -m 4G,maxmem=511G -monitor stdio
+ *
+ * Region compact-highmem=off compact-highmem=on
+ * ----------------------------------------------------------------
+ * RAM [1GB 512GB] [1GB 512GB]
+ * HIGH_GIC_REDISTS [512GB 512GB+64MB] [disabled]
+ * HIGH_PCIE_ECAM [512GB+256GB 512GB+512MB] [disabled]
+ * HIGH_PCIE_MMIO [disabled] [512GB 1TB]
*/
static MemMapEntry extended_memmap[] = {
/* Additional 64 MB redist region (can contain up to 512 redistributors) */
@@ -2353,6 +2374,20 @@ static void virt_set_highmem(Object *obj, bool value, Error **errp)
vms->highmem = value;
}
+static bool virt_get_compact_highmem(Object *obj, Error **errp)
+{
+ VirtMachineState *vms = VIRT_MACHINE(obj);
+
+ return vms->highmem_compact;
+}
+
+static void virt_set_compact_highmem(Object *obj, bool value, Error **errp)
+{
+ VirtMachineState *vms = VIRT_MACHINE(obj);
+
+ vms->highmem_compact = value;
+}
+
static bool virt_get_its(Object *obj, Error **errp)
{
VirtMachineState *vms = VIRT_MACHINE(obj);
@@ -2971,6 +3006,13 @@ static void virt_machine_class_init(ObjectClass *oc, void *data)
"Set on/off to enable/disable using "
"physical address space above 32 bits");
+ object_class_property_add_bool(oc, "compact-highmem",
+ virt_get_compact_highmem,
+ virt_set_compact_highmem);
+ object_class_property_set_description(oc, "compact-highmem",
+ "Set on/off to enable/disable compact "
+ "space for high memory regions");
+
object_class_property_add_str(oc, "gic-version", virt_get_gic_version,
virt_set_gic_version);
object_class_property_set_description(oc, "gic-version",
@@ -3055,6 +3097,7 @@ static void virt_instance_init(Object *obj)
/* High memory is enabled by default */
vms->highmem = true;
+ vms->highmem_compact = !vmc->no_highmem_compact;
vms->gic_version = VIRT_GIC_VERSION_NOSEL;
vms->highmem_ecam = !vmc->no_highmem_ecam;
@@ -3124,8 +3167,12 @@ DEFINE_VIRT_MACHINE_AS_LATEST(7, 2)
static void virt_machine_7_1_options(MachineClass *mc)
{
+ VirtMachineClass *vmc = VIRT_MACHINE_CLASS(OBJECT_CLASS(mc));
+
virt_machine_7_2_options(mc);
compat_props_add(mc->compat_props, hw_compat_7_1, hw_compat_7_1_len);
+ /* Compact space for high memory regions was introduced with 7.2 */
+ vmc->no_highmem_compact = true;
}
DEFINE_VIRT_MACHINE(7, 1)
diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
index 709f623741..c7dd59d7f1 100644
--- a/include/hw/arm/virt.h
+++ b/include/hw/arm/virt.h
@@ -125,6 +125,7 @@ struct VirtMachineClass {
bool no_pmu;
bool claim_edge_triggered_timers;
bool smbios_old_sys_ver;
+ bool no_highmem_compact;
bool no_highmem_ecam;
bool no_ged; /* Machines < 4.2 have no support for ACPI GED device */
bool kvm_no_adjvtime;
--
2.23.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* Re: [PATCH v5 6/6] hw/arm/virt: Add 'compact-highmem' property
2022-10-11 23:18 ` [PATCH v5 6/6] hw/arm/virt: Add 'compact-highmem' property Gavin Shan
@ 2022-10-19 14:00 ` Cornelia Huck
2022-10-19 23:08 ` Gavin Shan
2022-10-19 20:18 ` Eric Auger
1 sibling, 1 reply; 18+ messages in thread
From: Cornelia Huck @ 2022-10-19 14:00 UTC (permalink / raw)
To: Gavin Shan, qemu-arm
Cc: qemu-devel, maz, eric.auger, zhenyzha, richard.henderson,
peter.maydell, shan.gavin
On Wed, Oct 12 2022, Gavin Shan <gshan@redhat.com> wrote:
> After the improvement to high memory region address assignment is
> applied, the memory layout can be changed, introducing possible
> migration breakage. For example, VIRT_HIGH_PCIE_MMIO memory region
> is disabled or enabled when the optimization is applied or not, with
> the following configuration.
>
> pa_bits = 40;
> vms->highmem_redists = false;
> vms->highmem_ecam = false;
> vms->highmem_mmio = true;
>
> # qemu-system-aarch64 -accel kvm -cpu host \
> -machine virt-7.2,compact-highmem={on, off} \
> -m 4G,maxmem=511G -monitor stdio
>
> Region compact-highmem=off compact-highmem=on
> ----------------------------------------------------------------
> RAM [1GB 512GB] [1GB 512GB]
> HIGH_GIC_REDISTS [512GB 512GB+64MB] [disabled]
> HIGH_PCIE_ECAM [512GB+256MB 512GB+512MB] [disabled]
> HIGH_PCIE_MMIO [disabled] [512GB 1TB]
>
> In order to keep backwords compatibility, we need to disable the
> optimization on machines, which is virt-7.1 or ealier than it. It
> means the optimization is enabled by default from virt-7.2. Besides,
> 'compact-highmem' property is added so that the optimization can be
> explicitly enabled or disabled on all machine types by users.
>
> Signed-off-by: Gavin Shan <gshan@redhat.com>
> Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
> ---
> docs/system/arm/virt.rst | 4 ++++
> hw/arm/virt.c | 47 ++++++++++++++++++++++++++++++++++++++++
> include/hw/arm/virt.h | 1 +
> 3 files changed, 52 insertions(+)
>
> diff --git a/docs/system/arm/virt.rst b/docs/system/arm/virt.rst
> index 20442ea2c1..75bf5a4994 100644
> --- a/docs/system/arm/virt.rst
> +++ b/docs/system/arm/virt.rst
> @@ -94,6 +94,10 @@ highmem
> address space above 32 bits. The default is ``on`` for machine types
> later than ``virt-2.12``.
>
> +compact-highmem
> + Set ``on``/``off`` to enable/disable compact space for high memory regions.
Maybe s/compact space/the compact layout/ ?
> + The default is ``on`` for machine types later than ``virt-7.2``
> +
> gic-version
> Specify the version of the Generic Interrupt Controller (GIC) to provide.
> Valid values are:
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index c05cfb5314..8f1dba0ece 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -174,6 +174,27 @@ static const MemMapEntry base_memmap[] = {
> * Note the extended_memmap is sized so that it eventually also includes the
> * base_memmap entries (VIRT_HIGH_GIC_REDIST2 index is greater than the last
> * index of base_memmap).
> + *
> + * The addresses assigned to these regions are affected by 'compact-highmem'
> + * property, which is to enable or disable the compact space in the Highmem
s/space in/layout for/ ?
> + * IO regions. For example, VIRT_HIGH_PCIE_MMIO can be disabled or enabled
> + * depending on the property in the following scenario.
> + *
> + * pa_bits = 40;
> + * vms->highmem_redists = false;
> + * vms->highmem_ecam = false;
> + * vms->highmem_mmio = true;
> + *
> + * # qemu-system-aarch64 -accel kvm -cpu host \
> + * -machine virt-7.2,compact-highmem={on, off} \
> + * -m 4G,maxmem=511G -monitor stdio
> + *
> + * Region compact-highmem=off compact-highmem=on
> + * ----------------------------------------------------------------
> + * RAM [1GB 512GB] [1GB 512GB]
> + * HIGH_GIC_REDISTS [512GB 512GB+64MB] [disabled]
> + * HIGH_PCIE_ECAM [512GB+256GB 512GB+512MB] [disabled]
> + * HIGH_PCIE_MMIO [disabled] [512GB 1TB]
> */
> static MemMapEntry extended_memmap[] = {
> /* Additional 64 MB redist region (can contain up to 512 redistributors) */
(...)
> @@ -3124,8 +3167,12 @@ DEFINE_VIRT_MACHINE_AS_LATEST(7, 2)
>
> static void virt_machine_7_1_options(MachineClass *mc)
> {
> + VirtMachineClass *vmc = VIRT_MACHINE_CLASS(OBJECT_CLASS(mc));
> +
> virt_machine_7_2_options(mc);
> compat_props_add(mc->compat_props, hw_compat_7_1, hw_compat_7_1_len);
> + /* Compact space for high memory regions was introduced with 7.2 */
s/space/layout/ ?
> + vmc->no_highmem_compact = true;
> }
> DEFINE_VIRT_MACHINE(7, 1)
>
Other than the wording nits, lgtm.
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [PATCH v5 6/6] hw/arm/virt: Add 'compact-highmem' property
2022-10-19 14:00 ` Cornelia Huck
@ 2022-10-19 23:08 ` Gavin Shan
0 siblings, 0 replies; 18+ messages in thread
From: Gavin Shan @ 2022-10-19 23:08 UTC (permalink / raw)
To: Cornelia Huck, qemu-arm
Cc: qemu-devel, maz, eric.auger, zhenyzha, richard.henderson,
peter.maydell, shan.gavin
Hi Connie,
On 10/19/22 10:00 PM, Cornelia Huck wrote:
> On Wed, Oct 12 2022, Gavin Shan <gshan@redhat.com> wrote:
>
>> After the improvement to high memory region address assignment is
>> applied, the memory layout can be changed, introducing possible
>> migration breakage. For example, VIRT_HIGH_PCIE_MMIO memory region
>> is disabled or enabled when the optimization is applied or not, with
>> the following configuration.
>>
>> pa_bits = 40;
>> vms->highmem_redists = false;
>> vms->highmem_ecam = false;
>> vms->highmem_mmio = true;
>>
>> # qemu-system-aarch64 -accel kvm -cpu host \
>> -machine virt-7.2,compact-highmem={on, off} \
>> -m 4G,maxmem=511G -monitor stdio
>>
>> Region compact-highmem=off compact-highmem=on
>> ----------------------------------------------------------------
>> RAM [1GB 512GB] [1GB 512GB]
>> HIGH_GIC_REDISTS [512GB 512GB+64MB] [disabled]
>> HIGH_PCIE_ECAM [512GB+256MB 512GB+512MB] [disabled]
>> HIGH_PCIE_MMIO [disabled] [512GB 1TB]
>>
>> In order to keep backwords compatibility, we need to disable the
>> optimization on machines, which is virt-7.1 or ealier than it. It
>> means the optimization is enabled by default from virt-7.2. Besides,
>> 'compact-highmem' property is added so that the optimization can be
>> explicitly enabled or disabled on all machine types by users.
>>
>> Signed-off-by: Gavin Shan <gshan@redhat.com>
>> Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
>> ---
>> docs/system/arm/virt.rst | 4 ++++
>> hw/arm/virt.c | 47 ++++++++++++++++++++++++++++++++++++++++
>> include/hw/arm/virt.h | 1 +
>> 3 files changed, 52 insertions(+)
>>
>> diff --git a/docs/system/arm/virt.rst b/docs/system/arm/virt.rst
>> index 20442ea2c1..75bf5a4994 100644
>> --- a/docs/system/arm/virt.rst
>> +++ b/docs/system/arm/virt.rst
>> @@ -94,6 +94,10 @@ highmem
>> address space above 32 bits. The default is ``on`` for machine types
>> later than ``virt-2.12``.
>>
>> +compact-highmem
>> + Set ``on``/``off`` to enable/disable compact space for high memory regions.
>
> Maybe s/compact space/the compact layout/ ?
>
Yeah, 'compact layout' is better. I will amend in next respin.
>> + The default is ``on`` for machine types later than ``virt-7.2``
>> +
>> gic-version
>> Specify the version of the Generic Interrupt Controller (GIC) to provide.
>> Valid values are:
>> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
>> index c05cfb5314..8f1dba0ece 100644
>> --- a/hw/arm/virt.c
>> +++ b/hw/arm/virt.c
>> @@ -174,6 +174,27 @@ static const MemMapEntry base_memmap[] = {
>> * Note the extended_memmap is sized so that it eventually also includes the
>> * base_memmap entries (VIRT_HIGH_GIC_REDIST2 index is greater than the last
>> * index of base_memmap).
>> + *
>> + * The addresses assigned to these regions are affected by 'compact-highmem'
>> + * property, which is to enable or disable the compact space in the Highmem
>
> s/space in/layout for/ ?
>
Agreed.
>> + * IO regions. For example, VIRT_HIGH_PCIE_MMIO can be disabled or enabled
>> + * depending on the property in the following scenario.
>> + *
>> + * pa_bits = 40;
>> + * vms->highmem_redists = false;
>> + * vms->highmem_ecam = false;
>> + * vms->highmem_mmio = true;
>> + *
>> + * # qemu-system-aarch64 -accel kvm -cpu host \
>> + * -machine virt-7.2,compact-highmem={on, off} \
>> + * -m 4G,maxmem=511G -monitor stdio
>> + *
>> + * Region compact-highmem=off compact-highmem=on
>> + * ----------------------------------------------------------------
>> + * RAM [1GB 512GB] [1GB 512GB]
>> + * HIGH_GIC_REDISTS [512GB 512GB+64MB] [disabled]
>> + * HIGH_PCIE_ECAM [512GB+256GB 512GB+512MB] [disabled]
>> + * HIGH_PCIE_MMIO [disabled] [512GB 1TB]
>> */
>> static MemMapEntry extended_memmap[] = {
>> /* Additional 64 MB redist region (can contain up to 512 redistributors) */
>
> (...)
>
>> @@ -3124,8 +3167,12 @@ DEFINE_VIRT_MACHINE_AS_LATEST(7, 2)
>>
>> static void virt_machine_7_1_options(MachineClass *mc)
>> {
>> + VirtMachineClass *vmc = VIRT_MACHINE_CLASS(OBJECT_CLASS(mc));
>> +
>> virt_machine_7_2_options(mc);
>> compat_props_add(mc->compat_props, hw_compat_7_1, hw_compat_7_1_len);
>> + /* Compact space for high memory regions was introduced with 7.2 */
>
> s/space/layout/ ?
>
Ack.
>> + vmc->no_highmem_compact = true;
>> }
>> DEFINE_VIRT_MACHINE(7, 1)
>>
>
> Other than the wording nits, lgtm.
>
> Reviewed-by: Cornelia Huck <cohuck@redhat.com>
>
Thanks,
Gavin
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v5 6/6] hw/arm/virt: Add 'compact-highmem' property
2022-10-11 23:18 ` [PATCH v5 6/6] hw/arm/virt: Add 'compact-highmem' property Gavin Shan
2022-10-19 14:00 ` Cornelia Huck
@ 2022-10-19 20:18 ` Eric Auger
2022-10-19 23:57 ` Gavin Shan
1 sibling, 1 reply; 18+ messages in thread
From: Eric Auger @ 2022-10-19 20:18 UTC (permalink / raw)
To: Gavin Shan, qemu-arm
Cc: qemu-devel, maz, cohuck, zhenyzha, richard.henderson,
peter.maydell, shan.gavin
Hi Gavin,
On 10/12/22 01:18, Gavin Shan wrote:
> After the improvement to high memory region address assignment is
> applied, the memory layout can be changed, introducing possible
> migration breakage. For example, VIRT_HIGH_PCIE_MMIO memory region
> is disabled or enabled when the optimization is applied or not, with
> the following configuration.
>
> pa_bits = 40;
> vms->highmem_redists = false;
> vms->highmem_ecam = false;
> vms->highmem_mmio = true;
>
> # qemu-system-aarch64 -accel kvm -cpu host \
> -machine virt-7.2,compact-highmem={on, off} \
> -m 4G,maxmem=511G -monitor stdio
>
> Region compact-highmem=off compact-highmem=on
> ----------------------------------------------------------------
> RAM [1GB 512GB] [1GB 512GB]
> HIGH_GIC_REDISTS [512GB 512GB+64MB] [disabled]
> HIGH_PCIE_ECAM [512GB+256MB 512GB+512MB] [disabled]
> HIGH_PCIE_MMIO [disabled] [512GB 1TB]
>
> In order to keep backwords compatibility, we need to disable the
> optimization on machines, which is virt-7.1 or ealier than it. It
> means the optimization is enabled by default from virt-7.2. Besides,
> 'compact-highmem' property is added so that the optimization can be
> explicitly enabled or disabled on all machine types by users.
>
> Signed-off-by: Gavin Shan <gshan@redhat.com>
> Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
> ---
> docs/system/arm/virt.rst | 4 ++++
> hw/arm/virt.c | 47 ++++++++++++++++++++++++++++++++++++++++
> include/hw/arm/virt.h | 1 +
> 3 files changed, 52 insertions(+)
>
> diff --git a/docs/system/arm/virt.rst b/docs/system/arm/virt.rst
> index 20442ea2c1..75bf5a4994 100644
> --- a/docs/system/arm/virt.rst
> +++ b/docs/system/arm/virt.rst
> @@ -94,6 +94,10 @@ highmem
> address space above 32 bits. The default is ``on`` for machine types
> later than ``virt-2.12``.
>
> +compact-highmem
> + Set ``on``/``off`` to enable/disable compact space for high memory regions.
> + The default is ``on`` for machine types later than ``virt-7.2``
> +
> gic-version
> Specify the version of the Generic Interrupt Controller (GIC) to provide.
> Valid values are:
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index c05cfb5314..8f1dba0ece 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -174,6 +174,27 @@ static const MemMapEntry base_memmap[] = {
> * Note the extended_memmap is sized so that it eventually also includes the
> * base_memmap entries (VIRT_HIGH_GIC_REDIST2 index is greater than the last
> * index of base_memmap).
> + *
> + * The addresses assigned to these regions are affected by 'compact-highmem'
> + * property, which is to enable or disable the compact space in the Highmem
> + * IO regions. For example, VIRT_HIGH_PCIE_MMIO can be disabled or enabled
> + * depending on the property in the following scenario.
To me you shall rather explain here what is the so-called "compact"
space vs the legacy highmem layout.
If I understand correctly the example rather legitimates the use of a
compat option showing how the layout can be affected by the option. I
would put that in the commit msg instead. Also in your example I see
VIRT_HIGH_GIC_REDISTS is disabled but the code does not disable the
region excpet if it does not fit within the PA. This does not match your
example. Also the region is named VIRT_HIGH_GIC_REDIST2.
In v4, Marc also suggested to have individual options for each highmem
region.
https://lore.kernel.org/qemu-devel/0f8e6a58-0dde-fb80-6966-7bb32c4df552@redhat.com/
Have you considered that option?
Thanks
Eric
> + *
> + * pa_bits = 40;
> + * vms->highmem_redists = false;
> + * vms->highmem_ecam = false;
> + * vms->highmem_mmio = true;
> + *
> + * # qemu-system-aarch64 -accel kvm -cpu host \
> + * -machine virt-7.2,compact-highmem={on, off} \
> + * -m 4G,maxmem=511G -monitor stdio
> + *
> + * Region compact-highmem=off compact-highmem=on
> + * ----------------------------------------------------------------
> + * RAM [1GB 512GB] [1GB 512GB]
> + * HIGH_GIC_REDISTS [512GB 512GB+64MB] [disabled]
> + * HIGH_PCIE_ECAM [512GB+256GB 512GB+512MB] [disabled]
> + * HIGH_PCIE_MMIO [disabled] [512GB 1TB]
> */
> static MemMapEntry extended_memmap[] = {
> /* Additional 64 MB redist region (can contain up to 512 redistributors) */
> @@ -2353,6 +2374,20 @@ static void virt_set_highmem(Object *obj, bool value, Error **errp)
> vms->highmem = value;
> }
>
> +static bool virt_get_compact_highmem(Object *obj, Error **errp)
> +{
> + VirtMachineState *vms = VIRT_MACHINE(obj);
> +
> + return vms->highmem_compact;
> +}
> +
> +static void virt_set_compact_highmem(Object *obj, bool value, Error **errp)
> +{
> + VirtMachineState *vms = VIRT_MACHINE(obj);
> +
> + vms->highmem_compact = value;
> +}
> +
> static bool virt_get_its(Object *obj, Error **errp)
> {
> VirtMachineState *vms = VIRT_MACHINE(obj);
> @@ -2971,6 +3006,13 @@ static void virt_machine_class_init(ObjectClass *oc, void *data)
> "Set on/off to enable/disable using "
> "physical address space above 32 bits");
>
> + object_class_property_add_bool(oc, "compact-highmem",
> + virt_get_compact_highmem,
> + virt_set_compact_highmem);
> + object_class_property_set_description(oc, "compact-highmem",
> + "Set on/off to enable/disable compact "
> + "space for high memory regions");
> +
> object_class_property_add_str(oc, "gic-version", virt_get_gic_version,
> virt_set_gic_version);
> object_class_property_set_description(oc, "gic-version",
> @@ -3055,6 +3097,7 @@ static void virt_instance_init(Object *obj)
>
> /* High memory is enabled by default */
> vms->highmem = true;
> + vms->highmem_compact = !vmc->no_highmem_compact;
> vms->gic_version = VIRT_GIC_VERSION_NOSEL;
>
> vms->highmem_ecam = !vmc->no_highmem_ecam;
> @@ -3124,8 +3167,12 @@ DEFINE_VIRT_MACHINE_AS_LATEST(7, 2)
>
> static void virt_machine_7_1_options(MachineClass *mc)
> {
> + VirtMachineClass *vmc = VIRT_MACHINE_CLASS(OBJECT_CLASS(mc));
> +
> virt_machine_7_2_options(mc);
> compat_props_add(mc->compat_props, hw_compat_7_1, hw_compat_7_1_len);
> + /* Compact space for high memory regions was introduced with 7.2 */
> + vmc->no_highmem_compact = true;
> }
> DEFINE_VIRT_MACHINE(7, 1)
>
> diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
> index 709f623741..c7dd59d7f1 100644
> --- a/include/hw/arm/virt.h
> +++ b/include/hw/arm/virt.h
> @@ -125,6 +125,7 @@ struct VirtMachineClass {
> bool no_pmu;
> bool claim_edge_triggered_timers;
> bool smbios_old_sys_ver;
> + bool no_highmem_compact;
> bool no_highmem_ecam;
> bool no_ged; /* Machines < 4.2 have no support for ACPI GED device */
> bool kvm_no_adjvtime;
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [PATCH v5 6/6] hw/arm/virt: Add 'compact-highmem' property
2022-10-19 20:18 ` Eric Auger
@ 2022-10-19 23:57 ` Gavin Shan
2022-10-20 9:44 ` Marc Zyngier
0 siblings, 1 reply; 18+ messages in thread
From: Gavin Shan @ 2022-10-19 23:57 UTC (permalink / raw)
To: eric.auger, qemu-arm
Cc: qemu-devel, maz, cohuck, zhenyzha, richard.henderson,
peter.maydell, shan.gavin
Hi Eric,
On 10/20/22 4:18 AM, Eric Auger wrote:
> On 10/12/22 01:18, Gavin Shan wrote:
>> After the improvement to high memory region address assignment is
>> applied, the memory layout can be changed, introducing possible
>> migration breakage. For example, VIRT_HIGH_PCIE_MMIO memory region
>> is disabled or enabled when the optimization is applied or not, with
>> the following configuration.
>>
>> pa_bits = 40;
>> vms->highmem_redists = false;
>> vms->highmem_ecam = false;
>> vms->highmem_mmio = true;
>>
>> # qemu-system-aarch64 -accel kvm -cpu host \
>> -machine virt-7.2,compact-highmem={on, off} \
>> -m 4G,maxmem=511G -monitor stdio
>>
>> Region compact-highmem=off compact-highmem=on
>> ----------------------------------------------------------------
>> RAM [1GB 512GB] [1GB 512GB]
>> HIGH_GIC_REDISTS [512GB 512GB+64MB] [disabled]
>> HIGH_PCIE_ECAM [512GB+256MB 512GB+512MB] [disabled]
>> HIGH_PCIE_MMIO [disabled] [512GB 1TB]
>>
>> In order to keep backwords compatibility, we need to disable the
>> optimization on machines, which is virt-7.1 or ealier than it. It
>> means the optimization is enabled by default from virt-7.2. Besides,
>> 'compact-highmem' property is added so that the optimization can be
>> explicitly enabled or disabled on all machine types by users.
>>
>> Signed-off-by: Gavin Shan <gshan@redhat.com>
>> Tested-by: Zhenyu Zhang <zhenyzha@redhat.com>
>> ---
>> docs/system/arm/virt.rst | 4 ++++
>> hw/arm/virt.c | 47 ++++++++++++++++++++++++++++++++++++++++
>> include/hw/arm/virt.h | 1 +
>> 3 files changed, 52 insertions(+)
>>
>> diff --git a/docs/system/arm/virt.rst b/docs/system/arm/virt.rst
>> index 20442ea2c1..75bf5a4994 100644
>> --- a/docs/system/arm/virt.rst
>> +++ b/docs/system/arm/virt.rst
>> @@ -94,6 +94,10 @@ highmem
>> address space above 32 bits. The default is ``on`` for machine types
>> later than ``virt-2.12``.
>>
>> +compact-highmem
>> + Set ``on``/``off`` to enable/disable compact space for high memory regions.
>> + The default is ``on`` for machine types later than ``virt-7.2``
>> +
>> gic-version
>> Specify the version of the Generic Interrupt Controller (GIC) to provide.
>> Valid values are:
>> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
>> index c05cfb5314..8f1dba0ece 100644
>> --- a/hw/arm/virt.c
>> +++ b/hw/arm/virt.c
>> @@ -174,6 +174,27 @@ static const MemMapEntry base_memmap[] = {
>> * Note the extended_memmap is sized so that it eventually also includes the
>> * base_memmap entries (VIRT_HIGH_GIC_REDIST2 index is greater than the last
>> * index of base_memmap).
>> + *
>> + * The addresses assigned to these regions are affected by 'compact-highmem'
>> + * property, which is to enable or disable the compact space in the Highmem
>> + * IO regions. For example, VIRT_HIGH_PCIE_MMIO can be disabled or enabled
>> + * depending on the property in the following scenario.
> To me you shall rather explain here what is the so-called "compact"
> space vs the legacy highmem layout.
>
> If I understand correctly the example rather legitimates the use of a
> compat option showing how the layout can be affected by the option. I
> would put that in the commit msg instead. Also in your example I see
> VIRT_HIGH_GIC_REDISTS is disabled but the code does not disable the
> region excpet if it does not fit within the PA. This does not match your
> example. Also the region is named VIRT_HIGH_GIC_REDIST2.
>
> In v4, Marc also suggested to have individual options for each highmem
> region.
> https://lore.kernel.org/qemu-devel/0f8e6a58-0dde-fb80-6966-7bb32c4df552@redhat.com/
>
> Have you considered that option?
>
I think your comments make sense to me. So lets put the following comments
to the code and move the example to commit log.
/*
* The memory map for these Highmem IO Regions can be in legacy or compact
* layout, depending on 'compact-highmem' property. In legacy layout, the
* PA space for one specific region is always reserved, even the region has
* been disabled or doesn't fit into the PA space. However, the PA space for
* the region won't be reserved in these circumstances.
*/
You're correct about the example. VIRT_HIGH_GIC_REDIST2 should be used. Besides,
the configuration is only achievable by modifying source code at present, until
Marc's suggestion rolls in to allow users disable one particular high memory
regions by more properties. I will amend the commit log to have something like
below.
For example, VIRT_HIGH_PCIE_MMIO memory region is disabled or enabled when
the optimization is applied or not, with the following configuration. The
configuration is only achievable by modifying source code, until more properties
are added to allow user selectively disable those high memory regions.
For Marc's suggestion to add properties so that these high memory regions can
be disabled by users. I can add one patch after this one to introduce the following
3 properties. Could you please confirm the property names are good enough? It's
nice if Marc can help to confirm before I'm going to work on next revision.
"highmem-ecam": "on"/"off" on vms->highmem_ecam
"highmem-mmio": "on"/"off" on vms->highmem_mmio
"highmem-redists": "on"/"off" on vms->highmem_redists
>> + *
>> + * pa_bits = 40;
>> + * vms->highmem_redists = false;
>> + * vms->highmem_ecam = false;
>> + * vms->highmem_mmio = true;
>> + *
>> + * # qemu-system-aarch64 -accel kvm -cpu host \
>> + * -machine virt-7.2,compact-highmem={on, off} \
>> + * -m 4G,maxmem=511G -monitor stdio
>> + *
>> + * Region compact-highmem=off compact-highmem=on
>> + * ----------------------------------------------------------------
>> + * RAM [1GB 512GB] [1GB 512GB]
>> + * HIGH_GIC_REDISTS [512GB 512GB+64MB] [disabled]
>> + * HIGH_PCIE_ECAM [512GB+256GB 512GB+512MB] [disabled]
>> + * HIGH_PCIE_MMIO [disabled] [512GB 1TB]
>> */
>> static MemMapEntry extended_memmap[] = {
>> /* Additional 64 MB redist region (can contain up to 512 redistributors) */
>> @@ -2353,6 +2374,20 @@ static void virt_set_highmem(Object *obj, bool value, Error **errp)
>> vms->highmem = value;
>> }
>>
>> +static bool virt_get_compact_highmem(Object *obj, Error **errp)
>> +{
>> + VirtMachineState *vms = VIRT_MACHINE(obj);
>> +
>> + return vms->highmem_compact;
>> +}
>> +
>> +static void virt_set_compact_highmem(Object *obj, bool value, Error **errp)
>> +{
>> + VirtMachineState *vms = VIRT_MACHINE(obj);
>> +
>> + vms->highmem_compact = value;
>> +}
>> +
>> static bool virt_get_its(Object *obj, Error **errp)
>> {
>> VirtMachineState *vms = VIRT_MACHINE(obj);
>> @@ -2971,6 +3006,13 @@ static void virt_machine_class_init(ObjectClass *oc, void *data)
>> "Set on/off to enable/disable using "
>> "physical address space above 32 bits");
>>
>> + object_class_property_add_bool(oc, "compact-highmem",
>> + virt_get_compact_highmem,
>> + virt_set_compact_highmem);
>> + object_class_property_set_description(oc, "compact-highmem",
>> + "Set on/off to enable/disable compact "
>> + "space for high memory regions");
>> +
>> object_class_property_add_str(oc, "gic-version", virt_get_gic_version,
>> virt_set_gic_version);
>> object_class_property_set_description(oc, "gic-version",
>> @@ -3055,6 +3097,7 @@ static void virt_instance_init(Object *obj)
>>
>> /* High memory is enabled by default */
>> vms->highmem = true;
>> + vms->highmem_compact = !vmc->no_highmem_compact;
>> vms->gic_version = VIRT_GIC_VERSION_NOSEL;
>>
>> vms->highmem_ecam = !vmc->no_highmem_ecam;
>> @@ -3124,8 +3167,12 @@ DEFINE_VIRT_MACHINE_AS_LATEST(7, 2)
>>
>> static void virt_machine_7_1_options(MachineClass *mc)
>> {
>> + VirtMachineClass *vmc = VIRT_MACHINE_CLASS(OBJECT_CLASS(mc));
>> +
>> virt_machine_7_2_options(mc);
>> compat_props_add(mc->compat_props, hw_compat_7_1, hw_compat_7_1_len);
>> + /* Compact space for high memory regions was introduced with 7.2 */
>> + vmc->no_highmem_compact = true;
>> }
>> DEFINE_VIRT_MACHINE(7, 1)
>>
>> diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
>> index 709f623741..c7dd59d7f1 100644
>> --- a/include/hw/arm/virt.h
>> +++ b/include/hw/arm/virt.h
>> @@ -125,6 +125,7 @@ struct VirtMachineClass {
>> bool no_pmu;
>> bool claim_edge_triggered_timers;
>> bool smbios_old_sys_ver;
>> + bool no_highmem_compact;
>> bool no_highmem_ecam;
>> bool no_ged; /* Machines < 4.2 have no support for ACPI GED device */
>> bool kvm_no_adjvtime;
>
Thanks,
Gavin
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [PATCH v5 6/6] hw/arm/virt: Add 'compact-highmem' property
2022-10-19 23:57 ` Gavin Shan
@ 2022-10-20 9:44 ` Marc Zyngier
2022-10-20 11:13 ` Gavin Shan
0 siblings, 1 reply; 18+ messages in thread
From: Marc Zyngier @ 2022-10-20 9:44 UTC (permalink / raw)
To: Gavin Shan
Cc: eric.auger, qemu-arm, qemu-devel, cohuck, zhenyzha,
richard.henderson, peter.maydell, shan.gavin
On Thu, 20 Oct 2022 00:57:32 +0100,
Gavin Shan <gshan@redhat.com> wrote:
>
> For Marc's suggestion to add properties so that these high memory
> regions can be disabled by users. I can add one patch after this one
> to introduce the following 3 properties. Could you please confirm
> the property names are good enough? It's nice if Marc can help to
> confirm before I'm going to work on next revision.
>
> "highmem-ecam": "on"/"off" on vms->highmem_ecam
> "highmem-mmio": "on"/"off" on vms->highmem_mmio
> "highmem-redists": "on"/"off" on vms->highmem_redists
I think that'd be reasonable, and would give the user some actual
control over what gets exposed in the highmem region.
I guess that the annoying thing with these options is that they allow
the user to request conflicting settings (256 CPUs and
highmem-redists=off, for example). You'll need to make this fail more
or less gracefully.
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v5 6/6] hw/arm/virt: Add 'compact-highmem' property
2022-10-20 9:44 ` Marc Zyngier
@ 2022-10-20 11:13 ` Gavin Shan
0 siblings, 0 replies; 18+ messages in thread
From: Gavin Shan @ 2022-10-20 11:13 UTC (permalink / raw)
To: Marc Zyngier
Cc: eric.auger, qemu-arm, qemu-devel, cohuck, zhenyzha,
richard.henderson, peter.maydell, shan.gavin
Hi Marc,
On 10/20/22 5:44 PM, Marc Zyngier wrote:
> On Thu, 20 Oct 2022 00:57:32 +0100,
> Gavin Shan <gshan@redhat.com> wrote:
>>
>> For Marc's suggestion to add properties so that these high memory
>> regions can be disabled by users. I can add one patch after this one
>> to introduce the following 3 properties. Could you please confirm
>> the property names are good enough? It's nice if Marc can help to
>> confirm before I'm going to work on next revision.
>>
>> "highmem-ecam": "on"/"off" on vms->highmem_ecam
>> "highmem-mmio": "on"/"off" on vms->highmem_mmio
>> "highmem-redists": "on"/"off" on vms->highmem_redists
>
> I think that'd be reasonable, and would give the user some actual
> control over what gets exposed in the highmem region.
>
> I guess that the annoying thing with these options is that they allow
> the user to request conflicting settings (256 CPUs and
> highmem-redists=off, for example). You'll need to make this fail more
> or less gracefully.
>
Thanks for the quick confirm. The check is already existing in machvirt_init().
I think what we need is simply to calculate 'virt_max_cpus' with consideration
to 'highmem-redists' property there.
if (max_cpus > virt_max_cpus) {
error_report("Number of SMP CPUs requested (%d) exceeds max CPUs "
"supported by machine 'mach-virt' (%d)",
max_cpus, virt_max_cpus);
exit(1);
}
Thanks,
Gavin
^ permalink raw reply [flat|nested] 18+ messages in thread