* [PATCH] pc: q35: Bump max_cpus to 1728 vcpus
@ 2024-01-30 16:44 Ani Sinha
2024-01-30 16:47 ` Daniel P. Berrangé
0 siblings, 1 reply; 6+ messages in thread
From: Ani Sinha @ 2024-01-30 16:44 UTC (permalink / raw)
To: Paolo Bonzini, Richard Henderson, Eduardo Habkost,
Michael S. Tsirkin, Marcel Apfelbaum
Cc: Ani Sinha, Daniel P . Berrangé, Igor Mammedov,
Julia Suvorova, qemu-devel
Since commit f10a570b093e6 ("KVM: x86: Add CONFIG_KVM_MAX_NR_VCPUS to allow up to 4096 vCPUs")
Linux kernel can support upto a maximum number of 4096 vCPUS when MAXSMP is
enabled in the kernel. QEMU has been tested to correctly boot a linux guest
with 1728 vcpus both with edk2 and seabios firmwares. So bump up the max_cpus
value for q35 machines versions 9 and newer to 1728. Q35 machines versions
8.2 and older continue to support 1024 maximum vcpus as before for
compatibility.
If KVM is not able to support the specified number of vcpus, QEMU would
return the following error messages:
$ ./qemu-system-x86_64 -cpu host -accel kvm -machine q35 -smp 1728
qemu-system-x86_64: -accel kvm: warning: Number of SMP cpus requested (1728) exceeds the recommended cpus supported by KVM (12)
qemu-system-x86_64: -accel kvm: warning: Number of hotpluggable cpus requested (1728) exceeds the recommended cpus supported by KVM (12)
Number of SMP cpus requested (1728) exceeds the maximum cpus supported by KVM (1024)
Cc: Daniel P. Berrangé <berrange@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Julia Suvorova <jusual@redhat.com>
Signed-off-by: Ani Sinha <anisinha@redhat.com>
---
hw/i386/pc_q35.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index f43d5142b8..bfa627a70b 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -375,7 +375,7 @@ static void pc_q35_machine_options(MachineClass *m)
m->default_nic = "e1000e";
m->default_kernel_irqchip_split = false;
m->no_floppy = 1;
- m->max_cpus = 1024;
+ m->max_cpus = 1728;
m->no_parallel = !module_object_class_by_name(TYPE_ISA_PARALLEL);
machine_class_allow_dynamic_sysbus_dev(m, TYPE_AMD_IOMMU_DEVICE);
machine_class_allow_dynamic_sysbus_dev(m, TYPE_INTEL_IOMMU_DEVICE);
@@ -396,6 +396,7 @@ static void pc_q35_8_2_machine_options(MachineClass *m)
{
pc_q35_9_0_machine_options(m);
m->alias = NULL;
+ m->max_cpus = 1024;
compat_props_add(m->compat_props, hw_compat_8_2, hw_compat_8_2_len);
compat_props_add(m->compat_props, pc_compat_8_2, pc_compat_8_2_len);
}
--
2.42.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] pc: q35: Bump max_cpus to 1728 vcpus
2024-01-30 16:44 [PATCH] pc: q35: Bump max_cpus to 1728 vcpus Ani Sinha
@ 2024-01-30 16:47 ` Daniel P. Berrangé
2024-01-30 17:09 ` Ani Sinha
0 siblings, 1 reply; 6+ messages in thread
From: Daniel P. Berrangé @ 2024-01-30 16:47 UTC (permalink / raw)
To: Ani Sinha
Cc: Paolo Bonzini, Richard Henderson, Eduardo Habkost,
Michael S. Tsirkin, Marcel Apfelbaum, Igor Mammedov,
Julia Suvorova, qemu-devel
On Tue, Jan 30, 2024 at 10:14:28PM +0530, Ani Sinha wrote:
> Since commit f10a570b093e6 ("KVM: x86: Add CONFIG_KVM_MAX_NR_VCPUS to allow up to 4096 vCPUs")
> Linux kernel can support upto a maximum number of 4096 vCPUS when MAXSMP is
> enabled in the kernel. QEMU has been tested to correctly boot a linux guest
> with 1728 vcpus both with edk2 and seabios firmwares. So bump up the max_cpus
> value for q35 machines versions 9 and newer to 1728. Q35 machines versions
> 8.2 and older continue to support 1024 maximum vcpus as before for
> compatibility.
Where does the 1728 number come from ?
Did something break at 1729, or did the test machine simply not
have sufficient resources to do practical larger tests ?
>
> If KVM is not able to support the specified number of vcpus, QEMU would
> return the following error messages:
>
> $ ./qemu-system-x86_64 -cpu host -accel kvm -machine q35 -smp 1728
> qemu-system-x86_64: -accel kvm: warning: Number of SMP cpus requested (1728) exceeds the recommended cpus supported by KVM (12)
> qemu-system-x86_64: -accel kvm: warning: Number of hotpluggable cpus requested (1728) exceeds the recommended cpus supported by KVM (12)
> Number of SMP cpus requested (1728) exceeds the maximum cpus supported by KVM (1024)
>
> Cc: Daniel P. Berrangé <berrange@redhat.com>
> Cc: Igor Mammedov <imammedo@redhat.com>
> Cc: Michael S. Tsirkin <mst@redhat.com>
> Cc: Julia Suvorova <jusual@redhat.com>
> Signed-off-by: Ani Sinha <anisinha@redhat.com>
> ---
> hw/i386/pc_q35.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
> index f43d5142b8..bfa627a70b 100644
> --- a/hw/i386/pc_q35.c
> +++ b/hw/i386/pc_q35.c
> @@ -375,7 +375,7 @@ static void pc_q35_machine_options(MachineClass *m)
> m->default_nic = "e1000e";
> m->default_kernel_irqchip_split = false;
> m->no_floppy = 1;
> - m->max_cpus = 1024;
> + m->max_cpus = 1728;
> m->no_parallel = !module_object_class_by_name(TYPE_ISA_PARALLEL);
> machine_class_allow_dynamic_sysbus_dev(m, TYPE_AMD_IOMMU_DEVICE);
> machine_class_allow_dynamic_sysbus_dev(m, TYPE_INTEL_IOMMU_DEVICE);
> @@ -396,6 +396,7 @@ static void pc_q35_8_2_machine_options(MachineClass *m)
> {
> pc_q35_9_0_machine_options(m);
> m->alias = NULL;
> + m->max_cpus = 1024;
> compat_props_add(m->compat_props, hw_compat_8_2, hw_compat_8_2_len);
> compat_props_add(m->compat_props, pc_compat_8_2, pc_compat_8_2_len);
> }
> --
> 2.42.0
>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] pc: q35: Bump max_cpus to 1728 vcpus
2024-01-30 16:47 ` Daniel P. Berrangé
@ 2024-01-30 17:09 ` Ani Sinha
2024-01-30 17:13 ` Daniel P. Berrangé
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Ani Sinha @ 2024-01-30 17:09 UTC (permalink / raw)
To: "Daniel P. Berrangé"
Cc: Paolo Bonzini, Richard Henderson, Eduardo Habkost,
Michael S. Tsirkin, Marcel Apfelbaum, Igor Mammedov,
Julia Suvorova, QEMU Developers, Gerd Hoffmann
> On 30-Jan-2024, at 22:17, Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> On Tue, Jan 30, 2024 at 10:14:28PM +0530, Ani Sinha wrote:
>> Since commit f10a570b093e6 ("KVM: x86: Add CONFIG_KVM_MAX_NR_VCPUS to allow up to 4096 vCPUs")
>> Linux kernel can support upto a maximum number of 4096 vCPUS when MAXSMP is
>> enabled in the kernel. QEMU has been tested to correctly boot a linux guest
>> with 1728 vcpus both with edk2 and seabios firmwares. So bump up the max_cpus
>> value for q35 machines versions 9 and newer to 1728. Q35 machines versions
>> 8.2 and older continue to support 1024 maximum vcpus as before for
>> compatibility.
>
> Where does the 1728 number come from ?
>
> Did something break at 1729, or did the test machine simply not
> have sufficient resources to do practical larger tests ?
Actual limit currently is 1856 for EDK2. The HPE folks tested QEMU with edk2 and QEMU fails to boot beyond that limit.
There are RH internal bugs tracking this and Gerd is working on it from RH side [1].
We would ultimately like to go to 8192 vcpus for SAP HANA but 1728 vcpus is our immediate target for now. If you want, I can resend the patch with 1856 since that is currently the tested limit.
1. https://issues.redhat.com/browse/RHEL-22202
>
>>
>> If KVM is not able to support the specified number of vcpus, QEMU would
>> return the following error messages:
>>
>> $ ./qemu-system-x86_64 -cpu host -accel kvm -machine q35 -smp 1728
>> qemu-system-x86_64: -accel kvm: warning: Number of SMP cpus requested (1728) exceeds the recommended cpus supported by KVM (12)
>> qemu-system-x86_64: -accel kvm: warning: Number of hotpluggable cpus requested (1728) exceeds the recommended cpus supported by KVM (12)
>> Number of SMP cpus requested (1728) exceeds the maximum cpus supported by KVM (1024)
>>
>> Cc: Daniel P. Berrangé <berrange@redhat.com>
>> Cc: Igor Mammedov <imammedo@redhat.com>
>> Cc: Michael S. Tsirkin <mst@redhat.com>
>> Cc: Julia Suvorova <jusual@redhat.com>
>> Signed-off-by: Ani Sinha <anisinha@redhat.com>
>> ---
>> hw/i386/pc_q35.c | 3 ++-
>> 1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
>> index f43d5142b8..bfa627a70b 100644
>> --- a/hw/i386/pc_q35.c
>> +++ b/hw/i386/pc_q35.c
>> @@ -375,7 +375,7 @@ static void pc_q35_machine_options(MachineClass *m)
>> m->default_nic = "e1000e";
>> m->default_kernel_irqchip_split = false;
>> m->no_floppy = 1;
>> - m->max_cpus = 1024;
>> + m->max_cpus = 1728;
>> m->no_parallel = !module_object_class_by_name(TYPE_ISA_PARALLEL);
>> machine_class_allow_dynamic_sysbus_dev(m, TYPE_AMD_IOMMU_DEVICE);
>> machine_class_allow_dynamic_sysbus_dev(m, TYPE_INTEL_IOMMU_DEVICE);
>> @@ -396,6 +396,7 @@ static void pc_q35_8_2_machine_options(MachineClass *m)
>> {
>> pc_q35_9_0_machine_options(m);
>> m->alias = NULL;
>> + m->max_cpus = 1024;
>> compat_props_add(m->compat_props, hw_compat_8_2, hw_compat_8_2_len);
>> compat_props_add(m->compat_props, pc_compat_8_2, pc_compat_8_2_len);
>> }
>> --
>> 2.42.0
>>
>
> With regards,
> Daniel
> --
> |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o- https://fstop138.berrange.com :|
> |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] pc: q35: Bump max_cpus to 1728 vcpus
2024-01-30 17:09 ` Ani Sinha
@ 2024-01-30 17:13 ` Daniel P. Berrangé
2024-01-30 19:07 ` Philippe Mathieu-Daudé
2024-01-30 22:17 ` Michael S. Tsirkin
2 siblings, 0 replies; 6+ messages in thread
From: Daniel P. Berrangé @ 2024-01-30 17:13 UTC (permalink / raw)
To: Ani Sinha
Cc: Paolo Bonzini, Richard Henderson, Eduardo Habkost,
Michael S. Tsirkin, Marcel Apfelbaum, Igor Mammedov,
Julia Suvorova, QEMU Developers, Gerd Hoffmann
On Tue, Jan 30, 2024 at 10:39:51PM +0530, Ani Sinha wrote:
>
>
> > On 30-Jan-2024, at 22:17, Daniel P. Berrangé <berrange@redhat.com> wrote:
> >
> > On Tue, Jan 30, 2024 at 10:14:28PM +0530, Ani Sinha wrote:
> >> Since commit f10a570b093e6 ("KVM: x86: Add CONFIG_KVM_MAX_NR_VCPUS to allow up to 4096 vCPUs")
> >> Linux kernel can support upto a maximum number of 4096 vCPUS when MAXSMP is
> >> enabled in the kernel. QEMU has been tested to correctly boot a linux guest
> >> with 1728 vcpus both with edk2 and seabios firmwares. So bump up the max_cpus
> >> value for q35 machines versions 9 and newer to 1728. Q35 machines versions
> >> 8.2 and older continue to support 1024 maximum vcpus as before for
> >> compatibility.
> >
> > Where does the 1728 number come from ?
> >
> > Did something break at 1729, or did the test machine simply not
> > have sufficient resources to do practical larger tests ?
>
> Actual limit currently is 1856 for EDK2. The HPE folks tested QEMU with edk2 and QEMU fails to boot beyond that limit.
> There are RH internal bugs tracking this and Gerd is working on it from RH side [1].
>
> We would ultimately like to go to 8192 vcpus for SAP HANA but 1728 vcpus is our immediate target for now. If you want, I can resend the patch with 1856 since that is currently the tested limit.
Yes, could you resend with 1856, and include a description of
the blocking problem in the commit message for the historical
record, as this is the kind of thing that people will have
forgotten when re-visiting the patch later and wondering why
this limit was chosen.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] pc: q35: Bump max_cpus to 1728 vcpus
2024-01-30 17:09 ` Ani Sinha
2024-01-30 17:13 ` Daniel P. Berrangé
@ 2024-01-30 19:07 ` Philippe Mathieu-Daudé
2024-01-30 22:17 ` Michael S. Tsirkin
2 siblings, 0 replies; 6+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-01-30 19:07 UTC (permalink / raw)
To: Ani Sinha, Daniel P. Berrangé
Cc: Paolo Bonzini, Richard Henderson, Eduardo Habkost,
Michael S. Tsirkin, Marcel Apfelbaum, Igor Mammedov,
Julia Suvorova, QEMU Developers, Gerd Hoffmann
On 30/1/24 18:09, Ani Sinha wrote:
>
>
>> On 30-Jan-2024, at 22:17, Daniel P. Berrangé <berrange@redhat.com> wrote:
>>
>> On Tue, Jan 30, 2024 at 10:14:28PM +0530, Ani Sinha wrote:
>>> Since commit f10a570b093e6 ("KVM: x86: Add CONFIG_KVM_MAX_NR_VCPUS to allow up to 4096 vCPUs")
>>> Linux kernel can support upto a maximum number of 4096 vCPUS when MAXSMP is
>>> enabled in the kernel. QEMU has been tested to correctly boot a linux guest
>>> with 1728 vcpus both with edk2 and seabios firmwares. So bump up the max_cpus
>>> value for q35 machines versions 9 and newer to 1728. Q35 machines versions
>>> 8.2 and older continue to support 1024 maximum vcpus as before for
>>> compatibility.
>>
>> Where does the 1728 number come from ?
>>
>> Did something break at 1729, or did the test machine simply not
>> have sufficient resources to do practical larger tests ?
>
> Actual limit currently is 1856 for EDK2. The HPE folks tested QEMU with edk2 and QEMU fails to boot beyond that limit.
> There are RH internal bugs tracking this and Gerd is working on it from RH side [1].
>
> We would ultimately like to go to 8192 vcpus for SAP HANA but 1728 vcpus is our immediate target for now. If you want, I can resend the patch with 1856 since that is currently the tested limit.
>
> 1. https://issues.redhat.com/browse/RHEL-22202
Out of curiosity has the limit to be multiple of 64?
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] pc: q35: Bump max_cpus to 1728 vcpus
2024-01-30 17:09 ` Ani Sinha
2024-01-30 17:13 ` Daniel P. Berrangé
2024-01-30 19:07 ` Philippe Mathieu-Daudé
@ 2024-01-30 22:17 ` Michael S. Tsirkin
2 siblings, 0 replies; 6+ messages in thread
From: Michael S. Tsirkin @ 2024-01-30 22:17 UTC (permalink / raw)
To: Ani Sinha
Cc: "Daniel P. Berrangé", Paolo Bonzini,
Richard Henderson, Eduardo Habkost, Marcel Apfelbaum,
Igor Mammedov, Julia Suvorova, QEMU Developers, Gerd Hoffmann
On Tue, Jan 30, 2024 at 10:39:51PM +0530, Ani Sinha wrote:
>
>
> > On 30-Jan-2024, at 22:17, Daniel P. Berrangé <berrange@redhat.com> wrote:
> >
> > On Tue, Jan 30, 2024 at 10:14:28PM +0530, Ani Sinha wrote:
> >> Since commit f10a570b093e6 ("KVM: x86: Add CONFIG_KVM_MAX_NR_VCPUS to allow up to 4096 vCPUs")
> >> Linux kernel can support upto a maximum number of 4096 vCPUS when MAXSMP is
> >> enabled in the kernel. QEMU has been tested to correctly boot a linux guest
> >> with 1728 vcpus both with edk2 and seabios firmwares. So bump up the max_cpus
> >> value for q35 machines versions 9 and newer to 1728. Q35 machines versions
> >> 8.2 and older continue to support 1024 maximum vcpus as before for
> >> compatibility.
> >
> > Where does the 1728 number come from ?
> >
> > Did something break at 1729, or did the test machine simply not
> > have sufficient resources to do practical larger tests ?
>
> Actual limit currently is 1856 for EDK2. The HPE folks tested QEMU with edk2 and QEMU fails to boot beyond that limit.
> There are RH internal bugs tracking this and Gerd is working on it from RH side [1].
>
> We would ultimately like to go to 8192 vcpus for SAP HANA but 1728 vcpus is our immediate target for now. If you want, I can resend the patch with 1856 since that is currently the tested limit.
>
> 1. https://issues.redhat.com/browse/RHEL-22202
What is requested here is that you document the source of the number whatever
it is in a code comment and commit log.
>
> >
> >>
> >> If KVM is not able to support the specified number of vcpus, QEMU would
> >> return the following error messages:
> >>
> >> $ ./qemu-system-x86_64 -cpu host -accel kvm -machine q35 -smp 1728
> >> qemu-system-x86_64: -accel kvm: warning: Number of SMP cpus requested (1728) exceeds the recommended cpus supported by KVM (12)
> >> qemu-system-x86_64: -accel kvm: warning: Number of hotpluggable cpus requested (1728) exceeds the recommended cpus supported by KVM (12)
> >> Number of SMP cpus requested (1728) exceeds the maximum cpus supported by KVM (1024)
> >>
> >> Cc: Daniel P. Berrangé <berrange@redhat.com>
> >> Cc: Igor Mammedov <imammedo@redhat.com>
> >> Cc: Michael S. Tsirkin <mst@redhat.com>
> >> Cc: Julia Suvorova <jusual@redhat.com>
> >> Signed-off-by: Ani Sinha <anisinha@redhat.com>
> >> ---
> >> hw/i386/pc_q35.c | 3 ++-
> >> 1 file changed, 2 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
> >> index f43d5142b8..bfa627a70b 100644
> >> --- a/hw/i386/pc_q35.c
> >> +++ b/hw/i386/pc_q35.c
> >> @@ -375,7 +375,7 @@ static void pc_q35_machine_options(MachineClass *m)
> >> m->default_nic = "e1000e";
> >> m->default_kernel_irqchip_split = false;
> >> m->no_floppy = 1;
> >> - m->max_cpus = 1024;
> >> + m->max_cpus = 1728;
> >> m->no_parallel = !module_object_class_by_name(TYPE_ISA_PARALLEL);
> >> machine_class_allow_dynamic_sysbus_dev(m, TYPE_AMD_IOMMU_DEVICE);
> >> machine_class_allow_dynamic_sysbus_dev(m, TYPE_INTEL_IOMMU_DEVICE);
> >> @@ -396,6 +396,7 @@ static void pc_q35_8_2_machine_options(MachineClass *m)
> >> {
> >> pc_q35_9_0_machine_options(m);
> >> m->alias = NULL;
> >> + m->max_cpus = 1024;
> >> compat_props_add(m->compat_props, hw_compat_8_2, hw_compat_8_2_len);
> >> compat_props_add(m->compat_props, pc_compat_8_2, pc_compat_8_2_len);
> >> }
> >> --
> >> 2.42.0
> >>
> >
> > With regards,
> > Daniel
> > --
> > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
> > |: https://libvirt.org -o- https://fstop138.berrange.com :|
> > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2024-01-30 22:18 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-01-30 16:44 [PATCH] pc: q35: Bump max_cpus to 1728 vcpus Ani Sinha
2024-01-30 16:47 ` Daniel P. Berrangé
2024-01-30 17:09 ` Ani Sinha
2024-01-30 17:13 ` Daniel P. Berrangé
2024-01-30 19:07 ` Philippe Mathieu-Daudé
2024-01-30 22:17 ` Michael S. Tsirkin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).