How about increasing max_cpus for q35 ?

How about increasing max_cpus for q35 ?

[Dario Faggioli]

[-- Attachment #1: Type: text/plain, Size: 1604 bytes --]

Hello,

Sorry for the potentially naive question, but I'm not clear what the
process would be if, say, I'd like to raise the number of maximum CPUs
a q35 VM can have.

So, right now we have:

void pc_q35_2_7_machine_options(MachineClass *m) {
  ...
  m->max_cpus = 255;
}

And:

void pc_q35_machine_options(MachineClass *m)
{
  ...
  m->max_cpus = 288;
}

Focusing on the latter, it comes from this commit:

https://gitlab.com/qemu-project/qemu/-/commit/00d0f9fd6602a27b204f672ef5bc8e69736c7ff1
  
  pc: q35: Bump max_cpus to 288

  Along with it for machine versions 2.7 and older keep
  it at 255.

So, it was 255 and is now 288. This seems to me to be there since QEMU
2.8.0.

Now, as far as I understand, KVM can handle 1024, at least since this
commit (and a couple of other related ones):

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=074c82c8f7cf8a46c3b81965f122599e3a133450
"kvm: x86: Increase MAX_VCPUS to 1024"

Which basically does:

-#define KVM_MAX_VCPUS 288
+#define KVM_MAX_VCPUS 1024

And it's included in kernels >= 5.15.

So, what's the correct way of bumping up the limit again? Just changing
that assignment in pc_q35_machine_options() ? Or do we want a new
version of the machine type or something like that?

Thanks and Regards
-- 
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: How about increasing max_cpus for q35 ?

[Igor Mammedov]

On Wed, 9 Nov 2022 13:36:07 +0000
Dario Faggioli <dfaggioli@suse.com> wrote:

> Hello,
> 
> Sorry for the potentially naive question, but I'm not clear what the
> process would be if, say, I'd like to raise the number of maximum CPUs
> a q35 VM can have.
> 
> So, right now we have:
> 
> void pc_q35_2_7_machine_options(MachineClass *m) {
>   ...
>   m->max_cpus = 255;
> }
> 
> And:
> 
> void pc_q35_machine_options(MachineClass *m)
> {
>   ...
>   m->max_cpus = 288;
> }
> 
> Focusing on the latter, it comes from this commit:
> 
> https://gitlab.com/qemu-project/qemu/-/commit/00d0f9fd6602a27b204f672ef5bc8e69736c7ff1
>   
>   pc: q35: Bump max_cpus to 288
> 
>   Along with it for machine versions 2.7 and older keep
>   it at 255.
> 
> So, it was 255 and is now 288. This seems to me to be there since QEMU
> 2.8.0.
> 
> Now, as far as I understand, KVM can handle 1024, at least since this
> commit (and a couple of other related ones):
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=074c82c8f7cf8a46c3b81965f122599e3a133450
> "kvm: x86: Increase MAX_VCPUS to 1024"
> 
> Which basically does:
> 
> -#define KVM_MAX_VCPUS 288
> +#define KVM_MAX_VCPUS 1024
> 
> And it's included in kernels >= 5.15.
> 
> So, what's the correct way of bumping up the limit again? Just changing
> that assignment in pc_q35_machine_options()

that and preserve 288 limit for existing machine types.

Basically the same as above QEMU commit with difference
that pc_q35_2_8_machine_options() should be replaced by
7.2 equivalent.


PS:
we are still missing OVMF support for 1024,
but it's being worked on.

> ? Or do we want a new
> version of the machine type or something like that?
> 
> Thanks and Regards

Re: How about increasing max_cpus for q35 ?

[Daniel P. Berrangé]

On Wed, Nov 09, 2022 at 01:36:07PM +0000, Dario Faggioli wrote:
> Hello,
> 
> Sorry for the potentially naive question, but I'm not clear what the
> process would be if, say, I'd like to raise the number of maximum CPUs
> a q35 VM can have.
> 
> So, right now we have:
> 
> void pc_q35_2_7_machine_options(MachineClass *m) {
>   ...
>   m->max_cpus = 255;
> }
> 
> And:
> 
> void pc_q35_machine_options(MachineClass *m)
> {
>   ...
>   m->max_cpus = 288;
> }
> 
> Focusing on the latter, it comes from this commit:
> 
> https://gitlab.com/qemu-project/qemu/-/commit/00d0f9fd6602a27b204f672ef5bc8e69736c7ff1
>   
>   pc: q35: Bump max_cpus to 288
> 
>   Along with it for machine versions 2.7 and older keep
>   it at 255.
> 
> So, it was 255 and is now 288. This seems to me to be there since QEMU
> 2.8.0.
> 
> Now, as far as I understand, KVM can handle 1024, at least since this
> commit (and a couple of other related ones):
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=074c82c8f7cf8a46c3b81965f122599e3a133450
> "kvm: x86: Increase MAX_VCPUS to 1024"
> 
> Which basically does:
> 
> -#define KVM_MAX_VCPUS 288
> +#define KVM_MAX_VCPUS 1024
> 
> And it's included in kernels >= 5.15.
> 
> So, what's the correct way of bumping up the limit again? Just changing
> that assignment in pc_q35_machine_options() ? Or do we want a new
> version of the machine type or something like that?

Too late for this release, so we need to wait until the new release
cycle opens, with thue addition of the 8.0.0 machine types. Then
just set 'max_cpus' in the new machine type.

In RHEL downstream we've set max_cpus = 710 for x86_64, because this
is the largest that was possible without hitting SMBIOS limits. The
problem was the use of the 32-bit entry point on PC machine types.

If we switch Q35 to use the 64-bit entry point, we should be able
to go straight to 1024 IIUC.  We were blocked changing the entry
point previously by need to have support in both SeaBIOS and UEFI
builds. I'm not sure what the status of this is, for the ROMS we
currently have in QEMU git. If both SeaBIOS and UEFI ROMs now
support 64-bit SMBIOS, then we can switch Q35 to it.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: How about increasing max_cpus for q35 ?

[Igor Mammedov]

On Fri, 11 Nov 2022 10:47:20 +0000
Daniel P. Berrangé <berrange@redhat.com> wrote:

> On Wed, Nov 09, 2022 at 01:36:07PM +0000, Dario Faggioli wrote:
> > Hello,
> > 
> > Sorry for the potentially naive question, but I'm not clear what the
> > process would be if, say, I'd like to raise the number of maximum CPUs
> > a q35 VM can have.
> > 
> > So, right now we have:
> > 
> > void pc_q35_2_7_machine_options(MachineClass *m) {
> >   ...
> >   m->max_cpus = 255;
> > }
> > 
> > And:
> > 
> > void pc_q35_machine_options(MachineClass *m)
> > {
> >   ...
> >   m->max_cpus = 288;
> > }
> > 
> > Focusing on the latter, it comes from this commit:
> > 
> > https://gitlab.com/qemu-project/qemu/-/commit/00d0f9fd6602a27b204f672ef5bc8e69736c7ff1
> >   
> >   pc: q35: Bump max_cpus to 288
> > 
> >   Along with it for machine versions 2.7 and older keep
> >   it at 255.
> > 
> > So, it was 255 and is now 288. This seems to me to be there since QEMU
> > 2.8.0.
> > 
> > Now, as far as I understand, KVM can handle 1024, at least since this
> > commit (and a couple of other related ones):
> > 
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=074c82c8f7cf8a46c3b81965f122599e3a133450
> > "kvm: x86: Increase MAX_VCPUS to 1024"
> > 
> > Which basically does:
> > 
> > -#define KVM_MAX_VCPUS 288
> > +#define KVM_MAX_VCPUS 1024
> > 
> > And it's included in kernels >= 5.15.
> > 
> > So, what's the correct way of bumping up the limit again? Just changing
> > that assignment in pc_q35_machine_options() ? Or do we want a new
> > version of the machine type or something like that?  
> 
> Too late for this release, so we need to wait until the new release
> cycle opens, with thue addition of the 8.0.0 machine types. Then
> just set 'max_cpus' in the new machine type.
> 
> In RHEL downstream we've set max_cpus = 710 for x86_64, because this
> is the largest that was possible without hitting SMBIOS limits. The
> problem was the use of the 32-bit entry point on PC machine types.

Just merged

    05e27d74c7dc5318 hw/smbios: add core_count2 to smbios table type 4

should have fixed the remaining issue with SMBIOS.
So we can flip default entrypoint to 64bit one for new machine
type (it's late for 7.2 but we should be able to do that for the next
one)
 
> If we switch Q35 to use the 64-bit entry point, we should be able
> to go straight to 1024 IIUC.  We were blocked changing the entry
> point previously by need to have support in both SeaBIOS and UEFI
> builds. I'm not sure what the status of this is, for the ROMS we
> currently have in QEMU git. If both SeaBIOS and UEFI ROMs now
> support 64-bit SMBIOS, then we can switch Q35 to it.

there is UEFI issue (though not SMBIOS related one)
  https://bugzilla.redhat.com/show_bug.cgi?id=1983086
but now it seems that it could be fixed soon.

I'm not aware of other SeaBIOS/UEFI issues wrt large number of vCPUs.
 
> With regards,
> Daniel