[Qemu-devel] [PATCH 0/2] QEMU maxcpus support v3

[Qemu-devel] [PATCH 0/2] QEMU maxcpus support v3

[Jes Sorensen]

Anthony,

Here's an updated version of the maxcpus patch, which goes on top of
the most recent QEMU. My older version was against a 2 week old
version and I see you have applied some boot menu stuff since then
which did clash with the FW_CFG magic number I used. I suspect that
could have caused the problems.

Just did a full fresh rebuild and this one works fine for me with the
Ubuntu test image.

Jes

[Qemu-devel] [PATCH 1/2] Introduce -smp , maxcpus= flag to specify maximum number of CPUS.

[Jes Sorensen]

[-- Attachment #1: 0001-qemu-cfg-maxcpus.patch --]
[-- Type: text/plain, Size: 4262 bytes --]

Follow on patch will use it to determine the size of the MADT and
other BIOS tables.

Signed-off-by: Jes Sorensen <jes@sgi.com>

---
 hw/fw_cfg.c     |    1 +
 hw/fw_cfg.h     |    1 +
 qemu-options.hx |    5 ++++-
 sysemu.h        |    1 +
 vl.c            |   27 ++++++++++++++++++++++++++-
 5 files changed, 33 insertions(+), 2 deletions(-)

Index: qemu/hw/fw_cfg.c
===================================================================
--- qemu.orig/hw/fw_cfg.c
+++ qemu/hw/fw_cfg.c
@@ -279,6 +279,7 @@ void *fw_cfg_init(uint32_t ctl_port, uin
     fw_cfg_add_bytes(s, FW_CFG_UUID, qemu_uuid, 16);
     fw_cfg_add_i16(s, FW_CFG_NOGRAPHIC, (uint16_t)(display_type == DT_NOGRAPHIC));
     fw_cfg_add_i16(s, FW_CFG_NB_CPUS, (uint16_t)smp_cpus);
+    fw_cfg_add_i16(s, FW_CFG_MAX_CPUS, (uint16_t)max_cpus);
     fw_cfg_add_i16(s, FW_CFG_BOOT_MENU, (uint16_t)boot_menu);
 
     register_savevm("fw_cfg", -1, 1, fw_cfg_save, fw_cfg_load, s);
Index: qemu/hw/fw_cfg.h
===================================================================
--- qemu.orig/hw/fw_cfg.h
+++ qemu/hw/fw_cfg.h
@@ -16,6 +16,7 @@
 #define FW_CFG_BOOT_DEVICE      0x0c
 #define FW_CFG_NUMA             0x0d
 #define FW_CFG_BOOT_MENU        0x0e
+#define FW_CFG_MAX_CPUS         0x0f
 #define FW_CFG_MAX_ENTRY        0x10
 
 #define FW_CFG_WRITE_CHANNEL    0x4000
Index: qemu/qemu-options.hx
===================================================================
--- qemu.orig/qemu-options.hx
+++ qemu/qemu-options.hx
@@ -39,7 +39,10 @@ Select CPU model (-cpu ? for list and ad
 ETEXI
 
 DEF("smp", HAS_ARG, QEMU_OPTION_smp,
-    "-smp n          set the number of CPUs to 'n' [default=1]\n")
+    "-smp n[,maxcpus=cpus]\n"
+    "                set the number of CPUs to 'n' [default=1]\n"
+    "                maxcpus= maximum number of total cpus, including\n"
+    "                offline CPUs for hotplug etc.\n")
 STEXI
 @item -smp @var{n}
 Simulate an SMP system with @var{n} CPUs. On the PC target, up to 255
Index: qemu/sysemu.h
===================================================================
--- qemu.orig/sysemu.h
+++ qemu/sysemu.h
@@ -119,6 +119,7 @@ extern int usb_enabled;
 extern int virtio_balloon;
 extern const char *virtio_balloon_devaddr;
 extern int smp_cpus;
+extern int max_cpus;
 extern int cursor_hide;
 extern int graphic_rotate;
 extern int no_quit;
Index: qemu/vl.c
===================================================================
--- qemu.orig/vl.c
+++ qemu/vl.c
@@ -227,6 +227,7 @@ int rtc_td_hack = 0;
 int usb_enabled = 0;
 int singlestep = 0;
 int smp_cpus = 1;
+int max_cpus = 0;
 const char *vnc_display;
 int acpi_enabled = 1;
 int no_hpet = 0;
@@ -5458,12 +5459,29 @@ int main(int argc, char **argv, char **e
                 usb_devices_index++;
                 break;
             case QEMU_OPTION_smp:
-                smp_cpus = atoi(optarg);
+            {
+                char *p;
+                char option[128];
+                smp_cpus = strtol(optarg, &p, 10);
                 if (smp_cpus < 1) {
                     fprintf(stderr, "Invalid number of CPUs\n");
                     exit(1);
                 }
+                if (*p++ != ',')
+                    break;
+                if (get_param_value(option, 128, "maxcpus", p))
+                    max_cpus = strtol(option, NULL, 0);
+                if (max_cpus < smp_cpus) {
+                    fprintf(stderr, "maxcpus must be equal to or greater than "
+                            "smp\n");
+                    exit(1);
+                }
+                if (max_cpus > 255) {
+                    fprintf(stderr, "Unsupported number of maxcpus\n");
+                    exit(1);
+                }
                 break;
+            }
 	    case QEMU_OPTION_vnc:
                 display_type = DT_VNC;
 		vnc_display = optarg;
@@ -5644,6 +5662,13 @@ int main(int argc, char **argv, char **e
     }
 #endif
 
+    /*
+     * Default to max_cpus = smp_cpus, in case the user doesn't
+     * specify a max_cpus value.
+     */
+    if (!max_cpus)
+        max_cpus = smp_cpus;
+
     machine->max_cpus = machine->max_cpus ?: 1; /* Default to UP */
     if (smp_cpus > machine->max_cpus) {
         fprintf(stderr, "Number of SMP cpus requested (%d), exceeds max cpus "

[Qemu-devel] [PATCH 2/2] QEMU BOCHS bios patches to use maxcpus value.

[Jes Sorensen]

[-- Attachment #1: 0002-qemu-bios-patches.patch --]
[-- Type: text/plain, Size: 6458 bytes --]

Signed-off-by: Jes Sorensen <jes@sgi.com>
---
 pc-bios/bios-pq/0020-qemu-kvm-cfg-maxcpus.patch |   61 +++++++++++++++
 pc-bios/bios-pq/0021-qemu-madt-maxcpus.patch    |   96 ++++++++++++++++++++++++
 pc-bios/bios-pq/series                          |    2 
 3 files changed, 159 insertions(+)

Index: qemu/pc-bios/bios-pq/0020-qemu-kvm-cfg-maxcpus.patch
===================================================================
--- /dev/null
+++ qemu/pc-bios/bios-pq/0020-qemu-kvm-cfg-maxcpus.patch
@@ -0,0 +1,61 @@
+Read max_cpus variable from QEMU_CFG. If not provided, use value of
+smp_cpus.
+
+Signed-off-by: Jes Sorensen <jes@sgi.com>
+
+diff --git a/bios/rombios.h b/bios/rombios.h
+index 8ece2ee..dbf3bd3 100644
+--- a/bios/rombios.h
++++ b/bios/rombios.h
+@@ -65,6 +65,7 @@
+ #define QEMU_CFG_UUID             0x02
+ #define QEMU_CFG_NUMA             0x0d
+ #define QEMU_CFG_BOOT_MENU        0x0e
++#define QEMU_CFG_MAX_CPUS         0x0f
+ #define QEMU_CFG_ARCH_LOCAL       0x8000
+ #define QEMU_CFG_ACPI_TABLES      (QEMU_CFG_ARCH_LOCAL + 0)
+ #define QEMU_CFG_SMBIOS_ENTRIES   (QEMU_CFG_ARCH_LOCAL + 1)
+diff --git a/bios/rombios32.c b/bios/rombios32.c
+index 69e82b1..462884a 100644
+--- a/bios/rombios32.c
++++ b/bios/rombios32.c
+@@ -436,6 +436,7 @@ void delay_ms(int n)
+ }
+ 
+ uint16_t smp_cpus;
++uint16_t max_cpus;
+ uint32_t cpuid_signature;
+ uint32_t cpuid_features;
+ uint32_t cpuid_ext_features;
+@@ -526,6 +527,19 @@ static uint16_t smbios_entries(void)
+     return cnt;
+ }
+ 
++static uint16_t get_max_cpus(void)
++{
++    uint16_t cnt;
++
++    qemu_cfg_select(QEMU_CFG_MAX_CPUS);
++    qemu_cfg_read((uint8_t*)&cnt, sizeof(cnt));
++
++    if (!cnt)
++        cnt = smp_cpus;
++
++    return cnt;
++}
++
+ uint64_t qemu_cfg_get64 (void)
+ {
+     uint64_t ret;
+@@ -1645,6 +1659,11 @@ void acpi_bios_init(void)
+     uint16_t i, external_tables;
+     int nb_numa_nodes;
+ 
++#ifdef BX_QEMU
++    max_cpus = get_max_cpus();
++#else
++    max_cpus = smp_cpus;
++#endif
+     /* reserve memory space for tables */
+ #ifdef BX_USE_EBDA_TABLES
+     ebda_cur_addr = align(ebda_cur_addr, 16);
Index: qemu/pc-bios/bios-pq/0021-qemu-madt-maxcpus.patch
===================================================================
--- /dev/null
+++ qemu/pc-bios/bios-pq/0021-qemu-madt-maxcpus.patch
@@ -0,0 +1,96 @@
+Use max_cpus when building bios tables.
+
+Signed-off-by: Jes Sorensen <jes@sgi.com>
+
+diff --git a/bios/rombios32.c b/bios/rombios32.c
+index 462884a..1ec4a09 100644
+--- a/bios/rombios32.c
++++ b/bios/rombios32.c
+@@ -1144,13 +1144,13 @@ static void mptable_init(void)
+     putstr(&q, "0.1         "); /* vendor id */
+     putle32(&q, 0); /* OEM table ptr */
+     putle16(&q, 0); /* OEM table size */
+-    putle16(&q, smp_cpus + 18); /* entry count */
++    putle16(&q, max_cpus + 18); /* entry count */
+     putle32(&q, 0xfee00000); /* local APIC addr */
+     putle16(&q, 0); /* ext table length */
+     putb(&q, 0); /* ext table checksum */
+     putb(&q, 0); /* reserved */
+ 
+-    for(i = 0; i < smp_cpus; i++) {
++    for(i = 0; i < max_cpus; i++) {
+         putb(&q, 0); /* entry type = processor */
+         putb(&q, i); /* APIC id */
+         putb(&q, 0x11); /* local APIC version number */
+@@ -1177,7 +1177,7 @@ static void mptable_init(void)
+     putstr(&q, "ISA   ");
+ 
+     /* ioapic */
+-    ioapic_id = smp_cpus;
++    ioapic_id = max_cpus;
+     putb(&q, 2); /* entry type = I/O APIC */
+     putb(&q, ioapic_id); /* apic ID */
+     putb(&q, 0x11); /* I/O APIC version number */
+@@ -1577,7 +1577,7 @@ int acpi_build_processor_ssdt(uint8_t *ssdt)
+ {
+     uint8_t *ssdt_ptr = ssdt;
+     int i, length;
+-    int acpi_cpus = smp_cpus > 0xff ? 0xff : smp_cpus;
++    int acpi_cpus = max_cpus > 0xff ? 0xff : max_cpus;
+ 
+     ssdt_ptr[9] = 0; // checksum;
+     ssdt_ptr += sizeof(struct acpi_table_header);
+@@ -1714,7 +1714,7 @@ void acpi_bios_init(void)
+         addr = (addr + 7) & ~7;
+         srat_addr = addr;
+         srat_size = sizeof(*srat) +
+-            sizeof(struct srat_processor_affinity) * smp_cpus +
++            sizeof(struct srat_processor_affinity) * max_cpus +
+             sizeof(struct srat_memory_affinity) * (nb_numa_nodes + 2);
+         srat = (void *)(addr);
+         addr += srat_size;
+@@ -1727,7 +1727,7 @@ void acpi_bios_init(void)
+     addr = (addr + 7) & ~7;
+     madt_addr = addr;
+     madt_size = sizeof(*madt) +
+-        sizeof(struct madt_processor_apic) * smp_cpus +
++        sizeof(struct madt_processor_apic) * max_cpus +
+ #ifdef BX_QEMU
+         sizeof(struct madt_io_apic) + sizeof(struct madt_int_override);
+ #else
+@@ -1800,7 +1800,7 @@ void acpi_bios_init(void)
+         madt->local_apic_address = cpu_to_le32(0xfee00000);
+         madt->flags = cpu_to_le32(1);
+         apic = (void *)(madt + 1);
+-        for(i=0;i<smp_cpus;i++) {
++        for(i = 0;i < max_cpus; i++) {
+             apic->type = APIC_PROCESSOR;
+             apic->length = sizeof(*apic);
+             apic->processor_id = i;
+@@ -1811,7 +1811,7 @@ void acpi_bios_init(void)
+         io_apic = (void *)apic;
+         io_apic->type = APIC_IO;
+         io_apic->length = sizeof(*io_apic);
+-        io_apic->io_apic_id = smp_cpus;
++        io_apic->io_apic_id = max_cpus;
+         io_apic->address = cpu_to_le32(0xfec00000);
+         io_apic->interrupt = cpu_to_le32(0);
+ #ifdef BX_QEMU
+@@ -1845,7 +1845,7 @@ void acpi_bios_init(void)
+         srat->reserved1=1;
+  
+         core = (void*)(srat + 1);
+-        for (i = 0; i < smp_cpus; ++i) {
++        for (i = 0; i < max_cpus; ++i) {
+              core->type = SRAT_PROCESSOR;
+              core->length = sizeof(*core);
+              core->local_apic_id = i;
+@@ -2604,7 +2604,7 @@ void smbios_init(void)
+     add_struct(0, p);
+     add_struct(1, p);
+     add_struct(3, p);
+-    for (cpu_num = 1; cpu_num <= smp_cpus; cpu_num++)
++    for (cpu_num = 1; cpu_num <= max_cpus; cpu_num++)
+         add_struct(4, p, cpu_num);
+ 
+     /* Each 'memory device' covers up to 16GB of address space. */
Index: qemu/pc-bios/bios-pq/series
===================================================================
--- qemu.orig/pc-bios/bios-pq/series
+++ qemu/pc-bios/bios-pq/series
@@ -17,3 +17,5 @@
 0017-bochs-bios-Move-QEMU_CFG-constants-to-rombios.h.patch
 0018-bochs-bios-Make-boot-prompt-optional.patch
 0019-bios-fix-multiple-calls.patch
+0020-qemu-kvm-cfg-maxcpus.patch
+0021-qemu-madt-maxcpus.patch

Re: [Qemu-devel] [PATCH 0/2] QEMU maxcpus support v3

[Anthony Liguori]

Jes Sorensen wrote:
> Anthony,
>
> Here's an updated version of the maxcpus patch, which goes on top of
> the most recent QEMU. My older version was against a 2 week old
> version and I see you have applied some boot menu stuff since then
> which did clash with the FW_CFG magic number I used. I suspect that
> could have caused the problems.
>
> Just did a full fresh rebuild and this one works fine for me with the
> Ubuntu test image.
>   

I didn't realize that you changed maxcpus == smpcpus by default so my 
testing wasn't valid.

If I do the following:

x86_64-softmmu/qemu-system-x86_64 -hda ~/images/linux.img -snapshot -L 
~/git/qemu/pc-bios -smp 1,maxcpus=16

I get 'Not responding' errors from an Ubuntu server guest.

You can pull http://repo.or.cz/w/qemu/aliguori-queue.git for-jes

To get my built version of bios.bin.  Please try to reproduce this issue 
with these bits and let me know if you have trouble with it.

Regards,

Anthony Liguori

Re: [Qemu-devel] [PATCH 0/2] QEMU maxcpus support v3

[Jes Sorensen]

On 07/22/2009 05:10 PM, Anthony Liguori wrote:
> Jes Sorensen wrote:
> I didn't realize that you changed maxcpus == smpcpus by default so my
> testing wasn't valid.

This change was made at your request :)

> If I do the following:
>
> x86_64-softmmu/qemu-system-x86_64 -hda ~/images/linux.img -snapshot -L
> ~/git/qemu/pc-bios -smp 1,maxcpus=16

Just booted my tree with this line, works just fine:

~/src/qemu-bochs/qemu/x86_64-softmmu/qemu-system-x86_64 -smp 
1,maxcpus=15  -snapshot -L ~/src/qemu-bochs/test/ -serial stdio -cdrom 
~/Desktop/ubuntu-9.04-desktop-amd64.iso -boot d

> I get 'Not responding' errors from an Ubuntu server guest.
>
> You can pull http://repo.or.cz/w/qemu/aliguori-queue.git for-jes

jes@eye3:~/src/aligouri> git clone 
http://repo.or.cz/w/qemu/aliguori-queue.git
Initialized empty Git repository in 
/home/jes/src/aligouri/aliguori-queue/.git/
error: inflate: data stream error (incorrect header check)
error: inflate: data stream error (incorrect header check)
error: File 0000000000000000000000000000000000000000 
(http://repo.or.cz/w/qemu/aliguori-queue.git/objects/00/00000000000000000000000000000000000000) 
corrupt

> To get my built version of bios.bin. Please try to reproduce this issue
> with these bits and let me know if you have trouble with it.

Will try as soon as I can get to your git repo.

Cheers,
Jes