From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jan Kiszka Subject: Re: [PATCH] cpu hotplug issue Date: Sun, 24 Jul 2011 18:11:32 +0200 Message-ID: <4E2C4434.1060106@web.de> References: <20110720083507.GS2400@redhat.com> <20110721113342.GB3044@redhat.com> <4E281090.9070300@siemens.com> <20110721115118.GD3044@redhat.com> <20110721124512.GI3044@redhat.com> <4E29577A.9080909@siemens.com> <20110724115647.GR3044@redhat.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig77A482A7EB6C48346355F6F1" Cc: Vasilis Liaskovitis , "kvm@vger.kernel.org" , Markus Armbruster To: Gleb Natapov Return-path: Received: from fmmailgate01.web.de ([217.72.192.221]:54727 "EHLO fmmailgate01.web.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750769Ab1GXQLk (ORCPT ); Sun, 24 Jul 2011 12:11:40 -0400 In-Reply-To: <20110724115647.GR3044@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig77A482A7EB6C48346355F6F1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 2011-07-24 13:56, Gleb Natapov wrote: > On Fri, Jul 22, 2011 at 12:56:58PM +0200, Jan Kiszka wrote: >> On 2011-07-21 14:45, Gleb Natapov wrote: >>> On Thu, Jul 21, 2011 at 02:51:18PM +0300, Gleb Natapov wrote: >>>>>> Jan can you look at this please? >>>>> >>>>> I can't promise to do debugging myself. >>>>> >>>>> Also, as I never succeeded in getting anything working with CPU hot= plug, >>>>> even back in the days it was supposed to work, I'm a bit clueless /= wrt >>>>> to the right test cases. >>>>> >>>> CPU hotplug for Linux suppose to be easy (with allow_hotplug patch >>>> applied). But we have two bugs currently. One is that ACPI interrupt= >>>> is not send when cpu is onlined (at least this appears to be the cas= e). >>>> I will look at that one. Another is that after new cpu is detected i= t >>>> can't be onlined. >>>> >>>> After fixing the first bug the test should look like this: >>>> 1. start vm with -smp 1,macpus=3D2 >>>> 2. wait for it to boot >>>> 3. do "cpu 1 online" in monitor. >>>> 4. do "echo 1 > /sys/devices/system/cpu/cpu1/online" >>>> >>>> If step 4 should succeed. It fails now. >>>> >>> The first one was easy to solve. See patch below. Step 3 should be >>> "cpu_set 1 online". >>> >>> --- >>> >>> Trigger sci interrupt after cpu hotplug/unplug event. >>> >>> Signed-off-by: Gleb Natapov >>> diff --git a/hw/acpi_piix4.c b/hw/acpi_piix4.c >>> index c30a050..40f3fcd 100644 >>> --- a/hw/acpi_piix4.c >>> +++ b/hw/acpi_piix4.c >>> @@ -92,7 +92,8 @@ static void pm_update_sci(PIIX4PMState *s) >>> ACPI_BITMASK_POWER_BUTTON_ENABLE | >>> ACPI_BITMASK_GLOBAL_LOCK_ENABLE | >>> ACPI_BITMASK_TIMER_ENABLE)) !=3D 0) || >>> - (((s->gpe.sts[0] & s->gpe.en[0]) & PIIX4_PCI_HOTPLUG_STATUS)= !=3D 0); >>> + (((s->gpe.sts[0] & s->gpe.en[0]) & >>> + (PIIX4_PCI_HOTPLUG_STATUS | PIIX4_CPU_HOTPLUG_STATUS)) !=3D 0);=20 >>> =20 >>> qemu_set_irq(s->irq, sci_level); >>> /* schedule a timer interruption if needed */ >>> -- >>> Gleb. >> >> I had a closer look and identified two further issues, one generic, on= e >> CPU-hotplug-specific: >> - (qdev) devices that are hotplugged do not receive any reset. That >> does not only apply to the APIC in case of CPU hotplugging, it is >> also broken for NICs, storage controllers, etc. when doing PCI >> hot-add as I just checked via gdb. >> - CPU hotplugging was always (or at least for a fairly long time), >> well, fragile as it failed to make CPU thread creation and CPU >> initialization atomic against APIC addition and other initializatio= n >> steps. IOW, we need to create CPUs stopped, finish all init work, >> sync their states completely to the kernel >> (cpu_synchronize_post_init), and then kick them of. Actually I'm > Syncing the state to the kernel should be done by vcpu thread, so I it > cannot be stopped while the sync is done. May be I misunderstood what > you mean here. Stopped first of all means not entering kvm_cpu_exec before the whole setup and the initial sync are done. Syncing the initial state may also happen over the creating context as long as the vcpus are stopped (analogously to kvm_cpu_synchronize_post_init). Jan --------------enig77A482A7EB6C48346355F6F1 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4sRDkACgkQitSsb3rl5xQ+JgCeLRgV2s32VKZZ3IcWAsmyIreU SyIAn0R8dN8y/7KHeoZ/y6gPN+1ltkC4 =MEdv -----END PGP SIGNATURE----- --------------enig77A482A7EB6C48346355F6F1--