From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47129) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V2IFZ-0003xQ-1q for qemu-devel@nongnu.org; Thu, 25 Jul 2013 05:54:26 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1V2IFW-0002GF-Pv for qemu-devel@nongnu.org; Thu, 25 Jul 2013 05:54:20 -0400 Received: from mx1.redhat.com ([209.132.183.28]:30057) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V2IFW-0002Fv-3q for qemu-devel@nongnu.org; Thu, 25 Jul 2013 05:54:18 -0400 Date: Thu, 25 Jul 2013 11:54:09 +0200 From: Igor Mammedov Message-ID: <20130725115409.12db4fdd@nial.usersys.redhat.com> In-Reply-To: <51F0E45A.4030407@suse.de> References: <1374733640-9350-1-git-send-email-chen.fan.fnst@cn.fujitsu.com> <51F0E45A.4030407@suse.de> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [PATCH 1/1] cpu: Correct cpu-hotplug failure List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Andreas =?ISO-8859-1?B?RuRyYmVy?= Cc: Chen Fan , qemu-devel@nongnu.org, Eduardo Habkost On Thu, 25 Jul 2013 10:39:54 +0200 Andreas F=E4rber wrote: > Hi, >=20 > Am 25.07.2013 08:27, schrieb Chen Fan: > > When useing x86_64-softmmu --enable-kvm boot qemu, cpu-add command fail= s to add a vcpu, > > there show (KVM: setting VAPIC address failed). > >=20 > > The reason is that we use an uninitialized cpu->kvm-fd to ioctl. > > so we move realizing apic to the back of qemu_init_vcpu. > >=20 > > Signed-off-by: Chen Fan > > --- > > include/qom/cpu.h | 2 ++ > > qom/cpu.c | 13 +++++++++++++ > > target-i386/cpu.c | 10 ++++------ > > 3 files changed, 19 insertions(+), 6 deletions(-) >=20 > First, please CC the maintainer - in this case me. >=20 > The referenced commit is this one: > http://git.qemu.org/?p=3Dqemu.git;a=3Dcommit;h=3Dc643bed99 >=20 > Sorry for regressing. >=20 > > diff --git a/include/qom/cpu.h b/include/qom/cpu.h > > index daf1835..487a808 100644 > > --- a/include/qom/cpu.h > > +++ b/include/qom/cpu.h > > @@ -80,6 +80,7 @@ struct TranslationBlock; > > * @synchronize_from_tb: Callback for synchronizing state from a TCG > > * #TranslationBlock. > > * @get_phys_page_debug: Callback for obtaining a physical address. > > + * @apic_realize: Callback for realizing apic. > > * @vmsd: State description for migration. > > * > > * Represents a CPU family or model. > > @@ -108,6 +109,7 @@ typedef struct CPUClass { > > void (*set_pc)(CPUState *cpu, vaddr value); > > void (*synchronize_from_tb)(CPUState *cpu, struct TranslationBlock= *tb); > > hwaddr (*get_phys_page_debug)(CPUState *cpu, vaddr addr); > > + void (*apic_realize)(CPUState *cpu, Error **errp); > > =20 > > const struct VMStateDescription *vmsd; > > int (*write_elf64_note)(WriteCoreDumpFunction f, CPUState *cpu, > > diff --git a/qom/cpu.c b/qom/cpu.c > > index 5c45ab5..88c6028 100644 > > --- a/qom/cpu.c > > +++ b/qom/cpu.c > > @@ -213,12 +213,25 @@ static ObjectClass *cpu_common_class_by_name(cons= t char *cpu_model) > > return NULL; > > } > > =20 > > +static void cpu_apic_realize(CPUState *cpu, Error **errp) > > +{ > > + CPUClass *cc =3D CPU_GET_CLASS(cpu); > > + if (cc->apic_realize !=3D NULL) { > > + (*cc->apic_realize)(cpu, errp); > > + } > > +} > > + > > static void cpu_common_realizefn(DeviceState *dev, Error **errp) > > { > > CPUState *cpu =3D CPU(dev); > > =20 > > qemu_init_vcpu(cpu); > > =20 > > + cpu_apic_realize(cpu, errp); > > + if (error_is_set(errp)) { > > + return; > > + } > > + > > if (dev->hotplugged) { > > cpu_synchronize_post_init(cpu); > > notifier_list_notify(&cpu_added_notifiers, dev); >=20 > This is bogus since APIC is an x86 thing. It should stay in > target-i386/cpu.c. We could generalize it as a post-vCPU-init hook, but > I don't think that's needed here: >=20 > > diff --git a/target-i386/cpu.c b/target-i386/cpu.c > > index cd350cb..916d69e 100644 > > --- a/target-i386/cpu.c > > +++ b/target-i386/cpu.c > > @@ -2311,8 +2311,9 @@ static void x86_cpu_apic_create(X86CPU *cpu, Erro= r **errp) > > apic->cpu =3D cpu; > > } > > =20 > > -static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp) > > +static void x86_cpu_apic_realize(CPUState *s, Error **errp) > > { > > + X86CPU *cpu =3D X86_CPU(s); > > CPUX86State *env =3D &cpu->env; > > =20 > > if (env->apic_state =3D=3D NULL) { > > @@ -2326,7 +2327,7 @@ static void x86_cpu_apic_realize(X86CPU *cpu, Err= or **errp) > > } > > } > > #else > > -static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp) > > +static void x86_cpu_apic_realize(CPUState *s, Error **errp) > > { > > } > > #endif > > @@ -2388,10 +2389,6 @@ static void x86_cpu_realizefn(DeviceState *dev, = Error **errp) > > =20 > > mce_init(cpu); > > =20 > > - x86_cpu_apic_realize(cpu, &local_err); > > - if (local_err !=3D NULL) { > > - goto out; > > - } > > cpu_reset(CPU(cpu)); > > =20 > > xcc->parent_realize(dev, &local_err); >=20 > I think all that really needs to happen is to move the above four lines > to below xcc->parent_realize(). I doubt that a hot-add notifier is going > to fiddle with the APIC - we're just incrementing the number of CPUs in > the RTC today. that would push uninitialized apic state into KVM cpu_common_realizefn -> cpu_synchronize_post_init -> kvm_cpu_synchronize_post_init -> kvm_arch_put_registers -> kvm_put_apic -> kvm_put_apic_state kvm_vcpu_ioctl(CPU(cpu), KVM_SET_LAPIC, &kapic); which becomes initialized only after x86_cpu_apic_realize(). It works for startup CPUs only because system wide reset which calls cpu_synchronize_post_reset->...->kvm_put_apic_state() and qemu_system_reset->...->apic_reset_common->kvm_apic_vapic_base_update() in hotplug case flow was: x86_cpu_apic_realize->...->device_set_realized->realize() ->device_reset() ->apic_reset_common->kvm_a= pic_vapic_base_update() cpu_common_realizefn -> cpu_synchronize_post_init -> kvm_cpu_synchronize_post_init -> kvm_arch_put_registers -> kvm_put_apic =20 > Can you send a v2 doing so please? Thanks. >=20 > Regards, > Andreas >=20 > > @@ -2540,6 +2537,7 @@ static void x86_cpu_common_class_init(ObjectClass= *oc, void *data) > > cc->synchronize_from_tb =3D x86_cpu_synchronize_from_tb; > > cc->get_arch_id =3D x86_cpu_get_arch_id; > > cc->get_paging_enabled =3D x86_cpu_get_paging_enabled; > > + cc->apic_realize =3D x86_cpu_apic_realize; > > #ifndef CONFIG_USER_ONLY > > cc->get_memory_mapping =3D x86_cpu_get_memory_mapping; > > cc->get_phys_page_debug =3D x86_cpu_get_phys_page_debug; > >=20 >=20 >=20