* [patch 1/5] QEMU/KVM: properly copy the in-kernel apicbase value
2008-04-11 16:24 [patch 0/5] fix SMP migration and loadvm/savevm (V3) Marcelo Tosatti
@ 2008-04-11 16:24 ` Marcelo Tosatti
2008-04-11 16:24 ` [patch 2/5] QEMU/KVM: Add libkvm interface to get/set the mpstate Marcelo Tosatti
` (4 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Marcelo Tosatti @ 2008-04-11 16:24 UTC (permalink / raw)
To: Avi Kivity; +Cc: kvm-devel, Marcelo Tosatti
[-- Attachment #1: apicbase --]
[-- Type: text/plain, Size: 2125 bytes --]
The MSR_IA32_APICBASE_ENABLE/MSR_IA32_APICBASE_BSP bits in s->apicbase
are not initialized if in-kernel APIC emulation is used, so save the
actual value passed by cpu_set_apic_base() caller.
Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
Index: kvm-userspace.io/qemu/hw/apic.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/apic.c
+++ kvm-userspace.io/qemu/hw/apic.c
@@ -248,8 +248,11 @@ void cpu_set_apic_base(CPUState *env, ui
#ifdef DEBUG_APIC
printf("cpu_set_apic_base: %016" PRIx64 "\n", val);
#endif
- s->apicbase = (val & 0xfffff000) |
- (s->apicbase & (MSR_IA32_APICBASE_BSP | MSR_IA32_APICBASE_ENABLE));
+ if (kvm_enabled() && qemu_kvm_irqchip_in_kernel())
+ s->apicbase = val;
+ else
+ s->apicbase = (val & 0xfffff000) |
+ (s->apicbase & (MSR_IA32_APICBASE_BSP | MSR_IA32_APICBASE_ENABLE));
/* if disabled, cannot be enabled again */
if (!(val & MSR_IA32_APICBASE_ENABLE)) {
s->apicbase &= ~MSR_IA32_APICBASE_ENABLE;
Index: kvm-userspace.io/qemu/qemu-kvm-x86.c
===================================================================
--- kvm-userspace.io.orig/qemu/qemu-kvm-x86.c
+++ kvm-userspace.io/qemu/qemu-kvm-x86.c
@@ -248,13 +248,8 @@ void kvm_arch_load_regs(CPUState *env)
sregs.cr3 = env->cr[3];
sregs.cr4 = env->cr[4];
- if (kvm_irqchip_in_kernel(kvm_context)) {
- sregs.cr8 = kvm_get_cr8(kvm_context, env->cpu_index);
- sregs.apic_base = kvm_get_apic_base(kvm_context, env->cpu_index);
- } else {
- sregs.cr8 = cpu_get_apic_tpr(env);
- sregs.apic_base = cpu_get_apic_base(env);
- }
+ sregs.cr8 = cpu_get_apic_tpr(env);
+ sregs.apic_base = cpu_get_apic_base(env);
sregs.efer = env->efer;
--
-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
^ permalink raw reply [flat|nested] 8+ messages in thread* [patch 2/5] QEMU/KVM: Add libkvm interface to get/set the mpstate.
2008-04-11 16:24 [patch 0/5] fix SMP migration and loadvm/savevm (V3) Marcelo Tosatti
2008-04-11 16:24 ` [patch 1/5] QEMU/KVM: properly copy the in-kernel apicbase value Marcelo Tosatti
@ 2008-04-11 16:24 ` Marcelo Tosatti
2008-04-11 16:24 ` [patch 3/5] QEMU/KVM: save and load mp state Marcelo Tosatti
` (3 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Marcelo Tosatti @ 2008-04-11 16:24 UTC (permalink / raw)
To: Avi Kivity; +Cc: kvm-devel, Marcelo Tosatti
[-- Attachment #1: smpmig-libkvm --]
[-- Type: text/plain, Size: 2126 bytes --]
Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
Index: kvm-userspace.io/libkvm/libkvm.c
===================================================================
--- kvm-userspace.io.orig/libkvm/libkvm.c
+++ kvm-userspace.io/libkvm/libkvm.c
@@ -776,6 +776,28 @@ int kvm_set_sregs(kvm_context_t kvm, int
return ioctl(kvm->vcpu_fd[vcpu], KVM_SET_SREGS, sregs);
}
+#ifdef KVM_CAP_MP_STATE
+int kvm_get_mpstate(kvm_context_t kvm, int vcpu, struct kvm_mp_state *mp_state)
+{
+ int r;
+
+ r = ioctl(kvm->fd, KVM_CHECK_EXTENSION, KVM_CAP_MP_STATE);
+ if (r > 0)
+ return ioctl(kvm->vcpu_fd[vcpu], KVM_GET_MP_STATE, mp_state);
+ return -ENOSYS;
+}
+
+int kvm_set_mpstate(kvm_context_t kvm, int vcpu, struct kvm_mp_state *mp_state)
+{
+ int r;
+
+ r = ioctl(kvm->fd, KVM_CHECK_EXTENSION, KVM_CAP_MP_STATE);
+ if (r > 0)
+ return ioctl(kvm->vcpu_fd[vcpu], KVM_SET_MP_STATE, mp_state);
+ return -ENOSYS;
+}
+#endif
+
static int handle_mmio(kvm_context_t kvm, struct kvm_run *kvm_run)
{
unsigned long addr = kvm_run->mmio.phys_addr;
Index: kvm-userspace.io/libkvm/libkvm.h
===================================================================
--- kvm-userspace.io.orig/libkvm/libkvm.h
+++ kvm-userspace.io/libkvm/libkvm.h
@@ -300,6 +300,22 @@ int kvm_get_sregs(kvm_context_t kvm, int
*/
int kvm_set_sregs(kvm_context_t kvm, int vcpu, struct kvm_sregs *regs);
+#ifdef KVM_CAP_MP_STATE
+/*!
+ * * \brief Read VCPU MP state
+ *
+ */
+int kvm_get_mpstate(kvm_context_t kvm, int vcpu,
+ struct kvm_mp_state *mp_state);
+
+/*!
+ * * \brief Write VCPU MP state
+ *
+ */
+int kvm_set_mpstate(kvm_context_t kvm, int vcpu,
+ struct kvm_mp_state *mp_state);
+#endif
+
/*!
* \brief Simulate an external vectored interrupt
*
--
-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
^ permalink raw reply [flat|nested] 8+ messages in thread* [patch 3/5] QEMU/KVM: save and load mp state
2008-04-11 16:24 [patch 0/5] fix SMP migration and loadvm/savevm (V3) Marcelo Tosatti
2008-04-11 16:24 ` [patch 1/5] QEMU/KVM: properly copy the in-kernel apicbase value Marcelo Tosatti
2008-04-11 16:24 ` [patch 2/5] QEMU/KVM: Add libkvm interface to get/set the mpstate Marcelo Tosatti
@ 2008-04-11 16:24 ` Marcelo Tosatti
2008-04-11 16:24 ` [patch 4/5] QEMU/KVM: ignore SIG_IPI signals in userspace Marcelo Tosatti
` (2 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Marcelo Tosatti @ 2008-04-11 16:24 UTC (permalink / raw)
To: Avi Kivity; +Cc: kvm-devel, Marcelo Tosatti
[-- Attachment #1: smpmig-qemu --]
[-- Type: text/plain, Size: 4437 bytes --]
Use the new interface to save and restore MP_STATE for all vcpu's.
Increase version_id for cpu_load/cpu_save.
Fixes SMP migration.
Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
Index: kvm-userspace.io/qemu/hw/pc.c
===================================================================
--- kvm-userspace.io.orig/qemu/hw/pc.c
+++ kvm-userspace.io/qemu/hw/pc.c
@@ -750,7 +750,7 @@ CPUState *pc_new_cpu(int cpu, const char
/* XXX: enable it in all cases */
env->cpuid_features |= CPUID_APIC;
}
- register_savevm("cpu", cpu, 4, cpu_save, cpu_load, env);
+ register_savevm("cpu", cpu, 5, cpu_save, cpu_load, env);
qemu_register_reset(main_cpu_reset, env);
if (pci_enabled) {
apic_init(env);
Index: kvm-userspace.io/qemu/qemu-kvm-x86.c
===================================================================
--- kvm-userspace.io.orig/qemu/qemu-kvm-x86.c
+++ kvm-userspace.io/qemu/qemu-kvm-x86.c
@@ -277,6 +277,33 @@ void kvm_arch_load_regs(CPUState *env)
perror("kvm_set_msrs FAILED");
}
+void kvm_save_mpstate(CPUState *env)
+{
+#ifdef KVM_CAP_MP_STATE
+ int r;
+ struct kvm_mp_state mp_state;
+
+ r = kvm_get_mpstate(kvm_context, env->cpu_index, &mp_state);
+ if (r < 0)
+ env->mp_state = -1;
+ else
+ env->mp_state = mp_state.mp_state;
+#endif
+}
+
+void kvm_load_mpstate(CPUState *env)
+{
+#ifdef KVM_CAP_MP_STATE
+ struct kvm_mp_state mp_state = { .mp_state = env->mp_state };
+
+ /*
+ * -1 indicates that the host did not support GET_MP_STATE ioctl,
+ * so don't touch it.
+ */
+ if (env->mp_state != -1)
+ kvm_set_mpstate(kvm_context, env->cpu_index, &mp_state);
+#endif
+}
void kvm_arch_save_regs(CPUState *env)
{
Index: kvm-userspace.io/qemu/qemu-kvm.h
===================================================================
--- kvm-userspace.io.orig/qemu/qemu-kvm.h
+++ kvm-userspace.io/qemu/qemu-kvm.h
@@ -18,6 +18,8 @@ int kvm_init_ap(void);
void kvm_qemu_destroy(void);
void kvm_load_registers(CPUState *env);
void kvm_save_registers(CPUState *env);
+void kvm_load_mpstate(CPUState *env);
+void kvm_save_mpstate(CPUState *env);
int kvm_cpu_exec(CPUState *env);
int kvm_update_debugger(CPUState *env);
int kvm_qemu_init_env(CPUState *env);
Index: kvm-userspace.io/qemu/target-i386/cpu.h
===================================================================
--- kvm-userspace.io.orig/qemu/target-i386/cpu.h
+++ kvm-userspace.io/qemu/target-i386/cpu.h
@@ -599,6 +599,7 @@ typedef struct CPUX86State {
/* in order to simplify APIC support, we leave this pointer to the
user */
struct APICState *apic_state;
+ int mp_state;
} CPUX86State;
CPUX86State *cpu_x86_init(const char *cpu_model);
Index: kvm-userspace.io/qemu/vl.c
===================================================================
--- kvm-userspace.io.orig/qemu/vl.c
+++ kvm-userspace.io/qemu/vl.c
@@ -6655,8 +6655,10 @@ void cpu_save(QEMUFile *f, void *opaque)
uint32_t hflags;
int i;
- if (kvm_enabled())
+ if (kvm_enabled()) {
kvm_save_registers(env);
+ kvm_save_mpstate(env);
+ }
for(i = 0; i < CPU_NB_REGS; i++)
qemu_put_betls(f, &env->regs[i]);
@@ -6748,6 +6750,7 @@ void cpu_save(QEMUFile *f, void *opaque)
qemu_put_be32s(f, &env->kvm_interrupt_bitmap[i]);
}
qemu_put_be64s(f, &env->tsc);
+ qemu_put_be32s(f, &env->mp_state);
}
}
@@ -6782,7 +6785,7 @@ int cpu_load(QEMUFile *f, void *opaque,
uint32_t hflags;
uint16_t fpus, fpuc, fptag, fpregs_format;
- if (version_id != 3 && version_id != 4)
+ if (version_id < 3 || version_id > 5)
return -EINVAL;
for(i = 0; i < CPU_NB_REGS; i++)
qemu_get_betls(f, &env->regs[i]);
@@ -6900,6 +6903,10 @@ int cpu_load(QEMUFile *f, void *opaque,
}
qemu_get_be64s(f, &env->tsc);
kvm_load_registers(env);
+ if (version_id >= 5) {
+ qemu_get_be32s(f, &env->mp_state);
+ kvm_load_mpstate(env);
+ }
}
return 0;
}
--
-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
^ permalink raw reply [flat|nested] 8+ messages in thread* [patch 4/5] QEMU/KVM: ignore SIG_IPI signals in userspace
2008-04-11 16:24 [patch 0/5] fix SMP migration and loadvm/savevm (V3) Marcelo Tosatti
` (2 preceding siblings ...)
2008-04-11 16:24 ` [patch 3/5] QEMU/KVM: save and load mp state Marcelo Tosatti
@ 2008-04-11 16:24 ` Marcelo Tosatti
2008-04-11 16:24 ` [patch 5/5] KVM: add ioctls to save/store mpstate Marcelo Tosatti
2008-04-13 15:01 ` [patch 0/5] fix SMP migration and loadvm/savevm (V3) Avi Kivity
5 siblings, 0 replies; 8+ messages in thread
From: Marcelo Tosatti @ 2008-04-11 16:24 UTC (permalink / raw)
To: Avi Kivity; +Cc: kvm-devel, Marcelo Tosatti
[-- Attachment #1: ignore-sigipi --]
[-- Type: text/plain, Size: 997 bytes --]
Otherwise a signal can be received in userspace and a vcpu goes back
to the kernel while it should stay still.
Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
Index: kvm-userspace.io/qemu/qemu-kvm.c
===================================================================
--- kvm-userspace.io.orig/qemu/qemu-kvm.c
+++ kvm-userspace.io/qemu/qemu-kvm.c
@@ -350,7 +350,6 @@ static void *ap_main_loop(void *_env)
vcpu->env = env;
vcpu->env->thread_id = kvm_get_thread_id();
sigfillset(&signals);
- sigdelset(&signals, SIG_IPI);
sigprocmask(SIG_BLOCK, &signals, NULL);
kvm_create_vcpu(kvm_context, env->cpu_index);
kvm_qemu_init_env(env);
--
-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
^ permalink raw reply [flat|nested] 8+ messages in thread* [patch 5/5] KVM: add ioctls to save/store mpstate
2008-04-11 16:24 [patch 0/5] fix SMP migration and loadvm/savevm (V3) Marcelo Tosatti
` (3 preceding siblings ...)
2008-04-11 16:24 ` [patch 4/5] QEMU/KVM: ignore SIG_IPI signals in userspace Marcelo Tosatti
@ 2008-04-11 16:24 ` Marcelo Tosatti
2008-04-13 14:59 ` Avi Kivity
2008-04-13 15:01 ` [patch 0/5] fix SMP migration and loadvm/savevm (V3) Avi Kivity
5 siblings, 1 reply; 8+ messages in thread
From: Marcelo Tosatti @ 2008-04-11 16:24 UTC (permalink / raw)
To: Avi Kivity; +Cc: kvm-devel, Marcelo Tosatti
[-- Attachment #1: smp-mig-kernel.patch --]
[-- Type: text/plain, Size: 3806 bytes --]
So userspace can save/restore the mpstate during migration.
Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
Index: kvm/arch/x86/kvm/x86.c
===================================================================
--- kvm.orig/arch/x86/kvm/x86.c
+++ kvm/arch/x86/kvm/x86.c
@@ -814,6 +814,7 @@ int kvm_dev_ioctl_check_extension(long e
case KVM_CAP_CLOCKSOURCE:
case KVM_CAP_PIT:
case KVM_CAP_NOP_IO_DELAY:
+ case KVM_CAP_MP_STATE:
r = 1;
break;
case KVM_CAP_VAPIC:
@@ -3057,6 +3058,24 @@ int kvm_arch_vcpu_ioctl_get_sregs(struct
return 0;
}
+int kvm_arch_vcpu_ioctl_get_mpstate(struct kvm_vcpu *vcpu,
+ struct kvm_mp_state *mp_state)
+{
+ vcpu_load(vcpu);
+ mp_state->mp_state = vcpu->arch.mp_state;
+ vcpu_put(vcpu);
+ return 0;
+}
+
+int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
+ struct kvm_mp_state *mp_state)
+{
+ vcpu_load(vcpu);
+ vcpu->arch.mp_state = mp_state->mp_state;
+ vcpu_put(vcpu);
+ return 0;
+}
+
static void set_segment(struct kvm_vcpu *vcpu,
struct kvm_segment *var, int seg)
{
Index: kvm/include/linux/kvm.h
===================================================================
--- kvm.orig/include/linux/kvm.h
+++ kvm/include/linux/kvm.h
@@ -222,6 +222,11 @@ struct kvm_vapic_addr {
__u64 vapic_addr;
};
+/* for KVM_SET_MPSTATE */
+struct kvm_mp_state {
+ __u32 mp_state;
+};
+
struct kvm_s390_psw {
__u64 mask;
__u64 addr;
@@ -279,6 +284,7 @@ struct kvm_s390_interrupt {
#define KVM_CAP_PIT 11
#define KVM_CAP_NOP_IO_DELAY 12
#define KVM_CAP_PV_MMU 13
+#define KVM_CAP_MP_STATE 14
/*
* ioctls for VM fds
@@ -340,5 +346,7 @@ struct kvm_s390_interrupt {
#define KVM_S390_SET_INITIAL_PSW _IOW(KVMIO, 0x96, struct kvm_s390_psw)
/* initial reset for s390 */
#define KVM_S390_INITIAL_RESET _IO(KVMIO, 0x97)
+#define KVM_GET_MP_STATE _IOR(KVMIO, 0x98, struct kvm_mp_state)
+#define KVM_SET_MP_STATE _IOW(KVMIO, 0x99, struct kvm_mp_state)
#endif
Index: kvm/include/linux/kvm_host.h
===================================================================
--- kvm.orig/include/linux/kvm_host.h
+++ kvm/include/linux/kvm_host.h
@@ -236,6 +236,10 @@ int kvm_arch_vcpu_ioctl_get_sregs(struct
struct kvm_sregs *sregs);
int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,
struct kvm_sregs *sregs);
+int kvm_arch_vcpu_ioctl_get_mpstate(struct kvm_vcpu *vcpu,
+ struct kvm_mp_state *mp_state);
+int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
+ struct kvm_mp_state *mp_state);
int kvm_arch_vcpu_ioctl_debug_guest(struct kvm_vcpu *vcpu,
struct kvm_debug_guest *dbg);
int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run);
Index: kvm/virt/kvm/kvm_main.c
===================================================================
--- kvm.orig/virt/kvm/kvm_main.c
+++ kvm/virt/kvm/kvm_main.c
@@ -978,6 +978,30 @@ out_free2:
r = 0;
break;
}
+ case KVM_GET_MP_STATE: {
+ struct kvm_mp_state mp_state;
+
+ r = kvm_arch_vcpu_ioctl_get_mpstate(vcpu, &mp_state);
+ if (r)
+ goto out;
+ r = -EFAULT;
+ if (copy_to_user(argp, &mp_state, sizeof mp_state))
+ goto out;
+ r = 0;
+ break;
+ }
+ case KVM_SET_MP_STATE: {
+ struct kvm_mp_state mp_state;
+
+ r = -EFAULT;
+ if (copy_from_user(&mp_state, argp, sizeof mp_state))
+ goto out;
+ r = kvm_arch_vcpu_ioctl_set_mpstate(vcpu, &mp_state);
+ if (r)
+ goto out;
+ r = 0;
+ break;
+ }
case KVM_TRANSLATE: {
struct kvm_translation tr;
--
-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [patch 0/5] fix SMP migration and loadvm/savevm (V3)
2008-04-11 16:24 [patch 0/5] fix SMP migration and loadvm/savevm (V3) Marcelo Tosatti
` (4 preceding siblings ...)
2008-04-11 16:24 ` [patch 5/5] KVM: add ioctls to save/store mpstate Marcelo Tosatti
@ 2008-04-13 15:01 ` Avi Kivity
5 siblings, 0 replies; 8+ messages in thread
From: Avi Kivity @ 2008-04-13 15:01 UTC (permalink / raw)
To: Marcelo Tosatti; +Cc: kvm-devel
Marcelo Tosatti wrote:
> Address comments from Avi:
>
> - fold mp_state into a structure
> - IOW/IOR markers for ioctl definitions
> - return -ENOSYS if extension is not supported
>
>
Applied all, thanks.
--
error compiling committee.c: too many arguments to function
-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
^ permalink raw reply [flat|nested] 8+ messages in thread