APIC patch

APIC patch

[Dong, Eddie]

Greg/Dor:
	When thinking to merge APIC device model with PIC, a curious
question: Where does the code base come from? Xen or Qemu? I found there
are many diff from Xen (even excluding name difference). 
Eddie

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: APIC patch

[Gregory Haskins]

On Tue, 2007-06-26 at 11:02 +0800, Dong, Eddie wrote:
> Greg/Dor:
> 	When thinking to merge APIC device model with PIC, a curious
> question: Where does the code base come from? Xen or Qemu? I found there
> are many diff from Xen (even excluding name difference). 
> Eddie

I am fairly certain it originated in Xen (which might have originated in
Qemu??) but there have been many many changes since then.  I know Dor at
least did all the porting to Linux (including the hrtimer work).  I then
ported it to the irqdevice model and reworked a good amount of the code.
Between the two I am sure you will see lots of diffs from the original
Xen code.

-Greg


-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: APIC patch

[Dor Laor]

[-- Attachment #1: Type: text/plain, Size: 1358 bytes --]

>On Tue, 2007-06-26 at 11:02 +0800, Dong, Eddie wrote:
>> Greg/Dor:
>> 	When thinking to merge APIC device model with PIC, a curious
>> question: Where does the code base come from? Xen or Qemu? I found
there
>> are many diff from Xen (even excluding name difference).
>> Eddie
>
>I am fairly certain it originated in Xen (which might have originated
in
>Qemu??) but there have been many many changes since then.  I know Dor
at
>least did all the porting to Linux (including the hrtimer work).  I
then
>ported it to the irqdevice model and reworked a good amount of the
code.
>Between the two I am sure you will see lots of diffs from the original
>Xen code.
>
>-Greg

As Greg mentioned, the base code (lapic.c) is Xen based. I also stated
it clearly in file header.

Regarding merging it with mainline, if the AMD problems are over (as I
think they are) we can do it. There is one point regarding migration,
currently the apic state is not saved, it should be fixed. The irqdev
state should be also serialized. 
Avi should say his final word and he is lecturing in the OLS convention.

Oh, I just discovered old code of mine that handles apic save/restore.
I'm attaching the patch, its old and won't apply on the current sources
but it worked in the old days...

Btw, you guys are doing fantastic, important job. Kudos ;)

[-- Attachment #2: apic_save_restore_and_install.patch --]
[-- Type: application/octet-stream, Size: 20593 bytes --]

Index: kernel/kvm_apic.h
===================================================================
--- kernel/kvm_apic.h	(revision 3947)
+++ kernel/kvm_apic.h	(working copy)
@@ -131,5 +131,8 @@
 unsigned long kvm_apic_read_tpr(struct kvm_apic* apic);
 void kvm_apic_update_tpr(struct kvm_apic *apic, unsigned long cr8);
 int kvm_apic_receive_msg(struct kvm_vcpu *vcpu, struct kvm_apic_msg *msg);
+int kvm_apic_reset(struct kvm_apic *apic);
 
+int kvm_apic_get(struct kvm_apic *apic, struct kvm_apic_state* as);
+int kvm_apic_set(struct kvm_apic *apic, struct kvm_apic_state* as);
 #endif /* __KVM_APIC_H__ */
Index: kernel/kvm_main.c
===================================================================
--- kernel/kvm_main.c	(revision 3947)
+++ kernel/kvm_main.c	(working copy)
@@ -3113,6 +3113,7 @@
 
 	memcpy(sregs->interrupt_bitmap, vcpu->irq_pending,
 	       sizeof sregs->interrupt_bitmap);
+	sregs->pending_int = kvm_cpu_has_pending_irq(vcpu);
 
 	vcpu_put(vcpu);
 
@@ -3388,7 +3389,57 @@
 	return 0;
 }
 
+static int kvm_dev_ioctl_apic_reset(struct kvm *kvm, struct kvm_apic_reset *msg)
+{
+	struct kvm_vcpu *vcpu;
 
+	if (msg->vcpu < 0 || msg->vcpu >= KVM_MAX_VCPUS)
+		return -EINVAL;
+	vcpu = vcpu_load(kvm, msg->vcpu);
+	if (!vcpu)
+		return -ENOENT;
+
+	kvm_apic_reset(&vcpu->apic);
+
+	vcpu_put(vcpu);
+
+	return 0;
+}
+
+static int kvm_dev_ioctl_apic_set(struct kvm *kvm, struct kvm_apic_state *as)
+{
+	struct kvm_vcpu *vcpu;
+
+	if (as->vcpu < 0 || as->vcpu >= KVM_MAX_VCPUS)
+		return -EINVAL;
+	vcpu = vcpu_load(kvm, as->vcpu);
+	if (!vcpu)
+		return -ENOENT;
+
+	kvm_apic_set(&vcpu->apic, as);
+
+	vcpu_put(vcpu);
+
+	return 0;
+}
+
+static int kvm_dev_ioctl_apic_get(struct kvm *kvm, struct kvm_apic_state *as)
+{
+	struct kvm_vcpu *vcpu;
+
+	if (as->vcpu < 0 || as->vcpu >= KVM_MAX_VCPUS)
+		return -EINVAL;
+	vcpu = vcpu_load(kvm, as->vcpu);
+	if (!vcpu)
+		return -ENOENT;
+
+	kvm_apic_get(&vcpu->apic, as);
+
+	vcpu_put(vcpu);
+
+	return 0;
+}
+
 static int kvm_dev_ioctl_debug_guest(struct kvm *kvm,
 				     struct kvm_debug_guest *dbg)
 {
@@ -3564,6 +3615,48 @@
 		r = 0;
 		break;
 	}
+	case KVM_APIC_RESET: {
+		struct kvm_apic_reset msg;
+
+		r = -EFAULT;
+		if (copy_from_user(&msg, (void *)arg, sizeof msg))
+			goto out;
+		
+		r = kvm_dev_ioctl_apic_reset(kvm, &msg);
+		if (r)
+			goto out;
+		r = 0;
+		break;
+	}
+	case KVM_APIC_SET: {
+		struct kvm_apic_state msg;
+
+		r = -EFAULT;
+		if (copy_from_user(&msg, (void *)arg, sizeof msg))
+			goto out;
+		
+		r = kvm_dev_ioctl_apic_set(kvm, &msg);
+		if (r)
+			goto out;
+		r = 0;
+		break;
+	}
+	case KVM_APIC_GET: {
+		struct kvm_apic_state msg;
+
+		r = -EFAULT;
+		if (copy_from_user(&msg, (void *)arg, sizeof msg))
+			goto out;
+		
+		r = kvm_dev_ioctl_apic_get(kvm, &msg);
+		if (r)
+			goto out;
+		r = -EFAULT;
+		if (copy_to_user((void *)arg, &msg, sizeof msg))
+			goto out;
+		r = 0;
+		break;
+	}
 	case KVM_DEBUG_GUEST: {
 		struct kvm_debug_guest dbg;
 
Index: kernel/include/linux/kvm.h
===================================================================
--- kernel/include/linux/kvm.h	(revision 3947)
+++ kernel/include/linux/kvm.h	(working copy)
@@ -129,7 +129,7 @@
 struct kvm_sregs {
 	/* in */
 	__u32 vcpu;
-	__u32 padding;
+	__u32 pending_int;
 
 	/* out (KVM_GET_SREGS) / in (KVM_SET_SREGS) */
 	struct kvm_segment cs, ds, es, fs, gs, ss;
@@ -221,7 +221,41 @@
 	__u32 padding;
 };
 
+/* for KVM_APIC_RESET */
+struct kvm_apic_reset {
+	/* in */
+	__u32 vcpu;
+};
 
+/* for KVM_SAVE/LOAD_APIC */
+struct kvm_apic_state {
+	/* in/out */
+	__u64 apicbase;
+	__u64 irr[KVM_IRQ_BITMAP_SIZE(__u64)];
+	__u64 isr[KVM_IRQ_BITMAP_SIZE(__u64)];
+	__u64 tmr[KVM_IRQ_BITMAP_SIZE(__u64)];
+	__u32 vcpu; /* in only */
+	__u32 status;
+	__u32 ldr;
+	__u32 dfr;
+	__u32 id;
+	__u32 arb_id;
+	__u32 tpr;
+	__u32 ppr;
+	__u32 spurious_vec;
+	__u32 lvterr;
+	__u32 lvt0;
+	__u32 lvt1;
+	__u32 lvtpc;
+	__u32 lvtthmr;
+	__u32 lvtt;
+	__u32 esr; 
+	__u32 icr[2];
+        __u32 divide_conf;
+	__u32 initial_count;
+	__u32 padding;
+};
+
 #define KVMIO 0xAE
 
 #define KVM_RUN                   _IOWR(KVMIO, 2, struct kvm_run)
@@ -239,6 +273,9 @@
 #define KVM_SET_MSRS              _IOWR(KVMIO, 14, struct kvm_msrs)
 #define KVM_GET_MSR_INDEX_LIST    _IOWR(KVMIO, 15, struct kvm_msr_list)
 #define KVM_APIC_MSG		  _IOW(KVMIO, 16, struct kvm_apic_msg)
+#define KVM_APIC_RESET		  _IOW(KVMIO, 17, struct kvm_apic_reset)
+#define KVM_APIC_GET		  _IOR(KVMIO, 18, struct kvm_apic_state)
+#define KVM_APIC_SET		  _IOW(KVMIO, 19, struct kvm_apic_state)
 #define KVM_DUMP_VCPU             _IOW(KVMIO, 250, int /* vcpu_slot */)
 
 #endif
Index: kernel/kvm_apic.c
===================================================================
--- kernel/kvm_apic.c	(revision 3947)
+++ kernel/kvm_apic.c	(working copy)
@@ -254,7 +254,7 @@
 
 	case APIC_DM_INIT:
 		if (trig_mode && !(level & APIC_INT_ASSERT))     //Deassert
-			printk(KERN_WARNING "This kvm_apic is for P4, no work for De-assert init\n");
+			printk(KERN_INFO "This kvm_apic is for P4, no work for De-assert init\n");
 		else {
 			/* FIXME(xen) How to check the situation after vcpu reset? */
 			if (vcpu->launched) {
@@ -897,11 +897,126 @@
 	}
 }
 
-static int apic_reset(struct kvm_apic *apic)
+static void kvm_apic_dump_state(struct kvm_apic *apic)
 {
+	u64 *tmp;
+
+	printk(KERN_INFO "%s begin\n", __FUNCTION__);
+	
+	printk(KERN_INFO "status = 0x%08x\n", apic->status);
+	printk(KERN_INFO "apic_base_msr=0x%016llx, apicbase = 0x%08lx\n", apic->apic_base_msr, apic->base_address);
+	
+        tmp = (u64*)(apic->regs + APIC_IRR);
+	printk(KERN_INFO "IRR = 0x%016llx 0x%016llx 0x%016llx 0x%016llx\n", tmp[3], tmp[2], tmp[1], tmp[0]);
+	tmp = (u64*)(apic->regs + APIC_ISR);
+	printk(KERN_INFO "ISR = 0x%016llx 0x%016llx 0x%016llx 0x%016llx\n", tmp[3], tmp[2], tmp[1], tmp[0]);
+	tmp = (u64*)(apic->regs + APIC_TMR);
+	printk(KERN_INFO "TMR = 0x%016llx 0x%016llx 0x%016llx 0x%016llx\n", tmp[3], tmp[2], tmp[1], tmp[0]);
+
+	printk(KERN_INFO "APIC_ID=0x%08x\n", kvm_apic_get_reg(apic, APIC_ID));
+	printk(KERN_INFO "APIC_TASKPRI=0x%08x\n", kvm_apic_get_reg(apic, APIC_TASKPRI) & 0xff);
+	printk(KERN_INFO "APIC_PROCPRI=0x%08x\n", kvm_apic_get_reg(apic, APIC_PROCPRI));
+	
+	printk(KERN_INFO "APIC_DFR=0x%08x\n", kvm_apic_get_reg(apic, APIC_DFR) | 0x0FFFFFFF);
+	printk(KERN_INFO "APIC_LDR=0x%08x\n", kvm_apic_get_reg(apic, APIC_LDR) & APIC_LDR_MASK);
+	printk(KERN_INFO "APIC_SPIV=0x%08x\n", kvm_apic_get_reg(apic, APIC_SPIV) & 0x3ff);
+        printk(KERN_INFO "APIC_ESR=0x%08x\n", kvm_apic_get_reg(apic, APIC_ESR));
+	printk(KERN_INFO "APIC_ICR=0x%08x\n", kvm_apic_get_reg(apic, APIC_ICR) & ~(1 << 12));
+	printk(KERN_INFO "APIC_ICR2=0x%08x\n", kvm_apic_get_reg(apic, APIC_ICR2) & 0xff000000);
+
+	printk(KERN_INFO "APIC_LVTERR=0x%08x\n", kvm_apic_get_reg(apic, APIC_LVTERR));
+	printk(KERN_INFO "APIC_LVT1=0x%08x\n", kvm_apic_get_reg(apic, APIC_LVT1));
+	printk(KERN_INFO "APIC_LVT0=0x%08x\n", kvm_apic_get_reg(apic, APIC_LVT0));
+	printk(KERN_INFO "APIC_LVTPC=0x%08x\n", kvm_apic_get_reg(apic, APIC_LVTPC));
+	printk(KERN_INFO "APIC_LVTTHMR=0x%08x\n", kvm_apic_get_reg(apic, APIC_LVTTHMR));
+	printk(KERN_INFO "APIC_LVTT=0x%08x\n", kvm_apic_get_reg(apic, APIC_LVTT));
+	
+	printk(KERN_INFO "APIC_TMICT=0x%08x\n", kvm_apic_get_reg(apic, APIC_TMICT));
+	printk(KERN_INFO "APIC_TDCR=0x%08x\n", kvm_apic_get_reg(apic, APIC_TDCR));
+        
+	printk(KERN_INFO "%s end\n", __FUNCTION__);
+}
+
+
+int kvm_apic_set(struct kvm_apic *apic, struct kvm_apic_state* as)
+{
+	printk(KERN_DEBUG "%s begin\n", __FUNCTION__);
+	
+	apic->vcpu_id = as->vcpu;
+	apic->status = as->status;
+	kvm_apic_reset(apic);
+	kvm_apic_msr_set(apic, as->apicbase);
+	memset(&apic->intr_pending_count, 0, sizeof(int) * MAX_APIC_INT_VECTOR);
+	memcpy((void*)(apic->regs + APIC_IRR), &as->irr, sizeof(as->irr));
+	memcpy((void*)(apic->regs + APIC_ISR), &as->isr, sizeof(as->isr));
+	memcpy((void*)(apic->regs + APIC_TMR), &as->tmr, sizeof(as->tmr));
+	kvm_apic_set_reg(apic, APIC_ID, as->id);
+	kvm_apic_set_reg(apic, APIC_TASKPRI, as->tpr & 0xff);
+	kvm_apic_set_reg(apic, APIC_PROCPRI, as->ppr);
+	
+	kvm_apic_set_reg(apic, APIC_DFR, as->dfr | 0x0FFFFFFF);
+	kvm_apic_set_reg(apic, APIC_LDR, as->ldr & APIC_LDR_MASK);
+	kvm_apic_set_reg(apic, APIC_SPIV, as->spurious_vec & 0x3ff);
+        kvm_apic_set_reg(apic, APIC_ESR, as->esr);
+	kvm_apic_set_reg(apic, APIC_ICR, as->icr[0] & ~(1 << 12));
+	kvm_apic_set_reg(apic, APIC_ICR2, as->icr[1] & 0xff000000);
+
+	kvm_apic_set_reg(apic, APIC_LVTERR, as->lvterr);
+	kvm_apic_set_reg(apic, APIC_LVT1, as->lvt1);
+	kvm_apic_set_reg(apic, APIC_LVT0, as->lvt0);
+	kvm_apic_set_reg(apic, APIC_LVTPC, as->lvtpc);
+	kvm_apic_set_reg(apic, APIC_LVTTHMR, as->lvtthmr);
+	kvm_apic_set_reg(apic, APIC_LVTT, as->lvtt);
+	
+	kvm_apic_write(apic->vcpu, (unsigned long)(apic->regs + APIC_TDCR), 4, as->divide_conf);
+	kvm_apic_write(apic->vcpu, (unsigned long)(apic->regs + APIC_TMICT), 4, as->initial_count);
+
+	printk(KERN_DEBUG "%s end\n", __FUNCTION__);
+	kvm_apic_dump_state(apic);
+	return 0;
+}
+
+int kvm_apic_get(struct kvm_apic *apic, struct kvm_apic_state* as)
+{
+	printk(KERN_DEBUG "%s begin apic=%p\n", __FUNCTION__, apic);
+	
+	as->status = apic->status;
+	as->apicbase = apic->apic_base_msr;
+        memcpy(as->irr, (void*)(apic->regs + APIC_IRR), sizeof(as->irr));
+	memcpy(as->isr, (void*)(apic->regs + APIC_ISR), sizeof(as->isr));
+	memcpy(as->tmr, (void*)(apic->regs + APIC_TMR), sizeof(as->tmr));
+
+        as->id = kvm_apic_get_reg(apic, APIC_ID);
+	as->tpr = kvm_apic_get_reg(apic, APIC_TASKPRI) & 0xff;
+	as->ppr = kvm_apic_get_reg(apic, APIC_PROCPRI);
+        
+	as->dfr = kvm_apic_get_reg(apic, APIC_DFR) | 0x0FFFFFFF;
+	as->ldr = kvm_apic_get_reg(apic, APIC_LDR) & APIC_LDR_MASK;
+	as->spurious_vec = kvm_apic_get_reg(apic, APIC_SPIV) & 0x3ff;
+        as->esr = kvm_apic_get_reg(apic, APIC_ESR);
+	as->icr[0] = kvm_apic_get_reg(apic, APIC_ICR) & ~(1 << 12);
+	as->icr[1] = kvm_apic_get_reg(apic, APIC_ICR2) & 0xff000000;
+
+	as->lvterr = kvm_apic_get_reg(apic, APIC_LVTERR);
+	as->lvt1 = kvm_apic_get_reg(apic, APIC_LVT1);
+	as->lvt0 = kvm_apic_get_reg(apic, APIC_LVT0);
+	as->lvtpc = kvm_apic_get_reg(apic, APIC_LVTPC);
+	as->lvtthmr = kvm_apic_get_reg(apic, APIC_LVTTHMR);
+	as->lvtt = kvm_apic_get_reg(apic, APIC_LVTT);
+	as->initial_count = kvm_apic_get_reg(apic, APIC_TMICT);
+	as->divide_conf = kvm_apic_get_reg(apic, APIC_TDCR);
+        
+	kvm_apic_dump_state(apic);
+	return 0;
+}
+
+
+int kvm_apic_reset(struct kvm_apic *apic)
+{
 	struct kvm_vcpu *vcpu;
 	int i;
 
+	printk(KERN_INFO "%s\n", __FUNCTION__);
 	ASSERT(apic != NULL);
 	vcpu = apic->vcpu;
 	ASSERT(vcpu != NULL);
@@ -922,6 +1037,10 @@
 	apic->base_address = apic->apic_base_msr & MSR_IA32_APICBASE_BASE;
 	//FIXME kvm_vioapic_add_lapic(apic, v);
 
+	/* Stop the timer in case it's a reset an active apic */
+	if (apic->apic_timer.function)
+		hrtimer_try_to_cancel(&apic->apic_timer);
+
 	hrtimer_init(&apic->apic_timer, CLOCK_MONOTONIC, HRTIMER_ABS);
 	apic->apic_timer.function = apic_timer_fn;
 
@@ -937,7 +1056,7 @@
 	}
 #endif
 
-	printk(KERN_DEBUG "vcpu=%p, id=%d, apic_apic_base_msr=0x%016"PRIx64", "
+	printk(KERN_INFO  "vcpu=%p, id=%d, apic_apic_base_msr=0x%016"PRIx64", "
 			  "base_address=0x%0lx.\n",
 			  vcpu,  GET_APIC_ID(kvm_apic_get_reg(apic, APIC_ID)),
 			  apic->apic_base_msr, apic->base_address);
@@ -966,7 +1085,7 @@
 	memset(apic->regs, 0, PAGE_SIZE);
 	apic->vcpu = vcpu;
 	
-	apic_reset(apic);
+	kvm_apic_reset(apic);
 	return 0;
 }
 
Index: qemu/cpu-exec.c
===================================================================
--- qemu/cpu-exec.c	(revision 3947)
+++ qemu/cpu-exec.c	(working copy)
@@ -276,8 +276,10 @@
         /* disable halt condition */
         if ((env1->interrupt_request & CPU_INTERRUPT_HARD) &&
             (env1->eflags & IF_MASK)) {
+	    printf("%s: Stay inside\n", __FUNCTION__);
             env1->hflags &= ~HF_HALTED_MASK;
         } else {
+	    printf("%s: got out\n", __FUNCTION__);
             return EXCP_HALTED;
         }
     }
Index: qemu/target-i386/cpu.h
===================================================================
--- qemu/target-i386/cpu.h	(revision 3947)
+++ qemu/target-i386/cpu.h	(working copy)
@@ -536,6 +536,7 @@
 #ifdef USE_KVM
 #define BITS_PER_LONG (8 * sizeof (long))
 #define NR_IRQ_WORDS (256/ BITS_PER_LONG)
+    uint32_t kvm_pending_int;
     unsigned long kvm_interrupt_bitmap[NR_IRQ_WORDS];
 #endif
 
Index: qemu/hw/apic.c
===================================================================
--- qemu/hw/apic.c	(revision 3947)
+++ qemu/hw/apic.c	(working copy)
@@ -847,12 +847,101 @@
     return 0;
 }
 
-
 #endif //#ifndef NOAPIC
 
 
 #include "../qemu-kvm.h"
 
+static void apic_reset_slim(void *opaque)
+{
+    kvm_apic_reset(0); //assuming single cpu support
+}
+
+static void apic_save(QEMUFile *f, void *opaque)
+{
+    struct kvm_apic_state *s = opaque;
+    int i;
+
+    //get the data from the kvm
+    kvm_apic_save(s);
+
+    qemu_put_be64s(f, (uint64_t*)&s->apicbase);
+    qemu_put_be32s(f, &s->status);
+    qemu_put_be32s(f, &s->ldr);
+    qemu_put_be32s(f, &s->dfr);
+    qemu_put_be32s(f, &s->id);
+    qemu_put_be32s(f, &s->tpr);
+    qemu_put_be32s(f, &s->ppr);
+    qemu_put_be32s(f, &s->spurious_vec);
+    qemu_put_be32s(f, &s->lvterr);
+    qemu_put_be32s(f, &s->lvt0);
+    qemu_put_be32s(f, &s->lvt1);
+    qemu_put_be32s(f, &s->lvtpc);
+    qemu_put_be32s(f, &s->lvtthmr);
+    qemu_put_be32s(f, &s->lvtt);
+    qemu_put_be32s(f, &s->esr);
+    qemu_put_be32s(f, &s->icr[0]);
+    qemu_put_be32s(f, &s->icr[1]);
+    qemu_put_be32s(f, &s->divide_conf);
+    qemu_put_be32s(f, &s->initial_count);
+
+    for (i=0; i<KVM_IRQ_BITMAP_SIZE(uint64_t) ; i++)
+	qemu_put_be64s(f, (uint64_t*)&s->irr[i]);	
+    for (i=0; i<KVM_IRQ_BITMAP_SIZE(uint64_t) ; i++)
+	qemu_put_be64s(f, (uint64_t*)&s->isr[i]);	
+    for (i=0; i<KVM_IRQ_BITMAP_SIZE(uint64_t) ; i++)
+	qemu_put_be64s(f, (uint64_t*)&s->tmr[i]);	
+}
+
+static int apic_load(QEMUFile *f, void *opaque, int version_id)
+{
+    struct kvm_apic_state *s = opaque;
+    int i;
+
+    if (version_id != 1)
+        return -EINVAL;
+
+    s->apicbase = qemu_get_be64(f);
+    s->status = qemu_get_be32(f);
+    s->ldr = qemu_get_be32(f);
+    s->dfr= qemu_get_be32(f);
+    s->id = qemu_get_be32(f);
+    s->tpr = qemu_get_be32(f);
+    s->ppr = qemu_get_be32(f);
+    s->spurious_vec = qemu_get_be32(f);
+    s->lvterr = qemu_get_be32(f);
+    s->lvt0 = qemu_get_be32(f);
+    s->lvt1 = qemu_get_be32(f);
+    s->lvtpc = qemu_get_be32(f);
+    s->lvtthmr = qemu_get_be32(f);
+    s->lvtt = qemu_get_be32(f);
+    s->esr = qemu_get_be32(f);
+    s->icr[0] = qemu_get_be32(f);
+    s->icr[1] = qemu_get_be32(f);
+    s->divide_conf = qemu_get_be32(f);
+    s->initial_count = qemu_get_be32(f);
+
+    for (i=0; i<KVM_IRQ_BITMAP_SIZE(uint64_t) ; i++)
+	s->irr[0] = qemu_get_be64(f);
+    for (i=0; i<KVM_IRQ_BITMAP_SIZE(uint64_t) ; i++)
+	s->isr[0] = qemu_get_be64(f);
+    for (i=0; i<KVM_IRQ_BITMAP_SIZE(uint64_t) ; i++)
+	s->tmr[0] = qemu_get_be64(f);
+
+    //set the data to the kvm
+    return kvm_apic_load(s);
+}
+
+static struct kvm_apic_state local_apics[MAX_APICS + 1];
+
+int apic_init_slim()
+{
+    register_savevm("kvm_apic", 0, 1, apic_save, apic_load, &local_apics[0]);
+    qemu_register_reset(apic_reset_slim, NULL);
+    return 0;
+}
+
+
 static void ioapic_service(IOAPICState *s)
 {
     uint8_t i;
@@ -864,7 +953,7 @@
     uint8_t dest;
     uint8_t dest_mode;
     uint8_t polarity;
-    uint32_t deliver_bitmask[MAX_APIC_WORDS];
+    //uint32_t deliver_bitmask[MAX_APIC_WORDS];
 
     for (i = 0; i < IOAPIC_NUM_PINS; i++) {
         mask = 1 << i;
Index: qemu/hw/pc.c
===================================================================
--- qemu/hw/pc.c	(revision 3947)
+++ qemu/hw/pc.c	(working copy)
@@ -811,6 +811,7 @@
     register_ioport_write(0x92, 1, 1, ioport92_write, NULL);
 
     if (pci_enabled) {
+	apic_init_slim();
         ioapic = ioapic_init();
     }
     isa_pic = pic_init(pic_irq_request, first_cpu);
Index: qemu/qemu-kvm.c
===================================================================
--- qemu/qemu-kvm.c	(revision 3947)
+++ qemu/qemu-kvm.c	(working copy)
@@ -240,6 +240,9 @@
     kvm_get_sregs(kvm_context, 0, &sregs);
 
     memcpy(env->kvm_interrupt_bitmap, sregs.interrupt_bitmap, sizeof(env->kvm_interrupt_bitmap));
+    env->kvm_pending_int = sregs.pending_int;
+    if (sregs.pending_int) 
+	env->interrupt_request |= CPU_INTERRUPT_HARD;
 
 #define get_seg(var, seg) \
     env->seg.selector = sregs.var.selector; \
@@ -360,7 +363,7 @@
     for (i = 0; i < NR_IRQ_WORDS; ++i)
 	if (env->kvm_interrupt_bitmap[i])
 	    return 1;
-    return 0;
+    return env->kvm_pending_int;
 }
 
 static inline void push_interrupts(CPUState *env)
@@ -533,12 +536,17 @@
     env = envs[0];
     save_regs(env);
 
-    if (!((kvm_interrupt_pending(env) || 
+    if (0 && 
+	!((kvm_interrupt_pending(env) || 
 	   (env->interrupt_request & CPU_INTERRUPT_HARD)) && 
 	  (env->eflags & IF_MASK))) {
 	    env->hflags |= HF_HALTED_MASK;
 	    env->exception_index = EXCP_HLT;
     }
+    if (!(env->eflags & IF_MASK)) {
+	    env->hflags |= HF_HALTED_MASK;
+	    env->exception_index = EXCP_HLT;
+    }
     return 1;
 }
  
@@ -621,7 +629,24 @@
 	return kvm_apic_deliver(kvm_context, &msg);
 }
 
+int kvm_apic_reset(int vcpu)
+{
+	struct kvm_apic_reset msg;
+	
+	msg.vcpu = vcpu;
+	return kvm_apic_on_reset(kvm_context, &msg);
+}
 
+int kvm_apic_save(struct kvm_apic_state *state)
+{
+	state->vcpu = 0;
+	return kvm_apic_get(kvm_context, state);
+}
 
+int kvm_apic_load(struct kvm_apic_state *state)
+{
+	state->vcpu = 0;
+	return kvm_apic_set(kvm_context, state);
+}
 
 #endif
Index: qemu/qemu-kvm.h
===================================================================
--- qemu/qemu-kvm.h	(revision 3947)
+++ qemu/qemu-kvm.h	(working copy)
@@ -11,5 +11,7 @@
 int kvm_update_debugger(CPUState *env);
 int kvm_apic_msg(int vcpu, __u32 delivery_mode, __u64 dest,
 		 __u32 dest_mode, __u32 polarity, __u32 trig_mode, __u32 vector);
-
+int kvm_apic_reset(int vcpu);
+int kvm_apic_save(struct kvm_apic_state *state);
+int kvm_apic_load(struct kvm_apic_state *state);
 #endif
Index: qemu/vl.c
===================================================================
--- qemu/vl.c	(revision 3947)
+++ qemu/vl.c	(working copy)
@@ -4327,8 +4327,8 @@
         } else {
             ret = se->load_state(f, se->opaque, version_id);
             if (ret < 0) {
-                fprintf(stderr, "qemu: warning: error while loading state for instance 0x%x of device '%s'\n", 
-                        instance_id, idstr);
+                fprintf(stderr, "qemu: warning: error (%d)while loading state for instance 0x%x of device '%s'\n", 
+                        ret, instance_id, idstr);
             }
         }
         /* always seek to exact end of record */
Index: qemu/vl.h
===================================================================
--- qemu/vl.h	(revision 3947)
+++ qemu/vl.h	(working copy)
@@ -883,6 +883,7 @@
 typedef struct IOAPICState IOAPICState;
 
 int apic_init(CPUState *env);
+int apic_init_slim();
 int apic_get_interrupt(CPUState *env);
 IOAPICState *ioapic_init(void);
 void ioapic_set_irq(void *opaque, int vector, int level);
Index: user/kvmctl.c
===================================================================
--- user/kvmctl.c	(revision 3947)
+++ user/kvmctl.c	(working copy)
@@ -587,3 +587,29 @@
 	return ioctl(kvm->fd, KVM_APIC_MSG, msg);
 }
 
+int kvm_apic_on_reset(kvm_context_t kvm, struct kvm_apic_reset *msg)
+{
+	return ioctl(kvm->fd, KVM_APIC_RESET, msg);
+}
+
+int kvm_apic_get(kvm_context_t kvm, struct kvm_apic_state *msg)
+{
+	int rs;
+	printf("get start\n");
+	sleep(1);
+	rs = ioctl(kvm->fd, KVM_APIC_GET, msg);
+	sleep(1);
+	printf("get end\n");
+	return rs;
+}
+
+int kvm_apic_set(kvm_context_t kvm, struct kvm_apic_state *msg)
+{
+	int rs;
+	printf("set start\n");
+	sleep(1);
+	rs = ioctl(kvm->fd, KVM_APIC_SET, msg);
+	sleep(1);
+	printf("set end\n");
+	return rs;
+}
Index: user/kvmctl.h
===================================================================
--- user/kvmctl.h	(revision 3947)
+++ user/kvmctl.h	(working copy)
@@ -57,4 +57,7 @@
 			  unsigned long len);
 void kvm_get_dirty_pages(kvm_context_t, int slot, void *buf);
 int kvm_apic_deliver(kvm_context_t kvm, struct kvm_apic_msg *msg);
+int kvm_apic_on_reset(kvm_context_t kvm, struct kvm_apic_reset *msg);
+int kvm_apic_get(kvm_context_t kvm, struct kvm_apic_state *msg);
+int kvm_apic_set(kvm_context_t kvm, struct kvm_apic_state *msg);
 #endif

[-- Attachment #3: Type: text/plain, Size: 286 bytes --]

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

[-- Attachment #4: Type: text/plain, Size: 186 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Re: APIC patch

[Dong, Eddie]

Dor Laor wrote:
>> On Tue, 2007-06-26 at 11:02 +0800, Dong, Eddie wrote:
>>> Greg/Dor:
>>> 	When thinking to merge APIC device model with PIC, a curious
>>> question: Where does the code base come from? Xen or Qemu? I found
>>> there are many diff from Xen (even excluding name difference).
>>> Eddie
>> 
>> I am fairly certain it originated in Xen (which might have
>> originated in Qemu??) but there have been many many changes since
>> then.  I know Dor at least did all the porting to Linux (including
>> the hrtimer work).  I then ported it to the irqdevice model and
>> reworked a good amount of the code. Between the two I am sure you
>> will see lots of diffs from the original Xen code. 
>> 
>> -Greg
> 
> As Greg mentioned, the base code (lapic.c) is Xen based. I also
> stated it clearly in file header. 
> 
> Regarding merging it with mainline, if the AMD problems are over (as I
> think they are) we can do it. There is one point regarding migration,
> currently the apic state is not saved, it should be fixed. The irqdev
> state should be also serialized.

Like we discussed in the PIC patch thread, we need to use the new
user/kernel I/Fs which fixed the ABI
holes in V09. 

We discussed part of the irq injection path, the basic conclusion so far
is that we should not use push_back
mechanism which makes an architecture hole in PIC/APIC state when an irq
is pushed back.

But yes they are not difficult to fix. But I just think why we need the
irqdevice which occupy half of the patch in size and make life not very
easy? (Yes it has the advantage to provide a good abstraction.)

So I have 2 choices:
1: Port PIC to APIC patch which will use irqdevice abstraction layer and
V09 irq injection path + bug fix mentioned above.

2: Port APIC to PIC patch (have above mentioned issue fixed) which
eliminates irqdevice abstraction layer.
We can discuss the abstraction layer in future if we want, but we can
start from simple.

IMO #2 is a little bit easier from effort point of view, but I am not
sure what is the decision and up to the community. Or if somebody else
are willing to do the merge, I will be very happy to dig other problems.

> Avi should say his final word and he is lecturing in the OLS
> convention. 
> 
> Oh, I just discovered old code of mine that handles apic save/restore.
> I'm attaching the patch, its old and won't apply on the current
> sources but it worked in the old days...
> 
> Btw, you guys are doing fantastic, important job. Kudos ;)

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: APIC patch

[Gregory Haskins]

On Tue, 2007-06-26 at 00:36 -0700, Dor Laor wrote:
>
> Regarding merging it with mainline, if the AMD problems are over (as I
> think they are) we can do it. 

Did someone figure out what was wrong with the svm.c changes?  Cool!  Is
the patch checked into the branch?



-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: APIC patch

[Avi Kivity]

Dong, Eddie wrote:
> So I have 2 choices:
> 1: Port PIC to APIC patch which will use irqdevice abstraction layer and
> V09 irq injection path + bug fix mentioned above.
>
> 2: Port APIC to PIC patch (have above mentioned issue fixed) which
> eliminates irqdevice abstraction layer.
> We can discuss the abstraction layer in future if we want, but we can
> start from simple.
>
> IMO #2 is a little bit easier from effort point of view, but I am not
> sure what is the decision and up to the community. Or if somebody else
> are willing to do the merge, I will be very happy to dig other problems.
>
>   

We decided on #2 some time ago... why are we rehashing this again?



-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/