porting fixes regarding kvm-clock and lost irqs to stable qemu-kvm 0.12.4

porting fixes regarding kvm-clock and lost irqs to stable qemu-kvm 0.12.4

[Peter Lieven]

Hi,

I would like to get latest stable qemu-kvm (0.12.4) to a usable state
regarding live-migration.

Problems are fixed in git, but there is so much new stuff that has not
extensively tested and therefore I would like to stay at 0.12.4 at the 
moment.

Therefore I would appreciate your help regarding 2 bugs:

a) -cpu xxx,-kvmclock doesn't work in 0.12.4. It started working first
after applying a patchset from Avi that is not even in git yet. What
needs to be done to get it working in 0.12.4 ?

b) There is a bug in 0.12.4 which leads to lost irqs. It was reported
appearing in virtio_blk (#584131) and e1000 (#585113). It is finally
fixed in GIT. Can someone give me a hint with commit fixed this?

Thanks,
Peter

Re: porting fixes regarding kvm-clock and lost irqs to stable qemu-kvm 0.12.4

[Marcelo Tosatti]

On Wed, Jun 02, 2010 at 05:13:30PM +0200, Peter Lieven wrote:
> Hi,
> 
> I would like to get latest stable qemu-kvm (0.12.4) to a usable state
> regarding live-migration.
> 
> Problems are fixed in git, but there is so much new stuff that has not
> extensively tested and therefore I would like to stay at 0.12.4 at
> the moment.
> 
> Therefore I would appreciate your help regarding 2 bugs:
> 
> a) -cpu xxx,-kvmclock doesn't work in 0.12.4. It started working first
> after applying a patchset from Avi that is not even in git yet. What
> needs to be done to get it working in 0.12.4 ?

You want to disable kvmclock? -no-kvmclock kernel option.

Note migration should be working fine since kvm.git commit 
afbcf7ab8d1bc8c2d04792f6d9e786e0adeb328d.

> b) There is a bug in 0.12.4 which leads to lost irqs. It was reported
> appearing in virtio_blk (#584131) and e1000 (#585113). It is finally
> fixed in GIT. Can someone give me a hint with commit fixed this?

c3f8f61157625d0bb5bfc135047573de48fdc675.

Re: porting fixes regarding kvm-clock and lost irqs to stable qemu-kvm 0.12.4

[Peter Lieven]

Am 04.06.2010 um 17:31 schrieb Marcelo Tosatti:

> On Wed, Jun 02, 2010 at 05:13:30PM +0200, Peter Lieven wrote:
>> Hi,
>> 
>> I would like to get latest stable qemu-kvm (0.12.4) to a usable state
>> regarding live-migration.
>> 
>> Problems are fixed in git, but there is so much new stuff that has not
>> extensively tested and therefore I would like to stay at 0.12.4 at
>> the moment.
>> 
>> Therefore I would appreciate your help regarding 2 bugs:
>> 
>> a) -cpu xxx,-kvmclock doesn't work in 0.12.4. It started working first
>> after applying a patchset from Avi that is not even in git yet. What
>> needs to be done to get it working in 0.12.4 ?
> 
> You want to disable kvmclock? -no-kvmclock kernel option.
> 

i am afraid it is not, at least the last git i tested some days ago.
it's still crashing with kvm-clock.

i was looking for a generic way to disable it in the hypervisor
without the need to touch every guests kernel commandline.
this would be easy revertible once its all working as expected.

> Note migration should be working fine since kvm.git commit 
> afbcf7ab8d1bc8c2d04792f6d9e786e0adeb328d.

would it be easy to backport this to stable? hw/msix.c is looking quite different than
that of 0.12.4. 

> 
>> b) There is a bug in 0.12.4 which leads to lost irqs. It was reported
>> appearing in virtio_blk (#584131) and e1000 (#585113). It is finally
>> fixed in GIT. Can someone give me a hint with commit fixed this?
> 
> c3f8f61157625d0bb5bfc135047573de48fdc675.
> 
thanks

> 

Re: porting fixes regarding kvm-clock and lost irqs to stable qemu-kvm 0.12.4

[Faidon Liambotis]

Hi,

Peter Lieven wrote:
> i was looking for a generic way to disable it in the hypervisor
> without the need to touch every guests kernel commandline.
> this would be easy revertible once its all working as expected.
This is a huge hack but for the same reason I've made an LD_PRELOAD
wrapper that wraps ioctl masking the hsot kernel's kvmclock feature from
userspace kvm. I'll be happy to share it if you're interested.

Works like a charm, migration has worken reliably with qemu-kvm 0.11.3 +
the wrapper.

Regards,
Faidon

Re: porting fixes regarding kvm-clock and lost irqs to stable qemu-kvm 0.12.4

[Peter Lieven]

Am 05.06.2010 um 03:34 schrieb Faidon Liambotis:

> Hi,
> 
> Peter Lieven wrote:
>> i was looking for a generic way to disable it in the hypervisor
>> without the need to touch every guests kernel commandline.
>> this would be easy revertible once its all working as expected.
> This is a huge hack but for the same reason I've made an LD_PRELOAD
> wrapper that wraps ioctl masking the hsot kernel's kvmclock feature from
> userspace kvm. I'll be happy to share it if you're interested.

i would be grateful if you can sent it to me offlist. however, i came up with this hack,
that seems to work also:

diff -ur qemu-kvm-0.12.4.orig/qemu-kvm-x86.c qemu-kvm-0.12.4/qemu-kvm-x86.c
--- qemu-kvm-0.12.4.orig/qemu-kvm-x86.c	2010-05-09 13:05:19.000000000 +0200
+++ qemu-kvm-0.12.4/qemu-kvm-x86.c	2010-06-02 18:18:53.000000000 +0200
@@ -1240,7 +1240,7 @@
 	int feature;
 } para_features[] = {
 #ifdef KVM_CAP_CLOCKSOURCE
-	{ KVM_CAP_CLOCKSOURCE, KVM_FEATURE_CLOCKSOURCE },
+//	{ KVM_CAP_CLOCKSOURCE, KVM_FEATURE_CLOCKSOURCE },
 #endif
 #ifdef KVM_CAP_NOP_IO_DELAY
 	{ KVM_CAP_NOP_IO_DELAY, KVM_FEATURE_NOP_IO_DELAY },


> 
> Works like a charm, migration has worken reliably with qemu-kvm 0.11.3 +
> the wrapper.
> 
> Regards,
> Faidon
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: porting fixes regarding kvm-clock and lost irqs to stable qemu-kvm 0.12.4

[Faidon Liambotis]

Peter Lieven wrote:
>>> i was looking for a generic way to disable it in the hypervisor
>>> without the need to touch every guests kernel commandline.
>>> this would be easy revertible once its all working as expected.
>> This is a huge hack but for the same reason I've made an LD_PRELOAD
>> wrapper that wraps ioctl masking the hsot kernel's kvmclock feature from
>> userspace kvm. I'll be happy to share it if you're interested.
> 
> i would be grateful if you can sent it to me offlist.
http://people.debian.org/~paravoid/kvm-noclock-3.tar.gz

> however, i came up with this hack,
> that seems to work also:
<snip>

Yes, that's equivalent. We didn't use that since we wanted to use our
distribution's binaries and be able to upgrade without modifying the source.

Regards,
Faidon

Re: porting fixes regarding kvm-clock and lost irqs to stable qemu-kvm 0.12.4

[Peter Lieven]

Am 06.06.2010 um 16:53 schrieb Faidon Liambotis:

> Peter Lieven wrote:
>>>> i was looking for a generic way to disable it in the hypervisor
>>>> without the need to touch every guests kernel commandline.
>>>> this would be easy revertible once its all working as expected.
>>> This is a huge hack but for the same reason I've made an LD_PRELOAD
>>> wrapper that wraps ioctl masking the hsot kernel's kvmclock feature from
>>> userspace kvm. I'll be happy to share it if you're interested.
>> 
>> i would be grateful if you can sent it to me offlist.
> http://people.debian.org/~paravoid/kvm-noclock-3.tar.gz
> 
>> however, i came up with this hack,
>> that seems to work also:
> <snip>
> 
> Yes, that's equivalent. We didn't use that since we wanted to use our
> distribution's binaries and be able to upgrade without modifying the source.

thank you. 

i want to share this patch for stable until -cpu xxx,-kvmclock works in the stable
tree.

diff -ur qemu-kvm-0.12.4.orig/qemu-kvm.c qemu-kvm-0.12.4/qemu-kvm.c
--- qemu-kvm-0.12.4.orig/qemu-kvm.c	2010-05-09 13:05:19.000000000 +0200
+++ qemu-kvm-0.12.4/qemu-kvm.c	2010-06-06 16:58:12.000000000 +0200
@@ -54,7 +54,7 @@
 int kvm_pit = 1;
 int kvm_pit_reinject = 1;
 int kvm_nested = 0;
-
+int kvm_clock = 1;
 
 KVMState *kvm_state;
 kvm_context_t kvm_context;
diff -ur qemu-kvm-0.12.4.orig/qemu-kvm.h qemu-kvm-0.12.4/qemu-kvm.h
--- qemu-kvm-0.12.4.orig/qemu-kvm.h	2010-05-09 13:05:19.000000000 +0200
+++ qemu-kvm-0.12.4/qemu-kvm.h	2010-06-06 17:04:02.000000000 +0200
@@ -1035,6 +1035,7 @@
 extern int kvm_pit;
 extern int kvm_pit_reinject;
 extern int kvm_nested;
+extern int kvm_clock;
 extern kvm_context_t kvm_context;
 
 struct ioperm_data {
diff -ur qemu-kvm-0.12.4.orig/qemu-kvm-x86.c qemu-kvm-0.12.4/qemu-kvm-x86.c
--- qemu-kvm-0.12.4.orig/qemu-kvm-x86.c	2010-05-09 13:05:19.000000000 +0200
+++ qemu-kvm-0.12.4/qemu-kvm-x86.c	2010-06-06 16:46:08.000000000 +0200
@@ -1259,6 +1259,7 @@
 	int i, features = 0;
 
 	for (i = 0; i < ARRAY_SIZE(para_features)-1; i++) {
+                if (para_features[i].cap == KVM_CAP_CLOCKSOURCE && !kvm_clock) continue;
 		if (kvm_check_extension(kvm_state, para_features[i].cap))
 			features |= (1 << para_features[i].feature);
 	}
diff -ur qemu-kvm-0.12.4.orig/qemu-options.hx qemu-kvm-0.12.4/qemu-options.hx
--- qemu-kvm-0.12.4.orig/qemu-options.hx	2010-05-09 13:05:19.000000000 +0200
+++ qemu-kvm-0.12.4/qemu-options.hx	2010-06-06 16:38:46.000000000 +0200
@@ -1959,6 +1959,9 @@
     "-no-kvm-pit     disable KVM kernel mode PIT\n")
 DEF("no-kvm-pit-reinjection", 0, QEMU_OPTION_no_kvm_pit_reinjection,
     "-no-kvm-pit-reinjection disable KVM kernel mode PIT interrupt reinjection\n")
+DEF("no-kvm-clock", 0, QEMU_OPTION_no_kvm_clock,
+    "-no-kvm-clock   disable KVM pvclock\n")
+
 #if defined(TARGET_I386) || defined(TARGET_X86_64) || defined(TARGET_IA64) || defined(__linux__)
 DEF("pcidevice", HAS_ARG, QEMU_OPTION_pcidevice,
     "-pcidevice host=bus:dev.func[,dma=none][,name=string]\n"
diff -ur qemu-kvm-0.12.4.orig/vl.c qemu-kvm-0.12.4/vl.c
--- qemu-kvm-0.12.4.orig/vl.c	2010-05-09 13:05:19.000000000 +0200
+++ qemu-kvm-0.12.4/vl.c	2010-06-06 17:05:53.000000000 +0200
@@ -5530,6 +5530,10 @@
                 kvm_pit_reinject = 0;
                 break;
             }
+            case QEMU_OPTION_no_kvm_clock: {
+                kvm_clock = 0;
+                break;
+            }
 	    case QEMU_OPTION_enable_nesting: {
 		kvm_nested = 1;
 		break;


> 
> Regards,
> Faidon
> 

Re: porting fixes regarding kvm-clock and lost irqs to stable qemu-kvm 0.12.4

[Peter Lieven]

Am 04.06.2010 um 17:31 schrieb Marcelo Tosatti:

> On Wed, Jun 02, 2010 at 05:13:30PM +0200, Peter Lieven wrote:
>> Hi,
>> 
>> I would like to get latest stable qemu-kvm (0.12.4) to a usable state
>> regarding live-migration.
>> 
>> Problems are fixed in git, but there is so much new stuff that has not
>> extensively tested and therefore I would like to stay at 0.12.4 at
>> the moment.
>> 
>> Therefore I would appreciate your help regarding 2 bugs:
>> 
>> a) -cpu xxx,-kvmclock doesn't work in 0.12.4. It started working first
>> after applying a patchset from Avi that is not even in git yet. What
>> needs to be done to get it working in 0.12.4 ?
> 
> You want to disable kvmclock? -no-kvmclock kernel option.
> 
> Note migration should be working fine since kvm.git commit 
> afbcf7ab8d1bc8c2d04792f6d9e786e0adeb328d.

is this patch needed in the gust or on the host?
i have 2.6.33.3 on the host system and still observed crash conditions.

> 
>> b) There is a bug in 0.12.4 which leads to lost irqs. It was reported
>> appearing in virtio_blk (#584131) and e1000 (#585113). It is finally
>> fixed in GIT. Can someone give me a hint with commit fixed this?
> 
> c3f8f61157625d0bb5bfc135047573de48fdc675.
> 
>