[Qemu-devel] qmp block-mirror hang, lots of nfs op/s on nfs source disk

[Qemu-devel] qmp block-mirror hang, lots of nfs op/s on nfs source disk

[Alexandre DERUMIER]

Hi,

I'm currently playing with drive-mirror, (qemu 2.2)

and I have qmp hang when drive-mirror is starting.

just after qmp "drive-mirror" exec, qmp socket or hmp are not responding.

After some times it's working again, and I can see result of query-block-jobs.


The source volume is on nfs (v4), and it seem that the bigger the volume is, the bigger I need to wait before the mirror begin.


Looking at nfsiostat

   op/s		rpc bklog
1806.00 	   0.00
read:             ops/s		   kB/s		  kB/op		retrans		avg RTT (ms)	avg exe (ms)
		  0.000 	  0.000 	  0.000        0 (0.0%) 	  0.000 	  0.000
write:            ops/s		   kB/s		  kB/op		retrans		avg RTT (ms)	avg exe (ms)
		  0.000 	  0.000 	  0.000        0 (0.0%) 	  0.000 	  0.000


I'm seeing a lot of  nfs op/s on source disk (but not read).

I wonder if some kind of source file scan is done ? is it expected ?

I can reproduce it 100%

I found a report on redhat bugzilla here, which seem to have same behaviour (not on nfs):
https://bugzilla.redhat.com/show_bug.cgi?id=1185172

Re: [Qemu-devel] qmp block-mirror hang, lots of nfs op/s on nfs source disk

[Paolo Bonzini]

On 11/05/2015 13:38, Alexandre DERUMIER wrote:
> Hi,
> 
> I'm currently playing with drive-mirror, (qemu 2.2)
> 
> and I have qmp hang when drive-mirror is starting.
> 
> just after qmp "drive-mirror" exec, qmp socket or hmp are not
> responding.
> 
> After some times it's working again, and I can see result of
> query-block-jobs.
> 
> 
> The source volume is on nfs (v4), and it seem that the bigger the
> volume is, the bigger I need to wait before the mirror begin.

That is because drive-mirror checks the whole disk for allocated areas.

We are aware of that and Fam (CCed) is working on it.  Thanks for the
report nevertheless!

Paolo

Re: [Qemu-devel] qmp block-mirror hang, lots of nfs op/s on nfs source disk

[Alexandre DERUMIER]

>>That is because drive-mirror checks the whole disk for allocated areas. 

Oh, ok, that's why I'm seeing a lot of lseek in strace

lseek(21, 1447493632, 0x4 /* SEEK_??? */) = 107374182400
lseek(21, 1447559168, 0x3 /* SEEK_??? */) = 1447559168
lseek(21, 1447559168, 0x4 /* SEEK_??? */) = 107374182400
lseek(21, 1447624704, 0x3 /* SEEK_??? */) = 1447624704
lseek(21, 1447624704, 0x4 /* SEEK_??? */) = 107374182400
lseek(21, 1447690240, 0x3 /* SEEK_??? */) = 1447690240
lseek(21, 1447690240, 0x4 /* SEEK_??? */) = 107374182400
...

>>We are aware of that and Fam (CCed) is working on it. Thanks for the 
>>report nevertheless!

Ok, thanks !


----- Mail original -----
De: "pbonzini" <pbonzini@redhat.com>
À: "aderumier" <aderumier@odiso.com>, "qemu-devel" <qemu-devel@nongnu.org>, "Fam Zheng" <famz@redhat.com>
Envoyé: Lundi 11 Mai 2015 13:46:54
Objet: Re: qmp block-mirror hang, lots of nfs op/s on nfs source disk

On 11/05/2015 13:38, Alexandre DERUMIER wrote: 
> Hi, 
> 
> I'm currently playing with drive-mirror, (qemu 2.2) 
> 
> and I have qmp hang when drive-mirror is starting. 
> 
> just after qmp "drive-mirror" exec, qmp socket or hmp are not 
> responding. 
> 
> After some times it's working again, and I can see result of 
> query-block-jobs. 
> 
> 
> The source volume is on nfs (v4), and it seem that the bigger the 
> volume is, the bigger I need to wait before the mirror begin. 

That is because drive-mirror checks the whole disk for allocated areas. 

We are aware of that and Fam (CCed) is working on it. Thanks for the 
report nevertheless! 

Paolo