QEMU P2P migration speed

[Andrey Korolyov <andrey@xdel.ru>]

Hello,

I`ve got strange results during benchmarking migration speed for
different kinds of loads on source/target host: when source host is
'empty', migration takes approx. 30 percent longer than the same for
host already occupied by one VM with CPU overcommit ratio=1.

[src host, three equal vms, each with ability to eat all cores once]
[tgt host, one VM, with same appetite and limitations]

All VMs was put into cgroups with same cpu ceiling and cpu shares values.

Migration time is almost independent of VM RSS(varies by ten percent at
maximum), for situation when VM is active on target host, time is about
85 seconds to migrate 8G between hosts, and when it is turned off,
migration time *increasing* to 120s. For curious ones, frequency
management is completely inactive on both nodes, neither CStates
mechanism. Interconnection is relatively fast (20+Gbit/s by IPoIB).

Anyone have a suggestions on how to possibly explain/fix this?