From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37919) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XmLCC-0004Uy-J9 for qemu-devel@nongnu.org; Thu, 06 Nov 2014 06:25:49 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XmLC7-0004j9-J5 for qemu-devel@nongnu.org; Thu, 06 Nov 2014 06:25:44 -0500 Received: from mx1.redhat.com ([209.132.183.28]:38328) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XmLC7-0004j1-3d for qemu-devel@nongnu.org; Thu, 06 Nov 2014 06:25:39 -0500 Message-ID: <545B5AAE.7070004@redhat.com> Date: Thu, 06 Nov 2014 12:25:34 +0100 From: Eric Blake MIME-Version: 1.0 References: <1415272128-8273-1-git-send-email-liang.z.li@intel.com> <1415272128-8273-2-git-send-email-liang.z.li@intel.com> In-Reply-To: <1415272128-8273-2-git-send-email-liang.z.li@intel.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="re3IdECskPCsK6VUVuKR3BvT8osRwLPVN" Subject: Re: [Qemu-devel] [v2 1/2] docs: Add a doc about multiple compression threads List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Li Liang , qemu-devel@nongnu.org Cc: yang.z.zhang@intel.com, armbru@redhat.com, lcapitulino@redhat.com This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --re3IdECskPCsK6VUVuKR3BvT8osRwLPVN Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 11/06/2014 12:08 PM, Li Liang wrote: > Give some details about the multiple compression threads and how > to use it in live migration. >=20 > Signed-off-by: Li Liang > --- > docs/multiple-compression-threads.txt | 128 ++++++++++++++++++++++++++= ++++++++ > 1 file changed, 128 insertions(+) > create mode 100644 docs/multiple-compression-threads.txt >=20 > diff --git a/docs/multiple-compression-threads.txt b/docs/multiple-comp= ression-threads.txt > new file mode 100644 > index 0000000..a5e53de > --- /dev/null > +++ b/docs/multiple-compression-threads.txt > @@ -0,0 +1,128 @@ > +Use multiple (de)compression threads in live migration > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > +Copyright (C) 2014 Li Liang Asserting copyright without also mentioning an open license is awkward in open source (IANAL, but as I understand it, in some areas, asserting a copyright without also granting disclaimers merely gets the default non-open status where the file cannot be copied at all; the license is essential to make it obvious that the copyright holder INTENDS for the file to be copied in some circumstances). Thus, you need to explicitly call out GPLv2+ (even if it can be argued it is was implied by the top-level LICENSE) or some other compatible license to be safe. > + > + > +Contents: > +=3D=3D=3D=3D=3D=3D=3D=3D=3D > +* Introduction > +* When to use > +* Performance > +* Usage > +* TODO > + > +Introduction > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > +Instead of sending the guest memory directly, this solution will > +compress the ram page before sending, after receiving, the data will s/sending,/sending;/ > +be decompressed. Using compression in live migration can help > +to reduce the data transferred about 60%, this is very useful when the= > +bandwidth is limited, and the migration time can also be reduced about= > +70% in a typical case. > + > +The process of compression will consume additional CPU cycles, and the= > +extra CPU cycles will increase the migration time. On the other hand, > +the amount of data transferred will reduced, this factor can reduce > +the migration time. If the process of the compression is quick > +enough, then the total migration time can be reduced, and multiple > +compression threads can be used to accelerate the compression process.= > + > +The decompression speed of zlib is at least 4 times as quickly as s/quickly/quick/ > +compression, if the source and destination CPU have equal speed, > +keeping the compression thread count 4 times the decompression > +thread count can avoid CPU waste. > + > +Compression level can be used to control the compression speed and the= > +compression ratio. High compression ratio will take more time, level 0= > +stands for no compression, level 1 stands for the best compression > +speed, and level 9 stands for the best compression ratio. Users can > +select a level number between 0 and 9. > + > + > +When to use the multiple compression threads in live migration > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > +Compression of data will consume lot of extra CPU cycles, in a system s/lot of// s/cycles,/cycles; so/ > +with high overhead of CPU, avoid using this feature. When the network > +bandwidth is very limited and the CPU resource is adequate, use the s/use the/use of/ > +multiple compression threads will be very helpful. If both the CPU and= > +the network bandwidth are adequate, use multiple compression threads s/use/use of/ > +can still help to reduce the migration time. > + > +Performance > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > +Test environment: > + > +CPU: Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz > +Socket Count: 2 > +Ram: 128G > +NIC: Intel I350 (10/100/1000Mbps) > +Host OS: CentOS 7 64-bit > +Guest OS: Ubuntu 12.10 64-bit > +Parameter: qemu-system-x86_64 -enable-kvm -m 1024 > + /share/ia32e_ubuntu12.10.img -monitor stdio > + > +There is no additional application is running on the guest when doing > +the test. > + > + > +Speed limit: 32MB/s > +--------------------------------------------------------------- > + | original | compress thread: 8 > + | way | decompress thread: 2 > + | | compression level: 1 > +--------------------------------------------------------------- > +total time(msec): | 26561 | 7920 > +--------------------------------------------------------------- > +transferred ram(kB):| 877054 | 260641 > +--------------------------------------------------------------- > +throughput(mbps): | 270.53 | 269.68 > +--------------------------------------------------------------- > +total ram(kB): | 1057604 | 1057604 > +--------------------------------------------------------------- > + > + > +Speed limit: No > +--------------------------------------------------------------- > + | original | compress thread: 15 > + | way | decompress thread: 4 > + | | compression level: 1 > +--------------------------------------------------------------- > +total time(msec): | 7611 | 2888 > +--------------------------------------------------------------- > +transferred ram(kB):| 876761 | 262301 > +--------------------------------------------------------------- > +throughput(mbps): | 943.78 | 744.27 > +--------------------------------------------------------------- > +total ram(kB): | 1057604 | 1057604 > +--------------------------------------------------------------- > + > +Usage > +=3D=3D=3D=3D=3D=3D > +1. Verify the destination QEMU version is able to support the multiple= > +compression threads migration: > + {qemu} info_migrate_capablilites > + {qemu} ... compress: off ... > + > +2. Activate compression on the souce: > + {qemu} migrate_set_capability compress on > + > +3. Set the compression thread count on source: > + {qemu} migrate_set_compress_threads 10 > + > +4. Set the compression level on the source: > + {qemu} migrate_set_compress_level 1 > + > +5. Set the decompression thread count on destination: > + {qemu} migrate_set_decompress_threads 5 > + > +6. Start outgoing migration: > + {qemu} migrate -d tcp:destination.host:4444 > + {qemu} info migrate > + Capabilities: ... compress: on > + ... > + > +TODO > +=3D=3D=3D=3D > +Some faster compression/decompression method such as lz4 and quicklz > +can help to reduce the CPU consumption when doing (de)compression. > +Less (de)compression threads are needed when doing the migration. >=20 --=20 Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org --re3IdECskPCsK6VUVuKR3BvT8osRwLPVN Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg iQEcBAEBCAAGBQJUW1qvAAoJEKeha0olJ0NqvQoIAIiSIjTmPfwm9crwMfBPuvTJ qRnGz58VZZL8K9djmv9p9bUOUk62hHd2j4fQal8Xa/+XfLtcRP5P5HY8C21iC1oa TtnRljCj2+tYU8SFmxasqWOM8FZmb41dqu3jCt74FS3Y4X4BT6qJifBqfY9fRNsE atl+MMLO8XG1uelBGECHoKS2MUSsYbe1oUipPgn48C30ZDBLkKWmjZKbIMQe/ZzR LtWawmQxturLiZFps+lkTlfNa3TbRWIHJdGaX0oJUuEsom40fSWziwEZAT37sc9b s7H4im38Q5579yNh1lpvDnMwWGMW+cd7Rtor+XdILZHjNOvVyXVUFM/ud9swPTU= =8rk+ -----END PGP SIGNATURE----- --re3IdECskPCsK6VUVuKR3BvT8osRwLPVN--