xen with huffman coding

xen with huffman coding

[digvijay chauhan]

[-- Attachment #1.1: Type: text/plain, Size: 238 bytes --]

Hello all,
              Is it possible to integrate Xen pre-copy algorithm with
huffman coding compression algorithm. If during live migration of vm pages
are first compressed and then transferred at destination.

regards,
DigvijaySingh

[-- Attachment #1.2: Type: text/html, Size: 257 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: xen with huffman coding

[Mats Petersson]

On 14/12/12 14:32, digvijay chauhan wrote:
> Hello all,
>               Is it possible to integrate Xen pre-copy algorithm with 
> huffman coding compression algorithm. If during live migration of vm 
> pages are first compressed and then transferred at destination.
Short answer:
Probably.

Longer answer:
Have a look at the code, do some experimental changes, and see how much 
it improves. The question is if it's sufficiently more efficient to 
compress and then transfer the package, then uncompress it, vs. the 
uncompressed version. The answer to that depends, partly, on what the 
network setup is on the system, and how well the huffman coding is done.

For code and typical data, I'm not at all convinced that huffman 
encoding (which is based on run-lengths) is the best method. You may 
want to look at byte-pair encoding instead, which is more suitable for 
"stuff that contains byte values" - I believe it is also possible to do 
"in situ", meaning the encoding never takes more space than the original 
data - but I could have got that wrong.

Looking at different compression methods would be a good research project.

What is your goal (e.g. is downtime or overall transfer time the 
important factor)?
Generally, downtime is not very much affected by the amount of time it 
takes to copy the guest, except for the final iteration, which is a 
small portion of the total copy time. Of course, the total copy time for 
a large guest could easily be several minutes if the guest is also 
making lots of pages dirty.

What is your test setup?

What kind of network hardware are you using?

--
Mats
>
> regards,
> DigvijaySingh

Re: xen with huffman coding

[Keir Fraser]

On 14/12/2012 14:49, "Mats Petersson" <mats.petersson@citrix.com> wrote:

> For code and typical data, I'm not at all convinced that huffman
> encoding (which is based on run-lengths) is the best method.

Actually Huffman encoding is not a run-length scheme.

 -- Keir

Re: xen with huffman coding

[Mats Petersson]

On 14/12/12 16:27, Keir Fraser wrote:
> On 14/12/2012 14:49, "Mats Petersson" <mats.petersson@citrix.com> wrote:
>
>> For code and typical data, I'm not at all convinced that huffman
>> encoding (which is based on run-lengths) is the best method.
> Actually Huffman encoding is not a run-length scheme.
Ah, I'm confusing it with ccitt (or whatever it is that fax-machines 
use), which uses a fixed Huffman tree to encode a set of run lengths of 
black/white pixels.

Either way, looking at more than one compression mechanism may have some 
value.

--
Mats
>
>   -- Keir
>
>
>
>

Re: xen with huffman coding

[Keir Fraser]

On 14/12/2012 16:34, "Mats Petersson" <mats.petersson@citrix.com> wrote:

> On 14/12/12 16:27, Keir Fraser wrote:
>> On 14/12/2012 14:49, "Mats Petersson" <mats.petersson@citrix.com> wrote:
>> 
>>> For code and typical data, I'm not at all convinced that huffman
>>> encoding (which is based on run-lengths) is the best method.
>> Actually Huffman encoding is not a run-length scheme.
> Ah, I'm confusing it with ccitt (or whatever it is that fax-machines
> use), which uses a fixed Huffman tree to encode a set of run lengths of
> black/white pixels.
> 
> Either way, looking at more than one compression mechanism may have some
> value.

Yes it was a minor point really, and noone really uses Huffman on its own
afaik. More broadly I agree -- it looks like an interesting student project.

> --
> Mats
>> 
>>   -- Keir
>> 
>> 
>> 
>> 
>