Switch from tar.bz2 to tar.gz

Switch from tar.bz2 to tar.gz

[Cooper Jr., Franklin]

[-- Attachment #1: Type: text/plain, Size: 1406 bytes --]

In our current setup tar.bz2 is the default file type used by image recipes. In classic arago we used tar.gz.

I was always curious why some of the image builds were taking a long time. I also noticed that installing the SDK using the installer also took a very long time.

I took arago-amsdk-image contents and created and extracted tar.gz and tar.bz2 files and timed the results.

extract tar.bz2
real    2m20.402s
user    2m15.800s
sys     0m9.410s

extract tar.gz
real    0m47.641s
user    0m34.850s
sys     0m10.860s

create tar.gz
real    1m49.561s
user    1m39.130s
sys     0m4.420s

create tar.bz2
real    6m10.851s
user    6m8.490s
sys     0m4.420s

It seems tar.bz2 provides better compression at the cost of extraction and compression speed. From a SDK perspective  some savings in size is not worth a significant increase in time to bitbake image recipes and neither is it worth the extra time to install a full blown SDK. I would also note that these test are done on a 24 core machine. I'm unsure how much these different compression algorithms utilize all the different cores but I am sure times on a traditional machine will be significantly longer. So is there any issue with me changing the default file type for images from tar.bz2 to tar.gz?


Regards,
Franklin Cooper Jr.
Texas Instruments
Application Engineer
fcooper@ti.com<mailto:fcooper@ti.com>


[-- Attachment #2: Type: text/html, Size: 5932 bytes --]

Re: Switch from tar.bz2 to tar.gz

[Maupin, Chase]

[-- Attachment #1: Type: text/plain, Size: 1887 bytes --]

No issue from me.

Sincerely,
Chase Maupin
Software Applications
ARM MPU
e-mail: chase.maupin@ti.com
phone: (214) 567-2950

For support:
Forums - http://community.ti.com/forums/
Wiki - http://wiki.davincidsp.com/

From: meta-arago-bounces@arago-project.org [mailto:meta-arago-bounces@arago-project.org] On Behalf Of Cooper Jr., Franklin
Sent: Sunday, November 04, 2012 8:06 AM
To: meta-arago@arago-project.org
Subject: [meta-arago] Switch from tar.bz2 to tar.gz

In our current setup tar.bz2 is the default file type used by image recipes. In classic arago we used tar.gz.

I was always curious why some of the image builds were taking a long time. I also noticed that installing the SDK using the installer also took a very long time.

I took arago-amsdk-image contents and created and extracted tar.gz and tar.bz2 files and timed the results.

extract tar.bz2
real    2m20.402s
user    2m15.800s
sys     0m9.410s

extract tar.gz
real    0m47.641s
user    0m34.850s
sys     0m10.860s

create tar.gz
real    1m49.561s
user    1m39.130s
sys     0m4.420s

create tar.bz2
real    6m10.851s
user    6m8.490s
sys     0m4.420s

It seems tar.bz2 provides better compression at the cost of extraction and compression speed. From a SDK perspective  some savings in size is not worth a significant increase in time to bitbake image recipes and neither is it worth the extra time to install a full blown SDK. I would also note that these test are done on a 24 core machine. I'm unsure how much these different compression algorithms utilize all the different cores but I am sure times on a traditional machine will be significantly longer. So is there any issue with me changing the default file type for images from tar.bz2 to tar.gz?


Regards,
Franklin Cooper Jr.
Texas Instruments
Application Engineer
fcooper@ti.com<mailto:fcooper@ti.com>


[-- Attachment #2: Type: text/html, Size: 7666 bytes --]

Re: Switch from tar.bz2 to tar.gz

[Denys Dmytriyenko]

Actually, neither gzip nor bzip2 use multiple threads, so it's running on a 
single core and won't benefit from 24 or even 124 cores...

Other than that, I don't mind switching to gzip for speed, slightly losing in 
size.

Denys


On Sun, Nov 04, 2012 at 02:06:01PM +0000, Cooper Jr., Franklin wrote:
> In our current setup tar.bz2 is the default file type used by image recipes. In classic arago we used tar.gz.
> 
> I was always curious why some of the image builds were taking a long time. I also noticed that installing the SDK using the installer also took a very long time.
> 
> I took arago-amsdk-image contents and created and extracted tar.gz and tar.bz2 files and timed the results.
> 
> extract tar.bz2
> real    2m20.402s
> user    2m15.800s
> sys     0m9.410s
> 
> extract tar.gz
> real    0m47.641s
> user    0m34.850s
> sys     0m10.860s
> 
> create tar.gz
> real    1m49.561s
> user    1m39.130s
> sys     0m4.420s
> 
> create tar.bz2
> real    6m10.851s
> user    6m8.490s
> sys     0m4.420s
> 
> It seems tar.bz2 provides better compression at the cost of extraction and compression speed. From a SDK perspective  some savings in size is not worth a significant increase in time to bitbake image recipes and neither is it worth the extra time to install a full blown SDK. I would also note that these test are done on a 24 core machine. I'm unsure how much these different compression algorithms utilize all the different cores but I am sure times on a traditional machine will be significantly longer. So is there any issue with me changing the default file type for images from tar.bz2 to tar.gz?
> 
> 
> Regards,
> Franklin Cooper Jr.
> Texas Instruments
> Application Engineer
> fcooper@ti.com<mailto:fcooper@ti.com>
> 

> _______________________________________________
> meta-arago mailing list
> meta-arago@arago-project.org
> http://arago-project.org/cgi-bin/mailman/listinfo/meta-arago