Re: [PATCH 1/2] package_rpm.bbclass: use multithreaded xz compression

[Alexander Kanavin <alexander.kanavin@linux.intel.com>]

On 06/01/2017 10:02 PM, Andre McCurdy wrote:
>> RPM's default is single-threaded gz; the change greatly helps with
>> both buildtimes (when there is a small number of large-sized packages)
>> and disk space taken by resulting rpms.
>
> Do you have any test results for that? Obviously build times will be
> longer when building with a single CPU since xz compression needs more
> CPU than gzip, but I'm curious how many CPUs are required before the
> multiple threads win over and build times actually get shorter. 4? 8?

Yes; it's in the cover letter.

> One other concern with parallel xz compression is memory usage. Single
> threaded xz needs more memory than gzip and running multiple xz
> compression threads together will push the memory usage up again.
> Maybe not an issue if the peak is still acceptable, but something to
> be aware of (I've given up trying to build in 4GB, but hopefully 8GB
> build machines still have some life left in them...).

I haven't seen anything resembling low-memory or OOM in my testing, even 
when running 44 rpm instances at the same time, each with their own 
several threads, each thread handling a package, and compression for the 
package again being done in a multi-threaded way. But then I have 22 
cores and 32G RAM here.

So do try the patches please, and report the findings, YMMV.

Alex