Re: Recommended Hardware for building

[Martin Jansa <martin.jansa@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3820 bytes --]

On Thu, Oct 02, 2014 at 05:51:29PM +0100, Chris Tapp wrote:
> 
> On 2 Oct 2014, at 11:04, Burton, Ross <ross.burton@intel.com> wrote:
> 
> > On 2 October 2014 10:36, Oliver Novakovic <Oliver.Novakovic@alpine.de> wrote:
> >> Can anyone recommend a reasonable performant hardware setup to use ?
> >> 
> >> What should be considered ? Are there any pitfalls ? What about bottlenecks
> >> in the build system ?

you should start by saying what you're going to build, my experience is
quite different when building "small" images like console-image or even
x11-image and "big" images/feeds which contain whole qt5 stack, 3 webkits and
2 chromium builds.

In general: bitbake will better utilize all available performance with
bigger image (e.g. build time for console image won't change so much if
you go from 8 cores to 24, but building e.g. just webkit alone will be
more than twice faster on 24 cores).

Regards,

> >> Specifically:
> >> 
> >> How many cores are recommended ? And how much cache is necessary ?
> >> How much of the main memory does Yocto really use ? Is 32 GB sufficient or
> >> should I go for 64 ?
> >> 
> >> Does it make sense to use two SSDs as Raid0  to get builds faster ?
> > 
> > As much of everything as you can afford.  :)  The build isn't heavy in
> > any particular metric, so don't sacrifice RAM for SSDs for example.
> > 
> > RAID 0 over SSD would be nice and fast, but I prefer having a good
> > amount of RAM and a tuned ext4 (no journal, long commit delay) so data
> > doesn't actually hit the disk as frequently. Keeping the actual build
> > directories on a separate disk is good for performance and not causing
> > data loss when you lose a disk.
> > 
> > There are people that have 64GB in machines and then set TMPDIR to a
> > tmpfs.  Surprisingly this isn't that much faster (5% or so), but it's
> > a lot easier on the hardware and power consumption.
> 
> My experience:
> 
> I've got a quad core with hyper-threading (so 8 usable cores) running at about 3.8 GHz, 16GB of RAM and use multiple SSDs - one to hold the meta data, downloads and top level build areas (local.conf, etc) and have the TMPDIR on a second SSD (so, as Ross says, I don't get a surprise when it wears out!).
> 
> I can build my images (basically an x11 image) in just under 60 minutes (once all the files have been fetched). I run with BB_NUMBER_THREADS and PARALLEL_MAKE both set to 16 to make sure the cores are fully loaded as much as possible (other says that should be 8 and 8 to reduce scheduling overhead).
> 
> During the build the system is CPU bound quite a bit of the time (so more cores should help), but there are significant periods where the build dependency chain means this isn't the case and only two or three cores are active. Previously I recall comparing results with someone else and finding that having lots more cores (24, I think) didn't give a significant improvement in build time (certainly not for the 3x system build cost).
> 
> I've never seen peak memory usage go much above 9 GB during a build, and the peaks generally coincide with linking activities for "big" items (gcc, eglibc). This is likely to go higher with more active threads.
> 
> I started out with a RAID-0 SSD build array, but I didn't really see any difference over a single high-spec (consumer) SSD. As Ross said, running a fast file system on the disk is a good idea.
> 
> --
> 
> Chris Tapp
> opensource@keylevel.com
> www.keylevel.com
> 
> ----
> You can tell you're getting older when your car insurance gets real cheap!
> 
> -- 
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]