From mboxrd@z Thu Jan  1 00:00:00 1970
From: Bruce Richardson <bruce.richardson@intel.com>
Subject: Re: [PATCH] mk: optimize directory dependencies
Date: Tue, 24 Jan 2017 11:26:33 +0000
Message-ID: <20170124112633.GA156840@bricha3-MOBL3.ger.corp.intel.com>
References: <20170123181813.00567e47@platinum>
 <1485191953-2613-1-git-send-email-olivier.matz@6wind.com>
 <20170124111949.6jcif4uliwg6bgxv@6wind.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Cc: Olivier Matz <olivier.matz@6wind.com>, dev@dpdk.org,
 ferruh.yigit@intel.com, thomas.monjalon@6wind.com
To: Robin Jarry <robin.jarry@6wind.com>
Return-path: <dev-bounces@dpdk.org>
Received: from mga06.intel.com (mga06.intel.com [134.134.136.31])
 by dpdk.org (Postfix) with ESMTP id 1631A108D
 for <dev@dpdk.org>; Tue, 24 Jan 2017 12:26:37 +0100 (CET)
Content-Disposition: inline
In-Reply-To: <20170124111949.6jcif4uliwg6bgxv@6wind.com>
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

On Tue, Jan 24, 2017 at 12:19:49PM +0100, Robin Jarry wrote:
> Hi Olivier,
> 
> 2017-01-23, Olivier Matz:
> > Before this patch, the management of dependencies between directories
> > had several issues:
> > 
> > - the generation of .depdirs, done at configuration is slow: it can take
> >  more than one minute on some slow targets (usually ~10s on a standard
> >  PC).
> 
> Indeed, on a Qualcomm development board where disk I/O is quite slow:
> 
>  $ git describe
>  v17.02-rc1-3-g61207d014fc9
> 
>  $ time make config T=arm64-armv8a-linuxapp-gcc
>  real    1m4.308s
> 

Wow, what is the build time in that case?

> > - for instance, it is possible to expressed a dependency like:
> 
> s/expressed/express/
> 
> > - we cannot use "make -d" for debug, because the output of make is used
> > for the generation of .depdirs.
> 
> That is really annoying when debugging makefiles.
> 
> After applying this patch:
> 
>  $ git am mk-optimize-directory-dependencies.patch
>  Applying: mk: optimize directory dependencies
>  $ rm -rf build/
> 
>  $ time make config T=arm64-armv8a-linuxapp-gcc
>  real    0m0.111s
> 
> Almost 600 times faster than before!
> 
> I prefer this solution to the one proposed by Ferruh (which is interesting
> but requires to run parallel make). Here is a test with the other patch:
> 
>  $ git am mk-parallelize-make-config.patch
>  Applying: mk: parallelize make config
>  $ rm -rf build/
>  $ grep -c processor /proc/cpuinfo
>  24
> 
>  $ time make config T=arm64-armv8a-linuxapp-gcc -j24
>  real    0m11.253s
> 
> Here only 6 times faster than before, even when using 24 parallel processes.
> 
> Tested-by: Robin Jarry <robin.jarry@6wind.com>

Hi Robin,

what are the differences in the patches like when doing a build rather
than just a config? If the build is minutes long because of slow IO,
is the extra 10 seconds really going to make that much of a difference?

Regards,
/Bruce