From mboxrd@z Thu Jan  1 00:00:00 1970
From: Robin Jarry <robin.jarry@6wind.com>
Subject: Re: [PATCH] mk: optimize directory dependencies
Date: Tue, 24 Jan 2017 12:19:49 +0100
Message-ID: <20170124111949.6jcif4uliwg6bgxv@6wind.com>
References: <20170123181813.00567e47@platinum>
 <1485191953-2613-1-git-send-email-olivier.matz@6wind.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Cc: dev@dpdk.org, ferruh.yigit@intel.com, thomas.monjalon@6wind.com
To: Olivier Matz <olivier.matz@6wind.com>
Return-path: <dev-bounces@dpdk.org>
Received: from mail-wm0-f41.google.com (mail-wm0-f41.google.com [74.125.82.41])
 by dpdk.org (Postfix) with ESMTP id 4C2A41023
 for <dev@dpdk.org>; Tue, 24 Jan 2017 12:19:56 +0100 (CET)
Received: by mail-wm0-f41.google.com with SMTP id c206so203712856wme.0
 for <dev@dpdk.org>; Tue, 24 Jan 2017 03:19:56 -0800 (PST)
Content-Disposition: inline
In-Reply-To: <1485191953-2613-1-git-send-email-olivier.matz@6wind.com>
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

Hi Olivier,

2017-01-23, Olivier Matz:
>Before this patch, the management of dependencies between directories
>had several issues:
>
>- the generation of .depdirs, done at configuration is slow: it can take
>  more than one minute on some slow targets (usually ~10s on a standard
>  PC).

Indeed, on a Qualcomm development board where disk I/O is quite slow:

  $ git describe
  v17.02-rc1-3-g61207d014fc9

  $ time make config T=arm64-armv8a-linuxapp-gcc
  real    1m4.308s

>- for instance, it is possible to expressed a dependency like:

s/expressed/express/

>- we cannot use "make -d" for debug, because the output of make is used 
>for the generation of .depdirs.

That is really annoying when debugging makefiles.

After applying this patch:

  $ git am mk-optimize-directory-dependencies.patch
  Applying: mk: optimize directory dependencies
  $ rm -rf build/

  $ time make config T=arm64-armv8a-linuxapp-gcc
  real    0m0.111s

Almost 600 times faster than before!

I prefer this solution to the one proposed by Ferruh (which is 
interesting but requires to run parallel make). Here is a test with the 
other patch:

  $ git am mk-parallelize-make-config.patch
  Applying: mk: parallelize make config
  $ rm -rf build/
  $ grep -c processor /proc/cpuinfo
  24

  $ time make config T=arm64-armv8a-linuxapp-gcc -j24
  real    0m11.253s

Here only 6 times faster than before, even when using 24 parallel 
processes.

Tested-by: Robin Jarry <robin.jarry@6wind.com>