From mboxrd@z Thu Jan 1 00:00:00 1970 From: Neil Horman Subject: Re: [PATCH] validate_abi: build faster by augmenting make with job count Date: Thu, 21 Jul 2016 09:54:33 -0400 Message-ID: <20160721135433.GA9628@hmsreliant.think-freely.org> References: <1469034588-1847-1-git-send-email-nhorman@tuxdriver.com> <6330653.K6A9CLrOZy@xps13> <20160720174849.GE28844@hmsreliant.think-freely.org> <20160720201617.GA23515@hmsreliant.think-freely.org> <9309C1B3-D86C-4265-97A4-58E3D7958944@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Neil Horman , Thomas Monjalon , "dev@dpdk.org" , "Mcnamara, John" To: "Wiles, Keith" Return-path: Received: from smtp.tuxdriver.com (charlotte.tuxdriver.com [70.61.120.58]) by dpdk.org (Postfix) with ESMTP id E85775584 for ; Thu, 21 Jul 2016 15:54:50 +0200 (CEST) Content-Disposition: inline In-Reply-To: <9309C1B3-D86C-4265-97A4-58E3D7958944@intel.com> List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Wed, Jul 20, 2016 at 10:32:28PM +0000, Wiles, Keith wrote: >=20 > > On Jul 20, 2016, at 3:16 PM, Neil Horman wrot= e: > >=20 > > On Wed, Jul 20, 2016 at 07:47:32PM +0000, Wiles, Keith wrote: > >>=20 > >>> On Jul 20, 2016, at 12:48 PM, Neil Horman wrot= e: > >>>=20 > >>> On Wed, Jul 20, 2016 at 07:40:49PM +0200, Thomas Monjalon wrote: > >>>> 2016-07-20 13:09, Neil Horman: > >>>>> From: Neil Horman > >>>>>=20 > >>>>> John Mcnamara and I were discussing enhacing the validate_abi scr= ipt to build > >>>>> the dpdk tree faster with multiple jobs. Theres no reason not to= do it, so this > >>>>> implements that requirement. It uses a MAKE_JOBS variable that c= an be set by > >>>>> the user to limit the job count. By default the job count is set= to the number > >>>>> of online cpus. > >>>>=20 > >>>> Please could you use the variable name DPDK_MAKE_JOBS? > >>>> This name is already used in scripts/test-build.sh. > >>>>=20 > >>> Sure > >>>=20 > >>>>> +if [ -z "$MAKE_JOBS" ] > >>>>> +then > >>>>> + # This counts the number of cpus on the system > >>>>> + MAKE_JOBS=3D`lscpu -p=3Dcpu | grep -v "#" | wc -l` > >>>>> +fi > >>>>=20 > >>>> Is lscpu common enough? > >>>>=20 > >>> I'm not sure how to answer that. lscpu is part of the util-linux p= ackage, which > >>> is part of any base install. Theres a variant for BSD, but I'm not= sure how > >>> common it is there. > >>> Neil > >>>=20 > >>>> Another acceptable default would be just "-j" without any number. > >>>> It would make the number of jobs unlimited. > >>=20 > >> I think the best is just use -j as it tries to use the correct numbe= r of jobs based on the number of cores, right? > >>=20 > > -j with no argument (or -j 0), is sort of, maybe what you want. With= either of > > those options, make will just issue jobs as fast as it processes depe= ndencies. > > Dependent on how parallel the build is, that can lead to tons of wait= ing process > > (i.e. more than your number of online cpus), which can actually hurt = your build > > time. >=20 > I read the manual and looked at the code, which supports your statement= . (I think I had some statement on stack overflow and the last time I bel= ieve anything on the internet :-) I have not seen a lot of differences in= compile times with -j on my system. Mostly I suspect it is the number of= paths in the dependency, cores and memory on the system. >=20 > I have 72 lcores or 2 sockets, 18 cores per socket. Xeon 2.3Ghz cores. >=20 > $ export RTE_TARGET=3Dx86_64-native-linuxapp-gcc=20 >=20 > $ time make install T=3D${RTE_TARGET} > real 0m59.445s user 0m27.344s sys 0m7.040s >=20 > $ time make install T=3D${RTE_TARGET} -j > real 0m26.584s user 0m14.380s sys 0m5.120s >=20 > # Remove the x86_64-native-linuxapp-gcc >=20 > $ time make install T=3D${RTE_TARGET} -j 72 > real 0m23.454s user 0m10.832s sys 0m4.664s >=20 > $ time make install T=3D${RTE_TARGET} -j 8 > real 0m23.812s user 0m10.672s sys 0m4.276s >=20 > cd x86_64-native-linuxapp-gcc > $ make clean > $ time make > real 0m28.539s user 0m9.820s sys 0m3.620s >=20 > # Do a make clean between each build. >=20 > $ time make -j > real 0m7.217s user 0m6.532s sys 0m2.332s >=20 > $ time make -j 8 > real 0m8.256s user 0m6.472s sys 0m2.456s >=20 > $ time make -j 72 > real 0m6.866s user 0m6.184s sys 0m2.216s >=20 > Just the real time numbers in the following table. >=20 > processes real Time depdirs > no -j 59.4s Yes > -j 8 23.8s Yes > -j 72 23.5s Yes > -j 26.5s Yes >=20 > no -j 28.5s No > -j 8 8.2s No > -j 72 6.8s No > -j 7.2s No >=20 > Looks like the depdirs build time on my system: > $ make clean -j > $ rm .depdirs > $ time make -j > real 0m23.734s user 0m11.228s sys 0m4.844s >=20 > About 16 seconds, which is not a lot of savings. Now the difference fro= m no -j to -j is a lot, but the difference between -j and -j = is not a huge saving. This leads me back to over engineering the problem = when =E2=80=98-j=E2=80=99 would work just as well here. >=20 > Even on my MacBook Pro i7 system the difference is not that much 1m8s w= ithout depdirs build for -j in a VirtualBox with all 4 cores 8G RAM. Comp= ared to 1m13s with -j 4 option. >=20 > I just wonder if it makes a lot of sense to use cpuinfo in this given c= ase if it turns out to be -j works with the 80% rule? >=20 It may, but that seems to be reason to me to just set DPDK_MAKE_JOBS=3D0,= and you'll get that behavior Neil > On some other project with a lot more files like the FreeBSD or Linux d= istro, yes it would make a fair amount of real time difference. >=20 > Keith >=20 > >=20 > > While its fine in los of cases, its not always fine, and with this > > implementation you can still opt in to that behavior by setting DPDK_= MAKE_JOBS=3D0 > >=20 > > Neil > >=20 > >>=20 >=20