From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([209.51.188.92]:45329) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gmL9m-0001XF-SI for qemu-devel@nongnu.org; Wed, 23 Jan 2019 11:13:39 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gmL9l-0005CB-ES for qemu-devel@nongnu.org; Wed, 23 Jan 2019 11:13:38 -0500 Received: from mx1.redhat.com ([209.132.183.28]:52186) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gmL9l-00059e-4D for qemu-devel@nongnu.org; Wed, 23 Jan 2019 11:13:37 -0500 From: Laszlo Ersek References: <20190118223400.24311-1-lersek@redhat.com> <20190118223400.24311-5-lersek@redhat.com> <8d4dc6b0-d4d9-2919-194f-c27cb40699b3@redhat.com> <5cc9c1e9-4fd7-27c4-46bc-2388148945c5@redhat.com> Message-ID: <90bd9ea1-916e-f3ae-d4dc-f0f69b7400c2@redhat.com> Date: Wed, 23 Jan 2019 17:13:21 +0100 MIME-Version: 1.0 In-Reply-To: <5cc9c1e9-4fd7-27c4-46bc-2388148945c5@redhat.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [PATCH 4/5] tests/uefi-test-tools: add build scripts List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Peter Maydell , =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= Cc: "Michael S. Tsirkin" , Ard Biesheuvel , qemu devel list , Shannon Zhao , Gerd Hoffmann , Igor Mammedov On 01/22/19 13:02, Laszlo Ersek wrote: > On 01/21/19 20:30, Peter Maydell wrote: >> On Mon, 21 Jan 2019 at 19:09, Laszlo Ersek wrote: >>> It wasn't clear to me whether and how multi-threaded builds were >>> supposed to be used by maintainers, whenever they'd update >>> "tests/data/uefi-boot-images/*". >>> >>> I saw that "make" was invoked everywhere as $(MAKE), but that didn't >>> clarify any intent around "-j". So I didn't test "-j" at all, and in >>> fact I wouldn't have expected it to work: >> >> The usual assumption with make is that "-jN" should work >> and at least (if the thing being built can't actually >> be parallelized usefully) be no worse than if you'd not >> specified a -j option. We have occasionally had problems >> with -jN in 'make check' (usually because several test cases >> were trying to use the same temp filename or similar) but >> we've treated them as bugs and squashed them. >> >>> The "build" base tool in edk2 implements part of the job with generat= ed >>> makefiles and invoking "make" itself, thus, despite .NOTPARALLEL, it >>> likely inherits the outermost -j setting -- and it doesn't expect = such. >>> >>> So the best I can offer here is to check $MAKEFLAGS in "build.sh", an= d >>> exit with an early, explicit error if $MAKEFLAGS contains "-j", "-l",= or >>> their long variants (--jobs, --load-average). >> >> Could you sanitize MAKEFLAGS in build.sh instead to remove the >> parallelization options? >=20 > I've looked into MAKEFLAGS in a bit more depth now; both the > documentation and some debug prints. Manipulating MAKEFLAGS looks > somewhat brittle. >=20 > So, I was about to suggest that I use the .NOTPARALLEL special target > recommended by Phil, for the .efi binaries (which would ensure that no > two instances of the "build" base tool run at the same time), *plus* > that I submit a patch to edk2 so that the makefiles generated by the > "build" tool also contain .NOTPARALLEL. >=20 > (Because, to quote the make docs again, "If .NOTPARALLEL is mentioned a= s > a target, then this invocation of make will be run serially, even if th= e > =E2=80=98-j=E2=80=99 option is given. Any recursively invoked make comm= and will still > run recipes in parallel (unless its makefile also contains this target)= .") >=20 > However: when I wanted to see the actual error from using .NOTPARALLEL > *only* in "tests/uefi-test-tools/Makefile", and not in the > build-generated makefiles, I failed to get any error. All the output > images were built just fine. >=20 > Phil: when you wrote that "The following patch didn't help" -- referrin= g > to .NOTPARALLEL, added only to "tests/uefi-test-tools/Makefile"-- , did > you clean your tree first (with "make clean" or "git clean -ffdx")? >=20 > Because now I'm thinking that the *individual* makefiles generated by > edk2's "build" base tool might actually compatible with "-j", and your > testing of .NOTPARALLEL failed only because your build tree (for exampl= e > the Conf/ subdir -- list it with "-A") was in a messy state from your > previous -j4 attempt (where you didn't use .NOTPARALLEL at all). FWIW, I've asked - both on edk2-devel, about adding .NOTPARALLEL to the generated inner makefiles: "parallelism in the module-level, generated GNUmakefile's" https://lists.01.org/pipermail/edk2-devel/2019-January/035463.html - and on help-make, about safely filtering the job-related options from MAKEFLAGS, between the outer and inner "make" processes: "filtering job options from MAKEFLAGS, manually" http://lists.gnu.org/archive/html/help-make/2019-01/msg00003.html Thanks Laszlo