From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D3F3EC433FE for ; Wed, 12 Oct 2022 14:52:17 +0000 (UTC) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id D045B84DFB; Wed, 12 Oct 2022 16:52:14 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=konsulko.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=konsulko.com header.i=@konsulko.com header.b="uOMy9sMy"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id C337784DB6; Wed, 12 Oct 2022 16:52:12 +0200 (CEST) Received: from mail-qt1-x834.google.com (mail-qt1-x834.google.com [IPv6:2607:f8b0:4864:20::834]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 7A2D384E44 for ; Wed, 12 Oct 2022 16:52:09 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=konsulko.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=trini@konsulko.com Received: by mail-qt1-x834.google.com with SMTP id g11so5662290qts.1 for ; Wed, 12 Oct 2022 07:52:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=konsulko.com; s=google; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=h9Qdv9r9MDnqIHiZway2QuAlWiXdzwqWbaAgDPajptk=; b=uOMy9sMy05A9sBrlc4nwUD570P09GvHT0Dg3SgNhdWhMrXkc6ftm1w2zrMim4bRcaB SmbUtB9ocP8EBcH0aNs9JxZxLBdoBYemFuMCctxv3DSVGFgI5OiqVm/iiRNnV2zLd4Rn bfGjBE6kKcrwYFh04ZS4L1JnZ3PB6JHpmvx2o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=h9Qdv9r9MDnqIHiZway2QuAlWiXdzwqWbaAgDPajptk=; b=TBns+s/a+N8gjyNPslXYfuuxVSJfjPZS5dt4RM6EqkPg569xEWbTFFpCJdTj1EzafY kqal63YGZnsQg3LVw+4DQtlVGDOOUrFonYgQAXheFvIbij8UCEW9oi2YHSpdNFsqzRtw WQGNBWKX8gCMFi3XwjaYoEi4+6wCacTLiRwZpmR2/1za/zHA3JaBxpW6xg0EwP9zOeUO xnaQZ1v/lVVRPhpuYGZXvuI6LcRid3xd4FST/ZHrZEYNaDvSDsR23Akcgl7aBt64tDYE OwfRiLdNN9HAqJx74qPQ8tnRZ8zY/UJzrf1Wolr6Z+QYe6DbPrNWoGaTfcZPfVrOKIuz wRHQ== X-Gm-Message-State: ACrzQf0Kp5Gq9bsj3a0x0uflY/IhGYk2GnR9Rw4IbpfyuznJA2tMd165 bf3Pk/HTL0jzTmdYHgzMYezVW32GKOninQ== X-Google-Smtp-Source: AMsMyM714QFfLV2Dk18aCpOkq++VKoewgugMRsPpS/mVnjihB/XKA3oADMRYwDUC6KX+OXHL5ifChQ== X-Received: by 2002:a05:622a:1751:b0:35c:bf37:3a50 with SMTP id l17-20020a05622a175100b0035cbf373a50mr23833040qtk.22.1665586328152; Wed, 12 Oct 2022 07:52:08 -0700 (PDT) Received: from bill-the-cat (2603-6081-7b00-6400-295e-6c71-581f-29f1.res6.spectrum.com. [2603:6081:7b00:6400:295e:6c71:581f:29f1]) by smtp.gmail.com with ESMTPSA id a18-20020a05622a02d200b0039543f89109sm13619083qtx.96.2022.10.12.07.52.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 07:52:07 -0700 (PDT) Date: Wed, 12 Oct 2022 10:52:05 -0400 From: Tom Rini To: Simon Glass Cc: u-boot@lists.denx.de, Rasmus Villemoes Subject: Re: [PATCH 2/2] buildman: Add --allow-missing-binaries flag to build with BINMAN_ALLOW_MISSING=1 Message-ID: <20221012145205.GX2020586@bill-the-cat> References: <20221010151831.3376759-1-trini@konsulko.com> <20221010151831.3376759-2-trini@konsulko.com> <20221011162235.GF2020586@bill-the-cat> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="9ZRxqsK4bBEmgNeO" Content-Disposition: inline In-Reply-To: X-Clacks-Overhead: GNU Terry Pratchett X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean --9ZRxqsK4bBEmgNeO Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Oct 12, 2022 at 06:59:35AM -0600, Simon Glass wrote: > Hi Tom, >=20 > On Tue, 11 Oct 2022 at 10:22, Tom Rini wrote: > > > > On Mon, Oct 10, 2022 at 05:48:55PM -0600, Simon Glass wrote: > > > Hi Tom, > > > > > > On Mon, 10 Oct 2022 at 09:18, Tom Rini wrote: > > > > > > > > Add a new flag to buildman so that we will in turn pass > > > > BINMAN_ALLOW_MISSING=3D1 to 'make'. Make use of this flag in CI. > > > > > > > > Cc: Rasmus Villemoes > > > > Cc: Simon Glass > > > > Signed-off-by: Tom Rini > > > > --- > > > > .azure-pipelines.yml | 2 +- > > > > .gitlab-ci.yml | 6 +++--- > > > > tools/buildman/builder.py | 5 ++++- > > > > tools/buildman/builderthread.py | 2 ++ > > > > tools/buildman/cmdline.py | 3 +++ > > > > tools/buildman/control.py | 3 ++- > > > > 6 files changed, 15 insertions(+), 6 deletions(-) > > > > > > > > diff --git a/.azure-pipelines.yml b/.azure-pipelines.yml > > > > index f200b40dbb24..c932c2b3c619 100644 > > > > --- a/.azure-pipelines.yml > > > > +++ b/.azure-pipelines.yml > > > > @@ -553,7 +553,7 @@ stages: > > > > cat << "EOF" >> build.sh > > > > if [[ "${BUILDMAN}" !=3D "" ]]; then > > > > ret=3D0; > > > > - tools/buildman/buildman -o /tmp -P -E -W ${BUILDMAN}= ${OVERRIDE} || ret=3D$?; > > > > + tools/buildman/buildman -o /tmp -P -E -W --allow-mis= sing-binaries ${BUILDMAN} ${OVERRIDE} || ret=3D$?; > > > > > > This is fine for CI. > > > > I think one of the issues here is we need to agree on the common use > > cases. I don't think most people use buildman to build, they use make. >=20 > OK. With the new -I/--ide flag I use buildman directly for single > builds on some machines at least. > > Of the people that do use buildman, how many aren't already using a > > wrapper script? I know I always do. >=20 > I don't know, but I always use buildman directly for build-testing > multiple boards. I've been building for single machines with buildman for years, with --keep being passed to my wrapper script. And always use it for multiple boards. But this is you and I, to borrow a meme, we're builders georg and are outliers who should perhaps not be counted. > > Can we also not just deal with > > setting this flag in ~/.buildman ? Would it Just Work as: > > [builder] > > allow-missing-binaries =3D True > > > > Or is more code needed? >=20 > Yes I think that would work, although I think we would need two flags, > one for building current source (where people might want it off) and > another for building multiple boards (when presumably you want it on). I'm not sure. It's not "always use fake binaries" it's allow them. I'm honestly not sure how to tell buildman to use external binaries correctly, that's one of the cases where I use a different script that just manages output directories and known-good additional binaries. > ...although I don't see the advantage of this approach over the series > I sent. Basically, the build should fail if there are missing blobs. > The only question is whether we want to handle missing blobs by: >=20 > 1. failing immediately when we see the first missing blob (your approach) > 2. failing at the end after all binman processing is done (my approach) >=20 > The nice thing about 2 is that the build does complete and the exit > code lets buildman decide what to do afterwards (i.e. we can make > missing blobs be an error or a warning). >=20 > Option 1 has the benefit that we don't do any of the blob handling, so > it just dies right away. Perhaps this is a philosophical question, but > it is a little subtle and I'm not actually sure people would notice > the difference so long as they get the errors as expected. The way I'm thinking of it is there's two cases. The first case is someone who is testing on the hardware that requires these files. Stop ASAP because they either will know they forgot to pass X/Y/Z files or they'll re-read the board instructions page and see oh, they need to grab binaries X/Y/Z too. Waiting to collect up missing file errors doesn't save time. It's also still hugely likely I believe that they're using make and not buildman. The second case is someone building multiple boards at once to build-check (but not run-time check) something in multiple places. This is where passing a specific failure exit code can be helpful, yes. > BTW we need a one-character flag if we do this as it will be a very > common requirement. Sadly both -a and -A are taken. We could use -M for Missing ? > > > But having to provide a flag to do build testing seems like the tail > > > is wagging the dog. Boards should be discouraged to use blobs and we > > > don't want to make it hard for everyone else (who doesn't have the > > > blobs) to test whether their patch breaks a build. > > > > I'm not sure this ends up being a common case. If someone is build > > testing for something they can't run test, the error is going to be > > after whatever they compile-tested was compiled/linked, and if they're > > testing everything it's via CI. >=20 > I don't think people will be able to tell that. The 'make' fails with > an error so it looks like a failure. It happens to be at the end in > the binman step, but it is still a build failure. >=20 > My overall concern is constantly having to rerun a build because I > forgot to add a flag, when I don't have the blobs for the boards > anyway. Like the LTO thing that cost 4 seconds on every build, that > could get really annoying. Blobs should not get in the way of > development. The counter problem is this isn't the first time someone has come up and noted how much time they've wasted because we defaulted to fake binaries. I think we've optimized too much for the people that build a thousand configs all the time (us) instead of the people that build for 1 or two configs at a time (most people?). To that end, I am really curious what Rasmus has to say here, or anyone else that has a different workflow from you and I. --=20 Tom --9ZRxqsK4bBEmgNeO Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGzBAABCgAdFiEEGjx/cOCPqxcHgJu/FHw5/5Y0tywFAmNG1JEACgkQFHw5/5Y0 tyyD6wv/dTHzOY5UemwbQyN4A/GLuO1Ugfg+eXtJ661rUT+HrWyEe/gQzcCcg90X 3MSC2e5abHuF/17Jc/98qXirZpdnztsOtogtUcOxWL7v8uXdcP8Yu8nGqh8Yf6LL 16Z3HZY0B9uqlqVEjfDCqxYoFPnbk9FEnMCdC7U+D3Cx2RzGqnvI9S38p1yG2CtJ 1fJCnVgXJsYylaGP9nC+b+UbKH2grKrmpyn/LRarTMK6e0TSoLv+9kKsMdl9Ry7o Yfcr2SPM1GvncNldFEx7EvQl7dufM9Os1sbDN3l0EY+yfJm4QBE9/O86/iSyXrjj uaKm+gWkwYeoPuJOdzpTd8jjfgR1CUuNaUwDNmhzFKgZqR9F5Cw1GBZUCUnoGxU3 oxYYKdjJ7QUlQTuaV0d0OFtg8gwe4tNtzPTEXoF4NzVLLvX1EFN/Xp26jLXVPxo/ 0wgUMxHtaLnaaLs77yMfeqY0VVsE8Ld9zntO/hOPnYdOBp1LHBUMhYGVKhlWFX8Y CuWwIgVU =+kSg -----END PGP SIGNATURE----- --9ZRxqsK4bBEmgNeO--