From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_2 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 096E2C4338F for ; Thu, 29 Jul 2021 20:50:14 +0000 (UTC) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5286060F4B for ; Thu, 29 Jul 2021 20:50:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5286060F4B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=bootlin.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=busybox.net Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 10C5840108; Thu, 29 Jul 2021 20:50:13 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CjInRZgdmd4X; Thu, 29 Jul 2021 20:50:12 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp2.osuosl.org (Postfix) with ESMTP id 2F700401F0; Thu, 29 Jul 2021 20:50:11 +0000 (UTC) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 091411BF317 for ; Thu, 29 Jul 2021 20:50:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 0373340108 for ; Thu, 29 Jul 2021 20:50:09 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nIktu5ZK2qNH for ; Thu, 29 Jul 2021 20:50:07 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from relay7-d.mail.gandi.net (relay7-d.mail.gandi.net [217.70.183.200]) by smtp2.osuosl.org (Postfix) with ESMTPS id 7E9EC40151 for ; Thu, 29 Jul 2021 20:50:07 +0000 (UTC) Received: (Authenticated sender: thomas.petazzoni@bootlin.com) by relay7-d.mail.gandi.net (Postfix) with ESMTPSA id 00A0320009; Thu, 29 Jul 2021 20:50:02 +0000 (UTC) Date: Thu, 29 Jul 2021 22:50:00 +0200 From: Thomas Petazzoni To: Christian Stewart Message-ID: <20210729225000.750b323e@windsurf> In-Reply-To: References: <20201219153525.1361175-1-thomas.petazzoni@bootlin.com> <20201219153525.1361175-6-thomas.petazzoni@bootlin.com> Organization: Bootlin X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.33; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Subject: Re: [Buildroot] [PATCH v2 05/12] support/download/go-post-process: implement Go vendoring support X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Matt Weber , Patrick Havelange , Anisse Astier , Ryan Barnett , Buildroot List , "Yann E. MORIN" Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Hello Christian, On Thu, 29 Jul 2021 13:17:09 -0700 Christian Stewart wrote: > This works fine in most cases, however, this causes inconsistent hashes. > > On my system, the hash is correct after re-pack. But on 2 of my users > machines, the hash ends up different. > > So something specific to the host machine must be influencing the hash > of the repacked archive. > > It's either something in "go mod vendor" that's not deterministic, or > maybe host Go is being used instead of the one compiled by Buildroot, > or perhaps it's not possible to always get deterministic hashes this > way. Meh :-/ > Can we possibly switch to hashing the download /before/ re-packing it, > given that Go verifies the hashes in go.sum automatically as part of > the "go mod vendor" step? This is going to be difficult, because the whole principle of this patch series is to integrate the vendoring within the download step, i.e *before* the hash is verified. For example, the hash is verified even if you're just extracting, and not downloading. One possibility that we had discussed in the past, but that Yann didn't like was to create one tarball for the upstream code, and another for the vendored dependencies. Based on the difficulties, I would really like to understand a little bit why the archive is not deterministic. Could you compare the contents of the download stuff between your different machines, and the contents of the tarball, and see what changes? > Go mod vendor will fail if any of the dependencies are different, the > checksums are included in go.sum Is the presence of go.sum mandatory? Best regards, Thomas -- Thomas Petazzoni, co-owner and CEO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com _______________________________________________ buildroot mailing list buildroot@busybox.net http://lists.busybox.net/mailman/listinfo/buildroot