Re: [Buildroot] [PATCH v4 1/2] package/pkg-golang.mk: allow packages to override download GOPROXY

[Thomas Petazzoni via buildroot <buildroot@buildroot.org>]

Hello,

On Tue, 6 Aug 2024 09:13:50 -0700
Christian Stewart <christian@aperture.us> wrote:


> The reasoning for using GOPROXY=direct is as follows:
> 
> When we enable the proxy, the Go tool will download the sources for the
> dependencies from Google.
> 
> This is OK in general, but in the context of buildroot we typically
> download sources from their upstream releases or Git repositories.
> 
> GOPROXY is not broken.
> 
> That said, a lot of dependencies cause problems when they change what tag a
> git release points to retroactively (rare but happens), or delete a older
> commit from their repository (which happens extremely rarely).

Crap, those projects do that? They change a tag, or remove an older
commit from their repo? Yikes for reproducibility :-/

> The proxy smooths this over by always serving the same thing for a git tag
> or commit hash.
> 
> The problem with this is, now we have something different being returned
> for a dependency than what it says in go.mod.
> 
> The go.mod might say we are fetching foo at version 1.0.0, but in fact, if
> you go to the repository for foo, and go to version 1.0.0, it could be a
> completely different source tree from the one we get in the proxied
> download.

And that obviously sucks, and using a proxy doesn't seem like a great
workaround.

> This opens up the opportunity for bad actors to hide code in the proxied
> version and then retroactively change the git tag to hide that bad code.
> 
> For this reason the default was set to always use Git to fetch the
> dependencies for Buildroot as we would prefer to detect the mismatch and
> deal with it up front, possibly falling back to fetching the .tar.gz from
> our own mirror, rather than depend on this behavior from the Google proxy
> to always be there and save us.
> 
> As you can see we have never had an issue with this setting until now, when
> we have an actual broken dependency somewhere, and its when we are adding
> the package, not retroactively, that the issue is found and solved.

I'm not sure to follow you on this. Why would this only happen when we
add the package? If we use GOPROXY=direct, and 3 months after we've
added the package some upstream dependency decides to change its tag or
remove some commits, then it will start failing (of course,
sources.buildroot.net will have a backup, but sources.buildroot.net is
not meant to really be used in "normal" circumstances).

> This is my pitch for keeping GOPROXY=direct. If we want to use the Google
> proxy with all the cavets I mentioned above, we could add the option
> HOST_GO_GOPROXY and default it to the Google proxy. But personally I will
> still always override this value to direct in my projects.
> 
> For now we can just include the latest commit hash of tailscale until they
> make a new release and nothing else needs to be changed.

Don't understand this last part. What is your proposal to fix the
tailscale situation?

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot