Re: [Buildroot] [PATCH 2/4] package/pkg-golang: set dependencies to host-goc

Re: [Buildroot] [PATCH 2/4] package/pkg-golang: set dependencies to host-goc

[Christian Stewart via buildroot]

[-- Attachment #1.1: Type: text/plain, Size: 776 bytes --]

Hi Thomas,

On Mon, Oct 17, 2022, 7:31 AM Thomas Perale <thomas.perale@essensium.com>
wrote:

> Use 'host-goc' instead of 'host-go' to build one of the implementation
> of the 'goc' package instead of building the 'go' package by default.
>


> Also select 'goc' in every package that are built by the go compiler
> to keep the dependencies in the 'menuconfig' corrects.
>

Similar to my other email about this - why can't we keep using host-go and
avoid changing everything across the tree? It should be possible to instead
have a parameter for host go which changes the current default of compile
from source to downloading the prebuilt.

Also, nowhere in the Go ecosystem is it called 'goc' - this would add
significant confusion, in my opinion.

Thanks,
Christian Stewart

[-- Attachment #1.2: Type: text/html, Size: 1509 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 3/4] package/go-bin: new package

[Christian Stewart via buildroot]

[-- Attachment #1.1: Type: text/plain, Size: 1155 bytes --]

Hi Thomas,

On Mon, Oct 17, 2022, 7:31 AM Thomas Perale <thomas.perale@essensium.com>
wrote:

> This package provides a pre-built version of the go compiler for the
> host target.
> This enable the user to select between the pre-built and the compiled
> version of the go compiler. The pre-built version will be used by
> default when a go compiler is required.
>
> The virtual package 'goc' creates a menu that let the user select which
> variant of the go compiler to use.
>
> The supported host platform for the pre-built compiler are actually the
> same as the one for 'go-bootstrap' no further verification are required
> than the one already in place.
>
> For now GO_BIN_VERSION and GO_VERSION are set the same by hand but in
> next version this variable should be in common. Because the order of the
> '.mk' file is not guaranteed this will be investigated in a next patch.
>

Instead of adding new packages...

Continue using "host-go" and add parameters under a menu config for it.
Then you can simply use a single variable - GO_VERSION - rather than 2 new
packages.

I'm happy to send a prototype of this if you need.

Thanks,
Christian Stewart

[-- Attachment #1.2: Type: text/html, Size: 1757 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 3/4] package/go-bin: new package

[Thomas Petazzoni via buildroot]

On Mon, 17 Oct 2022 09:13:54 -0700
Christian Stewart <christian@paral.in> wrote:

> Instead of adding new packages...
> 
> Continue using "host-go" and add parameters under a menu config for it.
> Then you can simply use a single variable - GO_VERSION - rather than 2 new
> packages.
> 
> I'm happy to send a prototype of this if you need.

I also thought of doing this for the NodeJS package, instead of
separating host-nodejs-bin and host-nodejs-src. I went for the
virtual-package + 2 providers solution as this is what we had done with
Rust.

The main advantage I see with the single package solution is that the
version of NodeJS would be located at only one place. Although this
could also be solved by:

package/
 nodejs/
  nodejs.mk 	<= contains the version
  nodejs/   	<= virtual package
  nodejs-bin/	<= pre-compiled provider
  nodejs-src/   <= source-based provider

In my proposal I solved this duplicated version by simply having a
comment above the <pkg>_VERSION field to remind people that the other
package supporting NodeJS should be bumped at the same time.

The advantage of the 2 packages solution is obviously that each package
is simpler.

Best regards,

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 3/4] package/go-bin: new package

[Christian Stewart via buildroot]

[-- Attachment #1.1: Type: text/plain, Size: 456 bytes --]

Hi all,

This looks interesting, one request though...

On Mon, Oct 17, 2022, 7:29 AM Thomas Perale <thomas.perale@essensium.com>
wrote:
> +       # Default to host-go-bin

> +       default "host-go-bin" if BR2_PACKAGE_HOST_GO_BIN
> +       default "host-go" if BR2_PACKAGE_HOST_GO


Please don't default to downloading something predompiled. Especially when
the behavior currently defaults to compiling everything from source.

Thanks,
Christian Stewart

[-- Attachment #1.2: Type: text/html, Size: 978 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 3/4] package/go-bin: new package

[Yann E. MORIN]

Christian, All,

On 2022-10-17 09:05 -0700, Christian Stewart via buildroot spake thusly:
> Hi all,
> This looks interesting, one request though...
> 
> On Mon, Oct 17, 2022, 7:29 AM Thomas Perale < [1]thomas.perale@essensium.com> wrote:
> > +       # Default to host-go-bin
> 
>   +       default "host-go-bin" if BR2_PACKAGE_HOST_GO_BIN
>   +       default "host-go" if BR2_PACKAGE_HOST_GO
> 
> Please don't default to downloading something predompiled. Especially when the behavior currently defaults to compiling everything
> from source.

Agreed. The default should be to build everything from source when
sources are available. The pre-built packages are only a fallback as
a convenience for those that are too much in a hurry.

Regards,
Yann E. MORIN.

> Thanks,
> Christian Stewart
> 
> Links:
> 1. mailto:thomas.perale@essensium.com

> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot


-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 3/4] package/go-bin: new package

[Thomas Petazzoni via buildroot]

On Mon, 17 Oct 2022 20:03:50 +0200
"Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> Agreed. The default should be to build everything from source when
> sources are available. The pre-built packages are only a fallback as
> a convenience for those that are too much in a hurry.

Except we do exactly the opposite for Rust, and I'm not sure for the
host compiler of those languages it is a good idea to cause such a
major impact on the default build time faced by our users. I clearly
understand the need to be able from source and appreciate how important
it is, but I'm not so sure the default should be to build all those
compilers from source.

Best regards,

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 3/4] package/go-bin: new package

[Christian Stewart via buildroot]

Thomas,

On Mon, Oct 17, 2022 at 11:42 AM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
>
> On Mon, 17 Oct 2022 20:03:50 +0200
> "Yann E. MORIN" <yann.morin.1998@free.fr> wrote:
>
> > Agreed. The default should be to build everything from source when
> > sources are available. The pre-built packages are only a fallback as
> > a convenience for those that are too much in a hurry.
>
> Except we do exactly the opposite for Rust, and I'm not sure for the
> host compiler of those languages it is a good idea to cause such a
> major impact on the default build time faced by our users. I clearly
> understand the need to be able from source and appreciate how important
> it is, but I'm not so sure the default should be to build all those
> compilers from source.

As you mentioned in a reply on the other email thread-

The main thing that takes time with host-go from source is building
the C toolchain.

Provided that the C toolchain is pre-compiled, building host-go
doesn't take long (~1-2 mins).

Best regards,
Christian Stewart
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 0/4] Add support for using a pre-compiled Go compiler

[Thomas Perale via buildroot]

Improvement from the buildroot TODO list. This patch add supports for
pre-built version of the Go compiler.

Right now this patch introduce a new virtual package called 'goc' to
do the selection between two provider:
  - go
  - go-bin
Right now there is some variable from the 'go' package are used in 
multiple go provider packages.
In an upcoming patch all the go related packages will move to a 'go'
subfolder that will hold all the common variables.

Thomas Perale (4):
  package/goc: new virtual package
  package/pkg-golang: set dependencies to host-goc
  package/go-bin: new package
  support/testing: add tests for Go

 fs/oci/Config.in                         |  1 +
 package/Config.in.host                   |  2 ++
 package/balena-engine/Config.in          |  1 +
 package/containerd/Config.in             |  1 +
 package/crucible/Config.in               |  1 +
 package/delve/Config.in                  |  1 +
 package/docker-cli/Config.in             |  1 +
 package/docker-compose/Config.in         |  1 +
 package/docker-engine/Config.in          |  1 +
 package/docker-proxy/Config.in           |  1 +
 package/embiggen-disk/Config.in          |  1 +
 package/flannel/Config.in                |  1 +
 package/gitlab-runner/Config.in          |  1 +
 package/go-bin/Config.in.host            |  5 +++
 package/go-bin/go-bin.hash               |  4 +++
 package/go-bin/go-bin.mk                 | 20 ++++++++++++
 package/go/go.mk                         |  2 ++
 package/goc/Config.in.host               | 39 +++++++++++++++++++++++
 package/goc/goc.mk                       |  7 +++++
 package/gocryptfs/Config.in              |  1 +
 package/mender-artifact/Config.in.host   |  1 +
 package/mender-connect/Config.in         |  1 +
 package/mender/Config.in                 |  1 +
 package/moby-buildkit/Config.in          |  1 +
 package/moby-buildkit/Config.in.host     |  1 +
 package/nerdctl/Config.in                |  1 +
 package/pkg-golang.mk                    |  4 +--
 package/runc/Config.in                   |  1 +
 package/tinifier/Config.in               |  1 +
 package/wtfutil/Config.in                |  1 +
 support/testing/tests/package/test_go.py | 40 ++++++++++++++++++++++++
 31 files changed, 143 insertions(+), 2 deletions(-)
 create mode 100644 package/go-bin/Config.in.host
 create mode 100644 package/go-bin/go-bin.hash
 create mode 100644 package/go-bin/go-bin.mk
 create mode 100644 package/goc/Config.in.host
 create mode 100644 package/goc/goc.mk
 create mode 100644 support/testing/tests/package/test_go.py

-- 
2.37.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 3/4] package/go-bin: new package

[Thomas Perale via buildroot]

This package provides a pre-built version of the go compiler for the
host target.
This enable the user to select between the pre-built and the compiled
version of the go compiler. The pre-built version will be used by
default when a go compiler is required.

The virtual package 'goc' creates a menu that let the user select which
variant of the go compiler to use.

The supported host platform for the pre-built compiler are actually the
same as the one for 'go-bootstrap' no further verification are required
than the one already in place.

For now GO_BIN_VERSION and GO_VERSION are set the same by hand but in
next version this variable should be in common. Because the order of the
'.mk' file is not guaranteed this will be investigated in a next patch.

Signed-off-by: Thomas Perale <thomas.perale@mind.be>
---
 package/Config.in.host        |  1 +
 package/go-bin/Config.in.host |  5 +++++
 package/go-bin/go-bin.hash    |  4 ++++
 package/go-bin/go-bin.mk      | 20 ++++++++++++++++++++
 package/goc/Config.in.host    | 26 +++++++++++++++++++++++++-
 5 files changed, 55 insertions(+), 1 deletion(-)
 create mode 100644 package/go-bin/Config.in.host
 create mode 100644 package/go-bin/go-bin.hash
 create mode 100644 package/go-bin/go-bin.mk

diff --git a/package/Config.in.host b/package/Config.in.host
index 8c44ff0cb4..95b59bdacc 100644
--- a/package/Config.in.host
+++ b/package/Config.in.host
@@ -37,6 +37,7 @@ menu "Host utilities"
 	source "package/genpart/Config.in.host"
 	source "package/gnupg/Config.in.host"
 	source "package/go/Config.in.host"
+	source "package/go-bin/Config.in.host"
 	source "package/go-bootstrap/Config.in.host"
 	source "package/goc/Config.in.host"
 	source "package/google-breakpad/Config.in.host"
diff --git a/package/go-bin/Config.in.host b/package/go-bin/Config.in.host
new file mode 100644
index 0000000000..392060584f
--- /dev/null
+++ b/package/go-bin/Config.in.host
@@ -0,0 +1,5 @@
+config BR2_PACKAGE_HOST_GO_BIN_HOST_ARCH
+	string
+	default 386      if BR2_HOSTARCH = "x86"
+	default amd64    if BR2_HOSTARCH = "x86_64"
+	default arm64    if BR2_HOSTARCH = "arm"
diff --git a/package/go-bin/go-bin.hash b/package/go-bin/go-bin.hash
new file mode 100644
index 0000000000..91456d3e9a
--- /dev/null
+++ b/package/go-bin/go-bin.hash
@@ -0,0 +1,4 @@
+# sha256 checksum from https://go.dev/dl/
+sha256  9acc57342400c5b0c2da07b5b01b50da239dd4a7fad41a1fb56af8363ef4133f  go1.19.1.linux-386.tar.gz
+sha256  acc512fbab4f716a8f97a8b3fbaa9ddd39606a28be6c2515ef7c6c6311acffde  go1.19.1.linux-amd64.tar.gz
+sha256  49960821948b9c6b14041430890eccee58c76b52e2dbaafce971c3c38d43df9f  go1.19.1.linux-arm64.tar.gz
diff --git a/package/go-bin/go-bin.mk b/package/go-bin/go-bin.mk
new file mode 100644
index 0000000000..cf2db90532
--- /dev/null
+++ b/package/go-bin/go-bin.mk
@@ -0,0 +1,20 @@
+################################################################################
+#
+# go-bin
+#
+################################################################################
+
+GO_BIN_VERSION = 1.19.1
+GO_BIN_SITE = https://go.dev/dl
+GO_BIN_LICENSE = BSD-3-Clause
+GO_BIN_LICENSE_FILES = LICENSE
+
+HOST_GO_BIN_PROVIDES = host-goc
+
+HOST_GO_BIN_SOURCE = go$(GO_BIN_VERSION).linux-$(call qstrip, $(BR2_PACKAGE_HOST_GO_BIN_HOST_ARCH)).tar.gz
+
+define HOST_GO_BIN_INSTALL_CMDS
+	$(HOST_GO_INSTALL_CMDS)
+endef
+
+$(eval $(host-generic-package))
diff --git a/package/goc/Config.in.host b/package/goc/Config.in.host
index b0266b50ea..0bde50982f 100644
--- a/package/goc/Config.in.host
+++ b/package/goc/Config.in.host
@@ -8,8 +8,32 @@ config BR2_PACKAGE_HOST_GOC
 
 if BR2_PACKAGE_HOST_GOC
 
+choice
+	prompt "Go compiler variant"
+	default BR2_PACKAGE_HOST_GO_BIN
+	help
+	  Select a Go compiler variant.
+
+	  Default to 'host-go-bin'.
+
+config BR2_PACKAGE_HOST_GO
+	bool "host go"
+	depends on BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS
+	help
+	  This package will build the go compiler for the host.
+
+config BR2_PACKAGE_HOST_GO_BIN
+	bool "host go (pre-built)"
+	help
+	  This package will install pre-built versions of the compiler
+	  for the host.
+
+endchoice
+
 config BR2_PACKAGE_PROVIDES_HOST_GOC
 	string
-	default "host-go"
+	# Default to host-go-bin
+	default "host-go-bin" if BR2_PACKAGE_HOST_GO_BIN
+	default "host-go" if BR2_PACKAGE_HOST_GO
 
 endif
-- 
2.37.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot