[Buildroot] Using a go.mod in a subdirectory

[Buildroot] Using a go.mod in a subdirectory

[Bryce Johnson]

Hi All,
I was looking at adding a package for shellhub in buildroot.  Two
issues, first one is that the latest needs go 1.20.  I applied this
patch (https://patchwork.ozlabs.org/project/buildroot/list/?series=358451)
to get around that which seems to work so far.   CC: Christian since
I'm testing with his patch set.

The 2nd issue is the agent is in a subdirectory with its own go.mod
that is different from one the on the root directory.

Is there a way to do this better?  Or is the go.mod required to be in
the root of the repo?

SHELLHUB_VERSION = v0.12.3
SHELLHUB_SITE = $(call github,shellhub-io,shellhub,$(SHELLHUB_VERSION))
SHELLHUB_LICENSE = Apache-2.0
SHELLHUB_LICENSE_FILES = LICENSE.md
SHELLHUB_DEPENDENCIES = libxcrypt

SHELLHUB_GOMOD = github.com/shellhub-io/shellhub/agent
SHELLHUB_LDFLAGS =  -X main.AgentVersion=${SHELLHUB_VERSION}

# SHELLHUB_BUILD_TARGETS = agent
# SHELLHUB_INSTALL_BINS = $(notdir $(SHELLHUB_BUILD_TARGETS))

 $(eval $(golang-package))


I can export my output/host/usr/bin and do go build -ldflags "-X
main.AgentVersion=v0.12.3" and it is able to build in that way, so it
seems the host-go 1.20 patch is working.

Bryce
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Using a go.mod in a subdirectory

[Yann E. MORIN]

Bryce, All,

On 2023-06-23 10:59 -0600, Bryce Johnson spake thusly:
> Hi All,
> I was looking at adding a package for shellhub in buildroot.  Two
> issues, first one is that the latest needs go 1.20.  I applied this
> patch (https://patchwork.ozlabs.org/project/buildroot/list/?series=358451)
> to get around that which seems to work so far.   CC: Christian since
> I'm testing with his patch set.
> 
> The 2nd issue is the agent is in a subdirectory with its own go.mod
> that is different from one the on the root directory.
> 
> Is there a way to do this better?  Or is the go.mod required to be in
> the root of the repo?

When we introduced the go-package infra, we made the assumption that go
would be finely-grained packaged, with one single "go package" per
source tree, and that it would be at the root of the source tree.

The result of these assumptions are clearly visible in two location:

  - support/download/go-post-process expects that the vendoring and the
    go.mod are both at the root of the source tree; see lines 18 and 27;

  - package/pkg-golang.mk also expects that the go package lies at the
    root; see lines 111, 123, 139, and 148.

This restriction could certainly be lifted if support for FOO_SUB_DIR
was to be added to the go-package infra, but it is not straightforward.

When that is done, you'd need a buildroot package for each sub-part of
shellhub you need. Of course, in such a case, even though the different
parts would come from the same source tree, we could not share the
generated source archive with _DL_SUBDIR, becasue the different parts
would not be vendored similarly).

> SHELLHUB_VERSION = v0.12.3
> SHELLHUB_SITE = $(call github,shellhub-io,shellhub,$(SHELLHUB_VERSION))

We try to keep a "clean" version string without the decorum used to make
it a git tag, and make that part of the site URL, e.g.:

    SHELLHUB_VERSION = 0.12.3
    SHELLHUB_SITE = $(call github,shellhub-io,shellhub,v$(SHELLHUB_VERSION))

Regards,
Yann E. MORIN.

> SHELLHUB_LICENSE = Apache-2.0
> SHELLHUB_LICENSE_FILES = LICENSE.md
> SHELLHUB_DEPENDENCIES = libxcrypt
> 
> SHELLHUB_GOMOD = github.com/shellhub-io/shellhub/agent
> SHELLHUB_LDFLAGS =  -X main.AgentVersion=${SHELLHUB_VERSION}
> 
> # SHELLHUB_BUILD_TARGETS = agent
> # SHELLHUB_INSTALL_BINS = $(notdir $(SHELLHUB_BUILD_TARGETS))
> 
>  $(eval $(golang-package))
> 
> 
> I can export my output/host/usr/bin and do go build -ldflags "-X
> main.AgentVersion=v0.12.3" and it is able to build in that way, so it
> seems the host-go 1.20 patch is working.
> 
> Bryce
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Using a go.mod in a subdirectory

[Bryce Johnson]

Thanks for the info Yann.  I'm new to using go, so I'm not sure I'm
the best person to be making the mods, but I'll take a look.  In the
short term I might try and get it building locally by making a repo
with only the agent (the subdir package) and see if I can get that
building.

Bryce

On Sat, Jun 24, 2023 at 9:27 AM Yann E. MORIN <yann.morin.1998@free.fr> wrote:
>
> Bryce, All,
>
> On 2023-06-23 10:59 -0600, Bryce Johnson spake thusly:
> > Hi All,
> > I was looking at adding a package for shellhub in buildroot.  Two
> > issues, first one is that the latest needs go 1.20.  I applied this
> > patch (https://patchwork.ozlabs.org/project/buildroot/list/?series=358451)
> > to get around that which seems to work so far.   CC: Christian since
> > I'm testing with his patch set.
> >
> > The 2nd issue is the agent is in a subdirectory with its own go.mod
> > that is different from one the on the root directory.
> >
> > Is there a way to do this better?  Or is the go.mod required to be in
> > the root of the repo?
>
> When we introduced the go-package infra, we made the assumption that go
> would be finely-grained packaged, with one single "go package" per
> source tree, and that it would be at the root of the source tree.
>
> The result of these assumptions are clearly visible in two location:
>
>   - support/download/go-post-process expects that the vendoring and the
>     go.mod are both at the root of the source tree; see lines 18 and 27;
>
>   - package/pkg-golang.mk also expects that the go package lies at the
>     root; see lines 111, 123, 139, and 148.
>
> This restriction could certainly be lifted if support for FOO_SUB_DIR
> was to be added to the go-package infra, but it is not straightforward.
>
> When that is done, you'd need a buildroot package for each sub-part of
> shellhub you need. Of course, in such a case, even though the different
> parts would come from the same source tree, we could not share the
> generated source archive with _DL_SUBDIR, becasue the different parts
> would not be vendored similarly).
>
> > SHELLHUB_VERSION = v0.12.3
> > SHELLHUB_SITE = $(call github,shellhub-io,shellhub,$(SHELLHUB_VERSION))
>
> We try to keep a "clean" version string without the decorum used to make
> it a git tag, and make that part of the site URL, e.g.:
>
>     SHELLHUB_VERSION = 0.12.3
>     SHELLHUB_SITE = $(call github,shellhub-io,shellhub,v$(SHELLHUB_VERSION))
>
> Regards,
> Yann E. MORIN.
>
> > SHELLHUB_LICENSE = Apache-2.0
> > SHELLHUB_LICENSE_FILES = LICENSE.md
> > SHELLHUB_DEPENDENCIES = libxcrypt
> >
> > SHELLHUB_GOMOD = github.com/shellhub-io/shellhub/agent
> > SHELLHUB_LDFLAGS =  -X main.AgentVersion=${SHELLHUB_VERSION}
> >
> > # SHELLHUB_BUILD_TARGETS = agent
> > # SHELLHUB_INSTALL_BINS = $(notdir $(SHELLHUB_BUILD_TARGETS))
> >
> >  $(eval $(golang-package))
> >
> >
> > I can export my output/host/usr/bin and do go build -ldflags "-X
> > main.AgentVersion=v0.12.3" and it is able to build in that way, so it
> > seems the host-go 1.20 patch is working.
> >
> > Bryce
> > _______________________________________________
> > buildroot mailing list
> > buildroot@buildroot.org
> > https://lists.buildroot.org/mailman/listinfo/buildroot
>
> --
> .-----------------.--------------------.------------------.--------------------.
> |  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
> | +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
> | +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
> '------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot