[PATCH] go-mod.bbclass: Set GO_MOD_CACHE_DIR

[PATCH] go-mod.bbclass: Set GO_MOD_CACHE_DIR

[Christian Lindeberg]

From: Christian Lindeberg <christian.lindeberg@axis.com>

Set the GO_MOD_CACHE_DIR variable and move the location of the module
cache to enable the use of the go module fetchers for module
dependencies.
Also, clean out the module cache before unpacking.

Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com>
---
 meta/classes-recipe/go-mod.bbclass | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/meta/classes-recipe/go-mod.bbclass b/meta/classes-recipe/go-mod.bbclass
index ca3a690d05..71a6712c47 100644
--- a/meta/classes-recipe/go-mod.bbclass
+++ b/meta/classes-recipe/go-mod.bbclass
@@ -22,9 +22,9 @@ GOBUILDFLAGS:append = " -modcacherw"
 
 inherit go
 
+GO_MOD_CACHE_DIR ?= "${@os.path.join(os.path.basename(d.getVar('S')), 'pkg/mod')}"
+export GOMODCACHE = "${@os.path.join(os.path.dirname(d.getVar('S')), d.getVar('GO_MOD_CACHE_DIR'))}"
+do_unpack[cleandirs] += "${GOMODCACHE}"
+
 GO_WORKDIR ?= "${GO_IMPORT}"
 do_compile[dirs] += "${B}/src/${GO_WORKDIR}"
-
-export GOMODCACHE = "${B}/.mod"
-
-do_compile[cleandirs] += "${B}/.mod"
-- 
2.39.2

Re: [OE-core] [PATCH] go-mod.bbclass: Set GO_MOD_CACHE_DIR

[Jose Quaresma]

[-- Attachment #1: Type: text/plain, Size: 2314 bytes --]

Hi Christian,

Christian Lindeberg via lists.openembedded.org <christian.lindeberg=
axis.com@lists.openembedded.org> escreveu (quinta, 12/09/2024 à(s) 15:30):

> From: Christian Lindeberg <christian.lindeberg@axis.com>
>
> Set the GO_MOD_CACHE_DIR variable and move the location of the module
> cache to enable the use of the go module fetchers for module
> dependencies.
> Also, clean out the module cache before unpacking.
>
> Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com>
> ---
>  meta/classes-recipe/go-mod.bbclass | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/meta/classes-recipe/go-mod.bbclass
> b/meta/classes-recipe/go-mod.bbclass
> index ca3a690d05..71a6712c47 100644
> --- a/meta/classes-recipe/go-mod.bbclass
> +++ b/meta/classes-recipe/go-mod.bbclass
> @@ -22,9 +22,9 @@ GOBUILDFLAGS:append = " -modcacherw"
>
>  inherit go
>
> +GO_MOD_CACHE_DIR ?= "${@os.path.join(os.path.basename(d.getVar('S')),
> 'pkg/mod')}"
> +export GOMODCACHE = "${@os.path.join(os.path.dirname(d.getVar('S')),
> d.getVar('GO_MOD_CACHE_DIR'))}"
>

Using the default [1] bitbake S = "${WORKDIR}/${BP}" config, I think this
will be equal to:

GO_MOD_CACHE_DIR ?= "${BP}/pkg/mod"
export GOMODCACHE = "${WORKDIR}/${BP}/pkg/mod"

in the end this can be written as:

export GOMODCACHE ?= "${S}/pkg/mod"

Is it correct or am I seeing something wrong?

[1]
https://github.com/openembedded/openembedded-core/blob/40d2fbece1c2f6ecf62bffa44ad37850e90268cb/meta/conf/bitbake.conf#L405C6-L405C16

Jose

+do_unpack[cleandirs] += "${GOMODCACHE}"
> +
>  GO_WORKDIR ?= "${GO_IMPORT}"
>  do_compile[dirs] += "${B}/src/${GO_WORKDIR}"
> -
> -export GOMODCACHE = "${B}/.mod"
> -
> -do_compile[cleandirs] += "${B}/.mod"
> --
> 2.39.2
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#204444):
> https://lists.openembedded.org/g/openembedded-core/message/204444
> Mute This Topic: https://lists.openembedded.org/mt/108413937/5052612
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> quaresma.jose@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

-- 
Best regards,

José Quaresma

[-- Attachment #2: Type: text/html, Size: 4086 bytes --]

Re: [OE-core] [PATCH] go-mod.bbclass: Set GO_MOD_CACHE_DIR

[Ryan Eatmon]

On 9/12/2024 9:29 AM, Christian Lindeberg wrote:
> From: Christian Lindeberg <christian.lindeberg@axis.com>
> 
> Set the GO_MOD_CACHE_DIR variable and move the location of the module
> cache to enable the use of the go module fetchers for module
> dependencies.
> Also, clean out the module cache before unpacking.
> 
> Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com>
> ---
>   meta/classes-recipe/go-mod.bbclass | 8 ++++----
>   1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/meta/classes-recipe/go-mod.bbclass b/meta/classes-recipe/go-mod.bbclass
> index ca3a690d05..71a6712c47 100644
> --- a/meta/classes-recipe/go-mod.bbclass
> +++ b/meta/classes-recipe/go-mod.bbclass
> @@ -22,9 +22,9 @@ GOBUILDFLAGS:append = " -modcacherw"
>   
>   inherit go
>   
> +GO_MOD_CACHE_DIR ?= "${@os.path.join(os.path.basename(d.getVar('S')), 'pkg/mod')}"
> +export GOMODCACHE = "${@os.path.join(os.path.dirname(d.getVar('S')), d.getVar('GO_MOD_CACHE_DIR'))}"
> +do_unpack[cleandirs] += "${GOMODCACHE}"
> +
>   GO_WORKDIR ?= "${GO_IMPORT}"
>   do_compile[dirs] += "${B}/src/${GO_WORKDIR}"
> -
> -export GOMODCACHE = "${B}/.mod"
> -
> -do_compile[cleandirs] += "${B}/.mod"
> 


What is the reasoning behind moving the cache dir out of the build dir 
${B} and into the bitbake managed sources dir ${S} ?  There might be a 
good reason for doing it, but it was not explained in the commit message.




> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#204444): https://lists.openembedded.org/g/openembedded-core/message/204444
> Mute This Topic: https://lists.openembedded.org/mt/108413937/6551054
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [reatmon@ti.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 

-- 
Ryan Eatmon                reatmon@ti.com
-----------------------------------------
Texas Instruments, Inc.  -  LCPD  -  MGTS

RE: [OE-core] [PATCH] go-mod.bbclass: Set GO_MOD_CACHE_DIR

[Peter Kjellerstedt]

[-- Attachment #1: Type: text/plain, Size: 3587 bytes --]

[I can’t reply inline due to the HTML-formatting.]

I believe the goal is to have pkg/mod as a subdirectory of ${S}. At the same time, ${GO_MOD_CACHE_DIR} needs to be a relative path (as it is passed to the fetcher via the subdir= parameter). However, the suggested code only works for cases where ${S} matches ${WORKDIR}/subdirectory. In case it is ${WORKDIR}/sub/directory, it will fail. I would suggest to change the code like this:

export GOMODCACHE = "${S}/pkg/mod"
GO_MOD_CACHE_DIR = "${@os.path.relpath(d.getVar('GOMODCACHE'), d.getVar('WORKDIR'))}"
do_unpack[cleandirs] += "${GOMODCACHE}"

I have verified that the above fetches a Go module into the correct directory with S set to ${WORKDIR}/git/${BPN}. I have also verified that crucible (one of the recipes in meta-oe that uses the go-mod bbclass) still builds with the above. And as a bonus I now also have a patch that modifies the crucible recipe to use the gomod fetcher (which requires the above change and also the patch I just sent to the bitbake list for the gomod fetcher). Unfortunately I will not have time to finish it until I’m back from OSS in Vienna next week.

//Peter

From: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> On Behalf Of Jose Quaresma
Sent: den 12 september 2024 19:01
To: Christian Lindeberg <Christian.Lindeberg@axis.com>
Cc: openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH] go-mod.bbclass: Set GO_MOD_CACHE_DIR

Hi Christian,

Christian Lindeberg via lists.openembedded.org<http://lists.openembedded.org> <christian.lindeberg=axis.com@lists.openembedded.org<mailto:axis.com@lists.openembedded.org>> escreveu (quinta, 12/09/2024 à(s) 15:30):
From: Christian Lindeberg <christian.lindeberg@axis.com<mailto:christian.lindeberg@axis.com>>

Set the GO_MOD_CACHE_DIR variable and move the location of the module
cache to enable the use of the go module fetchers for module
dependencies.
Also, clean out the module cache before unpacking.

Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com<mailto:christian.lindeberg@axis.com>>
---
 meta/classes-recipe/go-mod.bbclass | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/meta/classes-recipe/go-mod.bbclass b/meta/classes-recipe/go-mod.bbclass
index ca3a690d05..71a6712c47 100644
--- a/meta/classes-recipe/go-mod.bbclass
+++ b/meta/classes-recipe/go-mod.bbclass
@@ -22,9 +22,9 @@ GOBUILDFLAGS:append = " -modcacherw"

 inherit go

+GO_MOD_CACHE_DIR ?= "${@os.path.join(os.path.basename(d.getVar('S')), 'pkg/mod')}<mailto:$%7b@os.path.join(os.path.basename(d.getVar('S')),%20'pkg/mod')%7d>"
+export GOMODCACHE = "${@os.path.join(os.path.dirname(d.getVar('S')), d.getVar('GO_MOD_CACHE_DIR'))}<mailto:$%7b@os.path.join(os.path.dirname(d.getVar('S')),%20d.getVar('GO_MOD_CACHE_DIR'))%7d>"

Using the default [1] bitbake S = "${WORKDIR}/${BP}" config, I think this will be equal to:

GO_MOD_CACHE_DIR ?= "${BP}/pkg/mod"
export GOMODCACHE = "${WORKDIR}/${BP}/pkg/mod"

in the end this can be written as:

export GOMODCACHE ?= "${S}/pkg/mod"

Is it correct or am I seeing something wrong?
[1] https://github.com/openembedded/openembedded-core/blob/40d2fbece1c2f6ecf62bffa44ad37850e90268cb/meta/conf/bitbake.conf#L405C6-L405C16

Jose

+do_unpack[cleandirs] += "${GOMODCACHE}"
+
 GO_WORKDIR ?= "${GO_IMPORT}"
 do_compile[dirs] += "${B}/src/${GO_WORKDIR}"
-
-export GOMODCACHE = "${B}/.mod"
-
-do_compile[cleandirs] += "${B}/.mod"
--
2.39.2





--
Best regards,

José Quaresma

[-- Attachment #2: Type: text/html, Size: 13368 bytes --]

Re: [PATCH] go-mod.bbclass: Set GO_MOD_CACHE_DIR

[Christian Lindeberg]

[-- Attachment #1: Type: text/plain, Size: 2213 bytes --]

On Thu, Sep 12, 2024 at 07:01 PM, Jose Quaresma wrote:

> 
> Hi Christian,
> 
> Christian Lindeberg via lists.openembedded.org (
> http://lists.openembedded.org ) <christian.lindeberg= axis.com@lists.openembedded.org
> > escreveu (quinta, 12/09/2024 à(s) 15:30):
> 
>> From: Christian Lindeberg < christian.lindeberg@axis.com >
>> 
>> Set the GO_MOD_CACHE_DIR variable and move the location of the module
>> cache to enable the use of the go module fetchers for module
>> dependencies.
>> Also, clean out the module cache before unpacking.
>> 
>> Signed-off-by: Christian Lindeberg < christian.lindeberg@axis.com >
>> ---
>> meta/classes-recipe/go-mod.bbclass | 8 ++++----
>> 1 file changed, 4 insertions(+), 4 deletions(-)
>> 
>> diff --git a/meta/classes-recipe/go-mod.bbclass
>> b/meta/classes-recipe/go-mod.bbclass
>> index ca3a690d05..71a6712c47 100644
>> --- a/meta/classes-recipe/go-mod.bbclass
>> +++ b/meta/classes-recipe/go-mod.bbclass
>> @@ -22,9 +22,9 @@ GOBUILDFLAGS:append = " -modcacherw"
>> 
>> inherit go
>> 
>> +GO_MOD_CACHE_DIR ?= "${@os.path.join(os.path.basename(d.getVar('S')),
>> 'pkg/mod')}"
>> +export GOMODCACHE = "${@os.path.join(os.path.dirname(d.getVar('S')),
>> d.getVar('GO_MOD_CACHE_DIR'))}"
> 
> 
> Using the default [1] bitbake S = "${WORKDIR}/${BP}" config, I think this
> will be equal to:
> 
> GO_MOD_CACHE_DIR ?= "${BP}/pkg/mod"
> export GOMODCACHE = "${WORKDIR}/${BP}/pkg/mod"
> 
> in the end this can be written as:
> 
> export GOMODCACHE ?= "${S}/pkg/mod"
> 
> Is it correct or am I seeing something wrong?
> 

Correct, I sent a new version with this and the suggestion from Peter Kjellerstedt.

Thanks,
Christian

> 
> 
> [1] https://github.com/openembedded/openembedded-core/blob/40d2fbece1c2f6ecf62bffa44ad37850e90268cb/meta/conf/bitbake.conf#L405C6-L405C16
> 
> 
> Jose
> 
> 
>> +do_unpack[cleandirs] += "${GOMODCACHE}"
>> +
>> GO_WORKDIR ?= "${GO_IMPORT}"
>> do_compile[dirs] += "${B}/src/${GO_WORKDIR}"
>> -
>> -export GOMODCACHE = "${B}/.mod"
>> -
>> -do_compile[cleandirs] += "${B}/.mod"
>> --
>> 2.39.2
>> 
>> 
>> 
>> 
> 
> 
> 
> 
> --
> Best regards,
> 
> José Quaresma
>

[-- Attachment #2: Type: text/html, Size: 3547 bytes --]

Re: [PATCH] go-mod.bbclass: Set GO_MOD_CACHE_DIR

[Christian Lindeberg]

[-- Attachment #1: Type: text/plain, Size: 2053 bytes --]

On Thu, Sep 12, 2024 at 10:14 PM, Ryan Eatmon wrote:

> 
> On 9/12/2024 9:29 AM, Christian Lindeberg wrote:
> 
>> From: Christian Lindeberg <christian.lindeberg@axis.com>
>> 
>> Set the GO_MOD_CACHE_DIR variable and move the location of the module
>> cache to enable the use of the go module fetchers for module
>> dependencies.
>> Also, clean out the module cache before unpacking.
>> 
>> Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com>
>> ---
>> meta/classes-recipe/go-mod.bbclass | 8 ++++----
>> 1 file changed, 4 insertions(+), 4 deletions(-)
>> 
>> diff --git a/meta/classes-recipe/go-mod.bbclass
>> b/meta/classes-recipe/go-mod.bbclass
>> index ca3a690d05..71a6712c47 100644
>> --- a/meta/classes-recipe/go-mod.bbclass
>> +++ b/meta/classes-recipe/go-mod.bbclass
>> @@ -22,9 +22,9 @@ GOBUILDFLAGS:append = " -modcacherw"
>> 
>> inherit go
>> 
>> +GO_MOD_CACHE_DIR ?= "${@os.path.join(os.path.basename(d.getVar('S')),
>> 'pkg/mod')}"
>> +export GOMODCACHE = "${@os.path.join(os.path.dirname(d.getVar('S')),
>> d.getVar('GO_MOD_CACHE_DIR'))}"
>> +do_unpack[cleandirs] += "${GOMODCACHE}"
>> +
>> GO_WORKDIR ?= "${GO_IMPORT}"
>> do_compile[dirs] += "${B}/src/${GO_WORKDIR}"
>> -
>> -export GOMODCACHE = "${B}/.mod"
>> -
>> -do_compile[cleandirs] += "${B}/.mod"
> 
> 
> What is the reasoning behind moving the cache dir out of the build dir
> ${B} and into the bitbake managed sources dir ${S} ? There might be a
> good reason for doing it, but it was not explained in the commit message.

In the review of the gomod and gomodgit fetchers concerns were raised about
letting the fetchers unpack under ${GOMODCACHE} instead of relative to the
unpack root directory.

With the fetchers there is now a way to have the module cache part of the
bitbake managed sources instead of having to resort to
do_compile[network] = "1".

Thanks,
Christian

> 
> 
> 
> 
> --
> Ryan Eatmon reatmon@ti.com
> -----------------------------------------
> Texas Instruments, Inc. - LCPD - MGTS

[-- Attachment #2: Type: text/html, Size: 2309 bytes --]