[langdale][kirkstone][PATCH] go-mod.bbclass: Allow network in do_compile

[langdale][kirkstone][PATCH] go-mod.bbclass: Allow network in do_compile

[Vyacheslav Yurkov]

This is a workaround to allow using go-mod until a proper solution is in
place. Possible solutions were discussued ealier this year in these
threads:

https://lists.openembedded.org/g/openembedded-core/topic/89464905
https://lists.openembedded.org/g/openembedded-architecture/topic/90782883

Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
---
 meta/classes-recipe/go-mod.bbclass | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/meta/classes-recipe/go-mod.bbclass b/meta/classes-recipe/go-mod.bbclass
index 61571596bc..da9a583288 100644
--- a/meta/classes-recipe/go-mod.bbclass
+++ b/meta/classes-recipe/go-mod.bbclass
@@ -24,3 +24,8 @@ inherit go
 
 GO_WORKDIR ?= "${GO_IMPORT}"
 do_compile[dirs] += "${B}/src/${GO_WORKDIR}"
+do_compile[network] = "1"
+
+do_compile:prepend() {
+    bbwarn "Using go-mod enables network during compilation to be able to fetch dependencies. This breaks reproducibility of the build!"
+}
-- 
2.35.1

Re: [OE-core] [langdale][kirkstone][PATCH] go-mod.bbclass: Allow network in do_compile

[Richard Purdie]

On Thu, 2022-12-22 at 16:02 +0100, Vyacheslav Yurkov wrote:
> This is a workaround to allow using go-mod until a proper solution is in
> place. Possible solutions were discussued ealier this year in these
> threads:
> 
> https://lists.openembedded.org/g/openembedded-core/topic/89464905
> https://lists.openembedded.org/g/openembedded-architecture/topic/90782883
> 
> Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
> ---
>  meta/classes-recipe/go-mod.bbclass | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/meta/classes-recipe/go-mod.bbclass b/meta/classes-recipe/go-mod.bbclass
> index 61571596bc..da9a583288 100644
> --- a/meta/classes-recipe/go-mod.bbclass
> +++ b/meta/classes-recipe/go-mod.bbclass
> @@ -24,3 +24,8 @@ inherit go
>  
>  GO_WORKDIR ?= "${GO_IMPORT}"
>  do_compile[dirs] += "${B}/src/${GO_WORKDIR}"
> +do_compile[network] = "1"
> +
> +do_compile:prepend() {
> +    bbwarn "Using go-mod enables network during compilation to be able to fetch dependencies. This breaks reproducibility of the build!"
> +}

This patch is not in master and is not a backport therefore not
eligible for kirkstone/langdale.

Cheers,

Richard

Re: [OE-core] [langdale][kirkstone][PATCH] go-mod.bbclass: Allow network in do_compile

[Bruce Ashfield]

On Thu, Dec 22, 2022 at 11:20 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Thu, 2022-12-22 at 16:02 +0100, Vyacheslav Yurkov wrote:
> > This is a workaround to allow using go-mod until a proper solution is in
> > place. Possible solutions were discussued ealier this year in these
> > threads:
> >
> > https://lists.openembedded.org/g/openembedded-core/topic/89464905
> > https://lists.openembedded.org/g/openembedded-architecture/topic/90782883
> >
> > Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
> > ---
> >  meta/classes-recipe/go-mod.bbclass | 5 +++++
> >  1 file changed, 5 insertions(+)
> >
> > diff --git a/meta/classes-recipe/go-mod.bbclass b/meta/classes-recipe/go-mod.bbclass
> > index 61571596bc..da9a583288 100644
> > --- a/meta/classes-recipe/go-mod.bbclass
> > +++ b/meta/classes-recipe/go-mod.bbclass
> > @@ -24,3 +24,8 @@ inherit go
> >
> >  GO_WORKDIR ?= "${GO_IMPORT}"
> >  do_compile[dirs] += "${B}/src/${GO_WORKDIR}"
> > +do_compile[network] = "1"
> > +
> > +do_compile:prepend() {
> > +    bbwarn "Using go-mod enables network during compilation to be able to fetch dependencies. This breaks reproducibility of the build!"
> > +}
>
> This patch is not in master and is not a backport therefore not
> eligible for kirkstone/langdale.
>

'nor should it go to master.

If someone wants to allow go to fetch sources during builds, it needs
to be done in their own layers.

Bruce

> Cheers,
>
> Richard
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#174968): https://lists.openembedded.org/g/openembedded-core/message/174968
> Mute This Topic: https://lists.openembedded.org/mt/95827446/1050810
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [langdale][kirkstone][PATCH] go-mod.bbclass: Allow network in do_compile

[Vyacheslav Yurkov]

On 22.12.2022 17:48, Bruce Ashfield wrote:
> On Thu, Dec 22, 2022 at 11:20 AM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
>> This patch is not in master and is not a backport therefore not
>> eligible for kirkstone/langdale.

My bad, the intention was to have it in all three branches: master, 
kirkstone, and langdale.

> 'nor should it go to master.
>
> If someone wants to allow go to fetch sources during builds, it needs
> to be done in their own layers.
>
> Bruce

Probably I misunderstood the outcome of the discussion in my first link. 
Bruce and Richard, you both suggested to have it in go-mod with the 
warning. I agree that it breaks reproducibility, and is considered a bad 
practice in general (I'm aware of a few more build systems with the same 
shortcoming).

My point is, until a proper solution is in place, it should be at least 
documented somewhere that this should be done in own layer/recipe. Any 
suggestions where this can be documented if not in OE-Core?

Vyacheslav

Re: [OE-core] [langdale][kirkstone][PATCH] go-mod.bbclass: Allow network in do_compile

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 5050 bytes --]

On Thu, Dec 22, 2022 at 7:39 PM Vyacheslav Yurkov <uvv.mail@gmail.com>
wrote:

> On 22.12.2022 17:48, Bruce Ashfield wrote:
> > On Thu, Dec 22, 2022 at 11:20 AM Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> >> This patch is not in master and is not a backport therefore not
> >> eligible for kirkstone/langdale.
>
> My bad, the intention was to have it in all three branches: master,
> kirkstone, and langdale.
>
> > 'nor should it go to master.
> >
> > If someone wants to allow go to fetch sources during builds, it needs
> > to be done in their own layers.
> >
> > Bruce
>
> Probably I misunderstood the outcome of the discussion in my first link.
> Bruce and Richard, you both suggested to have it in go-mod with the
> warning. I agree that it breaks reproducibility, and is considered a bad
> practice in general (I'm aware of a few more build systems with the same
> shortcoming).
>
> My point is, until a proper solution is in place, it should be at least
> documented somewhere that this should be done in own layer/recipe. Any
> suggestions where this can be documented if not in OE-Core?
>

Have you tried the changes submitted by Lukas/Stefan in:
https://lists.openembedded.org/g/openembedded-architecture/message/1539
?

It's not perfect, I was testing it on
https://github.com/influxdata/telegraf/blob/master/go.mod and I've found
some corner cases where it failed.

But it seems like very good start and we should work with Lukas/Stefan to
get it merged in master. Then all branches could consume recipes created in
master and only the exceptions would need to have network access (instead
of the bbclass enabling it for every go-mod user - go-vendor inherits
go-mod as well).

Lukas/Stefan: is there something we can do to help with v2 of your patches?

I wanted to submit better review on submitted patches after more debugging,
but here it is:

1) to resolve 2 deps:
{'Path': 'gonum.org/v1/gonum', 'Version': 'v0.12.0'} due to 404 "ERROR:
Error while fetching redirect page: HTTP Error 404: Not Found"

{'Path': 'code.cloudfoundry.org/clock', 'Version': 'v1.0.0', 'Indirect':
True} due to this being redirect to github.com and failing with "ERROR:
Error while fetching redirect page: <urlopen error [SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch,
certificate is not valid for 'code.cloudfoundry.org'. (_ssl.c:996)>"
parse

2) some issues with LICENSEs:
INFO: Please add the following line for 'docs/LICENSE_OF_DEPENDENCIES.md'
to a 'lib/recipetool/licenses.csv' and replace `Unknown` with the license:
c4807c5b058a56c02f592207b4e056b1,Unknown
Traceback (most recent call last):
  File "/OE/build/oe-core/openembedded-core/scripts/recipetool", line 111,
in <module>
    ret = main()
          ^^^^^^
  File "/OE/build/oe-core/openembedded-core/scripts/recipetool", line 100,
in main
    ret = args.func(args)
          ^^^^^^^^^^^^^^^
  File "/OE/layers/openembedded-core/scripts/lib/recipetool/create.py",
line 746, in create_recipe
    handler.process(srctree_use, classes, lines_before, lines_after,
handled, extravalues)
  File "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
line 349, in process
    self._rewrite_lic_uri(lines_before)
  File "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
line 380, in _rewrite_lic_uri
    updated, newlines = bb.utils.edit_metadata(lines_before,
['LIC_FILES_CHKSUM'], varfunc)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/OE/build/oe-core/bitbake/lib/bb/utils.py", line 1382, in
edit_metadata
    if handle_var_end():
       ^^^^^^^^^^^^^^^^
  File "/OE/build/oe-core/bitbake/lib/bb/utils.py", line 1280, in
handle_var_end
    (newvalue, newop, indent, minbreak) = varfunc(in_var, full_value, op,
newlines)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
line 372, in varfunc
    uri, chksum = license.split(';', 1)
    ^^^^^^^^^^^
ValueError: not enough values to unpack (expected 2, got 1)

(the license was just '/')

3) some SRCREVs set to None leading to parsing failure:

  File "/OE/layers/bitbake/lib/bb/fetch2/git.py", line 762, in
_latest_revision
    raise bb.fetch2.FetchError("Unable to resolve '%s' in upstream git
repository in git ls-remote output for %s" % \
bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression
was ${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError:
Fetcher failure: Unable to resolve 'None' in upstream git repository in git
ls-remote output for github.com/googleapis/google-cloud-go
The variable dependency chain for the failure is: SRCPV -> PV -> BP ->
FILESPATH

ERROR: Parsing newly created recipe failed, moving recipe to
/OE/build/oe-core/workspace/recipes/telegraf/telegraf_git.bb.parsefailed
for reference. If this looks to be caused by the recipe itself, please
report this error.

It takes long time to fetch all telegraf dependencies, so I'll create some
test recipe to more easily debug these 3 issues in isolation.

[-- Attachment #2: Type: text/html, Size: 6776 bytes --]

Re: [OE-core] [langdale][kirkstone][PATCH] go-mod.bbclass: Allow network in do_compile

[Vyacheslav Yurkov]

[-- Attachment #1: Type: text/plain, Size: 4716 bytes --]

Martin, I did see that patch series, but I was not sure about its state, 
therefore I didn't try it. If I understood correctly, that partially 
solves the issue, but the proper solution should involve the fetcher, 
that's why it's not yet merged to master. Am I missing something?

Regards,
Slava

On 22.12.2022 20:08, Martin Jansa wrote:
>
> Have you tried the changes submitted by Lukas/Stefan in:
> https://lists.openembedded.org/g/openembedded-architecture/message/1539
> ?
>
> It's not perfect, I was testing it on 
> https://github.com/influxdata/telegraf/blob/master/go.mod and I've 
> found some corner cases where it failed.
>
> But it seems like very good start and we should work with Lukas/Stefan 
> to get it merged in master. Then all branches could consume recipes 
> created in master and only the exceptions would need to have network 
> access (instead of the bbclass enabling it for every go-mod user - 
> go-vendor inherits go-mod as well).
>
> Lukas/Stefan: is there something we can do to help with v2 of your 
> patches?
>
> I wanted to submit better review on submitted patches after more 
> debugging, but here it is:
>
> 1) to resolve 2 deps:
> {'Path': 'gonum.org/v1/gonum <http://gonum.org/v1/gonum>', 'Version': 
> 'v0.12.0'} due to 404 "ERROR: Error while fetching redirect page: HTTP 
> Error 404: Not Found"
>
> {'Path': 'code.cloudfoundry.org/clock 
> <http://code.cloudfoundry.org/clock>', 'Version': 'v1.0.0', 
> 'Indirect': True} due to this being redirect to github.com 
> <http://github.com> and failing with "ERROR: Error while fetching 
> redirect page: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] 
> certificate verify failed: Hostname mismatch, certificate is not valid 
> for 'code.cloudfoundry.org <http://code.cloudfoundry.org>'. (_ssl.c:996)>"
> parse
>
> 2) some issues with LICENSEs:
> INFO: Please add the following line for 
> 'docs/LICENSE_OF_DEPENDENCIES.md' to a 'lib/recipetool/licenses.csv' 
> and replace `Unknown` with the license:
> c4807c5b058a56c02f592207b4e056b1,Unknown
> Traceback (most recent call last):
>   File "/OE/build/oe-core/openembedded-core/scripts/recipetool", line 
> 111, in <module>
>     ret = main()
>           ^^^^^^
>   File "/OE/build/oe-core/openembedded-core/scripts/recipetool", line 
> 100, in main
>     ret = args.func(args)
>           ^^^^^^^^^^^^^^^
>   File 
> "/OE/layers/openembedded-core/scripts/lib/recipetool/create.py", line 
> 746, in create_recipe
>     handler.process(srctree_use, classes, lines_before, lines_after, 
> handled, extravalues)
>   File 
> "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py", 
> line 349, in process
>     self._rewrite_lic_uri(lines_before)
>   File 
> "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py", 
> line 380, in _rewrite_lic_uri
>     updated, newlines = bb.utils.edit_metadata(lines_before, 
> ['LIC_FILES_CHKSUM'], varfunc)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "/OE/build/oe-core/bitbake/lib/bb/utils.py", line 1382, in 
> edit_metadata
>     if handle_var_end():
>        ^^^^^^^^^^^^^^^^
>   File "/OE/build/oe-core/bitbake/lib/bb/utils.py", line 1280, in 
> handle_var_end
>     (newvalue, newop, indent, minbreak) = varfunc(in_var, full_value, 
> op, newlines)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File 
> "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py", 
> line 372, in varfunc
>     uri, chksum = license.split(';', 1)
>     ^^^^^^^^^^^
> ValueError: not enough values to unpack (expected 2, got 1)
>
> (the license was just '/')
>
> 3) some SRCREVs set to None leading to parsing failure:
>
>   File "/OE/layers/bitbake/lib/bb/fetch2/git.py", line 762, in 
> _latest_revision
>     raise bb.fetch2.FetchError("Unable to resolve '%s' in upstream git 
> repository in git ls-remote output for %s" % \
> bb.data_smart.ExpansionError: Failure expanding variable SRCPV, 
> expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception 
> FetchError: Fetcher failure: Unable to resolve 'None' in upstream git 
> repository in git ls-remote output for 
> github.com/googleapis/google-cloud-go 
> <http://github.com/googleapis/google-cloud-go>
> The variable dependency chain for the failure is: SRCPV -> PV -> BP -> 
> FILESPATH
>
> ERROR: Parsing newly created recipe failed, moving recipe to 
> /OE/build/oe-core/workspace/recipes/telegraf/telegraf_git.bb.parsefailed 
> for reference. If this looks to be caused by the recipe itself, please 
> report this error.
>
> It takes long time to fetch all telegraf dependencies, so I'll create 
> some test recipe to more easily debug these 3 issues in isolation.
>

[-- Attachment #2: Type: text/html, Size: 7608 bytes --]

Re: [OE-core] [langdale][kirkstone][PATCH] go-mod.bbclass: Allow network in do_compile

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1396 bytes --]

On Mon, Dec 26, 2022 at 10:38 AM Vyacheslav Yurkov <uvv.mail@gmail.com>
wrote:

> Martin, I did see that patch series, but I was not sure about its state,
> therefore I didn't try it. If I understood correctly, that partially solves
> the issue, but the proper solution should involve the fetcher, that's why
> it's not yet merged to master. Am I missing something?
>

That's not my understanding.

The proposed changes for recipetool had some cosmetic and functional review
comments and it doesn't work correctly for every possible go dependency
(e.g. the 3 issues I've mentioned from telegram recipe), but it's very good
starting point and we should be able to get it merged in master (and then
everybody can more easily contribute fixes for the corner cases where it
might not work well yet).

Even missing documentation IMHO shouldn't block merging it at this point,
because there is a clear need this (as there are already 3 partial
implementations to fix this issue, neither 100% complete and nothing merged
in master - the recipetool changes are IMHO the best of 3 as it integrates
with devtool as well and seems to be most complete).

go fetcher in bitbake is another possibility for future, but generating the
long deps in SRC_URI is much better solution then enabling network in every
go-mod recipe.

Please try recipetool changes on your go-mod recipes and provide feedback.

Regards,

[-- Attachment #2: Type: text/html, Size: 1896 bytes --]

Re: [OE-core] [langdale][kirkstone][PATCH] go-mod.bbclass: Allow network in do_compile

[Vyacheslav Yurkov]

[-- Attachment #1: Type: text/plain, Size: 10839 bytes --]

On 26.12.2022 10:59, Martin Jansa wrote:
>
> Please try recipetool changes on your go-mod recipes and provide feedback.
>

Coincidentally, I also needed grpc-web recipe, which original patchset 
was tested on.
I applied the series on my kirkstone branch of poky and tried to run the 
command:

% recipetool create -S c1971f71a28950d212b7ddc8df07b5563723031d -B 
master https://github.com/improbable-eng/grpc-web.git

Unfortunately this failed with the following log:

WARNING: go.mod files generated by Go < 1.17 might have incomplete 
indirect dependencies.
Traceback (most recent call last):
   File "/home/uvv/projects/poky/scripts/recipetool", line 111, in <module>
     ret = main()
   File "/home/uvv/projects/poky/scripts/recipetool", line 100, in main
     ret = args.func(args)
   File "/home/uvv/projects/poky/scripts/lib/recipetool/create.py", line 
746, in create_recipe
     handler.process(srctree_use, classes, lines_before, lines_after, 
handled, extravalues)
   File "/home/uvv/projects/poky/scripts/lib/recipetool/create_go.py", 
line 332, in process
     go_mod = self._go_mod_patch(patchfile, go_import, srctree, 
localfilesdir, extravalues, d)
   File "/home/uvv/projects/poky/scripts/lib/recipetool/create_go.py", 
line 291, in _go_mod_patch
     bb.process.run("go mod tidy", stderr=subprocess.STDOUT, env=runenv, 
shell=True, cwd=srctree)
   File "/home/uvv/projects/poky/bitbake/lib/bb/process.py", line 189, 
in run
     raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
bb.process.ExecutionError: Execution of 'go mod tidy' failed with exit 
code 1:
go: finding module for package google.golang.org/grpc/metadata
go: finding module for package google.golang.org/grpc
go: finding module for package github.com/golang/protobuf/protoc-gen-go
go: finding module for package golang.org/x/net/context
go: finding module for package google.golang.org/grpc/codes
go: finding module for package google.golang.org/grpc/grpclog
go: downloading golang.org/x/net v0.4.0
go: downloading google.golang.org/grpc v1.51.0
go: downloading github.com/golang/protobuf v1.5.2
go: finding module for package github.com/cenkalti/backoff/v4
go: downloading github.com/cenkalti/backoff/v4 v4.2.0
go: finding module for package github.com/desertbit/timer
go: downloading github.com/cenkalti/backoff v2.2.1+incompatible
go: finding module for package github.com/rs/cors
go: finding module for package golang.org/x/net/http2
go: finding module for package google.golang.org/grpc/health/grpc_health_v1
go: finding module for package google.golang.org/grpc/status
go: finding module for package nhooyr.io/websocket
go: downloading github.com/desertbit/timer 
v0.0.0-20180107155436-c41aec40b27f
go: finding module for package github.com/grpc-ecosystem/go-grpc-middleware
go: finding module for package 
github.com/grpc-ecosystem/go-grpc-middleware/logging/logrus
go: downloading github.com/rs/cors v1.8.3
go: finding module for package github.com/grpc-ecosystem/go-grpc-prometheus
go: finding module for package github.com/mwitkow/go-conntrack
go: downloading nhooyr.io/websocket v1.8.7
go: finding module for package github.com/mwitkow/go-conntrack/connhelpers
go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: downloading github.com/grpc-ecosystem/go-grpc-middleware v1.3.0
go: finding module for package github.com/mwitkow/grpc-proxy/proxy
go: finding module for package 
github.com/prometheus/client_golang/prometheus/promhttp
go: downloading github.com/mwitkow/go-conntrack 
v0.0.0-20190716064945-2f068394615f
go: finding module for package github.com/sirupsen/logrus
go: finding module for package github.com/spf13/pflag
go: finding module for package golang.org/x/net/trace
go: finding module for package google.golang.org/grpc/credentials
go: finding module for package github.com/golang/protobuf/ptypes/empty
go: downloading github.com/mwitkow/grpc-proxy 
v0.0.0-20220126150247-db34e7bfee32
go: finding module for package github.com/golang/protobuf/proto
go: downloading github.com/spf13/pflag v1.0.5
go: downloading github.com/sirupsen/logrus v1.9.0
go: finding module for package github.com/stretchr/testify/assert
go: finding module for package github.com/stretchr/testify/require
go: finding module for package github.com/stretchr/testify/suite
go: finding module for package google.golang.org/grpc/health
go: finding module for package 
google.golang.org/protobuf/reflect/protoreflect
go: downloading github.com/prometheus/client_golang v1.14.0
go: downloading github.com/stretchr/testify v1.8.1
go: finding module for package google.golang.org/protobuf/runtime/protoimpl
go: finding module for package 
google.golang.org/protobuf/types/known/emptypb
go: downloading google.golang.org/protobuf v1.28.1
go: found github.com/golang/protobuf/protoc-gen-go in 
github.com/golang/protobuf v1.5.2
go: found golang.org/x/net/context in golang.org/x/net v0.4.0
go: found google.golang.org/grpc in google.golang.org/grpc v1.51.0
go: found google.golang.org/grpc/codes in google.golang.org/grpc v1.51.0
go: found google.golang.org/grpc/grpclog in google.golang.org/grpc v1.51.0
go: found google.golang.org/grpc/metadata in google.golang.org/grpc v1.51.0
go: found github.com/cenkalti/backoff/v4 in 
github.com/cenkalti/backoff/v4 v4.2.0
go: found github.com/desertbit/timer in github.com/desertbit/timer 
v0.0.0-20180107155436-c41aec40b27f
go: found github.com/rs/cors in github.com/rs/cors v1.8.3
go: found golang.org/x/net/http2 in golang.org/x/net v0.4.0
go: found google.golang.org/grpc/health/grpc_health_v1 in 
google.golang.org/grpc v1.51.0
go: found google.golang.org/grpc/status in google.golang.org/grpc v1.51.0
go: found nhooyr.io/websocket in nhooyr.io/websocket v1.8.7
go: found github.com/grpc-ecosystem/go-grpc-middleware in 
github.com/grpc-ecosystem/go-grpc-middleware v1.3.0
go: found github.com/grpc-ecosystem/go-grpc-middleware/logging/logrus in 
github.com/grpc-ecosystem/go-grpc-middleware v1.3.0
go: found github.com/grpc-ecosystem/go-grpc-prometheus in 
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: found github.com/mwitkow/go-conntrack in 
github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f
go: found github.com/mwitkow/go-conntrack/connhelpers in 
github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f
go: found github.com/mwitkow/grpc-proxy/proxy in 
github.com/mwitkow/grpc-proxy v0.0.0-20220126150247-db34e7bfee32
go: found github.com/prometheus/client_golang/prometheus/promhttp in 
github.com/prometheus/client_golang v1.14.0
go: found github.com/sirupsen/logrus in github.com/sirupsen/logrus v1.9.0
go: found github.com/spf13/pflag in github.com/spf13/pflag v1.0.5
go: found golang.org/x/net/trace in golang.org/x/net v0.4.0
go: found google.golang.org/grpc/credentials in google.golang.org/grpc 
v1.51.0
go: found github.com/golang/protobuf/ptypes/empty in 
github.com/golang/protobuf v1.5.2
go: found github.com/golang/protobuf/proto in github.com/golang/protobuf 
v1.5.2
go: found github.com/stretchr/testify/assert in 
github.com/stretchr/testify v1.8.1
go: found github.com/stretchr/testify/require in 
github.com/stretchr/testify v1.8.1
go: found github.com/stretchr/testify/suite in 
github.com/stretchr/testify v1.8.1
go: found google.golang.org/grpc/health in google.golang.org/grpc v1.51.0
go: found google.golang.org/protobuf/reflect/protoreflect in 
google.golang.org/protobuf v1.28.1
go: found google.golang.org/protobuf/runtime/protoimpl in 
google.golang.org/protobuf v1.28.1
go: found google.golang.org/protobuf/types/known/emptypb in 
google.golang.org/protobuf v1.28.1
go: downloading google.golang.org/genproto 
v0.0.0-20210401141331-865547bb08e2
go: downloading github.com/google/go-cmp v0.5.6
go: downloading github.com/klauspost/compress v1.10.3
go: downloading github.com/gin-gonic/gin v1.6.3
go: downloading github.com/gobwas/ws v1.0.2
go: downloading github.com/gorilla/websocket v1.4.1
go: downloading github.com/davecgh/go-spew v1.1.1
go: downloading github.com/pmezard/go-difflib v1.0.0
go: downloading gopkg.in/yaml.v3 v3.0.1
go: downloading github.com/jpillora/backoff v1.0.0
go: downloading github.com/prometheus/client_model v0.3.0
go: downloading github.com/prometheus/common v0.37.0
go: downloading golang.org/x/sys v0.3.0
go: downloading github.com/gin-contrib/sse v0.1.0
go: downloading github.com/mattn/go-isatty v0.0.12
go: downloading github.com/gobwas/httphead 
v0.0.0-20180130184737-2c6c146eadee
go: downloading github.com/gobwas/pool v0.2.0
go: downloading golang.org/x/text v0.5.0
go: downloading gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405
go: downloading github.com/beorn7/perks v1.0.1
go: downloading github.com/cespare/xxhash/v2 v2.1.2
go: downloading github.com/prometheus/procfs v0.8.0
go: downloading golang.org/x/sync v0.0.0-20210220032951-036812b2e83c
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: downloading golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1
go: downloading github.com/go-playground/validator/v10 v10.2.0
go: downloading github.com/ugorji/go/codec v1.1.7
go: downloading gopkg.in/yaml.v2 v2.4.0
go: downloading github.com/json-iterator/go v1.1.12
go: downloading github.com/ugorji/go v1.1.7
go: downloading github.com/gogo/protobuf v1.3.2
go: downloading github.com/go-playground/universal-translator v0.17.0
go: downloading github.com/leodido/go-urn v1.2.0
go: downloading github.com/modern-go/concurrent 
v0.0.0-20180228061459-e0a39a4cb421
go: downloading github.com/modern-go/reflect2 v1.0.2
go: downloading github.com/go-playground/locales v0.13.0
go: downloading github.com/google/go-cmp v0.5.8
go: downloading golang.org/x/sync v0.0.0-20220601150217-0de741cfad7f
go: downloading gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15
go: downloading github.com/modern-go/concurrent 
v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading github.com/kr/pretty v0.1.0
go: downloading github.com/kr/text v0.1.0
go: downloading golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4
github.com/improbable-eng/grpc-web/go/grpcwebproxy imports
         github.com/mwitkow/grpc-proxy/proxy tested by
         github.com/mwitkow/grpc-proxy/proxy.test imports
         github.com/mwitkow/grpc-proxy/testservice imports
         golang.org/x/sync/errgroup loaded from 
golang.org/x/sync@v0.0.0-20220601150217-0de741cfad7f,
         but go 1.16 would select v0.0.0-20220722155255-886fb9371eb4

To upgrade to the versions selected by go 1.16:
         go mod tidy -go=1.16 && go mod tidy -go=1.17
If reproducibility with go 1.16 is not needed:
         go mod tidy -compat=1.17
For other options, see:
https://golang.org/doc/modules/pruning


Kirkstone uses go1.17, but for some reason it's not detected properly.
Perhaps I'm doing something wrong?

Thanks,
Slava

[-- Attachment #2: Type: text/html, Size: 13162 bytes --]

Re: [OE-core] [langdale][kirkstone][PATCH] go-mod.bbclass: Allow network in do_compile

[Stefan Herbrechtsmeier]

[-- Attachment #1: Type: text/plain, Size: 2122 bytes --]

Hi Martin,

Am 26.12.2022 um 10:59 schrieb Martin Jansa via lists.openembedded.org:
> On Mon, Dec 26, 2022 at 10:38 AM Vyacheslav Yurkov 
> <uvv.mail@gmail.com> wrote:
>
>     Martin, I did see that patch series, but I was not sure about its
>     state, therefore I didn't try it. If I understood correctly, that
>     partially solves the issue, but the proper solution should involve
>     the fetcher, that's why it's not yet merged to master. Am I
>     missing something?
>
>
> That's not my understanding.
>
> The proposed changes for recipetool had some cosmetic and functional 
> review comments and it doesn't work correctly for every possible go 
> dependency (e.g. the 3 issues I've mentioned from telegram recipe), 
> but it's very good starting point and we should be able to get it 
> merged in master (and then everybody can more easily contribute fixes 
> for the corner cases where it might not work well yet).
>
> Even missing documentation IMHO shouldn't block merging it at this 
> point, because there is a clear need this (as there are already 3 
> partial implementations to fix this issue, neither 100% complete and 
> nothing merged in master - the recipetool changes are IMHO the best of 
> 3 as it integrates with devtool as well and seems to be most complete).
>
> go fetcher in bitbake is another possibility for future, but 
> generating the long deps in SRC_URI is much better solution then 
> enabling network in every go-mod recipe.

Does an agreement exist if we should use a package manager class which 
creates a wget fetcher URI (ex. pipi) or a specific fetcher (ex. crate 
and npm)? At the moment both specific fetchers have different URI 
styles, create their specific tar commands and doesn't reuse the wget 
fetcher. Furthermore, the crate fetcher doesn't check the integrity 
during fetch. The fetcher inside bitbake have the advantage that you can 
fetch the latest version (ex. AUTOREV), but it brings the risk of code 
duplication and missing features. If we have an agreement and a common 
style its easy to replace the function with a specific fetcher.

Regards
   Stefan


[-- Attachment #2: Type: text/html, Size: 3414 bytes --]

Re: [OE-core] [langdale][kirkstone][PATCH] go-mod.bbclass: Allow network in do_compile

[Lukas Funke]

Martin,

first of all: thank you for testing the patches. As usual the corner 
cases are the most tricky ones.

tl;dr: I'm working on an updated patch series to address the reported 
issues.


I've looked into the issues and would like to give some explanation.

1) the first module you mention ('gonum.org/v1/gonum 
<http://gonum.org/v1/gonum>') does not obey the go documentation for 
resolving the modules source-code repository (see 
https://go.dev/ref/mod#go-mod-download , Section 'Version control 
systems'). The documentation states that there should be a html page, 
queried by "?go-get=1", with a <meta>-tag which contains the original 
URL to the source-code repository. For 'gonum' there is only a 404-page. 
This page contains javascript-code that redirects to the actual gonum 
package page. Unnecessary hard to process. I'll open an issue for that. 
The second module 'code.cloudfoundry.org/clock 
<http://code.cloudfoundry.org/clock>' has a broken certificate. I 
already opend an issue on github for this.

I addressed this issue above by maintaining a list of modules and their 
actual repo URLs inside the go-recipetool as an absolute fallback. Would 
do you think about this solution?

2) yes, my bad :)

3) The problem is that some repositories have a prefix in their tags. 
The tags usually only contain the semantic version string. I'm curretly 
working on this issue.

Best regards

Lukas

Am 22.12.2022 um 20:08 schrieb Martin Jansa via lists.openembedded.org:
> On Thu, Dec 22, 2022 at 7:39 PM Vyacheslav Yurkov <uvv.mail@gmail.com> 
> wrote:
>
>     On 22.12.2022 17:48, Bruce Ashfield wrote:
>     > On Thu, Dec 22, 2022 at 11:20 AM Richard Purdie
>     > <richard.purdie@linuxfoundation.org> wrote:
>     >> This patch is not in master and is not a backport therefore not
>     >> eligible for kirkstone/langdale.
>
>     My bad, the intention was to have it in all three branches: master,
>     kirkstone, and langdale.
>
>     > 'nor should it go to master.
>     >
>     > If someone wants to allow go to fetch sources during builds, it
>     needs
>     > to be done in their own layers.
>     >
>     > Bruce
>
>     Probably I misunderstood the outcome of the discussion in my first
>     link.
>     Bruce and Richard, you both suggested to have it in go-mod with the
>     warning. I agree that it breaks reproducibility, and is considered
>     a bad
>     practice in general (I'm aware of a few more build systems with
>     the same
>     shortcoming).
>
>     My point is, until a proper solution is in place, it should be at
>     least
>     documented somewhere that this should be done in own layer/recipe.
>     Any
>     suggestions where this can be documented if not in OE-Core?
>
>
> Have you tried the changes submitted by Lukas/Stefan in:
> https://lists.openembedded.org/g/openembedded-architecture/message/1539
> ?
>
> It's not perfect, I was testing it on 
> https://github.com/influxdata/telegraf/blob/master/go.mod and I've 
> found some corner cases where it failed.
>
> But it seems like very good start and we should work with Lukas/Stefan 
> to get it merged in master. Then all branches could consume recipes 
> created in master and only the exceptions would need to have network 
> access (instead of the bbclass enabling it for every go-mod user - 
> go-vendor inherits go-mod as well).
>
> Lukas/Stefan: is there something we can do to help with v2 of your 
> patches?
>
> I wanted to submit better review on submitted patches after more 
> debugging, but here it is:
>
> 1) to resolve 2 deps:
> {'Path': 'gonum.org/v1/gonum <http://gonum.org/v1/gonum>', 'Version': 
> 'v0.12.0'} due to 404 "ERROR: Error while fetching redirect page: HTTP 
> Error 404: Not Found"
>
> {'Path': 'code.cloudfoundry.org/clock 
> <http://code.cloudfoundry.org/clock>', 'Version': 'v1.0.0', 
> 'Indirect': True} due to this being redirect to github.com 
> <http://github.com> and failing with "ERROR: Error while fetching 
> redirect page: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] 
> certificate verify failed: Hostname mismatch, certificate is not valid 
> for 'code.cloudfoundry.org <http://code.cloudfoundry.org>'. (_ssl.c:996)>"
> parse
>
> 2) some issues with LICENSEs:
> INFO: Please add the following line for 
> 'docs/LICENSE_OF_DEPENDENCIES.md' to a 'lib/recipetool/licenses.csv' 
> and replace `Unknown` with the license:
> c4807c5b058a56c02f592207b4e056b1,Unknown
> Traceback (most recent call last):
>   File "/OE/build/oe-core/openembedded-core/scripts/recipetool", line 
> 111, in <module>
>     ret = main()
>           ^^^^^^
>   File "/OE/build/oe-core/openembedded-core/scripts/recipetool", line 
> 100, in main
>     ret = args.func(args)
>           ^^^^^^^^^^^^^^^
>   File 
> "/OE/layers/openembedded-core/scripts/lib/recipetool/create.py", line 
> 746, in create_recipe
>     handler.process(srctree_use, classes, lines_before, lines_after, 
> handled, extravalues)
>   File 
> "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py", 
> line 349, in process
>     self._rewrite_lic_uri(lines_before)
>   File 
> "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py", 
> line 380, in _rewrite_lic_uri
>     updated, newlines = bb.utils.edit_metadata(lines_before, 
> ['LIC_FILES_CHKSUM'], varfunc)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "/OE/build/oe-core/bitbake/lib/bb/utils.py", line 1382, in 
> edit_metadata
>     if handle_var_end():
>        ^^^^^^^^^^^^^^^^
>   File "/OE/build/oe-core/bitbake/lib/bb/utils.py", line 1280, in 
> handle_var_end
>     (newvalue, newop, indent, minbreak) = varfunc(in_var, full_value, 
> op, newlines)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File 
> "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py", 
> line 372, in varfunc
>     uri, chksum = license.split(';', 1)
>     ^^^^^^^^^^^
> ValueError: not enough values to unpack (expected 2, got 1)
>
> (the license was just '/')
>
> 3) some SRCREVs set to None leading to parsing failure:
>
>   File "/OE/layers/bitbake/lib/bb/fetch2/git.py", line 762, in 
> _latest_revision
>     raise bb.fetch2.FetchError("Unable to resolve '%s' in upstream git 
> repository in git ls-remote output for %s" % \
> bb.data_smart.ExpansionError: Failure expanding variable SRCPV, 
> expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception 
> FetchError: Fetcher failure: Unable to resolve 'None' in upstream git 
> repository in git ls-remote output for 
> github.com/googleapis/google-cloud-go 
> <http://github.com/googleapis/google-cloud-go>
> The variable dependency chain for the failure is: SRCPV -> PV -> BP -> 
> FILESPATH
>
> ERROR: Parsing newly created recipe failed, moving recipe to 
> /OE/build/oe-core/workspace/recipes/telegraf/telegraf_git.bb.parsefailed 
> for reference. If this looks to be caused by the recipe itself, please 
> report this error.
>
> It takes long time to fetch all telegraf dependencies, so I'll create 
> some test recipe to more easily debug these 3 issues in isolation.
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#174971): https://lists.openembedded.org/g/openembedded-core/message/174971
> Mute This Topic: https://lists.openembedded.org/mt/95827446/7344993
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [lukas.funke-oss@weidmueller.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] [langdale][kirkstone][PATCH] go-mod.bbclass: Allow network in do_compile

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 9146 bytes --]

On Tue, Jan 3, 2023 at 5:03 PM Lukas Funke <lukas.funke-oss@weidmueller.com>
wrote:

> Martin,
>
> first of all: thank you for testing the patches. As usual the corner
> cases are the most tricky ones.
>
> tl;dr: I'm working on an updated patch series to address the reported
> issues.
>

Hi Lukas,

and update on this series?

It doesn't have to be perfect, other people can help with remaining corner
cases and right now there is nothing for this in oe-core, so any version
will be big improvement :) and it will make it easier for others to submit
incremental improvements - I have some as well, but haven't sent them as I
don't know what you've already updated locally, so I'm waiting for v2.

Mickledore is already in feature freeze, but maybe RP will make an
exception as this is new important functionality which isn't likely to
break other existing code.

Kind Regards,


I've looked into the issues and would like to give some explanation.
>
> 1) the first module you mention ('gonum.org/v1/gonum
> <http://gonum.org/v1/gonum>') does not obey the go documentation for
> resolving the modules source-code repository (see
> https://go.dev/ref/mod#go-mod-download , Section 'Version control
> systems'). The documentation states that there should be a html page,
> queried by "?go-get=1", with a <meta>-tag which contains the original
> URL to the source-code repository. For 'gonum' there is only a 404-page.
> This page contains javascript-code that redirects to the actual gonum
> package page. Unnecessary hard to process. I'll open an issue for that.
> The second module 'code.cloudfoundry.org/clock
> <http://code.cloudfoundry.org/clock>' has a broken certificate. I
> already opend an issue on github for this.
>
> I addressed this issue above by maintaining a list of modules and their
> actual repo URLs inside the go-recipetool as an absolute fallback. Would
> do you think about this solution?
>

Yes, that's what I was starting to do in:
https://git.openembedded.org/openembedded-core-contrib/commit/?h=jansa/go&id=b973c7f17c8a613233a1a18de0bf6daa352c47d8
https://git.openembedded.org/openembedded-core-contrib/commit/?h=jansa/go&id=61c6c9962b3b7d4c1cf6e6823de6c32b09998b00


> 2) yes, my bad :)
>

as work around I was using
https://git.openembedded.org/openembedded-core-contrib/commit/?h=jansa/go&id=5767819bfe96de29ab751d4a37a431a324f7e547


3) The problem is that some repositories have a prefix in their tags.
> The tags usually only contain the semantic version string. I'm curretly
> working on this issue.
>
> Best regards
>
> Lukas
>
> Am 22.12.2022 um 20:08 schrieb Martin Jansa via lists.openembedded.org:
> > On Thu, Dec 22, 2022 at 7:39 PM Vyacheslav Yurkov <uvv.mail@gmail.com>
> > wrote:
> >
> >     On 22.12.2022 17:48, Bruce Ashfield wrote:
> >     > On Thu, Dec 22, 2022 at 11:20 AM Richard Purdie
> >     > <richard.purdie@linuxfoundation.org> wrote:
> >     >> This patch is not in master and is not a backport therefore not
> >     >> eligible for kirkstone/langdale.
> >
> >     My bad, the intention was to have it in all three branches: master,
> >     kirkstone, and langdale.
> >
> >     > 'nor should it go to master.
> >     >
> >     > If someone wants to allow go to fetch sources during builds, it
> >     needs
> >     > to be done in their own layers.
> >     >
> >     > Bruce
> >
> >     Probably I misunderstood the outcome of the discussion in my first
> >     link.
> >     Bruce and Richard, you both suggested to have it in go-mod with the
> >     warning. I agree that it breaks reproducibility, and is considered
> >     a bad
> >     practice in general (I'm aware of a few more build systems with
> >     the same
> >     shortcoming).
> >
> >     My point is, until a proper solution is in place, it should be at
> >     least
> >     documented somewhere that this should be done in own layer/recipe.
> >     Any
> >     suggestions where this can be documented if not in OE-Core?
> >
> >
> > Have you tried the changes submitted by Lukas/Stefan in:
> > https://lists.openembedded.org/g/openembedded-architecture/message/1539
> > ?
> >
> > It's not perfect, I was testing it on
> > https://github.com/influxdata/telegraf/blob/master/go.mod and I've
> > found some corner cases where it failed.
> >
> > But it seems like very good start and we should work with Lukas/Stefan
> > to get it merged in master. Then all branches could consume recipes
> > created in master and only the exceptions would need to have network
> > access (instead of the bbclass enabling it for every go-mod user -
> > go-vendor inherits go-mod as well).
> >
> > Lukas/Stefan: is there something we can do to help with v2 of your
> > patches?
> >
> > I wanted to submit better review on submitted patches after more
> > debugging, but here it is:
> >
> > 1) to resolve 2 deps:
> > {'Path': 'gonum.org/v1/gonum <http://gonum.org/v1/gonum>', 'Version':
> > 'v0.12.0'} due to 404 "ERROR: Error while fetching redirect page: HTTP
> > Error 404: Not Found"
> >
> > {'Path': 'code.cloudfoundry.org/clock
> > <http://code.cloudfoundry.org/clock>', 'Version': 'v1.0.0',
> > 'Indirect': True} due to this being redirect to github.com
> > <http://github.com> and failing with "ERROR: Error while fetching
> > redirect page: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED]
> > certificate verify failed: Hostname mismatch, certificate is not valid
> > for 'code.cloudfoundry.org <http://code.cloudfoundry.org>'.
> (_ssl.c:996)>"
> > parse
> >
> > 2) some issues with LICENSEs:
> > INFO: Please add the following line for
> > 'docs/LICENSE_OF_DEPENDENCIES.md' to a 'lib/recipetool/licenses.csv'
> > and replace `Unknown` with the license:
> > c4807c5b058a56c02f592207b4e056b1,Unknown
> > Traceback (most recent call last):
> >   File "/OE/build/oe-core/openembedded-core/scripts/recipetool", line
> > 111, in <module>
> >     ret = main()
> >           ^^^^^^
> >   File "/OE/build/oe-core/openembedded-core/scripts/recipetool", line
> > 100, in main
> >     ret = args.func(args)
> >           ^^^^^^^^^^^^^^^
> >   File
> > "/OE/layers/openembedded-core/scripts/lib/recipetool/create.py", line
> > 746, in create_recipe
> >     handler.process(srctree_use, classes, lines_before, lines_after,
> > handled, extravalues)
> >   File
> > "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
> > line 349, in process
> >     self._rewrite_lic_uri(lines_before)
> >   File
> > "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
> > line 380, in _rewrite_lic_uri
> >     updated, newlines = bb.utils.edit_metadata(lines_before,
> > ['LIC_FILES_CHKSUM'], varfunc)
> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >   File "/OE/build/oe-core/bitbake/lib/bb/utils.py", line 1382, in
> > edit_metadata
> >     if handle_var_end():
> >        ^^^^^^^^^^^^^^^^
> >   File "/OE/build/oe-core/bitbake/lib/bb/utils.py", line 1280, in
> > handle_var_end
> >     (newvalue, newop, indent, minbreak) = varfunc(in_var, full_value,
> > op, newlines)
> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >   File
> > "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
> > line 372, in varfunc
> >     uri, chksum = license.split(';', 1)
> >     ^^^^^^^^^^^
> > ValueError: not enough values to unpack (expected 2, got 1)
> >
> > (the license was just '/')
> >
> > 3) some SRCREVs set to None leading to parsing failure:
> >
> >   File "/OE/layers/bitbake/lib/bb/fetch2/git.py", line 762, in
> > _latest_revision
> >     raise bb.fetch2.FetchError("Unable to resolve '%s' in upstream git
> > repository in git ls-remote output for %s" % \
> > bb.data_smart.ExpansionError: Failure expanding variable SRCPV,
> > expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception
> > FetchError: Fetcher failure: Unable to resolve 'None' in upstream git
> > repository in git ls-remote output for
> > github.com/googleapis/google-cloud-go
> > <http://github.com/googleapis/google-cloud-go>
> > The variable dependency chain for the failure is: SRCPV -> PV -> BP ->
> > FILESPATH
> >
> > ERROR: Parsing newly created recipe failed, moving recipe to
> > /OE/build/oe-core/workspace/recipes/telegraf/telegraf_git.bb.parsefailed
> > for reference. If this looks to be caused by the recipe itself, please
> > report this error.
> >
> > It takes long time to fetch all telegraf dependencies, so I'll create
> > some test recipe to more easily debug these 3 issues in isolation.
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#174971):
> https://lists.openembedded.org/g/openembedded-core/message/174971
> > Mute This Topic: https://lists.openembedded.org/mt/95827446/7344993
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> lukas.funke-oss@weidmueller.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
>

[-- Attachment #2: Type: text/html, Size: 13999 bytes --]

Re: [OE-core] [langdale][kirkstone][PATCH] go-mod.bbclass: Allow network in do_compile

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 9900 bytes --]

On Sun, Mar 12, 2023 at 10:15 AM Martin Jansa via lists.openembedded.org
<Martin.Jansa=gmail.com@lists.openembedded.org> wrote:

> On Tue, Jan 3, 2023 at 5:03 PM Lukas Funke <
> lukas.funke-oss@weidmueller.com> wrote:
>
>> Martin,
>>
>> first of all: thank you for testing the patches. As usual the corner
>> cases are the most tricky ones.
>>
>> tl;dr: I'm working on an updated patch series to address the reported
>> issues.
>>
>
> Hi Lukas, Stefan,
>
> and update on this series?
>
> It doesn't have to be perfect, other people can help with remaining corner
> cases and right now there is nothing for this in oe-core, so any version
> will be big improvement :) and it will make it easier for others to submit
> incremental improvements - I have some as well, but haven't sent them as I
> don't know what you've already updated locally, so I'm waiting for v2.
>
> Mickledore is already in feature freeze, but maybe RP will make an
> exception as this is new important functionality which isn't likely to
> break other existing code.
>

Hello Lukas,

any progress on updated patch series? Nanbield release is near and I don't
remember seeing the updated patch series in last couple months.

Is there anything I can do to help? The last version looked reasonably well
and IMHO could be used as starting point for everybody to help fixing those
corner cases where it didn't work well.

Regards,


> Kind Regards,
>
>
> I've looked into the issues and would like to give some explanation.
>>
>> 1) the first module you mention ('gonum.org/v1/gonum
>> <http://gonum.org/v1/gonum>') does not obey the go documentation for
>> resolving the modules source-code repository (see
>> https://go.dev/ref/mod#go-mod-download , Section 'Version control
>> systems'). The documentation states that there should be a html page,
>> queried by "?go-get=1", with a <meta>-tag which contains the original
>> URL to the source-code repository. For 'gonum' there is only a 404-page.
>> This page contains javascript-code that redirects to the actual gonum
>> package page. Unnecessary hard to process. I'll open an issue for that.
>> The second module 'code.cloudfoundry.org/clock
>> <http://code.cloudfoundry.org/clock>' has a broken certificate. I
>> already opend an issue on github for this.
>>
>> I addressed this issue above by maintaining a list of modules and their
>> actual repo URLs inside the go-recipetool as an absolute fallback. Would
>> do you think about this solution?
>>
>
> Yes, that's what I was starting to do in:
>
> https://git.openembedded.org/openembedded-core-contrib/commit/?h=jansa/go&id=b973c7f17c8a613233a1a18de0bf6daa352c47d8
>
> https://git.openembedded.org/openembedded-core-contrib/commit/?h=jansa/go&id=61c6c9962b3b7d4c1cf6e6823de6c32b09998b00
>
>
>> 2) yes, my bad :)
>>
>
> as work around I was using
>
> https://git.openembedded.org/openembedded-core-contrib/commit/?h=jansa/go&id=5767819bfe96de29ab751d4a37a431a324f7e547
>
>
> 3) The problem is that some repositories have a prefix in their tags.
>> The tags usually only contain the semantic version string. I'm curretly
>> working on this issue.
>>
>> Best regards
>>
>> Lukas
>>
>> Am 22.12.2022 um 20:08 schrieb Martin Jansa via lists.openembedded.org:
>> > On Thu, Dec 22, 2022 at 7:39 PM Vyacheslav Yurkov <uvv.mail@gmail.com>
>> > wrote:
>> >
>> >     On 22.12.2022 17:48, Bruce Ashfield wrote:
>> >     > On Thu, Dec 22, 2022 at 11:20 AM Richard Purdie
>> >     > <richard.purdie@linuxfoundation.org> wrote:
>> >     >> This patch is not in master and is not a backport therefore not
>> >     >> eligible for kirkstone/langdale.
>> >
>> >     My bad, the intention was to have it in all three branches: master,
>> >     kirkstone, and langdale.
>> >
>> >     > 'nor should it go to master.
>> >     >
>> >     > If someone wants to allow go to fetch sources during builds, it
>> >     needs
>> >     > to be done in their own layers.
>> >     >
>> >     > Bruce
>> >
>> >     Probably I misunderstood the outcome of the discussion in my first
>> >     link.
>> >     Bruce and Richard, you both suggested to have it in go-mod with the
>> >     warning. I agree that it breaks reproducibility, and is considered
>> >     a bad
>> >     practice in general (I'm aware of a few more build systems with
>> >     the same
>> >     shortcoming).
>> >
>> >     My point is, until a proper solution is in place, it should be at
>> >     least
>> >     documented somewhere that this should be done in own layer/recipe.
>> >     Any
>> >     suggestions where this can be documented if not in OE-Core?
>> >
>> >
>> > Have you tried the changes submitted by Lukas/Stefan in:
>> > https://lists.openembedded.org/g/openembedded-architecture/message/1539
>> > ?
>> >
>> > It's not perfect, I was testing it on
>> > https://github.com/influxdata/telegraf/blob/master/go.mod and I've
>> > found some corner cases where it failed.
>> >
>> > But it seems like very good start and we should work with Lukas/Stefan
>> > to get it merged in master. Then all branches could consume recipes
>> > created in master and only the exceptions would need to have network
>> > access (instead of the bbclass enabling it for every go-mod user -
>> > go-vendor inherits go-mod as well).
>> >
>> > Lukas/Stefan: is there something we can do to help with v2 of your
>> > patches?
>> >
>> > I wanted to submit better review on submitted patches after more
>> > debugging, but here it is:
>> >
>> > 1) to resolve 2 deps:
>> > {'Path': 'gonum.org/v1/gonum <http://gonum.org/v1/gonum>', 'Version':
>> > 'v0.12.0'} due to 404 "ERROR: Error while fetching redirect page: HTTP
>> > Error 404: Not Found"
>> >
>> > {'Path': 'code.cloudfoundry.org/clock
>> > <http://code.cloudfoundry.org/clock>', 'Version': 'v1.0.0',
>> > 'Indirect': True} due to this being redirect to github.com
>> > <http://github.com> and failing with "ERROR: Error while fetching
>> > redirect page: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED]
>> > certificate verify failed: Hostname mismatch, certificate is not valid
>> > for 'code.cloudfoundry.org <http://code.cloudfoundry.org>'.
>> (_ssl.c:996)>"
>> > parse
>> >
>> > 2) some issues with LICENSEs:
>> > INFO: Please add the following line for
>> > 'docs/LICENSE_OF_DEPENDENCIES.md' to a 'lib/recipetool/licenses.csv'
>> > and replace `Unknown` with the license:
>> > c4807c5b058a56c02f592207b4e056b1,Unknown
>> > Traceback (most recent call last):
>> >   File "/OE/build/oe-core/openembedded-core/scripts/recipetool", line
>> > 111, in <module>
>> >     ret = main()
>> >           ^^^^^^
>> >   File "/OE/build/oe-core/openembedded-core/scripts/recipetool", line
>> > 100, in main
>> >     ret = args.func(args)
>> >           ^^^^^^^^^^^^^^^
>> >   File
>> > "/OE/layers/openembedded-core/scripts/lib/recipetool/create.py", line
>> > 746, in create_recipe
>> >     handler.process(srctree_use, classes, lines_before, lines_after,
>> > handled, extravalues)
>> >   File
>> > "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
>> > line 349, in process
>> >     self._rewrite_lic_uri(lines_before)
>> >   File
>> > "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
>> > line 380, in _rewrite_lic_uri
>> >     updated, newlines = bb.utils.edit_metadata(lines_before,
>> > ['LIC_FILES_CHKSUM'], varfunc)
>> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> >   File "/OE/build/oe-core/bitbake/lib/bb/utils.py", line 1382, in
>> > edit_metadata
>> >     if handle_var_end():
>> >        ^^^^^^^^^^^^^^^^
>> >   File "/OE/build/oe-core/bitbake/lib/bb/utils.py", line 1280, in
>> > handle_var_end
>> >     (newvalue, newop, indent, minbreak) = varfunc(in_var, full_value,
>> > op, newlines)
>> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> >   File
>> > "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
>> > line 372, in varfunc
>> >     uri, chksum = license.split(';', 1)
>> >     ^^^^^^^^^^^
>> > ValueError: not enough values to unpack (expected 2, got 1)
>> >
>> > (the license was just '/')
>> >
>> > 3) some SRCREVs set to None leading to parsing failure:
>> >
>> >   File "/OE/layers/bitbake/lib/bb/fetch2/git.py", line 762, in
>> > _latest_revision
>> >     raise bb.fetch2.FetchError("Unable to resolve '%s' in upstream git
>> > repository in git ls-remote output for %s" % \
>> > bb.data_smart.ExpansionError: Failure expanding variable SRCPV,
>> > expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception
>> > FetchError: Fetcher failure: Unable to resolve 'None' in upstream git
>> > repository in git ls-remote output for
>> > github.com/googleapis/google-cloud-go
>> > <http://github.com/googleapis/google-cloud-go>
>> > The variable dependency chain for the failure is: SRCPV -> PV -> BP ->
>> > FILESPATH
>> >
>> > ERROR: Parsing newly created recipe failed, moving recipe to
>> >
>> /OE/build/oe-core/workspace/recipes/telegraf/telegraf_git.bb.parsefailed
>> > for reference. If this looks to be caused by the recipe itself, please
>> > report this error.
>> >
>> > It takes long time to fetch all telegraf dependencies, so I'll create
>> > some test recipe to more easily debug these 3 issues in isolation.
>> >
>> >
>> >
>> >
>>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#178380):
> https://lists.openembedded.org/g/openembedded-core/message/178380
> Mute This Topic: https://lists.openembedded.org/mt/95827446/3617156
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> Martin.Jansa@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 15151 bytes --]

Re: [OE-core] [langdale][kirkstone][PATCH] go-mod.bbclass: Allow network in do_compile

[Lukas Funke]

On 10.09.2023 23:26, Martin Jansa wrote:
> On Sun, Mar 12, 2023 at 10:15 AM Martin Jansa via lists.openembedded.org
> <Martin.Jansa=gmail.com@lists.openembedded.org> wrote:
> 
>> On Tue, Jan 3, 2023 at 5:03 PM Lukas Funke <
>> lukas.funke-oss@weidmueller.com> wrote:
>>
>>> Martin,
>>>
>>> first of all: thank you for testing the patches. As usual the corner
>>> cases are the most tricky ones.
>>>
>>> tl;dr: I'm working on an updated patch series to address the reported
>>> issues.
>>>
>>
>> Hi Lukas, Stefan,
>>
>> and update on this series?
>>
>> It doesn't have to be perfect, other people can help with remaining corner
>> cases and right now there is nothing for this in oe-core, so any version
>> will be big improvement :) and it will make it easier for others to submit
>> incremental improvements - I have some as well, but haven't sent them as I
>> don't know what you've already updated locally, so I'm waiting for v2.
>>
>> Mickledore is already in feature freeze, but maybe RP will make an
>> exception as this is new important functionality which isn't likely to
>> break other existing code.
>>
> 
> Hello Lukas,
> 
> any progress on updated patch series? Nanbield release is near and I don't
> remember seeing the updated patch series in last couple months.
> 
> Is there anything I can do to help? The last version looked reasonably well
> and IMHO could be used as starting point for everybody to help fixing those
> corner cases where it didn't work well.
> 
> Regards,

Hello Martin,

I'm currently working on an update, but can't make any promises 
regarding the timeline.

Thanks for offering help! Reviewing and testing is always helpful when 
the updated series is on it's way.

Regards
Lukas


> 
> 
>> Kind Regards,
>>
>>
>> I've looked into the issues and would like to give some explanation.
>>>
>>> 1) the first module you mention ('gonum.org/v1/gonum
>>> <http://gonum.org/v1/gonum>') does not obey the go documentation for
>>> resolving the modules source-code repository (see
>>> https://go.dev/ref/mod#go-mod-download , Section 'Version control
>>> systems'). The documentation states that there should be a html page,
>>> queried by "?go-get=1", with a <meta>-tag which contains the original
>>> URL to the source-code repository. For 'gonum' there is only a 404-page.
>>> This page contains javascript-code that redirects to the actual gonum
>>> package page. Unnecessary hard to process. I'll open an issue for that.
>>> The second module 'code.cloudfoundry.org/clock
>>> <http://code.cloudfoundry.org/clock>' has a broken certificate. I
>>> already opend an issue on github for this.
>>>
>>> I addressed this issue above by maintaining a list of modules and their
>>> actual repo URLs inside the go-recipetool as an absolute fallback. Would
>>> do you think about this solution?
>>>
>>
>> Yes, that's what I was starting to do in:
>>
>> https://git.openembedded.org/openembedded-core-contrib/commit/?h=jansa/go&id=b973c7f17c8a613233a1a18de0bf6daa352c47d8
>>
>> https://git.openembedded.org/openembedded-core-contrib/commit/?h=jansa/go&id=61c6c9962b3b7d4c1cf6e6823de6c32b09998b00
>>
>>
>>> 2) yes, my bad :)
>>>
>>
>> as work around I was using
>>
>> https://git.openembedded.org/openembedded-core-contrib/commit/?h=jansa/go&id=5767819bfe96de29ab751d4a37a431a324f7e547
>>
>>
>> 3) The problem is that some repositories have a prefix in their tags.
>>> The tags usually only contain the semantic version string. I'm curretly
>>> working on this issue.
>>>
>>> Best regards
>>>
>>> Lukas
>>>
>>> Am 22.12.2022 um 20:08 schrieb Martin Jansa via lists.openembedded.org:
>>>> On Thu, Dec 22, 2022 at 7:39 PM Vyacheslav Yurkov <uvv.mail@gmail.com>
>>>> wrote:
>>>>
>>>>      On 22.12.2022 17:48, Bruce Ashfield wrote:
>>>>      > On Thu, Dec 22, 2022 at 11:20 AM Richard Purdie
>>>>      > <richard.purdie@linuxfoundation.org> wrote:
>>>>      >> This patch is not in master and is not a backport therefore not
>>>>      >> eligible for kirkstone/langdale.
>>>>
>>>>      My bad, the intention was to have it in all three branches: master,
>>>>      kirkstone, and langdale.
>>>>
>>>>      > 'nor should it go to master.
>>>>      >
>>>>      > If someone wants to allow go to fetch sources during builds, it
>>>>      needs
>>>>      > to be done in their own layers.
>>>>      >
>>>>      > Bruce
>>>>
>>>>      Probably I misunderstood the outcome of the discussion in my first
>>>>      link.
>>>>      Bruce and Richard, you both suggested to have it in go-mod with the
>>>>      warning. I agree that it breaks reproducibility, and is considered
>>>>      a bad
>>>>      practice in general (I'm aware of a few more build systems with
>>>>      the same
>>>>      shortcoming).
>>>>
>>>>      My point is, until a proper solution is in place, it should be at
>>>>      least
>>>>      documented somewhere that this should be done in own layer/recipe.
>>>>      Any
>>>>      suggestions where this can be documented if not in OE-Core?
>>>>
>>>>
>>>> Have you tried the changes submitted by Lukas/Stefan in:
>>>> https://lists.openembedded.org/g/openembedded-architecture/message/1539
>>>> ?
>>>>
>>>> It's not perfect, I was testing it on
>>>> https://github.com/influxdata/telegraf/blob/master/go.mod and I've
>>>> found some corner cases where it failed.
>>>>
>>>> But it seems like very good start and we should work with Lukas/Stefan
>>>> to get it merged in master. Then all branches could consume recipes
>>>> created in master and only the exceptions would need to have network
>>>> access (instead of the bbclass enabling it for every go-mod user -
>>>> go-vendor inherits go-mod as well).
>>>>
>>>> Lukas/Stefan: is there something we can do to help with v2 of your
>>>> patches?
>>>>
>>>> I wanted to submit better review on submitted patches after more
>>>> debugging, but here it is:
>>>>
>>>> 1) to resolve 2 deps:
>>>> {'Path': 'gonum.org/v1/gonum <http://gonum.org/v1/gonum>', 'Version':
>>>> 'v0.12.0'} due to 404 "ERROR: Error while fetching redirect page: HTTP
>>>> Error 404: Not Found"
>>>>
>>>> {'Path': 'code.cloudfoundry.org/clock
>>>> <http://code.cloudfoundry.org/clock>', 'Version': 'v1.0.0',
>>>> 'Indirect': True} due to this being redirect to github.com
>>>> <http://github.com> and failing with "ERROR: Error while fetching
>>>> redirect page: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED]
>>>> certificate verify failed: Hostname mismatch, certificate is not valid
>>>> for 'code.cloudfoundry.org <http://code.cloudfoundry.org>'.
>>> (_ssl.c:996)>"
>>>> parse
>>>>
>>>> 2) some issues with LICENSEs:
>>>> INFO: Please add the following line for
>>>> 'docs/LICENSE_OF_DEPENDENCIES.md' to a 'lib/recipetool/licenses.csv'
>>>> and replace `Unknown` with the license:
>>>> c4807c5b058a56c02f592207b4e056b1,Unknown
>>>> Traceback (most recent call last):
>>>>    File "/OE/build/oe-core/openembedded-core/scripts/recipetool", line
>>>> 111, in <module>
>>>>      ret = main()
>>>>            ^^^^^^
>>>>    File "/OE/build/oe-core/openembedded-core/scripts/recipetool", line
>>>> 100, in main
>>>>      ret = args.func(args)
>>>>            ^^^^^^^^^^^^^^^
>>>>    File
>>>> "/OE/layers/openembedded-core/scripts/lib/recipetool/create.py", line
>>>> 746, in create_recipe
>>>>      handler.process(srctree_use, classes, lines_before, lines_after,
>>>> handled, extravalues)
>>>>    File
>>>> "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
>>>> line 349, in process
>>>>      self._rewrite_lic_uri(lines_before)
>>>>    File
>>>> "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
>>>> line 380, in _rewrite_lic_uri
>>>>      updated, newlines = bb.utils.edit_metadata(lines_before,
>>>> ['LIC_FILES_CHKSUM'], varfunc)
>>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>>>    File "/OE/build/oe-core/bitbake/lib/bb/utils.py", line 1382, in
>>>> edit_metadata
>>>>      if handle_var_end():
>>>>         ^^^^^^^^^^^^^^^^
>>>>    File "/OE/build/oe-core/bitbake/lib/bb/utils.py", line 1280, in
>>>> handle_var_end
>>>>      (newvalue, newop, indent, minbreak) = varfunc(in_var, full_value,
>>>> op, newlines)
>>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>>>    File
>>>> "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
>>>> line 372, in varfunc
>>>>      uri, chksum = license.split(';', 1)
>>>>      ^^^^^^^^^^^
>>>> ValueError: not enough values to unpack (expected 2, got 1)
>>>>
>>>> (the license was just '/')
>>>>
>>>> 3) some SRCREVs set to None leading to parsing failure:
>>>>
>>>>    File "/OE/layers/bitbake/lib/bb/fetch2/git.py", line 762, in
>>>> _latest_revision
>>>>      raise bb.fetch2.FetchError("Unable to resolve '%s' in upstream git
>>>> repository in git ls-remote output for %s" % \
>>>> bb.data_smart.ExpansionError: Failure expanding variable SRCPV,
>>>> expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception
>>>> FetchError: Fetcher failure: Unable to resolve 'None' in upstream git
>>>> repository in git ls-remote output for
>>>> github.com/googleapis/google-cloud-go
>>>> <http://github.com/googleapis/google-cloud-go>
>>>> The variable dependency chain for the failure is: SRCPV -> PV -> BP ->
>>>> FILESPATH
>>>>
>>>> ERROR: Parsing newly created recipe failed, moving recipe to
>>>>
>>> /OE/build/oe-core/workspace/recipes/telegraf/telegraf_git.bb.parsefailed
>>>> for reference. If this looks to be caused by the recipe itself, please
>>>> report this error.
>>>>
>>>> It takes long time to fetch all telegraf dependencies, so I'll create
>>>> some test recipe to more easily debug these 3 issues in isolation.
>>>>
>>>>
>>>>
>>>>
>>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#178380):
>> https://lists.openembedded.org/g/openembedded-core/message/178380
>> Mute This Topic: https://lists.openembedded.org/mt/95827446/3617156
>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
>> Martin.Jansa@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
>>
> 

Re: [OE-core] [langdale][kirkstone][PATCH] go-mod.bbclass: Allow network in do_compile

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 11365 bytes --]

On Wed, Sep 27, 2023 at 9:34 AM Lukas Funke <lukas.funke-oss@weidmueller.com>
wrote:

> On 10.09.2023 23:26, Martin Jansa wrote:
> > On Sun, Mar 12, 2023 at 10:15 AM Martin Jansa via lists.openembedded.org
> > <Martin.Jansa=gmail.com@lists.openembedded.org> wrote:
> >
> >> On Tue, Jan 3, 2023 at 5:03 PM Lukas Funke <
> >> lukas.funke-oss@weidmueller.com> wrote:
> >>
> >>> Martin,
> >>>
> >>> first of all: thank you for testing the patches. As usual the corner
> >>> cases are the most tricky ones.
> >>>
> >>> tl;dr: I'm working on an updated patch series to address the reported
> >>> issues.
> >>>
> >>
> >> Hi Lukas, Stefan,
> >>
> >> and update on this series?
> >>
> >> It doesn't have to be perfect, other people can help with remaining
> corner
> >> cases and right now there is nothing for this in oe-core, so any version
> >> will be big improvement :) and it will make it easier for others to
> submit
> >> incremental improvements - I have some as well, but haven't sent them
> as I
> >> don't know what you've already updated locally, so I'm waiting for v2.
> >>
> >> Mickledore is already in feature freeze, but maybe RP will make an
> >> exception as this is new important functionality which isn't likely to
> >> break other existing code.
> >>
> >
> > Hello Lukas,
> >
> > any progress on updated patch series? Nanbield release is near and I
> don't
> > remember seeing the updated patch series in last couple months.
> >
> > Is there anything I can do to help? The last version looked reasonably
> well
> > and IMHO could be used as starting point for everybody to help fixing
> those
> > corner cases where it didn't work well.
> >
> > Regards,
>
> Hello Martin,
>
> I'm currently working on an update, but can't make any promises
> regarding the timeline.
>

Just FYI: on meta-virt master there is a reworked script that is used to
update the .inc files for go projects in the layer.

It might be helpful, it might not be!

Bruce




>
> Thanks for offering help! Reviewing and testing is always helpful when
> the updated series is on it's way.
>
> Regards
> Lukas
>
>
> >
> >
> >> Kind Regards,
> >>
> >>
> >> I've looked into the issues and would like to give some explanation.
> >>>
> >>> 1) the first module you mention ('gonum.org/v1/gonum
> >>> <http://gonum.org/v1/gonum>') does not obey the go documentation for
> >>> resolving the modules source-code repository (see
> >>> https://go.dev/ref/mod#go-mod-download , Section 'Version control
> >>> systems'). The documentation states that there should be a html page,
> >>> queried by "?go-get=1", with a <meta>-tag which contains the original
> >>> URL to the source-code repository. For 'gonum' there is only a
> 404-page.
> >>> This page contains javascript-code that redirects to the actual gonum
> >>> package page. Unnecessary hard to process. I'll open an issue for that.
> >>> The second module 'code.cloudfoundry.org/clock
> >>> <http://code.cloudfoundry.org/clock>' has a broken certificate. I
> >>> already opend an issue on github for this.
> >>>
> >>> I addressed this issue above by maintaining a list of modules and their
> >>> actual repo URLs inside the go-recipetool as an absolute fallback.
> Would
> >>> do you think about this solution?
> >>>
> >>
> >> Yes, that's what I was starting to do in:
> >>
> >>
> https://git.openembedded.org/openembedded-core-contrib/commit/?h=jansa/go&id=b973c7f17c8a613233a1a18de0bf6daa352c47d8
> >>
> >>
> https://git.openembedded.org/openembedded-core-contrib/commit/?h=jansa/go&id=61c6c9962b3b7d4c1cf6e6823de6c32b09998b00
> >>
> >>
> >>> 2) yes, my bad :)
> >>>
> >>
> >> as work around I was using
> >>
> >>
> https://git.openembedded.org/openembedded-core-contrib/commit/?h=jansa/go&id=5767819bfe96de29ab751d4a37a431a324f7e547
> >>
> >>
> >> 3) The problem is that some repositories have a prefix in their tags.
> >>> The tags usually only contain the semantic version string. I'm curretly
> >>> working on this issue.
> >>>
> >>> Best regards
> >>>
> >>> Lukas
> >>>
> >>> Am 22.12.2022 um 20:08 schrieb Martin Jansa via lists.openembedded.org
> :
> >>>> On Thu, Dec 22, 2022 at 7:39 PM Vyacheslav Yurkov <uvv.mail@gmail.com
> >
> >>>> wrote:
> >>>>
> >>>>      On 22.12.2022 17:48, Bruce Ashfield wrote:
> >>>>      > On Thu, Dec 22, 2022 at 11:20 AM Richard Purdie
> >>>>      > <richard.purdie@linuxfoundation.org> wrote:
> >>>>      >> This patch is not in master and is not a backport therefore
> not
> >>>>      >> eligible for kirkstone/langdale.
> >>>>
> >>>>      My bad, the intention was to have it in all three branches:
> master,
> >>>>      kirkstone, and langdale.
> >>>>
> >>>>      > 'nor should it go to master.
> >>>>      >
> >>>>      > If someone wants to allow go to fetch sources during builds, it
> >>>>      needs
> >>>>      > to be done in their own layers.
> >>>>      >
> >>>>      > Bruce
> >>>>
> >>>>      Probably I misunderstood the outcome of the discussion in my
> first
> >>>>      link.
> >>>>      Bruce and Richard, you both suggested to have it in go-mod with
> the
> >>>>      warning. I agree that it breaks reproducibility, and is
> considered
> >>>>      a bad
> >>>>      practice in general (I'm aware of a few more build systems with
> >>>>      the same
> >>>>      shortcoming).
> >>>>
> >>>>      My point is, until a proper solution is in place, it should be at
> >>>>      least
> >>>>      documented somewhere that this should be done in own
> layer/recipe.
> >>>>      Any
> >>>>      suggestions where this can be documented if not in OE-Core?
> >>>>
> >>>>
> >>>> Have you tried the changes submitted by Lukas/Stefan in:
> >>>>
> https://lists.openembedded.org/g/openembedded-architecture/message/1539
> >>>> ?
> >>>>
> >>>> It's not perfect, I was testing it on
> >>>> https://github.com/influxdata/telegraf/blob/master/go.mod and I've
> >>>> found some corner cases where it failed.
> >>>>
> >>>> But it seems like very good start and we should work with Lukas/Stefan
> >>>> to get it merged in master. Then all branches could consume recipes
> >>>> created in master and only the exceptions would need to have network
> >>>> access (instead of the bbclass enabling it for every go-mod user -
> >>>> go-vendor inherits go-mod as well).
> >>>>
> >>>> Lukas/Stefan: is there something we can do to help with v2 of your
> >>>> patches?
> >>>>
> >>>> I wanted to submit better review on submitted patches after more
> >>>> debugging, but here it is:
> >>>>
> >>>> 1) to resolve 2 deps:
> >>>> {'Path': 'gonum.org/v1/gonum <http://gonum.org/v1/gonum>', 'Version':
> >>>> 'v0.12.0'} due to 404 "ERROR: Error while fetching redirect page: HTTP
> >>>> Error 404: Not Found"
> >>>>
> >>>> {'Path': 'code.cloudfoundry.org/clock
> >>>> <http://code.cloudfoundry.org/clock>', 'Version': 'v1.0.0',
> >>>> 'Indirect': True} due to this being redirect to github.com
> >>>> <http://github.com> and failing with "ERROR: Error while fetching
> >>>> redirect page: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED]
> >>>> certificate verify failed: Hostname mismatch, certificate is not valid
> >>>> for 'code.cloudfoundry.org <http://code.cloudfoundry.org>'.
> >>> (_ssl.c:996)>"
> >>>> parse
> >>>>
> >>>> 2) some issues with LICENSEs:
> >>>> INFO: Please add the following line for
> >>>> 'docs/LICENSE_OF_DEPENDENCIES.md' to a 'lib/recipetool/licenses.csv'
> >>>> and replace `Unknown` with the license:
> >>>> c4807c5b058a56c02f592207b4e056b1,Unknown
> >>>> Traceback (most recent call last):
> >>>>    File "/OE/build/oe-core/openembedded-core/scripts/recipetool", line
> >>>> 111, in <module>
> >>>>      ret = main()
> >>>>            ^^^^^^
> >>>>    File "/OE/build/oe-core/openembedded-core/scripts/recipetool", line
> >>>> 100, in main
> >>>>      ret = args.func(args)
> >>>>            ^^^^^^^^^^^^^^^
> >>>>    File
> >>>> "/OE/layers/openembedded-core/scripts/lib/recipetool/create.py", line
> >>>> 746, in create_recipe
> >>>>      handler.process(srctree_use, classes, lines_before, lines_after,
> >>>> handled, extravalues)
> >>>>    File
> >>>> "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
> >>>> line 349, in process
> >>>>      self._rewrite_lic_uri(lines_before)
> >>>>    File
> >>>> "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
> >>>> line 380, in _rewrite_lic_uri
> >>>>      updated, newlines = bb.utils.edit_metadata(lines_before,
> >>>> ['LIC_FILES_CHKSUM'], varfunc)
> >>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >>>>    File "/OE/build/oe-core/bitbake/lib/bb/utils.py", line 1382, in
> >>>> edit_metadata
> >>>>      if handle_var_end():
> >>>>         ^^^^^^^^^^^^^^^^
> >>>>    File "/OE/build/oe-core/bitbake/lib/bb/utils.py", line 1280, in
> >>>> handle_var_end
> >>>>      (newvalue, newop, indent, minbreak) = varfunc(in_var, full_value,
> >>>> op, newlines)
> >>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >>>>    File
> >>>> "/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
> >>>> line 372, in varfunc
> >>>>      uri, chksum = license.split(';', 1)
> >>>>      ^^^^^^^^^^^
> >>>> ValueError: not enough values to unpack (expected 2, got 1)
> >>>>
> >>>> (the license was just '/')
> >>>>
> >>>> 3) some SRCREVs set to None leading to parsing failure:
> >>>>
> >>>>    File "/OE/layers/bitbake/lib/bb/fetch2/git.py", line 762, in
> >>>> _latest_revision
> >>>>      raise bb.fetch2.FetchError("Unable to resolve '%s' in upstream
> git
> >>>> repository in git ls-remote output for %s" % \
> >>>> bb.data_smart.ExpansionError: Failure expanding variable SRCPV,
> >>>> expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception
> >>>> FetchError: Fetcher failure: Unable to resolve 'None' in upstream git
> >>>> repository in git ls-remote output for
> >>>> github.com/googleapis/google-cloud-go
> >>>> <http://github.com/googleapis/google-cloud-go>
> >>>> The variable dependency chain for the failure is: SRCPV -> PV -> BP ->
> >>>> FILESPATH
> >>>>
> >>>> ERROR: Parsing newly created recipe failed, moving recipe to
> >>>>
> >>>
> /OE/build/oe-core/workspace/recipes/telegraf/telegraf_git.bb.parsefailed
> >>>> for reference. If this looks to be caused by the recipe itself, please
> >>>> report this error.
> >>>>
> >>>> It takes long time to fetch all telegraf dependencies, so I'll create
> >>>> some test recipe to more easily debug these 3 issues in isolation.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>
> >> -=-=-=-=-=-=-=-=-=-=-=-
> >> Links: You receive all messages sent to this group.
> >> View/Reply Online (#178380):
> >> https://lists.openembedded.org/g/openembedded-core/message/178380
> >> Mute This Topic: https://lists.openembedded.org/mt/95827446/3617156
> >> Group Owner: openembedded-core+owner@lists.openembedded.org
> >> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> >> Martin.Jansa@gmail.com]
> >> -=-=-=-=-=-=-=-=-=-=-=-
> >>
> >>
> >
>
>

-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II

[-- Attachment #2: Type: text/html, Size: 18673 bytes --]

Re: [langdale][kirkstone][PATCH] go-mod.bbclass: Allow network in do_compile

[wonsang.ryu]

[-- Attachment #1: Type: text/plain, Size: 19764 bytes --]

I also tested using telegraf v1.21.4.
Kirkstone uses go1.17, so I tested this version.
Unfortunately, it failed with the following log.
Did I have something mistakes?

~$ recipetool create -S 583ee20a093acaa0781508b7840deb97a0fda622 -B release-1.21 https://github.com/influxdata/telegraf.git

Initialising tasks: 100% |################################################################################################| Time: 0:00:00
Sstate summary: Wanted 0 Local 0 Mirrors 0 Missed 0 Current 0 (0% match, 0% complete)
NOTE: No setscene tasks
NOTE: Executing Tasks
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 4 seconds
INFO: Resolved download path for import 'cloud.google.com/go/bigquery' => https://github.com/googleapis/google-cloud-go
INFO: Resolved download path for import 'cloud.google.com/go/monitoring' => https://github.com/googleapis/google-cloud-go
INFO: Resolved download path for import 'cloud.google.com/go/pubsub' => https://github.com/googleapis/google-cloud-go
INFO: Resolved download path for import 'collectd.org' => https://github.com/collectd/go-collectd
INFO: Resolved download path for import 'github.com/Azure/azure-event-hubs-go/v3' => https://github.com/Azure/azure-event-hubs-go
INFO: Resolved download path for import 'github.com/Azure/azure-kusto-go' => https://github.com/Azure/azure-kusto-go
INFO: Resolved download path for import 'github.com/Azure/azure-storage-queue-go' => https://github.com/Azure/azure-storage-queue-go
INFO: Resolved download path for import 'github.com/Azure/go-autorest/autorest' => https://github.com/Azure/go-autorest
INFO: Resolved download path for import 'github.com/Azure/go-autorest/autorest/adal' => https://github.com/Azure/go-autorest
INFO: Resolved download path for import 'github.com/Azure/go-autorest/autorest/azure/auth' => https://github.com/Azure/go-autorest
INFO: Resolved download path for import 'github.com/BurntSushi/toml' => https://github.com/BurntSushi/toml
INFO: Resolved download path for import 'github.com/Mellanox/rdmamap' => https://github.com/Mellanox/rdmamap
INFO: Resolved download path for import 'github.com/Shopify/sarama' => https://github.com/Shopify/sarama
INFO: Resolved download path for import 'github.com/aerospike/aerospike-client-go' => https://github.com/aerospike/aerospike-client-go
INFO: Resolved download path for import 'github.com/alecthomas/units' => https://github.com/alecthomas/units
INFO: Resolved download path for import 'github.com/aliyun/alibaba-cloud-sdk-go' => https://github.com/aliyun/alibaba-cloud-sdk-go
INFO: Resolved download path for import 'github.com/amir/raidman' => https://github.com/amir/raidman
INFO: Resolved download path for import 'github.com/antchfx/jsonquery' => https://github.com/antchfx/jsonquery
INFO: Resolved download path for import 'github.com/antchfx/xmlquery' => https://github.com/antchfx/xmlquery
INFO: Resolved download path for import 'github.com/antchfx/xpath' => https://github.com/antchfx/xpath
INFO: Resolved download path for import 'github.com/apache/thrift' => https://github.com/apache/thrift
INFO: Resolved download path for import 'github.com/aristanetworks/goarista' => https://github.com/aristanetworks/goarista
INFO: Resolved download path for import 'github.com/aws/aws-sdk-go-v2' => https://github.com/aws/aws-sdk-go-v2
INFO: Resolved download path for import 'github.com/aws/aws-sdk-go-v2/config' => https://github.com/aws/aws-sdk-go-v2
INFO: Resolved download path for import 'github.com/aws/aws-sdk-go-v2/credentials' => https://github.com/aws/aws-sdk-go-v2
INFO: Resolved download path for import 'github.com/aws/aws-sdk-go-v2/feature/ec2/imds' => https://github.com/aws/aws-sdk-go-v2
INFO: Resolved download path for import 'github.com/aws/aws-sdk-go-v2/service/cloudwatch' => https://github.com/aws/aws-sdk-go-v2
INFO: Resolved download path for import 'github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs' => https://github.com/aws/aws-sdk-go-v2
INFO: Resolved download path for import 'github.com/aws/aws-sdk-go-v2/service/dynamodb' => https://github.com/aws/aws-sdk-go-v2
INFO: Resolved download path for import 'github.com/aws/aws-sdk-go-v2/service/ec2' => https://github.com/aws/aws-sdk-go-v2
INFO: Resolved download path for import 'github.com/aws/aws-sdk-go-v2/service/kinesis' => https://github.com/aws/aws-sdk-go-v2
INFO: Resolved download path for import 'github.com/aws/aws-sdk-go-v2/service/sts' => https://github.com/aws/aws-sdk-go-v2
INFO: Resolved download path for import 'github.com/aws/aws-sdk-go-v2/service/timestreamwrite' => https://github.com/aws/aws-sdk-go-v2
INFO: Resolved download path for import 'github.com/aws/smithy-go' => https://github.com/aws/smithy-go
INFO: Resolved download path for import 'github.com/benbjohnson/clock' => https://github.com/benbjohnson/clock
INFO: Resolved download path for import 'github.com/bmatcuk/doublestar/v3' => https://github.com/bmatcuk/doublestar
INFO: Resolved download path for import 'github.com/caio/go-tdigest' => https://github.com/caio/go-tdigest
INFO: Resolved download path for import 'github.com/cisco-ie/nx-telemetry-proto' => https://github.com/cisco-ie/nx-telemetry-proto
INFO: Resolved download path for import 'github.com/coreos/go-semver' => https://github.com/coreos/go-semver
INFO: Resolved download path for import 'github.com/couchbase/go-couchbase' => https://github.com/couchbase/go-couchbase
INFO: Resolved download path for import 'github.com/denisenkom/go-mssqldb' => https://github.com/denisenkom/go-mssqldb
INFO: Resolved download path for import 'github.com/dimchansky/utfbom' => https://github.com/dimchansky/utfbom
INFO: Resolved download path for import 'github.com/djherbis/times' => https://github.com/djherbis/times
INFO: Resolved download path for import 'github.com/docker/docker' => https://github.com/docker/docker
INFO: Resolved download path for import 'github.com/doclambda/protobufquery' => https://github.com/doclambda/protobufquery
INFO: Resolved download path for import 'github.com/dynatrace-oss/dynatrace-metric-utils-go' => https://github.com/dynatrace-oss/dynatrace-metric-utils-go
INFO: Resolved download path for import 'github.com/eclipse/paho.mqtt.golang' => https://github.com/eclipse/paho.mqtt.golang
INFO: Resolved download path for import 'github.com/fatih/color' => https://github.com/fatih/color
INFO: Resolved download path for import 'github.com/ghodss/yaml' => https://github.com/ghodss/yaml
INFO: Resolved download path for import 'github.com/go-ldap/ldap/v3' => https://github.com/go-ldap/ldap
INFO: Resolved download path for import 'github.com/go-logfmt/logfmt' => https://github.com/go-logfmt/logfmt
INFO: Resolved download path for import 'github.com/go-ping/ping' => https://github.com/go-ping/ping
INFO: Resolved download path for import 'github.com/go-redis/redis' => https://github.com/go-redis/redis
INFO: Resolved download path for import 'github.com/go-sql-driver/mysql' => https://github.com/go-sql-driver/mysql
INFO: Resolved download path for import 'github.com/gobwas/glob' => https://github.com/gobwas/glob
INFO: Resolved download path for import 'github.com/gofrs/uuid' => https://github.com/gofrs/uuid
INFO: Resolved download path for import 'github.com/golang-jwt/jwt/v4' => https://github.com/golang-jwt/jwt
INFO: Resolved download path for import 'github.com/golang/geo' => https://github.com/golang/geo
INFO: Resolved download path for import 'github.com/golang/snappy' => https://github.com/golang/snappy
INFO: Resolved download path for import 'github.com/google/go-cmp' => https://github.com/google/go-cmp
INFO: Resolved download path for import 'github.com/google/go-github/v32' => https://github.com/google/go-github
INFO: Resolved download path for import 'github.com/gopcua/opcua' => https://github.com/gopcua/opcua
INFO: Resolved download path for import 'github.com/gophercloud/gophercloud' => https://github.com/gophercloud/gophercloud
INFO: Resolved download path for import 'github.com/gorilla/mux' => https://github.com/gorilla/mux
INFO: Resolved download path for import 'github.com/gorilla/websocket' => https://github.com/gorilla/websocket
INFO: Resolved download path for import 'github.com/gosnmp/gosnmp' => https://github.com/gosnmp/gosnmp
INFO: Resolved download path for import 'github.com/grid-x/modbus' => https://github.com/grid-x/modbus
INFO: Resolved download path for import 'github.com/gwos/tcg/sdk' => https://github.com/gwos/tcg
INFO: Resolved download path for import 'github.com/harlow/kinesis-consumer' => https://github.com/harlow/kinesis-consumer
INFO: Resolved download path for import 'github.com/hashicorp/consul/api' => https://github.com/hashicorp/consul
INFO: Resolved download path for import 'github.com/hashicorp/go-uuid' => https://github.com/hashicorp/go-uuid
INFO: Resolved download path for import 'github.com/influxdata/go-syslog/v3' => https://github.com/influxdata/go-syslog
INFO: Resolved download path for import 'github.com/influxdata/influxdb-observability/common' => https://github.com/influxdata/influxdb-observability
INFO: Resolved download path for import 'github.com/influxdata/influxdb-observability/influx2otel' => https://github.com/influxdata/influxdb-observability
INFO: Resolved download path for import 'github.com/influxdata/influxdb-observability/otel2influx' => https://github.com/influxdata/influxdb-observability
INFO: Resolved download path for import 'github.com/influxdata/tail' => https://github.com/influxdata/tail
INFO: Resolved download path for import 'github.com/influxdata/toml' => https://github.com/influxdata/toml
INFO: Resolved download path for import 'github.com/influxdata/wlog' => https://github.com/influxdata/wlog
INFO: Resolved download path for import 'github.com/intel/iaevents' => https://github.com/intel/iaevents
INFO: Resolved download path for import 'github.com/jackc/pgx/v4' => https://github.com/jackc/pgx
INFO: Resolved download path for import 'github.com/james4k/rcon' => https://github.com/james4k/rcon
INFO: Resolved download path for import 'github.com/jhump/protoreflect' => https://github.com/jhump/protoreflect
INFO: Resolved download path for import 'github.com/jmespath/go-jmespath' => https://github.com/jmespath/go-jmespath
INFO: Resolved download path for import 'github.com/kardianos/service' => https://github.com/kardianos/service
INFO: Resolved download path for import 'github.com/karrick/godirwalk' => https://github.com/karrick/godirwalk
INFO: Resolved download path for import 'github.com/kballard/go-shellquote' => https://github.com/kballard/go-shellquote
INFO: Resolved download path for import 'github.com/matttproud/golang_protobuf_extensions' => https://github.com/matttproud/golang_protobuf_extensions
INFO: Resolved download path for import 'github.com/mdlayher/apcupsd' => https://github.com/mdlayher/apcupsd
INFO: Resolved download path for import 'github.com/microsoft/ApplicationInsights-Go' => https://github.com/microsoft/ApplicationInsights-Go
INFO: Resolved download path for import 'github.com/miekg/dns' => https://github.com/miekg/dns
INFO: Resolved download path for import 'github.com/moby/ipvs' => https://github.com/moby/ipvs
INFO: Resolved download path for import 'github.com/multiplay/go-ts3' => https://github.com/multiplay/go-ts3
INFO: Resolved download path for import 'github.com/nats-io/nats-server/v2' => https://github.com/nats-io/nats-server
INFO: Resolved download path for import 'github.com/nats-io/nats.go' => https://github.com/nats-io/nats.go
INFO: Resolved download path for import 'github.com/newrelic/newrelic-telemetry-sdk-go' => https://github.com/newrelic/newrelic-telemetry-sdk-go
INFO: Resolved download path for import 'github.com/nsqio/go-nsq' => https://github.com/nsqio/go-nsq
INFO: Resolved download path for import 'github.com/olivere/elastic' => https://github.com/olivere/elastic
INFO: Resolved download path for import 'github.com/openconfig/gnmi' => https://github.com/openconfig/gnmi
INFO: Resolved download path for import 'github.com/opentracing/opentracing-go' => https://github.com/opentracing/opentracing-go
INFO: Resolved download path for import 'github.com/openzipkin-contrib/zipkin-go-opentracing' => https://github.com/openzipkin-contrib/zipkin-go-opentracing
INFO: Resolved download path for import 'github.com/openzipkin/zipkin-go' => https://github.com/openzipkin/zipkin-go
INFO: Resolved download path for import 'github.com/pion/dtls/v2' => https://github.com/pion/dtls
INFO: Resolved download path for import 'github.com/pkg/errors' => https://github.com/pkg/errors
INFO: Resolved download path for import 'github.com/prometheus/client_golang' => https://github.com/prometheus/client_golang
INFO: Resolved download path for import 'github.com/prometheus/client_model' => https://github.com/prometheus/client_model
INFO: Resolved download path for import 'github.com/prometheus/common' => https://github.com/prometheus/common
INFO: Resolved download path for import 'github.com/prometheus/procfs' => https://github.com/prometheus/procfs
INFO: Resolved download path for import 'github.com/prometheus/prometheus' => https://github.com/prometheus/prometheus
INFO: Resolved download path for import 'github.com/riemann/riemann-go-client' => https://github.com/riemann/riemann-go-client
INFO: Resolved download path for import 'github.com/safchain/ethtool' => https://github.com/safchain/ethtool
INFO: Resolved download path for import 'github.com/sensu/sensu-go/api/core/v2' => https://github.com/sensu/sensu-go
INFO: Resolved download path for import 'github.com/shirou/gopsutil/v3' => https://github.com/shirou/gopsutil
INFO: Resolved download path for import 'github.com/showwin/speedtest-go' => https://github.com/showwin/speedtest-go
INFO: Resolved download path for import 'github.com/signalfx/golib/v3' => https://github.com/signalfx/golib
INFO: Resolved download path for import 'github.com/sirupsen/logrus' => https://github.com/sirupsen/logrus
INFO: Resolved download path for import 'github.com/sleepinggenius2/gosmi' => https://github.com/sleepinggenius2/gosmi
INFO: Resolved download path for import 'github.com/snowflakedb/gosnowflake' => https://github.com/snowflakedb/gosnowflake
INFO: Resolved download path for import 'github.com/streadway/amqp' => https://github.com/streadway/amqp
INFO: Resolved download path for import 'github.com/stretchr/testify' => https://github.com/stretchr/testify
INFO: Resolved download path for import 'github.com/tbrandon/mbserver' => https://github.com/tbrandon/mbserver
INFO: Resolved download path for import 'github.com/testcontainers/testcontainers-go' => https://github.com/testcontainers/testcontainers-go
INFO: Resolved download path for import 'github.com/tidwall/gjson' => https://github.com/tidwall/gjson
INFO: Resolved download path for import 'github.com/tinylib/msgp' => https://github.com/tinylib/msgp
INFO: Resolved download path for import 'github.com/vapourismo/knx-go' => https://github.com/vapourismo/knx-go
INFO: Resolved download path for import 'github.com/vjeantet/grok' => https://github.com/vjeantet/grok
INFO: Resolved download path for import 'github.com/vmware/govmomi' => https://github.com/vmware/govmomi
INFO: Resolved download path for import 'github.com/wavefronthq/wavefront-sdk-go' => https://github.com/wavefronthq/wavefront-sdk-go
INFO: Resolved download path for import 'github.com/wvanbergen/kafka' => https://github.com/wvanbergen/kafka
INFO: Resolved download path for import 'github.com/xdg/scram' => https://github.com/xdg/scram
INFO: Resolved download path for import 'go.mongodb.org/mongo-driver' => https://github.com/mongodb/mongo-go-driver.git
INFO: Resolved download path for import 'go.opentelemetry.io/collector/model' => https://github.com/open-telemetry/opentelemetry-collector
INFO: Resolved download path for import 'go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc' => https://github.com/open-telemetry/opentelemetry-go
INFO: Resolved download path for import 'go.opentelemetry.io/otel/metric' => https://github.com/open-telemetry/opentelemetry-go
INFO: Resolved download path for import 'go.opentelemetry.io/otel/sdk/metric' => https://github.com/open-telemetry/opentelemetry-go
INFO: Resolved download path for import 'go.starlark.net' => https://github.com/google/starlark-go
INFO: Resolved download path for import 'golang.org/x/net' => https://go.googlesource.com/net
INFO: Resolved download path for import 'golang.org/x/oauth2' => https://go.googlesource.com/oauth2
INFO: Resolved download path for import 'golang.org/x/sync' => https://go.googlesource.com/sync
INFO: Resolved download path for import 'golang.org/x/sys' => https://go.googlesource.com/sys
INFO: Resolved download path for import 'golang.org/x/text' => https://go.googlesource.com/text
INFO: Resolved download path for import 'golang.zx2c4.com/wireguard/wgctrl' => https://github.com/WireGuard/wgctrl-go
INFO: Resolved download path for import 'google.golang.org/api' => https://github.com/googleapis/google-api-go-client
INFO: Resolved download path for import 'google.golang.org/genproto' => https://github.com/googleapis/go-genproto
INFO: Resolved download path for import 'google.golang.org/grpc' => https://github.com/grpc/grpc-go
INFO: Resolved download path for import 'google.golang.org/protobuf' => https://go.googlesource.com/protobuf
INFO: Resolved download path for import 'gopkg.in/gorethink/gorethink.v3' => https://gopkg.in/gorethink/gorethink.v3
INFO: Resolved download path for import 'gopkg.in/olivere/elastic.v5' => https://gopkg.in/olivere/elastic.v5
INFO: Resolved download path for import 'gopkg.in/tomb.v1' => https://gopkg.in/tomb.v1
INFO: Resolved download path for import 'gopkg.in/yaml.v2' => https://gopkg.in/yaml.v2
INFO: Resolved download path for import 'k8s.io/api' => https://github.com/kubernetes/api
INFO: Resolved download path for import 'k8s.io/apimachinery' => https://github.com/kubernetes/apimachinery
INFO: Resolved download path for import 'k8s.io/client-go' => https://github.com/kubernetes/client-go
INFO: Resolved download path for import 'modernc.org/sqlite' => https://gitlab.com/cznic/sqlite
INFO: Resolved download path for import 'cloud.google.com/go' => https://github.com/googleapis/google-cloud-go
INFO: Resolved download path for import 'cloud.google.com/go/compute' => https://github.com/googleapis/google-cloud-go
INFO: Resolved download path for import 'cloud.google.com/go/iam' => https://github.com/googleapis/google-cloud-go
Traceback (most recent call last):
File "/oe-core/scripts/recipetool", line 111, in <module>
ret = main()
File "/oe-core/scripts/recipetool", line 100, in main
ret = args.func(args)
File "/oe-core/scripts/lib/recipetool/create.py", line 746, in create_recipe
handler.process(srctree_use, classes, lines_before, lines_after, handled, extravalues)
File "/oe-core/scripts/lib/recipetool/create_go.py", line 336, in process
dep_src_uris, src_revs = self._handle_dependencies(d, srctree, go_mod)
File "/oe-core/scripts/lib/recipetool/create_go.py", line 261, in _handle_dependencies
repodata = self._resolve_repository(module_path)
File "/oe-core/scripts/lib/recipetool/create_go.py", line 184, in _resolve_repository
repodata = self._resolve_repository_dynamic(modulepath)
File "/oe-core/scripts/lib/recipetool/create_go.py", line 176, in _resolve_repository_dynamic
return GoImport(parser.rootpath, parser.vcs, parser.repourl, None)
File "/oe-core/scripts/lib/recipetool/create_go.py", line 151, in rootpath
return self.__srv[0]
IndexError: list index out of range

[-- Attachment #2: Type: text/html, Size: 22509 bytes --]