* Sumo to Thud migration: Fetch failure from gitsm.py
@ 2018-11-22 15:31 da Silva Filho, Josias Inacio
2018-11-23 10:30 ` Robert Yang
2018-11-26 14:08 ` Mark Hatle
0 siblings, 2 replies; 6+ messages in thread
From: da Silva Filho, Josias Inacio @ 2018-11-22 15:31 UTC (permalink / raw)
To: bitbake-devel@lists.openembedded.org
[-- Attachment #1: Type: text/plain, Size: 1355 bytes --]
Hi folks,
I'm working on migrating my environment from Sumo to Thud. After fixing a few quirks, it seems the fetch is failing for my foo.bb which uses gitsm.
.gitmodules content in foo repo:
[submodule "common-lib"]
path = common-lib
url = ssh://git@<url_to_common-lib>/common-lib.git
foo.bb:
SRC_URI="gitsm://git@b<url_to_foo>.git;protocol=ssh;branch=master"
SRCREV="f55fae9dc9b476f8ed193f15e2d5cd6747777451"
Digging inside gitsm.py it looks like module_hash is not being set correctly (empty) and that causes the issue:
http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/bitbake/lib/bb/fetch2/gitsm.py#n87
I added the following right after line 87, so I could dump the variables content. This is how I confirmed module_hash is empty:
raise bb.fetch2.FetchError('module: %s, submodules: %s, module_hash: %s, basecmd: %s, revisions: %s, paths: %s, clonedir: %s ' % ( module, submodules, module_hash, ud.basecmd, ud.revisions[name], paths[module], ud.clonedir))
And the output:
ERROR: Fetcher failure: module: common-lib, submodules: ['common-lib'], module_hash: , basecmd: git -c core.fsyncobjectfiles=0, revisions: f55fae9dc9b476f8ed193f15e2d5cd6747777451, paths: common-lib, clonedir: /localdata/yocto/downloads/git2/<foo's path>.git
Any ideas why module_hash is not being set correctly?
[-- Attachment #2: Type: text/html, Size: 4323 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: Sumo to Thud migration: Fetch failure from gitsm.py
2018-11-22 15:31 Sumo to Thud migration: Fetch failure from gitsm.py da Silva Filho, Josias Inacio
@ 2018-11-23 10:30 ` Robert Yang
2018-11-23 23:03 ` [**EXTERNAL**] " da Silva Filho, Josias Inacio
2018-11-26 14:08 ` Mark Hatle
1 sibling, 1 reply; 6+ messages in thread
From: Robert Yang @ 2018-11-23 10:30 UTC (permalink / raw)
To: da Silva Filho, Josias Inacio; +Cc: bitbake-devel@lists.openembedded.org
Hi,
Please refer to the following 2 commits in bitbake which may help:
commit a042179a784bab6a6f4bd5d63db80c1c02624b1c
Author: Mark Hatle <mark.hatle@windriver.com>
Date: Wed Oct 31 15:21:44 2018 -0400
bitbake: fetch2/gitsm.py: Fix the references when the module and path are
different
commit a28637abbceb05b52a6e38ab2340d3da9d0b1b37
Author: Mark Hatle <mark.hatle@windriver.com>
Date: Wed Oct 31 15:21:43 2018 -0400
bitbake: fetch2/gitsm.py: Disable branch checking on submodules
// Robert
On 11/22/18 11:31 PM, da Silva Filho, Josias Inacio wrote:
> Hi folks,
>
> I’m working on migrating my environment from Sumo to Thud. After fixing a few
> quirks, it seems the fetch is failing for my foo.bb which uses gitsm.
>
> *.gitmodules content in foo repo:*
>
> [submodule "common-lib"]
>
> path = common-lib
>
> url = ssh://git@<url_to_common-lib>/common-lib.git
>
> *foo.bb:*
>
> SRC_URI="gitsm://git@b<url_to_foo>.git;protocol=ssh;branch=master"
>
> SRCREV="f55fae9dc9b476f8ed193f15e2d5cd6747777451"
>
> Digging inside gitsm.py it looks like module_hash is not being set correctly
> (empty) and that causes the issue:
>
> http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/bitbake/lib/bb/fetch2/gitsm.py#n87
>
> I added the following right after line 87, so I could dump the variables
> content. This is how I confirmed module_hash is empty:
>
> *raise bb.fetch2.FetchError('module: %s, submodules: %s, module_hash: %s,
> basecmd: %s, revisions: %s, paths: %s, clonedir: %s ' % ( module, submodules,
> module_hash, ud.basecmd, ud.revisions[name], paths[module], ud.clonedir))*
>
> And the output:
>
> ERROR: Fetcher failure: module: common-lib, submodules: ['common-lib'],
> *module_hash*: , basecmd: git -c core.fsyncobjectfiles=0, revisions:
> f55fae9dc9b476f8ed193f15e2d5cd6747777451, paths: common-lib, clonedir:
> /localdata/yocto/downloads/git2/<foo's path>.git
>
> Any ideas why module_hash is not being set correctly?
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [**EXTERNAL**] Re: Sumo to Thud migration: Fetch failure from gitsm.py
2018-11-23 10:30 ` Robert Yang
@ 2018-11-23 23:03 ` da Silva Filho, Josias Inacio
0 siblings, 0 replies; 6+ messages in thread
From: da Silva Filho, Josias Inacio @ 2018-11-23 23:03 UTC (permalink / raw)
To: Robert Yang; +Cc: bitbake-devel@lists.openembedded.org
Hi Robert,
I had a look at those two commits before, also tried reverting them but got the same error.
I reverted a few other changes in gitsm.py for testing, but the error only went away once I reverted this one:
commit 5cd00e3e53819f3c10d1733480077ce9f3cc2ca8
Author: Mark Hatle <mark.hatle@windriver.com>
Date: Tue Sep 25 13:15:25 2018 -0400
bitbake: fetch2/gitsm.py: Rework the git submodule fetcher
-----Original Message-----
From: Robert Yang <liezhi.yang@windriver.com>
Sent: Friday, November 23, 2018 5:31 AM
To: da Silva Filho, Josias Inacio <jdasilva@ciena.com>
Cc: bitbake-devel@lists.openembedded.org; Mark Hatle <mark.hatle@windriver.com>
Subject: [**EXTERNAL**] Re: [bitbake-devel] Sumo to Thud migration: Fetch failure from gitsm.py
Hi,
Please refer to the following 2 commits in bitbake which may help:
commit a042179a784bab6a6f4bd5d63db80c1c02624b1c
Author: Mark Hatle <mark.hatle@windriver.com>
Date: Wed Oct 31 15:21:44 2018 -0400
bitbake: fetch2/gitsm.py: Fix the references when the module and path are different
commit a28637abbceb05b52a6e38ab2340d3da9d0b1b37
Author: Mark Hatle <mark.hatle@windriver.com>
Date: Wed Oct 31 15:21:43 2018 -0400
bitbake: fetch2/gitsm.py: Disable branch checking on submodules
// Robert
On 11/22/18 11:31 PM, da Silva Filho, Josias Inacio wrote:
> Hi folks,
>
> I'm working on migrating my environment from Sumo to Thud. After
> fixing a few quirks, it seems the fetch is failing for my foo.bb which uses gitsm.
>
> *.gitmodules content in foo repo:*
>
> [submodule "common-lib"]
>
> path = common-lib
>
> url = ssh://git@<url_to_common-lib>/common-lib.git
>
> *foo.bb:*
>
> SRC_URI="gitsm://git@b<url_to_foo>.git;protocol=ssh;branch=master"
>
> SRCREV="f55fae9dc9b476f8ed193f15e2d5cd6747777451"
>
> Digging inside gitsm.py it looks like module_hash is not being set
> correctly
> (empty) and that causes the issue:
>
> http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/bitbake/lib/bb/fet
> ch2/gitsm.py#n87
>
> I added the following right after line 87, so I could dump the
> variables content. This is how I confirmed module_hash is empty:
>
> *raise bb.fetch2.FetchError('module: %s, submodules: %s, module_hash:
> %s,
> basecmd: %s, revisions: %s, paths: %s, clonedir: %s ' % ( module,
> submodules, module_hash, ud.basecmd, ud.revisions[name],
> paths[module], ud.clonedir))*
>
> And the output:
>
> ERROR: Fetcher failure: module: common-lib, submodules:
> ['common-lib'],
> *module_hash*: , basecmd: git -c core.fsyncobjectfiles=0, revisions:
> f55fae9dc9b476f8ed193f15e2d5cd6747777451, paths: common-lib, clonedir:
> /localdata/yocto/downloads/git2/<foo's path>.git
>
> Any ideas why module_hash is not being set correctly?
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Sumo to Thud migration: Fetch failure from gitsm.py
2018-11-22 15:31 Sumo to Thud migration: Fetch failure from gitsm.py da Silva Filho, Josias Inacio
2018-11-23 10:30 ` Robert Yang
@ 2018-11-26 14:08 ` Mark Hatle
2018-12-04 20:50 ` [**EXTERNAL**] " da Silva Filho, Josias Inacio
1 sibling, 1 reply; 6+ messages in thread
From: Mark Hatle @ 2018-11-26 14:08 UTC (permalink / raw)
To: da Silva Filho, Josias Inacio,
bitbake-devel@lists.openembedded.org
On 11/22/18 9:31 AM, da Silva Filho, Josias Inacio wrote:
> Hi folks,
>
>
>
> I’m working on migrating my environment from Sumo to Thud. After fixing a few
> quirks, it seems the fetch is failing for my foo.bb which uses gitsm.
>
>
>
> *.gitmodules content in foo repo:*
>
> [submodule "common-lib"]
>
> path = common-lib
>
> url = ssh://git@<url_to_common-lib>/common-lib.git
>
>
>
> *foo.bb:*
>
> SRC_URI="gitsm://git@b<url_to_foo>.git;protocol=ssh;branch=master"
>
> SRCREV="f55fae9dc9b476f8ed193f15e2d5cd6747777451"
>
>
>
> Digging inside gitsm.py it looks like module_hash is not being set correctly
> (empty) and that causes the issue:
>
> http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/bitbake/lib/bb/fetch2/gitsm.py#n87
>
>
>
> I added the following right after line 87, so I could dump the variables
> content. This is how I confirmed module_hash is empty:
>
> *raise bb.fetch2.FetchError('module: %s, submodules: %s, module_hash: %s,
> basecmd: %s, revisions: %s, paths: %s, clonedir: %s ' % ( module, submodules,
> module_hash, ud.basecmd, ud.revisions[name], paths[module], ud.clonedir))*
Run the cloning steps manually. And run the same git instructions as what the
gitsm fetcher does to find the hash for the submodule.
git clone --bare <url> <destination>
cd <destination>
git show <commit>:.gitmodules
for each of the modules listed:
git ls-tree -z -d <commit> <module path>
It's the bit in the for loop that doesn't seem to be returning anything. The
only time I've ever seen this not work is when a module is defined in the
.gitmodules file, but was never actually added to the repository.
>
>
> And the output:
>
> ERROR: Fetcher failure: module: common-lib, submodules: ['common-lib'],
> *module_hash*: , basecmd: git -c core.fsyncobjectfiles=0, revisions:
> f55fae9dc9b476f8ed193f15e2d5cd6747777451, paths: common-lib, clonedir:
> /localdata/yocto/downloads/git2/<foo's path>.git
git clone --bare ssh://git@b<url_to_foo>.git my_repo
cd my_repo
git show f55fae9dc9b476f8ed193f15e2d5cd6747777451:.gitmodules
for each of the modules listed: (focusing on common-lib)
git ls-tree -z -d f55fae9dc9b476f8ed193f15e2d5cd6747777451 common-lib
BTW my suggestion is to throw an exception or at least do a warning in the for
loop if module_hash is empty... since this is clearly an invalid configuration.
(after the split line)
if not module_hash:
logger.warning("Submodule %s of %s does not appear to have a commit defined."
% ( ud.url, module ))
--Mark
> Any ideas why module_hash is not being set correctly?
>
>
>
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [**EXTERNAL**] Re: Sumo to Thud migration: Fetch failure from gitsm.py
2018-11-26 14:08 ` Mark Hatle
@ 2018-12-04 20:50 ` da Silva Filho, Josias Inacio
0 siblings, 0 replies; 6+ messages in thread
From: da Silva Filho, Josias Inacio @ 2018-12-04 20:50 UTC (permalink / raw)
To: Mark Hatle, bitbake-devel@lists.openembedded.org
Bingo! Indeed the module was added into .gitmodules but not added in the repo itself.
I will get someone to add the repo then I will re-test, but that looks to be the issue.
Thanks Mark!
-----Original Message-----
From: Mark Hatle <mark.hatle@windriver.com>
Sent: Monday, November 26, 2018 9:09 AM
To: da Silva Filho, Josias Inacio <jdasilva@ciena.com>; bitbake-devel@lists.openembedded.org
Subject: [**EXTERNAL**] Re: [bitbake-devel] Sumo to Thud migration: Fetch failure from gitsm.py
On 11/22/18 9:31 AM, da Silva Filho, Josias Inacio wrote:
> Hi folks,
>
>
>
> I'm working on migrating my environment from Sumo to Thud. After
> fixing a few quirks, it seems the fetch is failing for my foo.bb which uses gitsm.
>
>
>
> *.gitmodules content in foo repo:*
>
> [submodule "common-lib"]
>
> path = common-lib
>
> url = ssh://git@<url_to_common-lib>/common-lib.git
>
>
>
> *foo.bb:*
>
> SRC_URI="gitsm://git@b<url_to_foo>.git;protocol=ssh;branch=master"
>
> SRCREV="f55fae9dc9b476f8ed193f15e2d5cd6747777451"
>
>
>
> Digging inside gitsm.py it looks like module_hash is not being set
> correctly
> (empty) and that causes the issue:
>
> http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/bitbake/lib/bb/fet
> ch2/gitsm.py#n87
>
>
>
> I added the following right after line 87, so I could dump the
> variables content. This is how I confirmed module_hash is empty:
>
> *raise bb.fetch2.FetchError('module: %s, submodules: %s, module_hash:
> %s,
> basecmd: %s, revisions: %s, paths: %s, clonedir: %s ' % ( module,
> submodules, module_hash, ud.basecmd, ud.revisions[name],
> paths[module], ud.clonedir))*
Run the cloning steps manually. And run the same git instructions as what the gitsm fetcher does to find the hash for the submodule.
git clone --bare <url> <destination>
cd <destination>
git show <commit>:.gitmodules
for each of the modules listed:
git ls-tree -z -d <commit> <module path>
It's the bit in the for loop that doesn't seem to be returning anything. The only time I've ever seen this not work is when a module is defined in the .gitmodules file, but was never actually added to the repository.
>
>
> And the output:
>
> ERROR: Fetcher failure: module: common-lib, submodules:
> ['common-lib'],
> *module_hash*: , basecmd: git -c core.fsyncobjectfiles=0, revisions:
> f55fae9dc9b476f8ed193f15e2d5cd6747777451, paths: common-lib, clonedir:
> /localdata/yocto/downloads/git2/<foo's path>.git
git clone --bare ssh://git@b<url_to_foo>.git my_repo cd my_repo git show f55fae9dc9b476f8ed193f15e2d5cd6747777451:.gitmodules
for each of the modules listed: (focusing on common-lib)
git ls-tree -z -d f55fae9dc9b476f8ed193f15e2d5cd6747777451 common-lib
BTW my suggestion is to throw an exception or at least do a warning in the for loop if module_hash is empty... since this is clearly an invalid configuration.
(after the split line)
if not module_hash:
logger.warning("Submodule %s of %s does not appear to have a commit defined."
% ( ud.url, module ))
--Mark
> Any ideas why module_hash is not being set correctly?
>
>
>
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Sumo to Thud migration: Fetch failure from gitsm.py
@ 2018-11-20 23:01 da Silva Filho, Josias Inacio
0 siblings, 0 replies; 6+ messages in thread
From: da Silva Filho, Josias Inacio @ 2018-11-20 23:01 UTC (permalink / raw)
To: bitbake-devel@lists.openembedded.org
[-- Attachment #1: Type: text/plain, Size: 1353 bytes --]
Hi folks,
I'm working on migrating my environment from Sumo to Thud. After fixing a few quirks, it seems the fetch is failing for my foo.bb which uses gitsm.
.gitmodules content in foo repo:
[submodule "common-lib"]
path = common-lib
url = ssh://git@<url_to_common-lib>/common-lib.git
foo.bb:
SRC_URI="gitsm://git@b<url_to_foo>.git;protocol=ssh;branch=master"
SRCREV="f55fae9dc9b476f8ed193f15e2d5cd6747777451"
Digging inside gitsm.py it looks like module_hash is not being set correctly (empty) and that causes the issue:
http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/bitbake/lib/bb/fetch2/gitsm.py#n87
I added the following right after line 87, so I could dump the variables content. This is how I confirmed module_hash is empty:
raise bb.fetch2.FetchError('module: %s, submodules: %s, module_hash: %s, basecmd: %s, revisions: %s, paths: %s, clonedir: %s ' % ( module, submodules, module_hash, ud.basecmd, ud.revisions[name], paths[module], ud.clonedir))
And the output:
ERROR: Fetcher failure: module: common-lib, submodules: ['common-lib'], module_hash: , basecmd: git -c core.fsyncobjectfiles=0, revisions: f55fae9dc9b476f8ed193f15e2d5cd6747777451, paths: common-lib, clonedir: /localdata/yocto/downloads/git2/<foo's path>.git
Any ideas why module_hash is not being set correctly?
[-- Attachment #2: Type: text/html, Size: 4279 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2018-12-06 1:01 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-11-22 15:31 Sumo to Thud migration: Fetch failure from gitsm.py da Silva Filho, Josias Inacio
2018-11-23 10:30 ` Robert Yang
2018-11-23 23:03 ` [**EXTERNAL**] " da Silva Filho, Josias Inacio
2018-11-26 14:08 ` Mark Hatle
2018-12-04 20:50 ` [**EXTERNAL**] " da Silva Filho, Josias Inacio
-- strict thread matches above, loose matches on Subject: below --
2018-11-20 23:01 da Silva Filho, Josias Inacio
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox