From: "Böszörményi Zoltán" <zboszor@gmail.com>
To: Martin Jansa <martin.jansa@gmail.com>
Cc: openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] gitsm:// question
Date: Wed, 18 Oct 2023 18:18:07 +0200 [thread overview]
Message-ID: <b9d330f9-8789-4a8b-a078-00c7e062eb05@gmail.com> (raw)
In-Reply-To: <178F367FEBFA030F.14388@lists.openembedded.org>
Using a name for the main git:// SRC_URI entry and setting
SRCREV_FORMAT for it works.
The rest were partly my bugs: I had to fix two copy&paste errors
in that massive list of gitsm:// SRC_URI entries.
I also had to change a couple of them to use git:// due to the
submodules' incorrect declaration in their parent repos.
This is detected by Yocto and gave me the error below,
indicating the repos in question.
ERROR: python3-pytorch do_fetch: Fetcher failure: Submodule refers to the parent
repository. This will cause deadlock situation in current version of Bitbake.Consider
using git fetcher instead.
2023. 10. 18. 15:25 keltezéssel, Zoltan Boszormenyi via lists.openembedded.org írta:
> Thanks, I will try it.
>
> 2023. 10. 18. 15:16 keltezéssel, Martin Jansa írta:
>> Just add the name to pytorch repo url and use only that in SRCREV_FORMAT, you probably
>> don't care about representing the other repos in SRCPV if you're updating them only
>> together with main pytorch update.
>>
>> On Wed, Oct 18, 2023 at 3:05 PM Zoltan Boszormenyi <zboszor@gmail.com> wrote:
>>
>> Hi,
>>
>> I have a working python3-pytorch recipe which originally used this:
>>
>> =======================================================
>> addtask do_git_submodules after do_unpack before do_patch
>>
>> do_git_submodules[network] = "1"
>>
>> do_git_submodules () {
>> cd ${S}
>> git submodule update --init --recursive
>> }
>> =======================================================
>>
>> I would like to replace the above with a series of gitsm:// SRC_URI lines.
>> Here's a subset of submodules that shows the problem I am seeing:
>>
>> SRC_URI = " \
>> git://github.com/pytorch/pytorch.git;protocol=https;branch=release/2.1
>> <http://github.com/pytorch/pytorch.git;protocol=https;branch=release/2.1> \
>> ...
>> gitsm://github.com/pytorch/FBGEMM.git;protocol=https;name=fbgemm;nobranch=1;destsuffix=third_party/fbgemm
>> <http://github.com/pytorch/FBGEMM.git;protocol=https;name=fbgemm;nobranch=1;destsuffix=third_party/fbgemm>
>>
>> \
>> gitsm://github.com/asmjit/asmjit.git;protocol=https;name=fbgemmasmjit;destsuffix=third_party/fbgemm/third_party/asmjit
>> <http://github.com/asmjit/asmjit.git;protocol=https;name=fbgemmasmjit;destsuffix=third_party/fbgemm/third_party/asmjit>
>>
>> \
>> gitsm://github.com/pytorch/cpuinfo.git;protocol=https;name=fbgemmcpuinfo;destsuffix=third_party/fbgemm/third_party/cpuinfo
>> <http://github.com/pytorch/cpuinfo.git;protocol=https;name=fbgemmcpuinfo;destsuffix=third_party/fbgemm/third_party/cpuinfo>
>>
>> \
>> gitsm://github.com/NVIDIA/cutlass.git;protocol=https;name=fbgemmcutlass;destsuffix=third_party/fbgemm/third_party/cutlass
>> <http://github.com/NVIDIA/cutlass.git;protocol=https;name=fbgemmcutlass;destsuffix=third_party/fbgemm/third_party/cutlass>
>>
>> \
>> gitsm://github.com/google/googletest.git;protocol=https;name=fbgemmgtest;destsuffix=third_party/fbgemm/third_party/googletest
>> <http://github.com/google/googletest.git;protocol=https;name=fbgemmgtest;destsuffix=third_party/fbgemm/third_party/googletest>
>>
>> \
>> gitsm://github.com/ROCmSoftwarePlatform/hipify_torch.git;protocol=https;name=fbgemmhiptorch;destsuffix=third_party/fbgemm/third_party/hipify_torch
>> <http://github.com/ROCmSoftwarePlatform/hipify_torch.git;protocol=https;name=fbgemmhiptorch;destsuffix=third_party/fbgemm/third_party/hipify_torch>
>>
>> \
>> ...
>> "
>>
>> As you can see, there are recursively placed git submodules
>> and do_fetch fails with an error message about SRCREV_FORMAT
>> having to be set.
>>
>> The above is a limited subset of the complete list and
>> some of the repositories (like googletest, gloo, glog and pybind11)
>> occurs multiple times as child submodules of upper ones,
>> with different SRCREV values in different leaf submodules.
>>
>> I have not found a conclusive example to use SRCREV_FORMAT
>> with git:// + gitsm://
>>
>> Can someone enlighten me how to use SRCREV_FORMAT properly
>> for this case? Should I stick to the proven working nonstandard way
>> and just run git submodule update --init --recursive?
>>
>> As an easy way out, is there a flag to git:// to process its
>> submodules automatically?
>>
>> Thanks in advance,
>> Zoltán Böszörményi
>>
>>
>>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#189390): https://lists.openembedded.org/g/openembedded-core/message/189390
> Mute This Topic: https://lists.openembedded.org/mt/102038382/3617728
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [zboszor@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
prev parent reply other threads:[~2023-10-18 16:18 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-18 13:04 gitsm:// question Böszörményi Zoltán
2023-10-18 13:16 ` [OE-core] " Martin Jansa
2023-10-18 13:25 ` Böszörményi Zoltán
2023-10-18 13:28 ` Julien Stephan
2023-10-18 14:12 ` Böszörményi Zoltán
2023-10-18 14:24 ` Julien Stephan
2023-10-18 14:28 ` Martin Jansa
[not found] ` <178F367FEBFA030F.14388@lists.openembedded.org>
2023-10-18 16:18 ` Böszörményi Zoltán [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b9d330f9-8789-4a8b-a078-00c7e062eb05@gmail.com \
--to=zboszor@gmail.com \
--cc=martin.jansa@gmail.com \
--cc=openembedded-core@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox