Openembedded Core Discussions
 help / color / mirror / Atom feed
From: "Böszörményi Zoltán" <zboszor@gmail.com>
To: Martin Jansa <martin.jansa@gmail.com>
Cc: openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] gitsm:// question
Date: Wed, 18 Oct 2023 18:18:07 +0200	[thread overview]
Message-ID: <b9d330f9-8789-4a8b-a078-00c7e062eb05@gmail.com> (raw)
In-Reply-To: <178F367FEBFA030F.14388@lists.openembedded.org>

Using a name for the main git:// SRC_URI entry and setting
SRCREV_FORMAT for it works.

The rest were partly my bugs: I had to fix two copy&paste errors
in that massive list of gitsm:// SRC_URI entries.

I also had to change a couple of them to use git:// due to the
submodules' incorrect declaration in their parent repos.
This is detected by Yocto and gave me the error below,
indicating the repos in question.

ERROR: python3-pytorch do_fetch: Fetcher failure: Submodule refers to the parent 
repository. This will cause deadlock situation in current version of Bitbake.Consider 
using git fetcher instead.

2023. 10. 18. 15:25 keltezéssel, Zoltan Boszormenyi via lists.openembedded.org írta:
> Thanks, I will try it.
>
> 2023. 10. 18. 15:16 keltezéssel, Martin Jansa írta:
>> Just add the name to pytorch repo url and use only that in SRCREV_FORMAT, you probably 
>> don't care about representing the other repos in SRCPV if you're updating them only 
>> together with main pytorch update.
>>
>> On Wed, Oct 18, 2023 at 3:05 PM Zoltan Boszormenyi <zboszor@gmail.com> wrote:
>>
>>     Hi,
>>
>>     I have a working python3-pytorch recipe which originally used this:
>>
>>     =======================================================
>>     addtask do_git_submodules after do_unpack before do_patch
>>
>>     do_git_submodules[network] = "1"
>>
>>     do_git_submodules () {
>>              cd ${S}
>>              git submodule update --init --recursive
>>     }
>>     =======================================================
>>
>>     I would like to replace the above with a series of gitsm:// SRC_URI lines.
>>     Here's a subset of submodules that shows the problem I am seeing:
>>
>>     SRC_URI = " \
>> git://github.com/pytorch/pytorch.git;protocol=https;branch=release/2.1
>> <http://github.com/pytorch/pytorch.git;protocol=https;branch=release/2.1> \
>>     ...
>> gitsm://github.com/pytorch/FBGEMM.git;protocol=https;name=fbgemm;nobranch=1;destsuffix=third_party/fbgemm
>> <http://github.com/pytorch/FBGEMM.git;protocol=https;name=fbgemm;nobranch=1;destsuffix=third_party/fbgemm>
>>
>>     \
>> gitsm://github.com/asmjit/asmjit.git;protocol=https;name=fbgemmasmjit;destsuffix=third_party/fbgemm/third_party/asmjit
>> <http://github.com/asmjit/asmjit.git;protocol=https;name=fbgemmasmjit;destsuffix=third_party/fbgemm/third_party/asmjit>
>>
>>     \
>> gitsm://github.com/pytorch/cpuinfo.git;protocol=https;name=fbgemmcpuinfo;destsuffix=third_party/fbgemm/third_party/cpuinfo
>> <http://github.com/pytorch/cpuinfo.git;protocol=https;name=fbgemmcpuinfo;destsuffix=third_party/fbgemm/third_party/cpuinfo>
>>
>>     \
>> gitsm://github.com/NVIDIA/cutlass.git;protocol=https;name=fbgemmcutlass;destsuffix=third_party/fbgemm/third_party/cutlass
>> <http://github.com/NVIDIA/cutlass.git;protocol=https;name=fbgemmcutlass;destsuffix=third_party/fbgemm/third_party/cutlass>
>>
>>     \
>> gitsm://github.com/google/googletest.git;protocol=https;name=fbgemmgtest;destsuffix=third_party/fbgemm/third_party/googletest
>> <http://github.com/google/googletest.git;protocol=https;name=fbgemmgtest;destsuffix=third_party/fbgemm/third_party/googletest>
>>
>>     \
>> gitsm://github.com/ROCmSoftwarePlatform/hipify_torch.git;protocol=https;name=fbgemmhiptorch;destsuffix=third_party/fbgemm/third_party/hipify_torch
>> <http://github.com/ROCmSoftwarePlatform/hipify_torch.git;protocol=https;name=fbgemmhiptorch;destsuffix=third_party/fbgemm/third_party/hipify_torch>
>>
>>     \
>>     ...
>>     "
>>
>>     As you can see, there are recursively placed git submodules
>>     and do_fetch fails with an error message about SRCREV_FORMAT
>>     having to be set.
>>
>>     The above is a limited subset of the complete list and
>>     some of the repositories (like googletest, gloo, glog and pybind11)
>>     occurs multiple times as child submodules of upper ones,
>>     with different SRCREV values in different leaf submodules.
>>
>>     I have not found a conclusive example to use SRCREV_FORMAT
>>     with git:// + gitsm://
>>
>>     Can someone enlighten me how to use SRCREV_FORMAT properly
>>     for this case? Should I stick to the proven working nonstandard way
>>     and just run git submodule update --init --recursive?
>>
>>     As an easy way out, is there a flag to git:// to process its
>>     submodules automatically?
>>
>>     Thanks in advance,
>>     Zoltán Böszörményi
>>
>>
>>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#189390): https://lists.openembedded.org/g/openembedded-core/message/189390
> Mute This Topic: https://lists.openembedded.org/mt/102038382/3617728
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [zboszor@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>



      parent reply	other threads:[~2023-10-18 16:18 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-10-18 13:04 gitsm:// question Böszörményi Zoltán
2023-10-18 13:16 ` [OE-core] " Martin Jansa
2023-10-18 13:25   ` Böszörményi Zoltán
2023-10-18 13:28     ` Julien Stephan
2023-10-18 14:12       ` Böszörményi Zoltán
2023-10-18 14:24         ` Julien Stephan
2023-10-18 14:28           ` Martin Jansa
     [not found]   ` <178F367FEBFA030F.14388@lists.openembedded.org>
2023-10-18 16:18     ` Böszörményi Zoltán [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=b9d330f9-8789-4a8b-a078-00c7e062eb05@gmail.com \
    --to=zboszor@gmail.com \
    --cc=martin.jansa@gmail.com \
    --cc=openembedded-core@lists.openembedded.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox