Re: [OE-core] [PATCH] base.bbclass: make do_fetch depends on PREMIRRORS MIRRORS

[Changqing Li <changqing.li@windriver.com>]

[-- Attachment #1: Type: text/plain, Size: 4835 bytes --]


On 4/16/24 15:30, Alexander Kanavin wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
> I'm not sure why files in DL_DIR would symlink to the local
> (pre)mirrors instead of making an independent copy? Mirrors can indeed
> move or disappear altogether.
>
> Can you look into where the symlinking happens, and whether that can
> be modified or configured differently (maybe also the git history for
> that code)?

Current code is here: 
https://git.openembedded.org/bitbake/tree/lib/bb/fetch2/__init__.py#n1096

It is created during handle the (pre)mirrors in do_fetch.  And the 
history long time ago:

https://git.openembedded.org/bitbake/commit/?id=83f9445081595630488a5d3e170d282a13836c98

About if we can change to independent copy, I need to do more research, 
welcome to provide me some

background and notes.

Regards

Changqing

>
> Alex
>
> On Tue, 16 Apr 2024 at 03:48, Changqing Li via lists.openembedded.org
> <changqing.li=eng.windriver.com@lists.openembedded.org>  wrote:
>>
>> On 4/15/24 18:05, Richard Purdie wrote:
>>> CAUTION: This email comes from a non Wind River email account!
>>> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>> On Mon, 2024-04-15 at 16:30 +0800, Changqing Li via lists.openembedded.org wrote:
>>>> From: Changqing Li<changqing.li@windriver.com>
>>>>
>>>> When there are tarballs in local mirrorfile://, do_fetch will
>>>> create symlink under DL_DIR, eg:
>>>> pseudo-prebuilt-2.33.tar.xz -> /project/path/to/localmirrorA/pseudo-prebuilt-2.33.tar.xz
>>>> after user update path from localmirrorA to localmirrorB, because
>>>> do_fetch don't depends on PREMIRRORS/MIRRORS, do_fetch will not
>>>> be rerun, so the symlink will not update to:
>>>> pseudo-prebuilt-2.33.tar.xz -> /project/path/to/localmirrorB/pseudo-prebuilt-2.33.tar.xz
>>>> then, do_upack will failed since the tarball cannot be found.
>>>>
>>>> Signed-off-by: Changqing Li<changqing.li@windriver.com>
>>>> ---
>>>>    meta/classes-global/base.bbclass | 1 +
>>>>    1 file changed, 1 insertion(+)
>>>>
>>>> diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
>>>> index 0999b42daa..5094a1540e 100644
>>>> --- a/meta/classes-global/base.bbclass
>>>> +++ b/meta/classes-global/base.bbclass
>>>> @@ -139,6 +139,7 @@ do_fetch[file-checksums] ="${@bb.fetch.get_checksum_file_list(d)}"
>>>>    do_fetch[file-checksums] += " ${@get_lic_checksum_file_list(d)}"
>>>>    do_fetch[prefuncs] += "fetcher_hashes_dummyfunc"
>>>>    do_fetch[network] = "1"
>>>> +do_fetch[vardeps] += "PREMIRRORS MIRRORS"
>>>>    python base_do_fetch() {
>>>>
>>>>        src_uri = (d.getVar('SRC_URI') or "").split()
>>>>
>>> Changing PREMIRRORS or MIRRORS should not change the output of the
>>> task, ever, so do_fetch shouldn't depend upon them.
>> The failure case I met is like this:
>>
>> I have a project with PREMIRRORS setting, and the tarballs are under it.
>>
>> PREMIRRORS:append ="file:///project/path/to/localmirrorA"
>>
>> after first bitbake pseudo-native, there will a symlink under DL_DIR:
>>
>> pseudo-prebuilt-2.33.tar.xz ->
>> /project/path/to/localmirrorA/pseudo-prebuilt-2.33.tar.xz
>>
>> Then, Since all the tarballs are moved to /project/path/to/localmirrorB
>>
>> I changed PREMIRRORS:append ="file:///project/path/to/localmirrorB"
>>
>> Then bitbake pseudo-native again,   do_fetch  will not rerun since no
>> signature change.
>>
>> do_unpack rerun since one dependency  change, it will try to unpack
>> ${DL_DIR}/pseudo-prebuilt-2.33.tar.xz,
>>
>> but since do_fetch not rerun, ${DL_DIR}/pseudo-prebuilt-2.33.tar.xz
>> still point to localmirrorA, but localmirrorA not exist
>>
>> since localmirrorB replace it.  Then will cause do_unpck failure:
>>
>> NOTE: Unpacking /build/downloads/pseudo-prebuilt-2.33.tar.xz to
>> /build/tmp-glibc/work/x86_64-linux/pseudo-native/1.9.0+git xz:
>> /build/downloads/pseudo-prebuilt-2.33.tar.xz: No such file or directory
>>
>> So this is why I want to add this depends,  and want to make do_fetch
>> rerun and resymlink to /project/path/to/localmirrorB
>>
>> Regards
>>
>> Changqing
>>
>>> Cheers,
>>>
>>> Richard
>>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#198246):https://lists.openembedded.org/g/openembedded-core/message/198246
>> Mute This Topic:https://lists.openembedded.org/mt/105532159/1686489
>> Group Owner:openembedded-core+owner@lists.openembedded.org
>> Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub  [alex.kanavin@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>

[-- Attachment #2: Type: text/html, Size: 7314 bytes --]