Re: [OE-core] file checksums versus SRC_URI globs

["Rasmus Villemoes" <rasmus.villemoes@prevas.dk>]

On 09/07/2020 21.31, Rasmus Villemoes via lists.openembedded.org wrote:
> On 02/07/2020 08.42, Rasmus Villemoes via lists.openembedded.org wrote:
>> On 01/07/2020 16.03, Quentin Schulz wrote:
>>> Hi Rasmus,
>>>
>>> On Wed, Jul 01, 2020 at 03:51:19PM +0200, Rasmus Villemoes wrote:
>>>> Hi,
>>>>
>>>> We have a recipe that uses
>>>>
>>>>   SRC_URI += "file://somedir/*"
>>>>
>>>
>>> Glob aren't supported. Use "file://somedir/" instead.
>>
>> Thanks, that actually works for one of the cases we have (there are
>> others that use globs which cannot be solved quite that simply, but for
>> now I'm just listing files explicitly instead).
>>
>> However, I'm not sure that "globs aren't supported". The commit I
>> referenced clearly tried to make that work (better), it also "works" in
>> the sense of unpacking the expected things when building from scratch -
>> there's even
>>
>>     def test_local_wildcard(self):
>>         tree = self.fetchUnpack(['file://a', 'file://dir/*'])
>>         self.assertEqual(tree, ['a',  'dir/c', 'dir/d', 'dir/subdir/e'])
>>
>> in bitbake/lib/bb/tests/fetch.py. And the two upstream recipes
>> connman-gnome_0.7.bb and matchbox-desktop_2.2.bb both use that exact
>> pattern.
>>
>> So either
>>
>> - this is a plain bug in the signature computation,
> 
> I'm guessing the culprit is
> 
> commit 6c0706a28d72c591f1b75b6e3f3b645859387c7e
> Author: Richard Purdie <richard.purdie@linuxfoundation.org>
> Date:   Mon Dec 8 21:25:23 2014 +0000
> 
>     cache/fetch2/siggen: Ensure we track include history for file checksums
> 
> which was the one that introduced the ":True: or ":False" suffixing via
> 
> +                filelist.append(f + ":" + str(os.path.exists(f)))
> 
> and then also did
> 
> @@ -981,6 +980,10 @@ def get_file_checksums(filelist, pn):
> 
>      checksums = []
>      for pth in filelist.split():
> +        exist = pth.split(":")[1]
> +        if exist == "False":
> +            continue
> +        pth = pth.split(":")[0]
>          if '*' in pth:
>              # Handle globs
>              for f in glob.glob(pth):
> 
> which practically guaranteed that the "if '*' in pth" would be dead code.
> 
> Richard, do you agree that this is a bug in the signature computation?

Richard, ping?

Rasmus