Re: [bitbake-devel] [PATCH]] checksum: Support uri formatted file list

[Paulo Neves <ptsneves@gmail.com>]

I am not aware of it directly being done anywhere but files declared in 
the recipe are in URI form as far as i know. Where in the self test 
should i add such tests?

Paulo Neves

On 1/25/22 1:20 PM, Richard Purdie wrote:
> On Fri, 2022-01-21 at 16:39 +0100, Paulo Neves wrote:
>> Before this commit, if the file list for checksumming had
>> files names with spaces there would be a crash. This happened
>> due to filelist.split breaking on the file names instead of
>> on k:v boundaries. Now we validate this case and emit a fatal
>> error if such case is found. This needs to be fatal as the split
>> will generate broken k:v from then on.
>>
>> Instead of putting literal spaces in the file list the user
>> should urlencode the file names and if they contain coded spaces
>> they will be decoded. This is consistent with the current
>> practice where file names are urlencoded. A reproducer of the
>> issue this commit fixes, was to pass a do_compile[file-checksums]
>> list with files containing spaces in their names, urlencoded or
>> literal.
>>
>> Change-Id: I6ac4f1cffbb86e913883491d46e8cc69a028e992
>> Signed-off-by: Paulo Neves <ptsneves@gmail.com>
>> ---
>>   bitbake/lib/bb/checksum.py | 9 +++++++--
>>   1 file changed, 7 insertions(+), 2 deletions(-)
>>
>> diff --git a/bitbake/lib/bb/checksum.py b/bitbake/lib/bb/checksum.py
>> index fb8a77f6ab..97cf10825e 100644
>> --- a/bitbake/lib/bb/checksum.py
>> +++ b/bitbake/lib/bb/checksum.py
>> @@ -8,6 +8,7 @@
>>   import glob
>>   import operator
>>   import os
>> +import urllib.parse
>>   import stat
>>   import bb.utils
>>   import logging
>> @@ -110,10 +111,14 @@ class FileChecksumCache(MultiProcessCache):
>>   
>>           checksums = []
>>           for pth in filelist.split():
>> -            exist = pth.split(":")[1]
>> +            spl = pth.split(':')
>> +            if len(spl) != 2:
>> +                bb.fatal("found unformatted path in filelist " + pth)
>> +
>> +            exist = spl[1]
>>               if exist == "False":
>>                   continue
>> -            pth = pth.split(":")[0]
>> +            pth = urllib.parse.unquote(spl[0])
>>               if '*' in pth:
>>                   # Handle globs
>>                   for f in glob.glob(pth):
> Does any code currently urlencode the filepaths into this flag?
>
> I think we may also need to add some tests around this to bitbake-selftest...
>
> Cheers,
>
> Richard
>