Re: [OE-core] [PATCH 1/4] scripts/install-buildtools: improvements

Tim Orling
2020-04-03T23:22:23Z

[-- Attachment #1: Type: text/plain, Size: 3130 bytes --]

> On Apr 1, 2020, at 3:16 PM, Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
>
> On Wed, 2020-04-01 at 15:09 -0700, Andre McCurdy wrote:
>> On Wed, Apr 1, 2020 at 3:02 PM Tim Orling <ticotimo@gmail.com> wrote:
>>> On Wed, Apr 1, 2020 at 2:32 PM Tim Orling via
>>> lists.openembedded.org <ticotimo=gmail.com@lists.openembedded.org>
>>> wrote:
>>>> On Wed, Apr 1, 2020 at 1:09 PM Andre McCurdy <armccurdy@gmail.com
>>>>> wrote:
>>>>> It looks like there are some assumptions about the minimum
>>>>> version of
>>>>> python3 on the host in order to even be able to run
>>>>> install-buildtools. Should that be checked explicitly?
>>>>
>>>> To be honest, everything was written on CentOS-7, where ‘python3’
>>>> is now python36.
>>>>
>>>> The remaining supported holdout for 3.4 is Debian-8 (Jessie).
>>>> Which also happens to have tar 1.27. At least it has git 2.1.4,
>>>> so it’s not a complete mess.
>>>>
>>>> Python 3.4 is already EOL. We should not support it for the
>>>> complete build system, but I can see that the install-buildtools
>>>> script should ideally still run on python34. The whole point is
>>>> to make it easier to install the pre-built buildtools.
>>>
>>> Python 3.5 is now required in bitbake/lib/bb/__init__.py
>>> Which is needed for bb.utils.md5sum_file and sha256_file to check
>>> the checksum of the download. I’d rather not have to rewrite those
>>> functions.
>>
>> Looks like copy and pasting ~30 lines? Maybe not even much of a net
>> increase in code size if you can remove the existing code to check
>> that bitbake libs are available...
>>
>> def _hasher(method, filename):
>> import mmap
>>
>> with open(filename, "rb") as f:
>> try:
>> with mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) as
>> mm:
>> for chunk in iter(lambda: mm.read(8192), b''):
>> method.update(chunk)
>> except ValueError:
>> # You can't mmap() an empty file so silence this
>> exception
>> pass
>> return method.hexdigest()
>>
>> def md5_file(filename):
>> """
>> Return the hex string representation of the MD5 checksum of
>> filename.
>> """
>> import hashlib
>> return _hasher(hashlib.md5(), filename)
>>
>> def sha256_file(filename):
>> """
>> Return the hex string representation of the 256-bit SHA checksum
>> of
>> filename.
>> """
>> import hashlib
>> return _hasher(hashlib.sha256(), filename)
>
> Can you send a patch against master-next please? The buildtools
> installer is one place we probably do want to support older pythons and
> it shouldn't be too onnerous.
>

Submitted. The check for whether the install worked or not was the hardest part.
After many iterations, I just could not get subprocess.Popen to work as expected on python 3.4.
https://patchwork.openembedded.org/patch/171593/ <https://patchwork.openembedded.org/patch/171593/>

Thank you Andre for the testing and input. It takes a village to make a good release.

> Cheers,
>
> Richard
>
>

[-- Attachment #2: Type: text/html, Size: 5397 bytes --]