Re: [OE-core] [PATCH 1/4] scripts/install-buildtools: improvements

["Richard Purdie" <richard.purdie@linuxfoundation.org>]

On Wed, 2020-04-01 at 15:09 -0700, Andre McCurdy wrote:
> On Wed, Apr 1, 2020 at 3:02 PM Tim Orling <ticotimo@gmail.com> wrote:
> > On Wed, Apr 1, 2020 at 2:32 PM Tim Orling via
> > lists.openembedded.org <ticotimo=gmail.com@lists.openembedded.org>
> > wrote:
> > > On Wed, Apr 1, 2020 at 1:09 PM Andre McCurdy <armccurdy@gmail.com
> > > > wrote:
> > > > It looks like there are some assumptions about the minimum
> > > > version of
> > > > python3 on the host in order to even be able to run
> > > > install-buildtools. Should that be checked explicitly?
> > > 
> > > To be honest, everything was written on CentOS-7, where ‘python3’
> > > is now python36.
> > > 
> > > The remaining supported holdout for 3.4 is Debian-8 (Jessie).
> > > Which also happens to have tar 1.27. At least it has git 2.1.4,
> > > so it’s not a complete mess.
> > > 
> > > Python 3.4 is already EOL. We should not support it for the
> > > complete build system, but I can see that the install-buildtools
> > > script should ideally still run on python34. The whole point is
> > > to make it easier to install the pre-built buildtools.
> > 
> > Python 3.5 is now required in bitbake/lib/bb/__init__.py
> > Which is needed for bb.utils.md5sum_file and sha256_file to check
> > the checksum of the download. I’d rather not have to rewrite those
> > functions.
> 
> Looks like copy and pasting ~30 lines? Maybe not even much of a net
> increase in code size if you can remove the existing code to check
> that bitbake libs are available...
> 
> def _hasher(method, filename):
>     import mmap
> 
>     with open(filename, "rb") as f:
>         try:
>             with mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) as
> mm:
>                 for chunk in iter(lambda: mm.read(8192), b''):
>                     method.update(chunk)
>         except ValueError:
>             # You can't mmap() an empty file so silence this
> exception
>             pass
>     return method.hexdigest()
> 
> def md5_file(filename):
>     """
>     Return the hex string representation of the MD5 checksum of
> filename.
>     """
>     import hashlib
>     return _hasher(hashlib.md5(), filename)
> 
> def sha256_file(filename):
>     """
>     Return the hex string representation of the 256-bit SHA checksum
> of
>     filename.
>     """
>     import hashlib
>     return _hasher(hashlib.sha256(), filename)

Can you send a patch against master-next please? The buildtools
installer is one place we probably do want to support older pythons and
it shouldn't be too onnerous.

Cheers,

Richard