Re: [Buildroot] [PATCH v3 2/2] package/pkg-python: clean conflicting pep517 packages before install

["Yann E. MORIN" <yann.morin.1998@free.fr>]

James, All,

On 2022-08-22 01:54 -0600, James Hilliard spake thusly:
> The python installer package isn't able to overwrite files of packges
> that already exist, this causes problems when doing a rebuild or
> update without a full clean.
> 
> To fix this we can use functionality from importlib to identify and
> remove any conflicting python package files before installation.
> 
> Fixes:
> Traceback (most recent call last):
>   File "/home/buildroot/buildroot/support/scripts/pyinstaller.py", line 69, in <module>
>     main()
>   File "/home/buildroot/buildroot/support/scripts/pyinstaller.py", line 61, in main
>     install(
>   File "/home/buildroot/buildroot/output/host/lib/python3.10/site-packages/installer/_core.py", line 109, in install
>     record = destination.write_file(
>   File "/home/buildroot/buildroot/output/host/lib/python3.10/site-packages/installer/destinations.py", line 207, in write_file
>     return self.write_to_fs(scheme, path_, stream, is_executable)
>   File "/home/buildroot/buildroot/output/host/lib/python3.10/site-packages/installer/destinations.py", line 167, in write_to_fs
>     raise FileExistsError(message)
> FileExistsError: File already exists: /home/buildroot/buildroot/output/target/usr/lib/python3.10/site-packages/tinycss2/__init__.py
> 
> Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
> Tested-by: Marcus Hoffmann <marcus.hoffmann@othermo.de>
> ---
> Changes v2 -> v3:
>   - split out host pyinstaller.py change
> Changes v1 -> v2:
>   - remove unused os import
> ---
>  support/scripts/pyinstaller.py | 22 ++++++++++++++++++++++
>  1 file changed, 22 insertions(+)
> 
> diff --git a/support/scripts/pyinstaller.py b/support/scripts/pyinstaller.py
> index 6dd9242327..d9b77cac19 100644
> --- a/support/scripts/pyinstaller.py
> +++ b/support/scripts/pyinstaller.py
> @@ -2,12 +2,33 @@
>  
>  import argparse
>  import glob
> +import pathlib
> +
> +from importlib.machinery import PathFinder
> +from importlib.metadata import DistributionFinder
>  
>  from installer import install
> +from installer._core import _process_WHEEL_file

I've extended the commit log to explain why we need to use private
symbols.

>  from installer.destinations import SchemeDictionaryDestination
>  from installer.sources import WheelFile
>  
>  
> +def clean(source, destination):
> +    scheme = _process_WHEEL_file(source)
> +    scheme_path = destination.scheme_dict[scheme]
> +    context = DistributionFinder.Context(
> +        name=source.distribution,
> +        path=[scheme_path],
> +    )
> +    for path in PathFinder.find_distributions(context=context):
> +        if not path.files:

I've changed this for an explicit test against None.

Applied to master, thanks.

Regards,
Yann E. MORIN.

> +            continue
> +        for file in path.files:
> +            file_path = pathlib.Path(file.locate())
> +            if file_path.exists():
> +                file_path.unlink()
> +
> +
>  def main():
>      """Entry point for CLI."""
>      ap = argparse.ArgumentParser("python pyinstaller.py")
> @@ -58,6 +79,7 @@ def main():
>      )
>  
>      with WheelFile.open(glob.glob(args.wheel_file)[0]) as source:
> +        clean(source, destination)
>          install(
>              source=source,
>              destination=destination,
> -- 
> 2.34.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot