[Buildroot] [PATCH 1/1] package/python3: fix config directory pattern

[Buildroot] [PATCH 1/1] package/python3: fix config directory pattern

[Vincent Fazio]

If a platform triplet is not detected during the configure stage, the
config directory (LIBPL) defaults to `config-$LDVERSION`.

In this scenario, the `PYTHON3_REMOVE_USELESS_FILES` hook would fail due
to `find` incorrectly expecting a second dash and then the triplet.

Now, we glob anything after the version which will match in both cases.

Fixes: 54d48c8cad ("package/python3: miscellaneous fixups")

Signed-off-by: Vincent Fazio <vfazio@gmail.com>
---
 package/python3/python3.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/package/python3/python3.mk b/package/python3/python3.mk
index a0b9ed4437..a82bc47a57 100644
--- a/package/python3/python3.mk
+++ b/package/python3/python3.mk
@@ -204,7 +204,7 @@ PYTHON3_CONF_OPTS += \
 define PYTHON3_REMOVE_USELESS_FILES
 	rm -f $(TARGET_DIR)/usr/bin/python$(PYTHON3_VERSION_MAJOR)-config
 	rm -f $(TARGET_DIR)/usr/bin/python3-config
-	find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config-$(PYTHON3_VERSION_MAJOR)-*/ \
+	find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config-$(PYTHON3_VERSION_MAJOR)*/ \
 		-type f -not -name Makefile -exec rm -rf {} \;
 	find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/ -type d \
 		-name __pycache__ -exec rm -rf {} \;
-- 
2.34.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] package/python3: fix config directory pattern

[Yann E. MORIN]

Vincent, All,

On 2024-06-08 22:25 -0500, Vincent Fazio spake thusly:
> If a platform triplet is not detected during the configure stage,

Could you share a defconfig that exhibits the issue, please?

Regards,
Yann E. MORIN.

> the
> config directory (LIBPL) defaults to `config-$LDVERSION`.
> 
> In this scenario, the `PYTHON3_REMOVE_USELESS_FILES` hook would fail due
> to `find` incorrectly expecting a second dash and then the triplet.
> 
> Now, we glob anything after the version which will match in both cases.
> 
> Fixes: 54d48c8cad ("package/python3: miscellaneous fixups")
> 
> Signed-off-by: Vincent Fazio <vfazio@gmail.com>
> ---
>  package/python3/python3.mk | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/package/python3/python3.mk b/package/python3/python3.mk
> index a0b9ed4437..a82bc47a57 100644
> --- a/package/python3/python3.mk
> +++ b/package/python3/python3.mk
> @@ -204,7 +204,7 @@ PYTHON3_CONF_OPTS += \
>  define PYTHON3_REMOVE_USELESS_FILES
>  	rm -f $(TARGET_DIR)/usr/bin/python$(PYTHON3_VERSION_MAJOR)-config
>  	rm -f $(TARGET_DIR)/usr/bin/python3-config
> -	find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config-$(PYTHON3_VERSION_MAJOR)-*/ \
> +	find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config-$(PYTHON3_VERSION_MAJOR)*/ \
>  		-type f -not -name Makefile -exec rm -rf {} \;
>  	find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/ -type d \
>  		-name __pycache__ -exec rm -rf {} \;
> -- 
> 2.34.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] package/python3: fix config directory pattern

[Vincent Fazio]

All,

On Sun, Jun 9, 2024 at 12:47 AM Yann E. MORIN <yann.morin.1998@free.fr> wrote:
>
> Vincent, All,
>
> On 2024-06-08 22:25 -0500, Vincent Fazio spake thusly:
> > If a platform triplet is not detected during the configure stage,
>
> Could you share a defconfig that exhibits the issue, please?
>


Here is one I ran last night when testing python 3.12.4 but also fails on 3.11.

```
BR2_xtensa=y
BR2_XTENSA_CUSTOM=y
BR2_XTENSA_OVERLAY_FILE="https://github.com/jcmvbkbc/xtensa-toolchain-build/raw/95291b7c39e6f790d0b2f062c945a630290f2c81/overlays/xtensa_dc233c.tar.gz"
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_XTENSA_LX60_UCLIBC_STABLE=y
BR2_TARGET_GENERIC_GETTY_PORT="ttyS0"
BR2_SYSTEM_DHCP="eth0"
BR2_ROOTFS_POST_IMAGE_SCRIPT="board/qemu/post-image.sh"
BR2_ROOTFS_POST_SCRIPT_ARGS="$(BR2_DEFCONFIG)"
BR2_LINUX_KERNEL=y
BR2_LINUX_KERNEL_CUSTOM_VERSION=y
BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="6.6.18"
BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/xtensa-lx60/linux.config"
BR2_LINUX_KERNEL_IMAGE_TARGET_CUSTOM=y
BR2_LINUX_KERNEL_IMAGE_TARGET_NAME="Image"
BR2_LINUX_KERNEL_IMAGE_NAME="Image.elf"
BR2_PACKAGE_PYTHON3=y
BR2_TARGET_ROOTFS_INITRAMFS=y
# BR2_TARGET_ROOTFS_TAR is not set
BR2_PACKAGE_HOST_QEMU=y
BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
BR2_PACKAGE_HOST_UBOOT_TOOLS=y
```

I think this is fixable in other ways, but would probably require extending
Python's platform detection, which got rewritten in 3.13 and could be a hassle
to maintain, especially since uClibc isn't technically supported upstream.

configure.ac code:

https://github.com/python/cpython/blob/35c799d79177b962ddace2fa068101465570a29a/configure.ac#L5755

PLATFORM_TRIPLET and MULTIARCH values drive this, however

https://github.com/python/cpython/blob/35c799d79177b962ddace2fa068101465570a29a/configure.ac#L1090

```
checking for the platform triplet based on compiler characteristics... none
checking for multiarch...

```


> Regards,
> Yann E. MORIN.
>
> > the
> > config directory (LIBPL) defaults to `config-$LDVERSION`.
> >
> > In this scenario, the `PYTHON3_REMOVE_USELESS_FILES` hook would fail due
> > to `find` incorrectly expecting a second dash and then the triplet.
> >
> > Now, we glob anything after the version which will match in both cases.
> >
> > Fixes: 54d48c8cad ("package/python3: miscellaneous fixups")
> >
> > Signed-off-by: Vincent Fazio <vfazio@gmail.com>
> > ---
> >  package/python3/python3.mk | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/package/python3/python3.mk b/package/python3/python3.mk
> > index a0b9ed4437..a82bc47a57 100644
> > --- a/package/python3/python3.mk
> > +++ b/package/python3/python3.mk
> > @@ -204,7 +204,7 @@ PYTHON3_CONF_OPTS += \
> >  define PYTHON3_REMOVE_USELESS_FILES
> >       rm -f $(TARGET_DIR)/usr/bin/python$(PYTHON3_VERSION_MAJOR)-config
> >       rm -f $(TARGET_DIR)/usr/bin/python3-config
> > -     find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config-$(PYTHON3_VERSION_MAJOR)-*/ \
> > +     find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config-$(PYTHON3_VERSION_MAJOR)*/ \
> >               -type f -not -name Makefile -exec rm -rf {} \;
> >       find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/ -type d \
> >               -name __pycache__ -exec rm -rf {} \;
> > --
> > 2.34.1
> >
> > _______________________________________________
> > buildroot mailing list
> > buildroot@buildroot.org
> > https://lists.buildroot.org/mailman/listinfo/buildroot
>
> --
> .-----------------.--------------------.------------------.--------------------.
> |  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
> | +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
> | +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
> '------------------------------^-------^------------------^--------------------'


-Vincent
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] package/python3: fix config directory pattern

[Yann E. MORIN]

Vincent, All,

On 2024-06-08 22:25 -0500, Vincent Fazio spake thusly:
> If a platform triplet is not detected during the configure stage, the
> config directory (LIBPL) defaults to `config-$LDVERSION`.

I found the $LDVERSION to be a bit strange, even though that is actually
what the code uses, but here, in the contect of a Buildrot commit log,
it does not look right, as it rather hints at the version of ld, which
does not make sense...

So I just dropped the LD to keep $VERSION, which makes mor esense I
believe.

> In this scenario, the `PYTHON3_REMOVE_USELESS_FILES` hook would fail due
> to `find` incorrectly expecting a second dash and then the triplet.
> 
> Now, we glob anything after the version which will match in both cases.

Woot, 11MiB dropped here! 👍

> Fixes: 54d48c8cad ("package/python3: miscellaneous fixups")

In fact, it was already broken before that: master, which does not have
54d48c8cad, alrady suffered from that issue, except the error is ignore,
as te find is in a sub-shell evaluated as the list for a for-loop.

Would you care to send the same fix for master, please?

> Signed-off-by: Vincent Fazio <vfazio@gmail.com>

Applied to next, thanks.

Regards,
Yann E. MORIN.

> ---
>  package/python3/python3.mk | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/package/python3/python3.mk b/package/python3/python3.mk
> index a0b9ed4437..a82bc47a57 100644
> --- a/package/python3/python3.mk
> +++ b/package/python3/python3.mk
> @@ -204,7 +204,7 @@ PYTHON3_CONF_OPTS += \
>  define PYTHON3_REMOVE_USELESS_FILES
>  	rm -f $(TARGET_DIR)/usr/bin/python$(PYTHON3_VERSION_MAJOR)-config
>  	rm -f $(TARGET_DIR)/usr/bin/python3-config
> -	find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config-$(PYTHON3_VERSION_MAJOR)-*/ \
> +	find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config-$(PYTHON3_VERSION_MAJOR)*/ \
>  		-type f -not -name Makefile -exec rm -rf {} \;
>  	find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/ -type d \
>  		-name __pycache__ -exec rm -rf {} \;
> -- 
> 2.34.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 1/1] package/python3: fix config directory pattern

[Vincent Fazio]

Previously, when running `PYTHON3_REMOVE_USELESS_FILES`, the hook to
clean up files from the python config directory assumed a pattern of
"config-$(VERSION)m-$(PLATFORM_TRIPLET)".

However, the "m" ABI suffix was dropped in python 3.8, so the hook would
never actually find files to delete. No error was raised due to the use
of a subshell to invoke find.

Also, if a platform triplet is not detected during the configure stage,
the config directory (LIBPL) defaults to `config-$VERSION`, and has no
trailing `-$PLATFORM_TRIPLET`.

Now, we glob anything after the version to ensure files get deleted.

Signed-off-by: Vincent Fazio <vfazio@gmail.com>
---
 package/python3/python3.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/package/python3/python3.mk b/package/python3/python3.mk
index 5d9d77af50..950006698b 100644
--- a/package/python3/python3.mk
+++ b/package/python3/python3.mk
@@ -204,7 +204,7 @@ define PYTHON3_REMOVE_USELESS_FILES
 	rm -f $(TARGET_DIR)/usr/bin/python3-config
 	rm -f $(TARGET_DIR)/usr/bin/smtpd.py.3
 	rm -f $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/distutils/command/wininst*.exe
-	for i in `find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config-$(PYTHON3_VERSION_MAJOR)m-*/ \
+	for i in `find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config-$(PYTHON3_VERSION_MAJOR)*/ \
 		-type f -not -name Makefile` ; do \
 		rm -f $$i ; \
 	done
-- 
2.34.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] package/python3: fix config directory pattern

[Vincent Fazio]

Yann, all

On Sun, Jun 9, 2024 at 10:51 AM Yann E. MORIN <yann.morin.1998@free.fr> wrote:
>
> Vincent, All,
>
> On 2024-06-08 22:25 -0500, Vincent Fazio spake thusly:
> > If a platform triplet is not detected during the configure stage, the
> > config directory (LIBPL) defaults to `config-$LDVERSION`.
>
> I found the $LDVERSION to be a bit strange, even though that is actually
> what the code uses, but here, in the contect of a Buildrot commit log,
> it does not look right, as it rather hints at the version of ld, which
> does not make sense...
>
> So I just dropped the LD to keep $VERSION, which makes mor esense I
> believe.
>
> > In this scenario, the `PYTHON3_REMOVE_USELESS_FILES` hook would fail due
> > to `find` incorrectly expecting a second dash and then the triplet.
> >
> > Now, we glob anything after the version which will match in both cases.
>
> Woot, 11MiB dropped here! 👍
>
> > Fixes: 54d48c8cad ("package/python3: miscellaneous fixups")
>
> In fact, it was already broken before that: master, which does not have
> 54d48c8cad, alrady suffered from that issue, except the error is ignore,
> as te find is in a sub-shell evaluated as the list for a for-loop.
>
> Would you care to send the same fix for master, please?
>

Sure thing: https://patchwork.ozlabs.org/project/buildroot/patch/20240609174048.31062-1-vfazio@gmail.com/


> > Signed-off-by: Vincent Fazio <vfazio@gmail.com>
>
> Applied to next, thanks.
>
> Regards,
> Yann E. MORIN.
>
> > ---
> >  package/python3/python3.mk | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/package/python3/python3.mk b/package/python3/python3.mk
> > index a0b9ed4437..a82bc47a57 100644
> > --- a/package/python3/python3.mk
> > +++ b/package/python3/python3.mk
> > @@ -204,7 +204,7 @@ PYTHON3_CONF_OPTS += \
> >  define PYTHON3_REMOVE_USELESS_FILES
> >       rm -f $(TARGET_DIR)/usr/bin/python$(PYTHON3_VERSION_MAJOR)-config
> >       rm -f $(TARGET_DIR)/usr/bin/python3-config
> > -     find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config-$(PYTHON3_VERSION_MAJOR)-*/ \
> > +     find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config-$(PYTHON3_VERSION_MAJOR)*/ \
> >               -type f -not -name Makefile -exec rm -rf {} \;
> >       find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/ -type d \
> >               -name __pycache__ -exec rm -rf {} \;
> > --
> > 2.34.1
> >
> > _______________________________________________
> > buildroot mailing list
> > buildroot@buildroot.org
> > https://lists.buildroot.org/mailman/listinfo/buildroot
>
> --
> .-----------------.--------------------.------------------.--------------------.
> |  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
> | +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
> | +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
> '------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] package/python3: fix config directory pattern

[Yann E. MORIN]

Vincent, All,

On 2024-06-09 12:40 -0500, Vincent Fazio spake thusly:
> Previously, when running `PYTHON3_REMOVE_USELESS_FILES`, the hook to
> clean up files from the python config directory assumed a pattern of
> "config-$(VERSION)m-$(PLATFORM_TRIPLET)".
> 
> However, the "m" ABI suffix was dropped in python 3.8, so the hook would
> never actually find files to delete. No error was raised due to the use
> of a subshell to invoke find.
> 
> Also, if a platform triplet is not detected during the configure stage,
> the config directory (LIBPL) defaults to `config-$VERSION`, and has no
> trailing `-$PLATFORM_TRIPLET`.
> 
> Now, we glob anything after the version to ensure files get deleted.
> 
> Signed-off-by: Vincent Fazio <vfazio@gmail.com>

Applied to master, thank you!

Regards,
Yann E. MORIN.

> ---
>  package/python3/python3.mk | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/package/python3/python3.mk b/package/python3/python3.mk
> index 5d9d77af50..950006698b 100644
> --- a/package/python3/python3.mk
> +++ b/package/python3/python3.mk
> @@ -204,7 +204,7 @@ define PYTHON3_REMOVE_USELESS_FILES
>  	rm -f $(TARGET_DIR)/usr/bin/python3-config
>  	rm -f $(TARGET_DIR)/usr/bin/smtpd.py.3
>  	rm -f $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/distutils/command/wininst*.exe
> -	for i in `find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config-$(PYTHON3_VERSION_MAJOR)m-*/ \
> +	for i in `find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config-$(PYTHON3_VERSION_MAJOR)*/ \
>  		-type f -not -name Makefile` ; do \
>  		rm -f $$i ; \
>  	done
> -- 
> 2.34.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] package/python3: fix config directory pattern

[Peter Korsgaard]

>>>>> "Vincent" == Vincent Fazio <vfazio@gmail.com> writes:

 > Previously, when running `PYTHON3_REMOVE_USELESS_FILES`, the hook to
 > clean up files from the python config directory assumed a pattern of
 > "config-$(VERSION)m-$(PLATFORM_TRIPLET)".

 > However, the "m" ABI suffix was dropped in python 3.8, so the hook would
 > never actually find files to delete. No error was raised due to the use
 > of a subshell to invoke find.

 > Also, if a platform triplet is not detected during the configure stage,
 > the config directory (LIBPL) defaults to `config-$VERSION`, and has no
 > trailing `-$PLATFORM_TRIPLET`.

 > Now, we glob anything after the version to ensure files get deleted.

 > Signed-off-by: Vincent Fazio <vfazio@gmail.com>

Committed to 2024.02.x, thanks.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot