[Buildroot] [PATCH 1/2] package/python-pyasyncore: enable host package

[Buildroot] [PATCH 1/2] package/python-pyasyncore: enable host package

[Bernd Kuhls]

Needed for python-emailproxy.

Signed-off-by: Bernd Kuhls <bernd@kuhls.net>
---
 package/python-pyasyncore/python-pyasyncore.mk | 1 +
 1 file changed, 1 insertion(+)

diff --git a/package/python-pyasyncore/python-pyasyncore.mk b/package/python-pyasyncore/python-pyasyncore.mk
index 4b1e3c98b6..5a178c37b8 100644
--- a/package/python-pyasyncore/python-pyasyncore.mk
+++ b/package/python-pyasyncore/python-pyasyncore.mk
@@ -12,3 +12,4 @@ PYTHON_PYASYNCORE_LICENSE = PSF-2.0
 PYTHON_PYASYNCORE_LICENSE_FILES = LICENSE
 
 $(eval $(python-package))
+$(eval $(host-python-package))
-- 
2.39.5

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH v2 2/2] package/python-emailproxy: new package

[Bernd Kuhls]

Signed-off-by: Bernd Kuhls <bernd@kuhls.net>
---
v2: bump version to 2024.11.11

 DEVELOPERS                                       |  1 +
 package/Config.in                                |  1 +
 package/python-emailproxy/Config.in              | 14 ++++++++++++++
 package/python-emailproxy/python-emailproxy.hash |  5 +++++
 package/python-emailproxy/python-emailproxy.mk   | 15 +++++++++++++++
 5 files changed, 36 insertions(+)
 create mode 100644 package/python-emailproxy/Config.in
 create mode 100644 package/python-emailproxy/python-emailproxy.hash
 create mode 100644 package/python-emailproxy/python-emailproxy.mk

diff --git a/DEVELOPERS b/DEVELOPERS
index f0006cfa35..532e86acc8 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -482,6 +482,7 @@ F:	package/privoxy/
 F:	package/pure-ftpd/
 F:	package/python-couchdb/
 F:	package/python-cssutils/
+F:	package/python-emailproxy/
 F:	package/python-glslang/
 F:	package/python-mako/
 F:	package/python-mwclient/
diff --git a/package/Config.in b/package/Config.in
index dfae37e1bc..588f3da9c9 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1102,6 +1102,7 @@ menu "External python modules"
 	source "package/python-dpkt/Config.in"
 	source "package/python-dtschema/Config.in"
 	source "package/python-ecdsa/Config.in"
+	source "package/python-emailproxy/Config.in"
 	source "package/python-email-validator/Config.in"
 	source "package/python-engineio/Config.in"
 	source "package/python-entrypoints/Config.in"
diff --git a/package/python-emailproxy/Config.in b/package/python-emailproxy/Config.in
new file mode 100644
index 0000000000..f5c72d3a18
--- /dev/null
+++ b/package/python-emailproxy/Config.in
@@ -0,0 +1,14 @@
+config BR2_PACKAGE_PYTHON_EMAILPROXY
+	bool "python-emailproxy"
+	depends on BR2_PACKAGE_HOST_RUSTC_TARGET_ARCH_SUPPORTS # python-cryptography
+	select BR2_PACKAGE_HOST_PYTHON3
+	select BR2_PACKAGE_HOST_PYTHON3_SSL
+	select BR2_PACKAGE_PYTHON_CRYPTOGRAPHY # runtime
+	select BR2_PACKAGE_PYTHON_PROMPT_TOOLKIT # runtime
+	select BR2_PACKAGE_PYTHON_PYASYNCORE # runtime
+	help
+	  An IMAP/POP/SMTP proxy that transparently adds OAuth 2.0
+	  authentication for email clients that don't support this
+	  method.
+
+	  https://github.com/simonrob/email-oauth2-proxy
diff --git a/package/python-emailproxy/python-emailproxy.hash b/package/python-emailproxy/python-emailproxy.hash
new file mode 100644
index 0000000000..da3f63639f
--- /dev/null
+++ b/package/python-emailproxy/python-emailproxy.hash
@@ -0,0 +1,5 @@
+# md5, sha256 from https://pypi.org/pypi/emailproxy/json
+md5  fa1c82907718c7c643c541606aec947d  emailproxy-2024.11.11.tar.gz
+sha256  97b59c5f9cd5afb1d2146aff9c5259a3ab0d46b074cb50952b9fb4a4d8b4df35  emailproxy-2024.11.11.tar.gz
+# Locally computed sha256 checksums
+sha256  c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4  LICENSE
diff --git a/package/python-emailproxy/python-emailproxy.mk b/package/python-emailproxy/python-emailproxy.mk
new file mode 100644
index 0000000000..58bb1fb7ee
--- /dev/null
+++ b/package/python-emailproxy/python-emailproxy.mk
@@ -0,0 +1,15 @@
+################################################################################
+#
+# python-emailproxy
+#
+################################################################################
+
+PYTHON_EMAILPROXY_VERSION = 2024.11.11
+PYTHON_EMAILPROXY_SOURCE = emailproxy-$(PYTHON_EMAILPROXY_VERSION).tar.gz
+PYTHON_EMAILPROXY_SITE = https://files.pythonhosted.org/packages/2b/49/386e675f2b0706083fdf02ad3666b98d0875a2e1cdcabff8df9fe641d2e3
+PYTHON_EMAILPROXY_DEPENDENCIES = host-python-cryptography host-python-pyasyncore
+PYTHON_EMAILPROXY_SETUP_TYPE = setuptools
+PYTHON_EMAILPROXY_LICENSE = Apache-2.0
+PYTHON_EMAILPROXY_LICENSE_FILES = LICENSE
+
+$(eval $(python-package))
-- 
2.39.5

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v2 2/2] package/python-emailproxy: new package

[Thomas Petazzoni via buildroot]

Hello Bernd,

On Wed, 11 Dec 2024 20:16:30 +0100
Bernd Kuhls <bernd@kuhls.net> wrote:

> +PYTHON_EMAILPROXY_VERSION = 2024.11.11
> +PYTHON_EMAILPROXY_SOURCE = emailproxy-$(PYTHON_EMAILPROXY_VERSION).tar.gz
> +PYTHON_EMAILPROXY_SITE = https://files.pythonhosted.org/packages/2b/49/386e675f2b0706083fdf02ad3666b98d0875a2e1cdcabff8df9fe641d2e3
> +PYTHON_EMAILPROXY_DEPENDENCIES = host-python-cryptography host-python-pyasyncore

I am still not happy with host-python-cryptography
host-python-pyasyncore being dependencies, while they definitely
shouldn't be needed. I know James asked you to use
--skip-dependency-check, and it apparently wasn't working with:

  File "/home/bernd/buildroot/output/build/python-emailproxy-2024.9.12/
emailproxy.py", line 52, in <module>
    import asyncore
ModuleNotFoundError: No module named 'asyncore'

Do we have an explanation why?

Adding those dependencies really feels like a hack.

Best regards,

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v2 2/2] package/python-emailproxy: new package

[Fiona Klute via buildroot]

Am 29.12.24 um 22:12 schrieb Thomas Petazzoni via buildroot:
> Hello Bernd,
>
> On Wed, 11 Dec 2024 20:16:30 +0100
> Bernd Kuhls <bernd@kuhls.net> wrote:
>
>> +PYTHON_EMAILPROXY_VERSION = 2024.11.11
>> +PYTHON_EMAILPROXY_SOURCE = emailproxy-$(PYTHON_EMAILPROXY_VERSION).tar.gz
>> +PYTHON_EMAILPROXY_SITE = https://files.pythonhosted.org/packages/2b/49/386e675f2b0706083fdf02ad3666b98d0875a2e1cdcabff8df9fe641d2e3
>> +PYTHON_EMAILPROXY_DEPENDENCIES = host-python-cryptography host-python-pyasyncore
>
> I am still not happy with host-python-cryptography
> host-python-pyasyncore being dependencies, while they definitely
> shouldn't be needed. I know James asked you to use
> --skip-dependency-check, and it apparently wasn't working with:
>
>    File "/home/bernd/buildroot/output/build/python-emailproxy-2024.9.12/
> emailproxy.py", line 52, in <module>
>      import asyncore
> ModuleNotFoundError: No module named 'asyncore'
>
> Do we have an explanation why?
>
> Adding those dependencies really feels like a hack.
Looks like the upstream pyproject.toml requires importing the module to
read the version:
https://github.com/simonrob/email-oauth2-proxy/blob/3ed7dfdcd1d7a9dc2a392aecaae7bb5d913acec0/pyproject.toml#L45

My guess is that cross-build wasn't a consideration there. Might be
worth trying to convince upstream to change that, otherwise the question
is whether it's preferable for Buildroot to have hackish host
dependencies, or a patch to make the version static.

Best regards,
Fiona

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [External] - Re: [PATCH v2 2/2] package/python-emailproxy: new package

[Bernd Kuhls]

Hi Vincent,

Am 13.01.25 um 20:38 schrieb Vincent Fazio:
>>>> Am 29.12.24 um 22:12 schrieb Thomas Petazzoni via buildroot:
>>>>> I am still not happy with host-python-cryptography host-
>>>>> python-pyasyncore being dependencies, while they definitely 
>>>>> shouldn't be needed. I know James asked you to use --skip-
>>>>> dependency-check, and it apparently wasn't working with:
> 
> If we wanted to rely on setuptools' dependency resolution, we need
> to remove the `--no-isolation` flag in the build, otherwise it
> assumes the dependencies are available within host-python (which is
> why the host packages are necessary). I tested dropping `-n` from
> the build command and it worked OK. I'm guessing we do this so that
> BR is an island and self-contained.

thanks for the hint. Yes, removing "-n" from PKG_PYTHON_PEP517_BUILD_CMD 
fixes the build of python-emailproxy but breaks dozens of other python 
packages. Adding "-n" to _BUILD_OPTS of these packages fixes them but it 
will require lots of patches.

> If we wanted to avoid the host dependencies, we could just patch the
> pyproject to hardcode the version and not rely on setuptools'
> dynamic attribute resolution, though it might be best to do this on
> the latest tag and not 2024.9.12?
Did not test it yet but would such a patch be accepted by buildroot 
without sending it upstream?

Regards, Bernd

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [External] - Re: [PATCH v2 2/2] package/python-emailproxy: new package

[James Hilliard]

On Mon, Jan 27, 2025 at 2:57 AM Bernd Kuhls <bernd@kuhls.net> wrote:
>
> Hi Vincent,
>
> Am 13.01.25 um 20:38 schrieb Vincent Fazio:
> >>>> Am 29.12.24 um 22:12 schrieb Thomas Petazzoni via buildroot:
> >>>>> I am still not happy with host-python-cryptography host-
> >>>>> python-pyasyncore being dependencies, while they definitely
> >>>>> shouldn't be needed. I know James asked you to use --skip-
> >>>>> dependency-check, and it apparently wasn't working with:
> >
> > If we wanted to rely on setuptools' dependency resolution, we need
> > to remove the `--no-isolation` flag in the build, otherwise it
> > assumes the dependencies are available within host-python (which is
> > why the host packages are necessary). I tested dropping `-n` from
> > the build command and it worked OK. I'm guessing we do this so that
> > BR is an island and self-contained.
>
> thanks for the hint. Yes, removing "-n" from PKG_PYTHON_PEP517_BUILD_CMD
> fixes the build of python-emailproxy but breaks dozens of other python
> packages. Adding "-n" to _BUILD_OPTS of these packages fixes them but it
> will require lots of patches.

Yeah, so I don't think removing -n is the correct approach here since we
want to manage all the build dependencies as part of buildroot.

>
> > If we wanted to avoid the host dependencies, we could just patch the
> > pyproject to hardcode the version and not rely on setuptools'
> > dynamic attribute resolution, though it might be best to do this on
> > the latest tag and not 2024.9.12?
> Did not test it yet but would such a patch be accepted by buildroot
> without sending it upstream?

Was this a bug in the setuptools ast parser maybe? These dynamic
attribute resolvers usually try ast parsing for versions and fallback to
traditional imports only if the ast parsing fails.

>
> Regards, Bernd
>
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [External] - Re: [PATCH v2 2/2] package/python-emailproxy: new package

[Vincent Fazio]

> -----Original Message-----
> From: James Hilliard <james.hilliard1@gmail.com>
> Sent: Sunday, January 26, 2025 7:49 PM
> To: Bernd Kuhls <bernd@kuhls.net>
> Cc: Vincent Fazio <vfazio@xes-inc.com>; Thomas Petazzoni
> <thomas.petazzoni@bootlin.com>; buildroot@buildroot.org; Fiona Klute
> <fiona.klute@gmx.de>
> Subject: Re: [Buildroot] [External] - Re: [PATCH v2 2/2] package/python-
> emailproxy: new package
> 
> On Mon, Jan 27, 2025 at 2:57 AM Bernd Kuhls <bernd@kuhls.net> wrote:
> >
> > Hi Vincent,
> >
> > Am 13.01.25 um 20:38 schrieb Vincent Fazio:
> > >
> > > If we wanted to rely on setuptools' dependency resolution, we need
> > > to remove the `--no-isolation` flag in the build, otherwise it
> > > assumes the dependencies are available within host-python (which is
> > > why the host packages are necessary). I tested dropping `-n` from
> > > the build command and it worked OK. I'm guessing we do this so that
> > > BR is an island and self-contained.
> >
> > thanks for the hint. Yes, removing "-n" from
> > PKG_PYTHON_PEP517_BUILD_CMD fixes the build of python-emailproxy
> but
> > breaks dozens of other python packages. Adding "-n" to _BUILD_OPTS of
> > these packages fixes them but it will require lots of patches.
> 
> Yeah, so I don't think removing -n is the correct approach here since we want
> to manage all the build dependencies as part of buildroot.
> 

Sorry I wasn't clear here, I was not advocating that we remove this, I only did this as a test to prove out
that was what was requiring the host dependencies

> >
> > > If we wanted to avoid the host dependencies, we could just patch the
> > > pyproject to hardcode the version and not rely on setuptools'
> > > dynamic attribute resolution, though it might be best to do this on
> > > the latest tag and not 2024.9.12?
> > Did not test it yet but would such a patch be accepted by buildroot
> > without sending it upstream?

IMO, since this is a requirement of the Buildroot build system, it's fine to add a patch for this with the trailer Upstream: N/A

We just have to deal with repatching with every release (hence my suggestion to start with the latest tag)

> 
> Was this a bug in the setuptools ast parser maybe? These dynamic attribute
> resolvers usually try ast parsing for versions and fallback to traditional imports
> only if the ast parsing fails.

I was going to suggest talking to the setuptools group. I haven't looked too closely into this.

It would obviously be ideal if it just looked at the AST and didn't try to load everything from the module.



> 
> >
> > Regards, Bernd
> >
> > _______________________________________________
> > buildroot mailing list
> > buildroot@buildroot.org
> > https://lists.buildroot.org/mailman/listinfo/buildroot
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [External] - Re: [PATCH v2 2/2] package/python-emailproxy: new package

[Vincent Fazio]

> -----Original Message-----
> From: buildroot <buildroot-bounces@buildroot.org> On Behalf Of Vincent
> Fazio
> Sent: Monday, January 27, 2025 7:10 AM
> To: James Hilliard <james.hilliard1@gmail.com>; Bernd Kuhls
> <bernd@kuhls.net>
> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>;
> buildroot@buildroot.org; Fiona Klute <fiona.klute@gmx.de>
> Subject: Re: [Buildroot] [External] - Re: [PATCH v2 2/2] package/python-
> emailproxy: new package
> 
> 
> 
> > -----Original Message-----
> > From: James Hilliard <james.hilliard1@gmail.com>
> > Sent: Sunday, January 26, 2025 7:49 PM
> > To: Bernd Kuhls <bernd@kuhls.net>
> > Cc: Vincent Fazio <vfazio@xes-inc.com>; Thomas Petazzoni
> > <thomas.petazzoni@bootlin.com>; buildroot@buildroot.org; Fiona Klute
> > <fiona.klute@gmx.de>
> > Subject: Re: [Buildroot] [External] - Re: [PATCH v2 2/2]
> > package/python-
> > emailproxy: new package
> >
> > On Mon, Jan 27, 2025 at 2:57 AM Bernd Kuhls <bernd@kuhls.net> wrote:
> > >
> > > Hi Vincent,
> > >
> > > Am 13.01.25 um 20:38 schrieb Vincent Fazio:
> > > >
> > > > If we wanted to rely on setuptools' dependency resolution, we need
> > > > to remove the `--no-isolation` flag in the build, otherwise it
> > > > assumes the dependencies are available within host-python (which
> > > > is why the host packages are necessary). I tested dropping `-n`
> > > > from the build command and it worked OK. I'm guessing we do this
> > > > so that BR is an island and self-contained.
> > >
> > > thanks for the hint. Yes, removing "-n" from
> > > PKG_PYTHON_PEP517_BUILD_CMD fixes the build of python-emailproxy
> > but
> > > breaks dozens of other python packages. Adding "-n" to _BUILD_OPTS
> > > of these packages fixes them but it will require lots of patches.
> >
> > Yeah, so I don't think removing -n is the correct approach here since
> > we want to manage all the build dependencies as part of buildroot.
> >
> 
> Sorry I wasn't clear here, I was not advocating that we remove this, I only did
> this as a test to prove out that was what was requiring the host dependencies
> 
> > >
> > > > If we wanted to avoid the host dependencies, we could just patch
> > > > the pyproject to hardcode the version and not rely on setuptools'
> > > > dynamic attribute resolution, though it might be best to do this
> > > > on the latest tag and not 2024.9.12?
> > > Did not test it yet but would such a patch be accepted by buildroot
> > > without sending it upstream?
> 
> IMO, since this is a requirement of the Buildroot build system, it's fine to add a
> patch for this with the trailer Upstream: N/A
> 
> We just have to deal with repatching with every release (hence my suggestion
> to start with the latest tag)
> 
> >
> > Was this a bug in the setuptools ast parser maybe? These dynamic
> > attribute resolvers usually try ast parsing for versions and fallback
> > to traditional imports only if the ast parsing fails.

Apparently, part of the problem is that AST parsing expects the attribute to be a fixed string and not "calculated"

Traceback (most recent call last):
  File "/mnt/development/email-oauth2-proxy/venv/lib/python3.10/site-packages/setuptools/config/expand.py", line 191, in read_attr
    return getattr(StaticModule(module_name, spec), attr_name)
  File "/mnt/development/email-oauth2-proxy/venv/lib/python3.10/site-packages/setuptools/config/expand.py", line 87, in __getattr__
    raise AttributeError(f"{self.name} has no attribute {attr}") from e
AttributeError: emailproxy has no attribute __package_version__


Also, the setuptools version requirement is incorrect, the pyproject.toml is defined with features introduced in 62.6.0:
https://setuptools.pypa.io/en/stable/history.html#v62-6-0

It could be argued that the upstream package could be "fixed" to not require crypto or asynccore/pyasynccore as a build dep if they just hardcoded the attribute instead of interpreting it from `__version__` so the dynamic attribute could leverage the AST instead of trying to import the module.

Their pyproject.toml does complain that it's necessary and implies they do not like this current behavior so may be an easy sell.

> 
> I was going to suggest talking to the setuptools group. I haven't looked too
> closely into this.
> 
> It would obviously be ideal if it just looked at the AST and didn't try to load
> everything from the module.
> 
> 
> 
> >
> > >
> > > Regards, Bernd
> > >
> > > _______________________________________________
> > > buildroot mailing list
> > > buildroot@buildroot.org
> > > https://lists.buildroot.org/mailman/listinfo/buildroot
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v2 2/2] package/python-emailproxy: new package

[James Hilliard]

On Mon, Jan 13, 2025 at 8:17 PM Fiona Klute <fiona.klute@gmx.de> wrote:
>
> Am 29.12.24 um 22:12 schrieb Thomas Petazzoni via buildroot:
> > Hello Bernd,
> >
> > On Wed, 11 Dec 2024 20:16:30 +0100
> > Bernd Kuhls <bernd@kuhls.net> wrote:
> >
> >> +PYTHON_EMAILPROXY_VERSION = 2024.11.11
> >> +PYTHON_EMAILPROXY_SOURCE = emailproxy-$(PYTHON_EMAILPROXY_VERSION).tar.gz
> >> +PYTHON_EMAILPROXY_SITE = https://files.pythonhosted.org/packages/2b/49/386e675f2b0706083fdf02ad3666b98d0875a2e1cdcabff8df9fe641d2e3
> >> +PYTHON_EMAILPROXY_DEPENDENCIES = host-python-cryptography host-python-pyasyncore
> >
> > I am still not happy with host-python-cryptography
> > host-python-pyasyncore being dependencies, while they definitely
> > shouldn't be needed. I know James asked you to use
> > --skip-dependency-check, and it apparently wasn't working with:
> >
> >    File "/home/bernd/buildroot/output/build/python-emailproxy-2024.9.12/
> > emailproxy.py", line 52, in <module>
> >      import asyncore
> > ModuleNotFoundError: No module named 'asyncore'
> >
> > Do we have an explanation why?
> >
> > Adding those dependencies really feels like a hack.
> Looks like the upstream pyproject.toml requires importing the module to
> read the version:
> https://github.com/simonrob/email-oauth2-proxy/blob/3ed7dfdcd1d7a9dc2a392aecaae7bb5d913acec0/pyproject.toml#L45
>
> My guess is that cross-build wasn't a consideration there. Might be
> worth trying to convince upstream to change that, otherwise the question
> is whether it's preferable for Buildroot to have hackish host
> dependencies, or a patch to make the version static.

I hacked together a quick PR that should fix this:
https://github.com/simonrob/email-oauth2-proxy/pull/322

>
> Best regards,
> Fiona
>
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot