[Buildroot] [PATCH 0/2] Fix fail2ban

[Buildroot] [PATCH 0/2] Fix fail2ban

[Angelo Compagnucci]

This series fixes fail2ban not installing configuration files in /etc anymore.
While fixing the problem, I discovered a bug in the way we invoke the
pyinstaller.py in pep517 compatibility.
Basically, pyinstaller.py will install data_files defined in setup.py
and tarballed in the wheel inside the directory pointed by the --data parameter,
which is nowadays pointing to /usr of TARGET/STAGING directory.

I consider this wrong, because the data folder shall point to the
TARGET_DIR/STAGING_DIR to be correct and we shall trust the destination path
expressed in the setup.py.

In the case of fail2ban, configuration files end up being installed in
/usr/etc/fail2ban instead of /etc/fail2ban.

v1 -> v2:
Fix signed off with correct email.
v2 -> v3:
Fix commit author too


Angelo Compagnucci (2):
  package/fail2ban: fix data_files installation path
  package/pkg-python: fix pep517 data files installation

 ...001-setup.py-fix-data-relative-paths.patch | 73 +++++++++++++++++++
 package/pkg-python.mk                         |  4 +-
 2 files changed, 75 insertions(+), 2 deletions(-)
 create mode 100644 package/fail2ban/0001-setup.py-fix-data-relative-paths.patch

-- 
2.34.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 1/2] package/fail2ban: fix data_files installation path

[Angelo Compagnucci]

When packaging data_files in a wheel, those files shall have the destination
path expressed as relative to the wheel installation.
The reason for that is the wheel installation shall be contained inside a
specific folder and/or in site-packages.
When building the wheel file by the bdist_wheel command, python will package
files with absolute pats as python files ending up installing them in
site-packages.

Fixes:
http://autobuild.buildroot.net/results/36ac5278d19195a21c3d02d087965e08f49228ef

Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
---
 ...001-setup.py-fix-data-relative-paths.patch | 73 +++++++++++++++++++
 1 file changed, 73 insertions(+)
 create mode 100644 package/fail2ban/0001-setup.py-fix-data-relative-paths.patch

diff --git a/package/fail2ban/0001-setup.py-fix-data-relative-paths.patch b/package/fail2ban/0001-setup.py-fix-data-relative-paths.patch
new file mode 100644
index 0000000000..e1b5bc490b
--- /dev/null
+++ b/package/fail2ban/0001-setup.py-fix-data-relative-paths.patch
@@ -0,0 +1,73 @@
+From 7b7fa3c02d4575c09362336bc310835c37859f1a Mon Sep 17 00:00:00 2001
+From: Angelo Compagnucci <angelo.compagnucci@gmail.com>
+Date: Fri, 19 Apr 2024 22:47:20 +0200
+Subject: [PATCH] setup.py: fix data relative paths
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+As per distutils documentation:
+
+The directory should be a relative path. It is interpreted relative to the
+installation prefix (Python’s sys.prefix for system installations;
+site.USER_BASE for user installations). Distutils allows directory to be an
+absolute installation path, but this is discouraged since it is incompatible
+with the wheel packaging format [1].
+
+When packaging to a bdist_wheel indeed, the files in the "data_files" dictionary
+are not picked at all as data files if they have an absolute path,
+but they are instead added to the wheel as python files.
+This has the effect of installing them as part of python files in the
+site-packages folder when the wheel is installed.
+
+This patch cannot be sent upstream because it breaks the legacy way of
+installing the package with *python setup.py install* and it is needed only to
+comply with the pep517 buildroot compatibility layer.
+
+[1] https://docs.python.org/3.10/distutils/setupscript.html#installing-additional-files
+Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
+---
+ setup.py | 14 +++++++-------
+ 1 file changed, 7 insertions(+), 7 deletions(-)
+
+diff --git a/setup.py b/setup.py
+index 91f71cf2..99ba2e32 100755
+--- a/setup.py
++++ b/setup.py
+@@ -238,26 +238,26 @@ setup(
+ 				for f in w[2]]
+ 	} if with_tests else {},
+ 	data_files = [
+-		('/etc/fail2ban',
++		('etc/fail2ban',
+ 			glob("config/*.conf")
+ 		),
+-		('/etc/fail2ban/filter.d',
++		('etc/fail2ban/filter.d',
+ 			glob("config/filter.d/*.conf")
+ 		),
+-		('/etc/fail2ban/filter.d/ignorecommands',
++		('etc/fail2ban/filter.d/ignorecommands',
+ 			[p for p in glob("config/filter.d/ignorecommands/*") if isfile(p)]
+ 		),
+-		('/etc/fail2ban/action.d',
++		('etc/fail2ban/action.d',
+ 			glob("config/action.d/*.conf") +
+ 			glob("config/action.d/*.py")
+ 		),
+-		('/etc/fail2ban/fail2ban.d',
++		('etc/fail2ban/fail2ban.d',
+ 			''
+ 		),
+-		('/etc/fail2ban/jail.d',
++		('etc/fail2ban/jail.d',
+ 			''
+ 		),
+-		('/var/lib/fail2ban',
++		('var/lib/fail2ban',
+ 			''
+ 		),
+ 	] + data_files_extra,
+-- 
+2.34.1
+
-- 
2.34.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/2] package/fail2ban: fix data_files installation path

[Fiona Klute via buildroot]

Am 19.04.24 um 23:32 schrieb Angelo Compagnucci:
> When packaging data_files in a wheel, those files shall have the destination
> path expressed as relative to the wheel installation.
> The reason for that is the wheel installation shall be contained inside a
> specific folder and/or in site-packages.
> When building the wheel file by the bdist_wheel command, python will package
> files with absolute pats as python files ending up installing them in
> site-packages.
>
> Fixes:
> http://autobuild.buildroot.net/results/36ac5278d19195a21c3d02d087965e08f49228ef
>
> Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>

Tested-by: Fiona Klute <fiona.klute+wiwa@gmx.de>

Testing included: Build for aarch64, configuring fail2ban for sshd,
testing that a host indeed gets banned after repeated failed login
attempts (password auth), with syslog-ng (the default fail2ban patterns
don't work with Busybox syslogd) and nftables.

Thanks for fixing the build, Angelo!

Best regards,
Fiona

> ---
>   ...001-setup.py-fix-data-relative-paths.patch | 73 +++++++++++++++++++
>   1 file changed, 73 insertions(+)
>   create mode 100644 package/fail2ban/0001-setup.py-fix-data-relative-paths.patch
>
> diff --git a/package/fail2ban/0001-setup.py-fix-data-relative-paths.patch b/package/fail2ban/0001-setup.py-fix-data-relative-paths.patch
> new file mode 100644
> index 0000000000..e1b5bc490b
> --- /dev/null
> +++ b/package/fail2ban/0001-setup.py-fix-data-relative-paths.patch
> @@ -0,0 +1,73 @@
> +From 7b7fa3c02d4575c09362336bc310835c37859f1a Mon Sep 17 00:00:00 2001
> +From: Angelo Compagnucci <angelo.compagnucci@gmail.com>
> +Date: Fri, 19 Apr 2024 22:47:20 +0200
> +Subject: [PATCH] setup.py: fix data relative paths
> +MIME-Version: 1.0
> +Content-Type: text/plain; charset=UTF-8
> +Content-Transfer-Encoding: 8bit
> +
> +As per distutils documentation:
> +
> +The directory should be a relative path. It is interpreted relative to the
> +installation prefix (Python’s sys.prefix for system installations;
> +site.USER_BASE for user installations). Distutils allows directory to be an
> +absolute installation path, but this is discouraged since it is incompatible
> +with the wheel packaging format [1].
> +
> +When packaging to a bdist_wheel indeed, the files in the "data_files" dictionary
> +are not picked at all as data files if they have an absolute path,
> +but they are instead added to the wheel as python files.
> +This has the effect of installing them as part of python files in the
> +site-packages folder when the wheel is installed.
> +
> +This patch cannot be sent upstream because it breaks the legacy way of
> +installing the package with *python setup.py install* and it is needed only to
> +comply with the pep517 buildroot compatibility layer.
> +
> +[1] https://docs.python.org/3.10/distutils/setupscript.html#installing-additional-files
> +Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
> +---
> + setup.py | 14 +++++++-------
> + 1 file changed, 7 insertions(+), 7 deletions(-)
> +
> +diff --git a/setup.py b/setup.py
> +index 91f71cf2..99ba2e32 100755
> +--- a/setup.py
> ++++ b/setup.py
> +@@ -238,26 +238,26 @@ setup(
> + 				for f in w[2]]
> + 	} if with_tests else {},
> + 	data_files = [
> +-		('/etc/fail2ban',
> ++		('etc/fail2ban',
> + 			glob("config/*.conf")
> + 		),
> +-		('/etc/fail2ban/filter.d',
> ++		('etc/fail2ban/filter.d',
> + 			glob("config/filter.d/*.conf")
> + 		),
> +-		('/etc/fail2ban/filter.d/ignorecommands',
> ++		('etc/fail2ban/filter.d/ignorecommands',
> + 			[p for p in glob("config/filter.d/ignorecommands/*") if isfile(p)]
> + 		),
> +-		('/etc/fail2ban/action.d',
> ++		('etc/fail2ban/action.d',
> + 			glob("config/action.d/*.conf") +
> + 			glob("config/action.d/*.py")
> + 		),
> +-		('/etc/fail2ban/fail2ban.d',
> ++		('etc/fail2ban/fail2ban.d',
> + 			''
> + 		),
> +-		('/etc/fail2ban/jail.d',
> ++		('etc/fail2ban/jail.d',
> + 			''
> + 		),
> +-		('/var/lib/fail2ban',
> ++		('var/lib/fail2ban',
> + 			''
> + 		),
> + 	] + data_files_extra,
> +--
> +2.34.1
> +
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/2] package/fail2ban: fix data_files installation path

[Thomas Petazzoni via buildroot]

Hello Angelo,

On Fri, 19 Apr 2024 23:32:15 +0200
Angelo Compagnucci <angelo.compagnucci@gmail.com> wrote:


> diff --git a/package/fail2ban/0001-setup.py-fix-data-relative-paths.patch b/package/fail2ban/0001-setup.py-fix-data-relative-paths.patch
> new file mode 100644
> index 0000000000..e1b5bc490b
> --- /dev/null
> +++ b/package/fail2ban/0001-setup.py-fix-data-relative-paths.patch
> @@ -0,0 +1,73 @@
> +From 7b7fa3c02d4575c09362336bc310835c37859f1a Mon Sep 17 00:00:00 2001
> +From: Angelo Compagnucci <angelo.compagnucci@gmail.com>
> +Date: Fri, 19 Apr 2024 22:47:20 +0200
> +Subject: [PATCH] setup.py: fix data relative paths
> +MIME-Version: 1.0
> +Content-Type: text/plain; charset=UTF-8
> +Content-Transfer-Encoding: 8bit
> +
> +As per distutils documentation:
> +
> +The directory should be a relative path. It is interpreted relative to the
> +installation prefix (Python’s sys.prefix for system installations;
> +site.USER_BASE for user installations). Distutils allows directory to be an
> +absolute installation path, but this is discouraged since it is incompatible
> +with the wheel packaging format [1].
> +
> +When packaging to a bdist_wheel indeed, the files in the "data_files" dictionary
> +are not picked at all as data files if they have an absolute path,
> +but they are instead added to the wheel as python files.
> +This has the effect of installing them as part of python files in the
> +site-packages folder when the wheel is installed.
> +
> +This patch cannot be sent upstream because it breaks the legacy way of
> +installing the package with *python setup.py install* and it is needed only to
> +comply with the pep517 buildroot compatibility layer.
> +
> +[1] https://docs.python.org/3.10/distutils/setupscript.html#installing-additional-files
> +Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>

Thanks for the patch!

However, this patch has no Upstream: tag, causing a check-package
warning when being applied:

WARNING: package/fail2ban/0001-setup.py-fix-data-relative-paths.patch:0: missing Upstream in the header (https://nightly.buildroot.org/#_additional_patch_documentation)

What is the upstream status of this patch?

Thanks!

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/2] package/fail2ban: fix data_files installation path

[Fiona Klute via buildroot]

Am 09.07.24 um 22:28 schrieb Thomas Petazzoni via buildroot:
> Hello Angelo,
>
> On Fri, 19 Apr 2024 23:32:15 +0200
> Angelo Compagnucci <angelo.compagnucci@gmail.com> wrote:
>
>
>> diff --git a/package/fail2ban/0001-setup.py-fix-data-relative-paths.patch b/package/fail2ban/0001-setup.py-fix-data-relative-paths.patch
>> new file mode 100644
>> index 0000000000..e1b5bc490b
>> --- /dev/null
>> +++ b/package/fail2ban/0001-setup.py-fix-data-relative-paths.patch
>> @@ -0,0 +1,73 @@
>> +From 7b7fa3c02d4575c09362336bc310835c37859f1a Mon Sep 17 00:00:00 2001
>> +From: Angelo Compagnucci <angelo.compagnucci@gmail.com>
>> +Date: Fri, 19 Apr 2024 22:47:20 +0200
>> +Subject: [PATCH] setup.py: fix data relative paths
>> +MIME-Version: 1.0
>> +Content-Type: text/plain; charset=UTF-8
>> +Content-Transfer-Encoding: 8bit
>> +
>> +As per distutils documentation:
>> +
>> +The directory should be a relative path. It is interpreted relative to the
>> +installation prefix (Python’s sys.prefix for system installations;
>> +site.USER_BASE for user installations). Distutils allows directory to be an
>> +absolute installation path, but this is discouraged since it is incompatible
>> +with the wheel packaging format [1].
>> +
>> +When packaging to a bdist_wheel indeed, the files in the "data_files" dictionary
>> +are not picked at all as data files if they have an absolute path,
>> +but they are instead added to the wheel as python files.
>> +This has the effect of installing them as part of python files in the
>> +site-packages folder when the wheel is installed.
>> +
>> +This patch cannot be sent upstream because it breaks the legacy way of
>> +installing the package with *python setup.py install* and it is needed only to
>> +comply with the pep517 buildroot compatibility layer.
>> +
>> +[1] https://docs.python.org/3.10/distutils/setupscript.html#installing-additional-files
>> +Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
>
> Thanks for the patch!
>
> However, this patch has no Upstream: tag, causing a check-package
> warning when being applied:
>
> WARNING: package/fail2ban/0001-setup.py-fix-data-relative-paths.patch:0: missing Upstream in the header (https://nightly.buildroot.org/#_additional_patch_documentation)
>
> What is the upstream status of this patch?

I believe the last paragraph of the patch description answers the
question, maybe you could add a shortened version as the Upstream tag? E.g.:
Upstream: N/A (breaks upstream install, only for Buildroot compatibility)

Best regards,
Fiona

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 2/2] package/pkg-python: fix pep517 data files installation

[Angelo Compagnucci]

When installing a wheel in pep517 compatibility mode, the pyinstaller.py is
invoked with the --data parameter which point to the directory in which data
files should be installed.
Actually the --data parameter points to the /usr subdirectory which is indeed
wrong cause it shall point to the root directory where the wheel will be
installed.
This fixes the problem of having configuration files installed in /usr/etc
instead of /etc.

Fixes:
http://autobuild.buildroot.net/results/36ac5278d19195a21c3d02d087965e08f49228ef

Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
---
 package/pkg-python.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/package/pkg-python.mk b/package/pkg-python.mk
index 839f728e2f..4f9bee4b90 100644
--- a/package/pkg-python.mk
+++ b/package/pkg-python.mk
@@ -70,7 +70,7 @@ PKG_PYTHON_PEP517_INSTALL_TARGET_CMD = \
 	--purelib=$(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/site-packages \
 	--headers=$(TARGET_DIR)/usr/include/python$(PYTHON3_VERSION_MAJOR) \
 	--scripts=$(TARGET_DIR)/usr/bin \
-	--data=$(TARGET_DIR)/usr
+	--data=$(TARGET_DIR)
 
 PKG_PYTHON_PEP517_INSTALL_STAGING_CMD = \
 	$(TOPDIR)/support/scripts/pyinstaller.py \
@@ -79,7 +79,7 @@ PKG_PYTHON_PEP517_INSTALL_STAGING_CMD = \
 	--purelib=$(STAGING_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/site-packages \
 	--headers=$(STAGING_DIR)/usr/include/python$(PYTHON3_VERSION_MAJOR) \
 	--scripts=$(STAGING_DIR)/usr/bin \
-	--data=$(STAGING_DIR)/usr
+	--data=$(STAGING_DIR)
 
 PKG_PYTHON_PEP517_DEPENDENCIES = \
 	host-python-pypa-build \
-- 
2.34.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 2/2] package/pkg-python: fix pep517 data files installation

[Fiona Klute via buildroot]

Am 19.04.24 um 23:32 schrieb Angelo Compagnucci:
> When installing a wheel in pep517 compatibility mode, the pyinstaller.py is
> invoked with the --data parameter which point to the directory in which data
> files should be installed.
> Actually the --data parameter points to the /usr subdirectory which is indeed
> wrong cause it shall point to the root directory where the wheel will be
> installed.
> This fixes the problem of having configuration files installed in /usr/etc
> instead of /etc.
>
> Fixes:
> http://autobuild.buildroot.net/results/36ac5278d19195a21c3d02d087965e08f49228ef
>
> Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>

Tested-by: Fiona Klute <fiona.klute+wiwa@gmx.de>

Testing included: Build for aarch64, configuring fail2ban for sshd,
testing that a host indeed gets banned after repeated failed login
attempts (password auth), with syslog-ng (the default fail2ban patterns
don't work with Busybox syslogd) and nftables.

> ---
>   package/pkg-python.mk | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/package/pkg-python.mk b/package/pkg-python.mk
> index 839f728e2f..4f9bee4b90 100644
> --- a/package/pkg-python.mk
> +++ b/package/pkg-python.mk
> @@ -70,7 +70,7 @@ PKG_PYTHON_PEP517_INSTALL_TARGET_CMD = \
>   	--purelib=$(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/site-packages \
>   	--headers=$(TARGET_DIR)/usr/include/python$(PYTHON3_VERSION_MAJOR) \
>   	--scripts=$(TARGET_DIR)/usr/bin \
> -	--data=$(TARGET_DIR)/usr
> +	--data=$(TARGET_DIR)
>
>   PKG_PYTHON_PEP517_INSTALL_STAGING_CMD = \
>   	$(TOPDIR)/support/scripts/pyinstaller.py \
> @@ -79,7 +79,7 @@ PKG_PYTHON_PEP517_INSTALL_STAGING_CMD = \
>   	--purelib=$(STAGING_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/site-packages \
>   	--headers=$(STAGING_DIR)/usr/include/python$(PYTHON3_VERSION_MAJOR) \
>   	--scripts=$(STAGING_DIR)/usr/bin \
> -	--data=$(STAGING_DIR)/usr
> +	--data=$(STAGING_DIR)
>
>   PKG_PYTHON_PEP517_DEPENDENCIES = \
>   	host-python-pypa-build \

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 0/2] Fix fail2ban

[Arnout Vandecappelle via buildroot]

On 19/04/2024 23:32, Angelo Compagnucci wrote:
> This series fixes fail2ban not installing configuration files in /etc anymore.
> While fixing the problem, I discovered a bug in the way we invoke the
> pyinstaller.py in pep517 compatibility.
> Basically, pyinstaller.py will install data_files defined in setup.py
> and tarballed in the wheel inside the directory pointed by the --data parameter,
> which is nowadays pointing to /usr of TARGET/STAGING directory.
> 
> I consider this wrong, because the data folder shall point to the
> TARGET_DIR/STAGING_DIR to be correct and we shall trust the destination path
> expressed in the setup.py.
> 
> In the case of fail2ban, configuration files end up being installed in
> /usr/etc/fail2ban instead of /etc/fail2ban.
> 
> v1 -> v2:
> Fix signed off with correct email.
> v2 -> v3:
> Fix commit author too
> 
> 
> Angelo Compagnucci (2):
>    package/fail2ban: fix data_files installation path
>    package/pkg-python: fix pep517 data files installation

  Applied both to master, thanks.

  I added the Upstream: tag as suggested by Fiona. Still, I think it would be 
good to report the issue upstream, so they can take it into account when they 
eventually get around to migrating to a pep517 build system.

  I also changed the order of the two commits, and removed the Fixes entry from 
the one in pkg-python.

  Do you know if this issue also occurs on 2024.02.x and 2024.05.x?

  Regards,
  Arnout

> 
>   ...001-setup.py-fix-data-relative-paths.patch | 73 +++++++++++++++++++
>   package/pkg-python.mk                         |  4 +-
>   2 files changed, 75 insertions(+), 2 deletions(-)
>   create mode 100644 package/fail2ban/0001-setup.py-fix-data-relative-paths.patch
> 
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 0/2] Fix fail2ban

[Angelo Compagnucci]

[-- Attachment #1.1: Type: text/plain, Size: 2135 bytes --]

Il giorno ven 12 lug 2024 alle ore 15:42 Arnout Vandecappelle <
arnout@mind.be> ha scritto:

>
>
> On 19/04/2024 23:32, Angelo Compagnucci wrote:
> > This series fixes fail2ban not installing configuration files in /etc
> anymore.
> > While fixing the problem, I discovered a bug in the way we invoke the
> > pyinstaller.py in pep517 compatibility.
> > Basically, pyinstaller.py will install data_files defined in setup.py
> > and tarballed in the wheel inside the directory pointed by the --data
> parameter,
> > which is nowadays pointing to /usr of TARGET/STAGING directory.
> >
> > I consider this wrong, because the data folder shall point to the
> > TARGET_DIR/STAGING_DIR to be correct and we shall trust the destination
> path
> > expressed in the setup.py.
> >
> > In the case of fail2ban, configuration files end up being installed in
> > /usr/etc/fail2ban instead of /etc/fail2ban.
> >
> > v1 -> v2:
> > Fix signed off with correct email.
> > v2 -> v3:
> > Fix commit author too
> >
> >
> > Angelo Compagnucci (2):
> >    package/fail2ban: fix data_files installation path
> >    package/pkg-python: fix pep517 data files installation
>
>   Applied both to master, thanks.
>
>   I added the Upstream: tag as suggested by Fiona. Still, I think it would
> be
> good to report the issue upstream, so they can take it into account when
> they
> eventually get around to migrating to a pep517 build system.
>
>   I also changed the order of the two commits, and removed the Fixes entry
> from
> the one in pkg-python.
>
>   Do you know if this issue also occurs on 2024.02.x and 2024.05.x?
>

I sent the patch on April 19 to silence the numerous package failings I was
already receiving. Not sure for the 2024.05, but surely the 2024.05 is
affected.

>
>   Regards,
>   Arnout
>
> >
> >   ...001-setup.py-fix-data-relative-paths.patch | 73 +++++++++++++++++++
> >   package/pkg-python.mk                         |  4 +-
> >   2 files changed, 75 insertions(+), 2 deletions(-)
> >   create mode 100644
> package/fail2ban/0001-setup.py-fix-data-relative-paths.patch
> >
>


-- 
Profile: http://it.linkedin.com/in/compagnucciangelo

[-- Attachment #1.2: Type: text/html, Size: 3161 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 0/2] Fix fail2ban

[Peter Korsgaard]

>>>>> "Angelo" == Angelo Compagnucci <angelo.compagnucci@gmail.com> writes:

Hi,

>> Do you know if this issue also occurs on 2024.02.x and 2024.05.x?
 >> 

 > I sent the patch on April 19 to silence the numerous package failings I was
 > already receiving. Not sure for the 2024.05, but surely the 2024.05 is
 > affected.

I was unable to reproduce it on 2024.02.x, but it indeed also fails on
2024.05.x. I believe it was introduced by commit 8937db8dd595e
(package/pkg-python.mk: migrate setuptools to pep517).

So committed both to 2024.05.x, thanks.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 0/2] Fix fail2ban

[Angelo Compagnucci]

From: Angelo Compagnucci <angelo@amarulasolutions.com>

This series fixes fail2ban not installing configuration files in /etc anymore.
While fixing the problem, I discovered a bug in the way we invoke the
pyinstaller.py in pep517 compatibility.
Basically, pyinstaller.py will install data_files defined in setup.py
and tarballed in the wheel inside the directory pointed by the --data parameter,
which is nowadays pointing to /usr of TARGET/STAGING directory.

I consider this wrong, because the data folder shall point to the
TARGET_DIR/STAGING_DIR to be correct and we shall trust the destination path
expressed in the setup.py.

In the case of fail2ban, configuration files end up being installed in
/usr/etc/fail2ban instead of /etc/fail2ban.

v1 -> v2:
Fix signed off with correct email.

Angelo Compagnucci (2):
  package/fail2ban: fix data_files installation path
  package/pkg-python: fix pep517 data files installation

 ...001-setup.py-fix-data-relative-paths.patch | 73 +++++++++++++++++++
 package/pkg-python.mk                         |  4 +-
 2 files changed, 75 insertions(+), 2 deletions(-)
 create mode 100644 package/fail2ban/0001-setup.py-fix-data-relative-paths.patch

-- 
2.34.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 1/2] package/fail2ban: fix data_files installation path

[Angelo Compagnucci]

From: Angelo Compagnucci <angelo@amarulasolutions.com>

When packaging data_files in a wheel, those files shall have the destination
path expressed as relative to the wheel installation.
The reason for that is the wheel installation shall be contained inside a
specific folder and/or in site-packages.
When building the wheel file by the bdist_wheel command, python will package
files with absolute pats as python files ending up installing them in
site-packages.

Fixes:
http://autobuild.buildroot.net/results/36ac5278d19195a21c3d02d087965e08f49228ef

Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
---
 ...001-setup.py-fix-data-relative-paths.patch | 73 +++++++++++++++++++
 1 file changed, 73 insertions(+)
 create mode 100644 package/fail2ban/0001-setup.py-fix-data-relative-paths.patch

diff --git a/package/fail2ban/0001-setup.py-fix-data-relative-paths.patch b/package/fail2ban/0001-setup.py-fix-data-relative-paths.patch
new file mode 100644
index 0000000000..e1b5bc490b
--- /dev/null
+++ b/package/fail2ban/0001-setup.py-fix-data-relative-paths.patch
@@ -0,0 +1,73 @@
+From 7b7fa3c02d4575c09362336bc310835c37859f1a Mon Sep 17 00:00:00 2001
+From: Angelo Compagnucci <angelo.compagnucci@gmail.com>
+Date: Fri, 19 Apr 2024 22:47:20 +0200
+Subject: [PATCH] setup.py: fix data relative paths
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+As per distutils documentation:
+
+The directory should be a relative path. It is interpreted relative to the
+installation prefix (Python’s sys.prefix for system installations;
+site.USER_BASE for user installations). Distutils allows directory to be an
+absolute installation path, but this is discouraged since it is incompatible
+with the wheel packaging format [1].
+
+When packaging to a bdist_wheel indeed, the files in the "data_files" dictionary
+are not picked at all as data files if they have an absolute path,
+but they are instead added to the wheel as python files.
+This has the effect of installing them as part of python files in the
+site-packages folder when the wheel is installed.
+
+This patch cannot be sent upstream because it breaks the legacy way of
+installing the package with *python setup.py install* and it is needed only to
+comply with the pep517 buildroot compatibility layer.
+
+[1] https://docs.python.org/3.10/distutils/setupscript.html#installing-additional-files
+Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
+---
+ setup.py | 14 +++++++-------
+ 1 file changed, 7 insertions(+), 7 deletions(-)
+
+diff --git a/setup.py b/setup.py
+index 91f71cf2..99ba2e32 100755
+--- a/setup.py
++++ b/setup.py
+@@ -238,26 +238,26 @@ setup(
+ 				for f in w[2]]
+ 	} if with_tests else {},
+ 	data_files = [
+-		('/etc/fail2ban',
++		('etc/fail2ban',
+ 			glob("config/*.conf")
+ 		),
+-		('/etc/fail2ban/filter.d',
++		('etc/fail2ban/filter.d',
+ 			glob("config/filter.d/*.conf")
+ 		),
+-		('/etc/fail2ban/filter.d/ignorecommands',
++		('etc/fail2ban/filter.d/ignorecommands',
+ 			[p for p in glob("config/filter.d/ignorecommands/*") if isfile(p)]
+ 		),
+-		('/etc/fail2ban/action.d',
++		('etc/fail2ban/action.d',
+ 			glob("config/action.d/*.conf") +
+ 			glob("config/action.d/*.py")
+ 		),
+-		('/etc/fail2ban/fail2ban.d',
++		('etc/fail2ban/fail2ban.d',
+ 			''
+ 		),
+-		('/etc/fail2ban/jail.d',
++		('etc/fail2ban/jail.d',
+ 			''
+ 		),
+-		('/var/lib/fail2ban',
++		('var/lib/fail2ban',
+ 			''
+ 		),
+ 	] + data_files_extra,
+-- 
+2.34.1
+
-- 
2.34.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 0/2] Fix fail2ban

[Angelo Compagnucci]

From: Angelo Compagnucci <angelo@amarulasolutions.com>

This series fixes fail2ban not installing configuration files in /etc anymore.
While fixing the problem, I discovered a bug in the way we invoke the
pyinstaller.py in pep517 compatibility.
Basically, pyinstaller.py will install data_files defined in setup.py
and tarballed in the wheel inside the directory pointed by the --data parameter,
which is nowadays pointing to /usr of TARGET/STAGING directory.

I consider this wrong, because the data folder shall point to the
TARGET_DIR/STAGING_DIR to be correct and we shall trust the destination path
expressed in the setup.py.

In the case of fail2ban, configuration files end up being installed in
/usr/etc/fail2ban instead of /etc/fail2ban.

Angelo Compagnucci (2):
  package/fail2ban: fix data_files installation path
  package/pkg-python: fix pep517 data files installation

 ...001-setup.py-fix-data-relative-paths.patch | 73 +++++++++++++++++++
 package/pkg-python.mk                         |  4 +-
 2 files changed, 75 insertions(+), 2 deletions(-)
 create mode 100644 package/fail2ban/0001-setup.py-fix-data-relative-paths.patch

-- 
2.34.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 1/2] package/fail2ban: fix data_files installation path

[Angelo Compagnucci]

From: Angelo Compagnucci <angelo@amarulasolutions.com>

When packaging data_files in a wheel, those files shall have the destination
path expressed as relative to the wheel installation.
The reason for that is the wheel installation shall be contained inside a
specific folder and/or in site-packages.
When building the wheel file by the bdist_wheel command, python will package
files with absolute pats as python files ending up installing them in
site-packages.

Fixes:
http://autobuild.buildroot.net/results/36ac5278d19195a21c3d02d087965e08f49228ef

Signed-off-by: Angelo Compagnucci <angelo@amarulasolutions.com>
---
 ...001-setup.py-fix-data-relative-paths.patch | 73 +++++++++++++++++++
 1 file changed, 73 insertions(+)
 create mode 100644 package/fail2ban/0001-setup.py-fix-data-relative-paths.patch

diff --git a/package/fail2ban/0001-setup.py-fix-data-relative-paths.patch b/package/fail2ban/0001-setup.py-fix-data-relative-paths.patch
new file mode 100644
index 0000000000..e1b5bc490b
--- /dev/null
+++ b/package/fail2ban/0001-setup.py-fix-data-relative-paths.patch
@@ -0,0 +1,73 @@
+From 7b7fa3c02d4575c09362336bc310835c37859f1a Mon Sep 17 00:00:00 2001
+From: Angelo Compagnucci <angelo.compagnucci@gmail.com>
+Date: Fri, 19 Apr 2024 22:47:20 +0200
+Subject: [PATCH] setup.py: fix data relative paths
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+As per distutils documentation:
+
+The directory should be a relative path. It is interpreted relative to the
+installation prefix (Python’s sys.prefix for system installations;
+site.USER_BASE for user installations). Distutils allows directory to be an
+absolute installation path, but this is discouraged since it is incompatible
+with the wheel packaging format [1].
+
+When packaging to a bdist_wheel indeed, the files in the "data_files" dictionary
+are not picked at all as data files if they have an absolute path,
+but they are instead added to the wheel as python files.
+This has the effect of installing them as part of python files in the
+site-packages folder when the wheel is installed.
+
+This patch cannot be sent upstream because it breaks the legacy way of
+installing the package with *python setup.py install* and it is needed only to
+comply with the pep517 buildroot compatibility layer.
+
+[1] https://docs.python.org/3.10/distutils/setupscript.html#installing-additional-files
+Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
+---
+ setup.py | 14 +++++++-------
+ 1 file changed, 7 insertions(+), 7 deletions(-)
+
+diff --git a/setup.py b/setup.py
+index 91f71cf2..99ba2e32 100755
+--- a/setup.py
++++ b/setup.py
+@@ -238,26 +238,26 @@ setup(
+ 				for f in w[2]]
+ 	} if with_tests else {},
+ 	data_files = [
+-		('/etc/fail2ban',
++		('etc/fail2ban',
+ 			glob("config/*.conf")
+ 		),
+-		('/etc/fail2ban/filter.d',
++		('etc/fail2ban/filter.d',
+ 			glob("config/filter.d/*.conf")
+ 		),
+-		('/etc/fail2ban/filter.d/ignorecommands',
++		('etc/fail2ban/filter.d/ignorecommands',
+ 			[p for p in glob("config/filter.d/ignorecommands/*") if isfile(p)]
+ 		),
+-		('/etc/fail2ban/action.d',
++		('etc/fail2ban/action.d',
+ 			glob("config/action.d/*.conf") +
+ 			glob("config/action.d/*.py")
+ 		),
+-		('/etc/fail2ban/fail2ban.d',
++		('etc/fail2ban/fail2ban.d',
+ 			''
+ 		),
+-		('/etc/fail2ban/jail.d',
++		('etc/fail2ban/jail.d',
+ 			''
+ 		),
+-		('/var/lib/fail2ban',
++		('var/lib/fail2ban',
+ 			''
+ 		),
+ 	] + data_files_extra,
+-- 
+2.34.1
+
-- 
2.34.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot