[Buildroot] [PATCH 1/4] support/testing: add python-mako runtime test

[Buildroot] [PATCH 1/4] support/testing: add python-mako runtime test

[Romain Naour]

This new runtime test allows to reproduce the issue
reported by https://bugs.busybox.net/show_bug.cgi?id=15628
with a simple python script.

Traceback (most recent call last):
  File "/bin/piglit", line 107, in <module>
    import framework.programs.summary as summary
  File "/usr/lib64/piglit/framework/programs/summary.py", line 30, in <module>
    from framework import summary, status, core, backends, exceptions
  File "/usr/lib64/piglit/framework/summary/__init__.py", line 31, in <module>
    from .html_ import html, feat
  File "/usr/lib64/piglit/framework/summary/html_.py", line 37, in <module>
    from mako.lookup import TemplateLookup
  File "/usr/lib/python3.11/site-packages/mako/lookup.py", line 13, in <module>
  File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 292, in <module>
  File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 289, in _install_highlighting
  File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 277, in _install_fallback
  File "/usr/lib/python3.11/site-packages/mako/filters.py", line 14, in <module>
ModuleNotFoundError: No module named 'markupsafe

Signed-off-by: Romain Naour <romain.naour@gmail.com>
---
 support/testing/tests/package/sample_python_mako.py |  1 +
 support/testing/tests/package/test_python_mako.py   | 12 ++++++++++++
 2 files changed, 13 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_mako.py
 create mode 100644 support/testing/tests/package/test_python_mako.py

diff --git a/support/testing/tests/package/sample_python_mako.py b/support/testing/tests/package/sample_python_mako.py
new file mode 100644
index 0000000000..aedae7ebad
--- /dev/null
+++ b/support/testing/tests/package/sample_python_mako.py
@@ -0,0 +1 @@
+from mako import runtime
diff --git a/support/testing/tests/package/test_python_mako.py b/support/testing/tests/package/test_python_mako.py
new file mode 100644
index 0000000000..b6075bde6a
--- /dev/null
+++ b/support/testing/tests/package/test_python_mako.py
@@ -0,0 +1,12 @@
+from tests.package.test_python import TestPythonPackageBase
+
+
+class TestPythonPy3Mako(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_MAKO=y
+        """
+    sample_scripts = ["tests/package/sample_python_mako.py"]
+    timeout = 30
-- 
2.40.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 2/4] package/python-mako: add optionnal runtime dependency on python-pygments

[Romain Naour]

Signed-off-by: Romain Naour <romain.naour@gmail.com>
---
 package/python-mako/Config.in | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/package/python-mako/Config.in b/package/python-mako/Config.in
index e6a10c9c2e..2e1094189f 100644
--- a/package/python-mako/Config.in
+++ b/package/python-mako/Config.in
@@ -7,3 +7,14 @@ config BR2_PACKAGE_PYTHON_MAKO
 	  for maximum performance.
 
 	  https://pypi.python.org/pypi/Mako
+
+if BR2_PACKAGE_PYTHON_MAKO
+
+config BR2_PACKAGE_PYTHON_MAKO_EXT_PYGMENTPLUGIN
+	bool "mako extension pygmentplugin"
+	select BR2_PACKAGE_PYTHON_PYGMENTS # runtime
+	help
+	  Provide python-pygments support for
+	  mako.ext.pygmentplugin
+
+endif
-- 
2.40.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 2/4] package/python-mako: add optionnal runtime dependency on python-pygments

[Thomas Petazzoni via buildroot]

On Fri,  9 Jun 2023 19:49:53 +0200
Romain Naour <romain.naour@gmail.com> wrote:

> Signed-off-by: Romain Naour <romain.naour@gmail.com>
> ---
>  package/python-mako/Config.in | 11 +++++++++++
>  1 file changed, 11 insertions(+)

Applied to master, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 3/4] package/python-mako: add optionnal runtime dependency on python-babel

[Romain Naour]

Signed-off-by: Romain Naour <romain.naour@gmail.com>
---
 package/python-mako/Config.in | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/package/python-mako/Config.in b/package/python-mako/Config.in
index 2e1094189f..7bba31943e 100644
--- a/package/python-mako/Config.in
+++ b/package/python-mako/Config.in
@@ -17,4 +17,11 @@ config BR2_PACKAGE_PYTHON_MAKO_EXT_PYGMENTPLUGIN
 	  Provide python-pygments support for
 	  mako.ext.pygmentplugin
 
+config BR2_PACKAGE_PYTHON_MAKO_EXT_BABELPLUGIN
+	bool "mako extension babelplugin"
+	select BR2_PACKAGE_PYTHON_BABEL # runtime
+	help
+	  Provide python-babel support for
+	  mako.ext.babelplugin
+
 endif
-- 
2.40.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 3/4] package/python-mako: add optionnal runtime dependency on python-babel

[Thomas Petazzoni via buildroot]

On Fri,  9 Jun 2023 19:49:54 +0200
Romain Naour <romain.naour@gmail.com> wrote:

> Signed-off-by: Romain Naour <romain.naour@gmail.com>
> ---
>  package/python-mako/Config.in | 7 +++++++
>  1 file changed, 7 insertions(+)

Applied to master, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 4/4] support/testing: TestPythonPy3MakoExt: add a test for mako external plugin (pygmentsplugin, babel)

[Romain Naour]

Mako provide some external plugins that requires additionnal and
optionnal runtime dependencies.

Signed-off-by: Romain Naour <romain.naour@gmail.com>
---
 .../testing/tests/package/sample_python_mako_ext.py  |  3 +++
 support/testing/tests/package/test_python_mako.py    | 12 ++++++++++++
 2 files changed, 15 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_mako_ext.py

diff --git a/support/testing/tests/package/sample_python_mako_ext.py b/support/testing/tests/package/sample_python_mako_ext.py
new file mode 100644
index 0000000000..5b06a28181
--- /dev/null
+++ b/support/testing/tests/package/sample_python_mako_ext.py
@@ -0,0 +1,3 @@
+from mako import runtime
+import mako.ext.pygmentplugin
+import mako.ext.babelplugin
diff --git a/support/testing/tests/package/test_python_mako.py b/support/testing/tests/package/test_python_mako.py
index b6075bde6a..f1031d1d59 100644
--- a/support/testing/tests/package/test_python_mako.py
+++ b/support/testing/tests/package/test_python_mako.py
@@ -10,3 +10,15 @@ class TestPythonPy3Mako(TestPythonPackageBase):
         """
     sample_scripts = ["tests/package/sample_python_mako.py"]
     timeout = 30
+
+class TestPythonPy3MakoExt(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_MAKO=y
+        BR2_PACKAGE_PYTHON_MAKO_EXT_PYGMENTPLUGIN=y
+        BR2_PACKAGE_PYTHON_MAKO_EXT_BABELPLUGIN=y
+        """
+    sample_scripts = ["tests/package/sample_python_mako_ext.py"]
+    timeout = 30
-- 
2.40.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 4/4] support/testing: TestPythonPy3MakoExt: add a test for mako external plugin (pygmentsplugin, babel)

[Thomas Petazzoni via buildroot]

On Fri,  9 Jun 2023 19:49:55 +0200
Romain Naour <romain.naour@gmail.com> wrote:

> Mako provide some external plugins that requires additionnal and
> optionnal runtime dependencies.
> 
> Signed-off-by: Romain Naour <romain.naour@gmail.com>
> ---
>  .../testing/tests/package/sample_python_mako_ext.py  |  3 +++
>  support/testing/tests/package/test_python_mako.py    | 12 ++++++++++++
>  2 files changed, 15 insertions(+)
>  create mode 100644 support/testing/tests/package/sample_python_mako_ext.py

Applied to master after adding entries in the DEVELOPERS file, and
fixing flake8 warnings. Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/4] support/testing: add python-mako runtime test

[Thomas Petazzoni via buildroot]

Hello Romain,

On Fri,  9 Jun 2023 19:49:52 +0200
Romain Naour <romain.naour@gmail.com> wrote:

> This new runtime test allows to reproduce the issue
> reported by https://bugs.busybox.net/show_bug.cgi?id=15628
> with a simple python script.
> 
> Traceback (most recent call last):
>   File "/bin/piglit", line 107, in <module>
>     import framework.programs.summary as summary
>   File "/usr/lib64/piglit/framework/programs/summary.py", line 30, in <module>
>     from framework import summary, status, core, backends, exceptions
>   File "/usr/lib64/piglit/framework/summary/__init__.py", line 31, in <module>
>     from .html_ import html, feat
>   File "/usr/lib64/piglit/framework/summary/html_.py", line 37, in <module>
>     from mako.lookup import TemplateLookup
>   File "/usr/lib/python3.11/site-packages/mako/lookup.py", line 13, in <module>
>   File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 292, in <module>
>   File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 289, in _install_highlighting
>   File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 277, in _install_fallback
>   File "/usr/lib/python3.11/site-packages/mako/filters.py", line 14, in <module>
> ModuleNotFoundError: No module named 'markupsafe
> 
> Signed-off-by: Romain Naour <romain.naour@gmail.com>

I'm unclear on a number of things:

- How does this interact with the patch proposed by Sebastian at
  https://patchwork.ozlabs.org/project/buildroot/patch/20230608140933.1814667-1-sebastian.weyer@smile.fr/,
  which takes a different approach ?

- How does your patch series fixes bug #15628 ? The patch from
  Sebastian fixes it, but not your series as far as I can see.

- I don't see in your series what ensures that the test
  TestPythonPy3Mako properly works. It would work with Sebastian patch
  applied, but Sebastian's patch collides with your series because
  Sebastian patch makes BR2_PACKAGE_PYTHON_PYGMENTS a mandatory
  dependency of python-mako, while your patch series makes it an
  optional dependency, enabled when
  BR2_PACKAGE_PYTHON_MAKO_EXT_PYGMENTPLUGIN is enabled.

I might not have grasped the full scale of the issue, but my feeling is
that:

- python-mako needs markupsafe unconditionally, as demonstrated by your
  test TestPythonPy3Mako. So python-mako should unconditionally select
  BR2_PACKAGE_PYTHON_MARKUPSAFE

- python-mako only needs pygments/babel as optional dependencies so
  your patches 2/4 and 3/4 are fine

- in order to fix bug #15628, we need to ensure that the piglit package
  selects BR2_PACKAGE_PYTHON_MAKO_EXT_PYGMENTPLUGIN, so that
  python-mako is installed with pygments support

Does that make sense ?

Best regards,

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/4] support/testing: add python-mako runtime test

[Thomas Petazzoni via buildroot]

On Wed, 12 Jul 2023 21:43:57 +0200
Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote:

> I might not have grasped the full scale of the issue, but my feeling is
> that:
> 
> - python-mako needs markupsafe unconditionally, as demonstrated by your
>   test TestPythonPy3Mako. So python-mako should unconditionally select
>   BR2_PACKAGE_PYTHON_MARKUPSAFE
> 
> - python-mako only needs pygments/babel as optional dependencies so
>   your patches 2/4 and 3/4 are fine
> 
> - in order to fix bug #15628, we need to ensure that the piglit package
>   selects BR2_PACKAGE_PYTHON_MAKO_EXT_PYGMENTPLUGIN, so that
>   python-mako is installed with pygments support
> 
> Does that make sense ?

I got it wrong, your basic test shows that both pygments and markupsafe
are needed for a basic usage of python-mako:

$ ./support/testing/run-tests -o ../outputs/ -k tests.package.test_python_mako.TestPythonPy3Mako
21:32:04 TestPythonPy3Mako                        Starting
21:32:04 TestPythonPy3Mako                        Building
21:34:55 TestPythonPy3Mako                        Building done
21:35:01 TestPythonPy3Mako                        Cleaning up
F
======================================================================
FAIL: test_run (tests.package.test_python_mako.TestPythonPy3Mako)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/thomas/buildroot/buildroot/support/testing/tests/package/test_python.py", line 137, in test_run
    self.run_sample_scripts()
  File "/home/thomas/buildroot/buildroot/support/testing/tests/package/test_python.py", line 132, in run_sample_scripts
    self.assertRunOk(cmd, timeout=self.timeout)
  File "/home/thomas/buildroot/buildroot/support/testing/infra/basetest.py", line 95, in assertRunOk
    self.assertEqual(
AssertionError: 1 != 0 : 
Failed to run: python sample_python_mako.py
output was:
  Traceback (most recent call last):
    File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 287, in _install_highlighting
    File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 271, in _install_pygments
    File "/usr/lib/python3.11/site-packages/mako/ext/pygmentplugin.py", line 7, in <module>
  ModuleNotFoundError: No module named 'pygments'
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "/root/sample_python_mako.py", line 1, in <module>
      from mako import runtime
    File "/usr/lib/python3.11/site-packages/mako/runtime.py", line 15, in <module>
    File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 292, in <module>
    File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 289, in _install_highlighting
    File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 277, in _install_fallback
    File "/usr/lib/python3.11/site-packages/mako/filters.py", line 14, in <module>
  ModuleNotFoundError: No module named 'markupsafe'

----------------------------------------------------------------------
Ran 1 test in 177.157s

FAILED (failures=1)

So to me it looks like the patch from Sebastian is correct, and
therefore your PATCH 2/4 adding an option to enable pygments support
does not make sense because pygments is needed unconditionally by
python-mako.

Thoughts?

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/4] support/testing: add python-mako runtime test

[Romain Naour]

Hello Thomas, All,

Le 12/07/2023 à 21:46, Thomas Petazzoni via buildroot a écrit :
> On Wed, 12 Jul 2023 21:43:57 +0200
> Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote:
> 
>> I might not have grasped the full scale of the issue, but my feeling is
>> that:
>>
>> - python-mako needs markupsafe unconditionally, as demonstrated by your
>>   test TestPythonPy3Mako. So python-mako should unconditionally select
>>   BR2_PACKAGE_PYTHON_MARKUPSAFE
>>
>> - python-mako only needs pygments/babel as optional dependencies so
>>   your patches 2/4 and 3/4 are fine
>>
>> - in order to fix bug #15628, we need to ensure that the piglit package
>>   selects BR2_PACKAGE_PYTHON_MAKO_EXT_PYGMENTPLUGIN, so that
>>   python-mako is installed with pygments support
>>
>> Does that make sense ?
> 
> I got it wrong, your basic test shows that both pygments and markupsafe
> are needed for a basic usage of python-mako:
> 
> $ ./support/testing/run-tests -o ../outputs/ -k tests.package.test_python_mako.TestPythonPy3Mako
> 21:32:04 TestPythonPy3Mako                        Starting
> 21:32:04 TestPythonPy3Mako                        Building
> 21:34:55 TestPythonPy3Mako                        Building done
> 21:35:01 TestPythonPy3Mako                        Cleaning up
> F
> ======================================================================
> FAIL: test_run (tests.package.test_python_mako.TestPythonPy3Mako)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/home/thomas/buildroot/buildroot/support/testing/tests/package/test_python.py", line 137, in test_run
>     self.run_sample_scripts()
>   File "/home/thomas/buildroot/buildroot/support/testing/tests/package/test_python.py", line 132, in run_sample_scripts
>     self.assertRunOk(cmd, timeout=self.timeout)
>   File "/home/thomas/buildroot/buildroot/support/testing/infra/basetest.py", line 95, in assertRunOk
>     self.assertEqual(
> AssertionError: 1 != 0 : 
> Failed to run: python sample_python_mako.py
> output was:
>   Traceback (most recent call last):
>     File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 287, in _install_highlighting
>     File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 271, in _install_pygments
>     File "/usr/lib/python3.11/site-packages/mako/ext/pygmentplugin.py", line 7, in <module>
>   ModuleNotFoundError: No module named 'pygments'
>   
>   During handling of the above exception, another exception occurred:
>   
>   Traceback (most recent call last):
>     File "/root/sample_python_mako.py", line 1, in <module>
>       from mako import runtime
>     File "/usr/lib/python3.11/site-packages/mako/runtime.py", line 15, in <module>
>     File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 292, in <module>
>     File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 289, in _install_highlighting
>     File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 277, in _install_fallback
>     File "/usr/lib/python3.11/site-packages/mako/filters.py", line 14, in <module>
>   ModuleNotFoundError: No module named 'markupsafe'
> 
> ----------------------------------------------------------------------
> Ran 1 test in 177.157s
> 
> FAILED (failures=1)
> 
> So to me it looks like the patch from Sebastian is correct, and
> therefore your PATCH 2/4 adding an option to enable pygments support
> does not make sense because pygments is needed unconditionally by
> python-mako.
> 
> Thoughts?

Actually the patch from Sebastian should be part of this series but without
selecting pygments.

It's unclear for me if pygments and babel are really mandatory runtime
dependencies of mako. Indeed, pygments and babel are used by "mako extension"
and may not be used at all when using python-mako.

Do we really want to handle such "mako extension" ?

Otherwise, I'm fine to add them as runtime dependencies of mako.

Best regards,
Romain

> 
> Thomas

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/4] support/testing: add python-mako runtime test

[Romain Naour]

Le 13/07/2023 à 09:36, Romain Naour a écrit :
> Hello Thomas, All,
> 
> Le 12/07/2023 à 21:46, Thomas Petazzoni via buildroot a écrit :
>> On Wed, 12 Jul 2023 21:43:57 +0200
>> Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote:
>>
>>> I might not have grasped the full scale of the issue, but my feeling is
>>> that:
>>>
>>> - python-mako needs markupsafe unconditionally, as demonstrated by your
>>>   test TestPythonPy3Mako. So python-mako should unconditionally select
>>>   BR2_PACKAGE_PYTHON_MARKUPSAFE
>>>
>>> - python-mako only needs pygments/babel as optional dependencies so
>>>   your patches 2/4 and 3/4 are fine
>>>
>>> - in order to fix bug #15628, we need to ensure that the piglit package
>>>   selects BR2_PACKAGE_PYTHON_MAKO_EXT_PYGMENTPLUGIN, so that
>>>   python-mako is installed with pygments support
>>>
>>> Does that make sense ?
>>
>> I got it wrong, your basic test shows that both pygments and markupsafe
>> are needed for a basic usage of python-mako:
>>
>> $ ./support/testing/run-tests -o ../outputs/ -k tests.package.test_python_mako.TestPythonPy3Mako
>> 21:32:04 TestPythonPy3Mako                        Starting
>> 21:32:04 TestPythonPy3Mako                        Building
>> 21:34:55 TestPythonPy3Mako                        Building done
>> 21:35:01 TestPythonPy3Mako                        Cleaning up
>> F
>> ======================================================================
>> FAIL: test_run (tests.package.test_python_mako.TestPythonPy3Mako)
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>>   File "/home/thomas/buildroot/buildroot/support/testing/tests/package/test_python.py", line 137, in test_run
>>     self.run_sample_scripts()
>>   File "/home/thomas/buildroot/buildroot/support/testing/tests/package/test_python.py", line 132, in run_sample_scripts
>>     self.assertRunOk(cmd, timeout=self.timeout)
>>   File "/home/thomas/buildroot/buildroot/support/testing/infra/basetest.py", line 95, in assertRunOk
>>     self.assertEqual(
>> AssertionError: 1 != 0 : 
>> Failed to run: python sample_python_mako.py
>> output was:
>>   Traceback (most recent call last):
>>     File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 287, in _install_highlighting
>>     File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 271, in _install_pygments
>>     File "/usr/lib/python3.11/site-packages/mako/ext/pygmentplugin.py", line 7, in <module>
>>   ModuleNotFoundError: No module named 'pygments'

IIRC, I didn't have this error while testing with TestPythonPy3Mako with
Sebastian patch applied and ammanded to remove pygments dependency.

Best regards,
Romain


>>   
>>   During handling of the above exception, another exception occurred:
>>   
>>   Traceback (most recent call last):
>>     File "/root/sample_python_mako.py", line 1, in <module>
>>       from mako import runtime
>>     File "/usr/lib/python3.11/site-packages/mako/runtime.py", line 15, in <module>
>>     File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 292, in <module>
>>     File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 289, in _install_highlighting
>>     File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 277, in _install_fallback
>>     File "/usr/lib/python3.11/site-packages/mako/filters.py", line 14, in <module>
>>   ModuleNotFoundError: No module named 'markupsafe'
>>
>> ----------------------------------------------------------------------
>> Ran 1 test in 177.157s
>>
>> FAILED (failures=1)
>>
>> So to me it looks like the patch from Sebastian is correct, and
>> therefore your PATCH 2/4 adding an option to enable pygments support
>> does not make sense because pygments is needed unconditionally by
>> python-mako.
>>
>> Thoughts?
> 
> Actually the patch from Sebastian should be part of this series but without
> selecting pygments.
> 
> It's unclear for me if pygments and babel are really mandatory runtime
> dependencies of mako. Indeed, pygments and babel are used by "mako extension"
> and may not be used at all when using python-mako.
> 
> Do we really want to handle such "mako extension" ?
> 
> Otherwise, I'm fine to add them as runtime dependencies of mako.
> 
> Best regards,
> Romain
> 
>>
>> Thomas
> 

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/4] support/testing: add python-mako runtime test

[Sebastian WEYER]

Hello all,


On Thu, Jul 13, 2023 at 9:42 AM Romain Naour <romain.naour@smile.fr> wrote:
>
> Le 13/07/2023 à 09:36, Romain Naour a écrit :
> > Hello Thomas, All,
> >
> > Le 12/07/2023 à 21:46, Thomas Petazzoni via buildroot a écrit :
> >> On Wed, 12 Jul 2023 21:43:57 +0200
> >> Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote:
> >>
> >>> I might not have grasped the full scale of the issue, but my feeling is
> >>> that:
> >>>
> >>> - python-mako needs markupsafe unconditionally, as demonstrated by your
> >>>   test TestPythonPy3Mako. So python-mako should unconditionally select
> >>>   BR2_PACKAGE_PYTHON_MARKUPSAFE
> >>>
> >>> - python-mako only needs pygments/babel as optional dependencies so
> >>>   your patches 2/4 and 3/4 are fine
> >>>
> >>> - in order to fix bug #15628, we need to ensure that the piglit package
> >>>   selects BR2_PACKAGE_PYTHON_MAKO_EXT_PYGMENTPLUGIN, so that
> >>>   python-mako is installed with pygments support
> >>>
> >>> Does that make sense ?
> >>
> >> I got it wrong, your basic test shows that both pygments and markupsafe
> >> are needed for a basic usage of python-mako:


After checking again I'm not even sure that is necessarily the case.
When you don't
select either you get the error that both are missing. However it
seems that just
adding one of BR2_PACKAGE_PYTHON_MARKUPSAFE=y or
BR2_PACKAGE_PYTHON_PYGMENTS=y in the TestPythonPy3Mako config
makes the test work. I don't know if we should add both or just one of them as a
default dependency in this case. It does seem though that markupsafe
is necessary
If the optional babel dependency is selected.

./support/testing/run-tests -o test-output -d test-dl/ -k
tests.package.test_python_mako.TestPythonPy3MakoExt
11:16:50 TestPythonPy3MakoExt                     Starting
11:16:59 TestPythonPy3MakoExt                     Cleaning up
F
======================================================================
FAIL: test_run (tests.package.test_python_mako.TestPythonPy3MakoExt)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/sewey/projects/buildroot/support/testing/tests/package/test_python.py",
line 137, in test_run
    self.run_sample_scripts()
  File "/home/sewey/projects/buildroot/support/testing/tests/package/test_python.py",
line 132, in run_sample_scripts
    self.assertRunOk(cmd, timeout=self.timeout)
  File "/home/sewey/projects/buildroot/support/testing/infra/basetest.py",
line 95, in assertRunOk
    self.assertEqual(
AssertionError: 1 != 0 :
Failed to run: python sample_python_mako_ext.py
output was:
  Traceback (most recent call last):
    File "/root/sample_python_mako_ext.py", line 3, in <module>
      import mako.ext.babelplugin
    File "/usr/lib/python3.11/site-packages/mako/ext/babelplugin.py",
line 10, in <module>
    File "/usr/lib/python3.11/site-packages/mako/ext/extract.py", line
11, in <module>
    File "/usr/lib/python3.11/site-packages/mako/lexer.py", line 13, in <module>
    File "/usr/lib/python3.11/site-packages/mako/parsetree.py", line
13, in <module>
    File "/usr/lib/python3.11/site-packages/mako/filters.py", line 14,
in <module>
  ModuleNotFoundError: No module named 'markupsafe'

----------------------------------------------------------------------
Ran 1 test in 9.392s

FAILED (failures=1)


Best regards,
Sebastian


> >>
> >> $ ./support/testing/run-tests -o ../outputs/ -k tests.package.test_python_mako.TestPythonPy3Mako
> >> 21:32:04 TestPythonPy3Mako                        Starting
> >> 21:32:04 TestPythonPy3Mako                        Building
> >> 21:34:55 TestPythonPy3Mako                        Building done
> >> 21:35:01 TestPythonPy3Mako                        Cleaning up
> >> F
> >> ======================================================================
> >> FAIL: test_run (tests.package.test_python_mako.TestPythonPy3Mako)
> >> ----------------------------------------------------------------------
> >> Traceback (most recent call last):
> >>   File "/home/thomas/buildroot/buildroot/support/testing/tests/package/test_python.py", line 137, in test_run
> >>     self.run_sample_scripts()
> >>   File "/home/thomas/buildroot/buildroot/support/testing/tests/package/test_python.py", line 132, in run_sample_scripts
> >>     self.assertRunOk(cmd, timeout=self.timeout)
> >>   File "/home/thomas/buildroot/buildroot/support/testing/infra/basetest.py", line 95, in assertRunOk
> >>     self.assertEqual(
> >> AssertionError: 1 != 0 :
> >> Failed to run: python sample_python_mako.py
> >> output was:
> >>   Traceback (most recent call last):
> >>     File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 287, in _install_highlighting
> >>     File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 271, in _install_pygments
> >>     File "/usr/lib/python3.11/site-packages/mako/ext/pygmentplugin.py", line 7, in <module>
> >>   ModuleNotFoundError: No module named 'pygments'
>
> IIRC, I didn't have this error while testing with TestPythonPy3Mako with
> Sebastian patch applied and ammanded to remove pygments dependency.
>
> Best regards,
> Romain
>
>
> >>
> >>   During handling of the above exception, another exception occurred:
> >>
> >>   Traceback (most recent call last):
> >>     File "/root/sample_python_mako.py", line 1, in <module>
> >>       from mako import runtime
> >>     File "/usr/lib/python3.11/site-packages/mako/runtime.py", line 15, in <module>
> >>     File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 292, in <module>
> >>     File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 289, in _install_highlighting
> >>     File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 277, in _install_fallback
> >>     File "/usr/lib/python3.11/site-packages/mako/filters.py", line 14, in <module>
> >>   ModuleNotFoundError: No module named 'markupsafe'
> >>
> >> ----------------------------------------------------------------------
> >> Ran 1 test in 177.157s
> >>
> >> FAILED (failures=1)
> >>
> >> So to me it looks like the patch from Sebastian is correct, and
> >> therefore your PATCH 2/4 adding an option to enable pygments support
> >> does not make sense because pygments is needed unconditionally by
> >> python-mako.
> >>
> >> Thoughts?
> >
> > Actually the patch from Sebastian should be part of this series but without
> > selecting pygments.
> >
> > It's unclear for me if pygments and babel are really mandatory runtime
> > dependencies of mako. Indeed, pygments and babel are used by "mako extension"
> > and may not be used at all when using python-mako.
> >
> > Do we really want to handle such "mako extension" ?
> >
> > Otherwise, I'm fine to add them as runtime dependencies of mako.
> >
> > Best regards,
> > Romain
> >
> >>
> >> Thomas
> >
>


-- 
Pour la planète, n'imprimez ce mail que si c'est nécessaire


10-12 Rue Nicolas Appert
44100 NANTES

Sebastian WEYER
Stagiaire production

sebastian.weyer@smile.fr
http://www.smile.eu
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/4] support/testing: add python-mako runtime test

[Thomas Petazzoni via buildroot]

On Fri,  9 Jun 2023 19:49:52 +0200
Romain Naour <romain.naour@gmail.com> wrote:

> This new runtime test allows to reproduce the issue
> reported by https://bugs.busybox.net/show_bug.cgi?id=15628
> with a simple python script.
> 
> Traceback (most recent call last):
>   File "/bin/piglit", line 107, in <module>
>     import framework.programs.summary as summary
>   File "/usr/lib64/piglit/framework/programs/summary.py", line 30, in <module>
>     from framework import summary, status, core, backends, exceptions
>   File "/usr/lib64/piglit/framework/summary/__init__.py", line 31, in <module>
>     from .html_ import html, feat
>   File "/usr/lib64/piglit/framework/summary/html_.py", line 37, in <module>
>     from mako.lookup import TemplateLookup
>   File "/usr/lib/python3.11/site-packages/mako/lookup.py", line 13, in <module>
>   File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 292, in <module>
>   File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 289, in _install_highlighting
>   File "/usr/lib/python3.11/site-packages/mako/exceptions.py", line 277, in _install_fallback
>   File "/usr/lib/python3.11/site-packages/mako/filters.py", line 14, in <module>
> ModuleNotFoundError: No module named 'markupsafe
> 
> Signed-off-by: Romain Naour <romain.naour@gmail.com>
> ---
>  support/testing/tests/package/sample_python_mako.py |  1 +
>  support/testing/tests/package/test_python_mako.py   | 12 ++++++++++++
>  2 files changed, 13 insertions(+)
>  create mode 100644 support/testing/tests/package/sample_python_mako.py
>  create mode 100644 support/testing/tests/package/test_python_mako.py

I've applied after changing the commit log, which was confusing,
because the problem does not reproduce with this test, as it has been
fixed by the previous commit. Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot