Re: [PATCH 0/4] Improve pylibfdt python packaging

[Bruce Ashfield <bruce.ashfield-gjFFaj9aHVfQT0dZR+AlfA@public.gmane.org>]

On 11/11/2021 9:08 AM, Rob Herring wrote:
> On Wed, Nov 10, 2021 at 9:41 PM David Gibson
> <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
>>
>> On Wed, Nov 10, 2021 at 07:11:31PM -0600, Rob Herring wrote:
>>> I'm interested in getting pylibfdt into PyPI and ran into a few issues
>>> with pylibfdt using the python packaging tools. Primarily, pip didn't
>>> work nor did setup.py sdist and bdist_wheel subcommands. This series
>>> fixes those issues.
>>>
>>> I've left meson calling setup.py intact for now, but think it's the
>>> wrong way around. In fact, there's actually some efforts to make meson
>>> the backend for pip/setuptools. I made several attempts to completely
>>> eliminate putting files in the source tree without success. Also, I
>>> noticed a meson install builds pylibfdt twice (though make may too).
>>>
>>> I don't think I broke anything. Tests and installs both work with make
>>> and meson.
>>
>> Applied, it certainly looks better to me.
>>
>> However, I've just spotted another nasty problem.  I think it must
>> have started with moving to Fedora 35 on my laptop.  A bunch of the
>> Python tests now fail like this:
>>
>> ======================================================================
>> ERROR: testGetIntProperties (__main__.PyLibfdtBasicTests)
>> Test that we can access properties as integers
>> ----------------------------------------------------------------------
>> SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats
>>
>> The above exception was the direct cause of the following exception:
>>
>> Traceback (most recent call last):
>>    File "/home/dwg/src/dtc/tests/./pylibfdt_tests.py", line 378, in testGetIntProperties
>>      self.assertEqual(0xdeadbeef, self.get_prop("prop-hex32").as_uint32())
>>    File "/home/dwg/src/dtc/tests/./pylibfdt_tests.py", line 374, in get_prop
>>      return self.fdt2.getprop(0, name)
>>    File "/home/dwg/src/dtc/tests/../pylibfdt/libfdt.py", line 451, in getprop
>>      pdata = check_err_null(fdt_getprop(self._fdt, nodeoffset, prop_name),
>>    File "/home/dwg/src/dtc/tests/../pylibfdt/libfdt.py", line 1279, in fdt_getprop
>>      return _libfdt.fdt_getprop(fdt, nodeoffset, name)
>> SystemError: <built-in function fdt_getprop> returned a result with an exception set
>>
>> Any ideas?
> 
> Python 3.10? Only guessing because I'm on 3.9. Otherwise, I have no clue.
> 

FWIW, and in case you haven't already sorted it out.

We just fixed this in the yocto integration of dtc/libfdt this
morning (I case the patch hasn't made it upstream to you yet).

It is indeed python 3.10, and in particular:

https://docs.python.org/3.10/whatsnew/3.10.html#id2

(and again, a patch should arrive shortly if it hasn't already).

Cheers,

Bruce

> I was going to look at making '.setup.py test' work as testing is
> intertwined with meson too. Most python CI testing runs against a
> matrix of python versions which would help here.
> 
>> Also, Rob, did you have patches to finish the conversion of the
>> Makefiles to wrappers around meson?
> 
> That was Marc-André...
> 
>> If so, I'm sorry I've lost track
>> of them.  Can you repost please?
> 
> One of the issues you had with Travis CI. Are you still using Travis
> CI after their move? I found it easier to just move to GH workflows
> than move given I always seem to hit login token issues (maybe that's
> just group projects with multiple users).
> 
>> That README addition with meson
>> native building instructions would also be really good to have.
> 
> I can look at that. It's gotten a bit more straight-forward with newer
> meson versions as you don't have to run ninja directly. So it is just:
> 
> One-time init:
> meson <builddir>
> 
> And then:
> meson compile|test|install -C <builddir>
> 
> (I find specifying the builddir every time pretty annoying. Guess I
> need a wrapper script.)
> 
> Rob
>