Re: [PATCH 0/4] Improve pylibfdt python packaging

[Rob Herring <robh-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>]

On Thu, Nov 11, 2021 at 8:43 PM David Gibson
<david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
>
> On Thu, Nov 11, 2021 at 08:08:08AM -0600, Rob Herring wrote:
> > On Wed, Nov 10, 2021 at 9:41 PM David Gibson
> > <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
> > >
> > > On Wed, Nov 10, 2021 at 07:11:31PM -0600, Rob Herring wrote:
> > > > I'm interested in getting pylibfdt into PyPI and ran into a few issues
> > > > with pylibfdt using the python packaging tools. Primarily, pip didn't
> > > > work nor did setup.py sdist and bdist_wheel subcommands. This series
> > > > fixes those issues.
> > > >
> > > > I've left meson calling setup.py intact for now, but think it's the
> > > > wrong way around. In fact, there's actually some efforts to make meson
> > > > the backend for pip/setuptools. I made several attempts to completely
> > > > eliminate putting files in the source tree without success. Also, I
> > > > noticed a meson install builds pylibfdt twice (though make may too).
> > > >
> > > > I don't think I broke anything. Tests and installs both work with make
> > > > and meson.
> > >
> > > Applied, it certainly looks better to me.
> > >
> > > However, I've just spotted another nasty problem.  I think it must
> > > have started with moving to Fedora 35 on my laptop.  A bunch of the
> > > Python tests now fail like this:
> > >
> > > ======================================================================
> > > ERROR: testGetIntProperties (__main__.PyLibfdtBasicTests)
> > > Test that we can access properties as integers
> > > ----------------------------------------------------------------------
> > > SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats
> > >
> > > The above exception was the direct cause of the following exception:
> > >
> > > Traceback (most recent call last):
> > >   File "/home/dwg/src/dtc/tests/./pylibfdt_tests.py", line 378, in testGetIntProperties
> > >     self.assertEqual(0xdeadbeef, self.get_prop("prop-hex32").as_uint32())
> > >   File "/home/dwg/src/dtc/tests/./pylibfdt_tests.py", line 374, in get_prop
> > >     return self.fdt2.getprop(0, name)
> > >   File "/home/dwg/src/dtc/tests/../pylibfdt/libfdt.py", line 451, in getprop
> > >     pdata = check_err_null(fdt_getprop(self._fdt, nodeoffset, prop_name),
> > >   File "/home/dwg/src/dtc/tests/../pylibfdt/libfdt.py", line 1279, in fdt_getprop
> > >     return _libfdt.fdt_getprop(fdt, nodeoffset, name)
> > > SystemError: <built-in function fdt_getprop> returned a result with an exception set
> > >
> > > Any ideas?
> >
> > Python 3.10? Only guessing because I'm on 3.9. Otherwise, I have no clue.
>
> It appears so; I've now merged Ross Burton's fix.
>
> > I was going to look at making '.setup.py test' work as testing is
> > intertwined with meson too. Most python CI testing runs against a
> > matrix of python versions which would help here.
> >
> > > Also, Rob, did you have patches to finish the conversion of the
> > > Makefiles to wrappers around meson?
> >
> > That was Marc-André...
>
> Oops, sorry, I got confused.
>
> > > If so, I'm sorry I've lost track
> > > of them.  Can you repost please?
> >
> > One of the issues you had with Travis CI. Are you still using Travis
> > CI after their move? I found it easier to just move to GH workflows
> > than move given I always seem to hit login token issues (maybe that's
> > just group projects with multiple users).
>
> It stopped working after the move, and I haven't looked into what
> would be needed to make it go again.  I was actually thinking I'd move
> over to GitLan - I'm more familiar with its CI stuff from qemu, and I
> like what I've seen.  I haven't actually had a chance to do anything
> on that front, though.

I'm assuming you mean GitLab. I'm using both. The DT spec and dtschema
test/pkging are using workflows. Schema validation runs with the
kernel tree are on GitLab (which actually run in docker on my
machines).

GH workflows have recipes (written by random folks) to do various
things like python version matrix testing or upload to pypi whereas
GitLab is more 'here's a container'. dtc could go either way I think.

Rob