[libgpiod] Feedback from the Raspberry Pi community

[libgpiod] Feedback from the Raspberry Pi community

[Kent Gibson]

I've been poking around various forums looking for libgpiod related
problems recently, and most recently stuck my nose into
forums.raspberrypi.com.  What was I thinking??

Turns out the Raspberry Pi community is in a bit of turmoil.
Historically they have had three options for userspace GPIO; sysfs,
gpiomem (which exposes the hardware registers to userspace) and
libgpiod.  The Pi devs have finally seen the light and are moving away from
the sysfs interface.  That only took what, eight years?
At the same time the recently released RPi5 is incompatible with the existing
gpiomem approaches as their new architecture has a very different register map.
Rather than continuing down that path, the Pi devs have realised
the problems inherent in that approach and are now endorsing the
standard Linux interfaces as the correct way.  So libgpiod has gone from
being their last resort GPIO interface, since sysfs and poking at
registers is so much easier, to now essentially being the only choice.

Of course this means libgpiod is getting a lot more of their attention
and, as it doesn't provide a seamless plug-in replacement for their
existing solutions, they are not happy and have taken to blaming libgpiod
for their wows.

Here is a short list of their grievances:
 - forum
 - packaging v2
 - persistence
 - ownership

I'm not claiming that to be exhaustive, but those are the major issues.

In more detail:

Forum:

While I've gotten them to understand (fingers crossed) that the linux-gpio
mailing list is the contact point for libgpiod, that isn't very
approachable for the average user who would be more comfortable with a
github-type experience with an issue tracker, wiki, etc.

I agree with that.  I've raised it before and again now.
Having an interactive website of some form would improve community
engagement enormously.
I'm not sure what form that should take, and I'm not suggesting moving
the primary repo from kernel.org or changing the development process,
but I don't see how, say, having a github mirror that could double as
the community engagement hub could hurt.
(ok, on second thought after re-reading that, I can see how that
**could** hurt, but I was thinking in terms of increasing engagement, not
having to deal with some of the cr*p that would inevitibly land there.)

Anything that would help address the misunderstandings,
misinformation, and outright propaganda I've seen out there can only be
a good thing, right :-| ?

Packaging:

Raspberry Pi OS is based on Debian, currently bookworm.
The problem there is that libgpiod v2 didn't make it into bookworm so they
are still on 1.6.3.  The problem I see there is that their user base will
migrate onto v1, and then have to face another migration to v2 somewhere down
the track, when we would obviously prefer they migrate direct to v2.

The problem as they see it is that they like the gpioget --as-is option
in v2, as it works with the driver/persistence solution that I will get
to later, but that option is not available in v1.

Either way, the Raspberry Pi devs appreciate that v2 would be preferable
and appear interested in packaging libgpiod v2 themselves, rather
than waiting for Debian, and IIUC are looking into doing that.
They currently package libgpiod2 (libgpiod v1.6.3) and gpiod (libgpiod
tools v1.6.3).  I figure they can add a libgpiod3 package (libgpiod
v2.1) so they can install both library versions at once.  Wrt the tools,
update gpiod to contain the new tools and depend on libgpiod3, and allow
the user to pick which rev of the gpiod package they want to install, if
they want to support v1 or v2.
Requiring both versions of the tools seems like an extreme corner case
to me, and could be handled by having the user download, build and install
v1.6.3 into a non-standard location.  Alternatively they can package
them independently and rename the binaries from one...

I don't think there is anything we need to do here, and with any luck
this will be resolved in the near future.  Maybe just keep an eye on it.

Persistance:

Most Raspberry Pi users want to be able to call something from shell to
set an output line and have it stick after it returns.
This means they absolutely hate gpioset as it either doesn't stick
(depending on the driver) or doesn't return.  The Raspberry Pi devs have
addressed this by patching their GPIO driver to NOT revert GPIO lines to
their default state when released.  This solves most of their use cases,
other than the gpioget --as-is.  Raspberry Pi users are accustomed to being
able to read back the last value written to an output line, and insist
vociferously that any other solution is a bodge. :-).

The two solutions I see are the --as-is option and the daemon.  Sadly
neither is currently available to them via standard packages.

So two things we could do; the first is backport the --as-is to v1.
Ironically there is a patch[1] for this early in the v2 development that
I had thought was also in the v1 tree, but apparently not.
Perhaps backport that to v1, possibly making it closer to the v2 option
if possible?
Not at all sure this would be any faster than the Pi devs packaging v2
and it might even reduce their incentive to package v2, so maybe just
focus on that....

The second solution is the dbus daemon.  Which beings us to...

Ownership:

Pi users are used to shared access to GPIO lines and are content to use
GPIO output lines to share state.  A line being held by one app being
inaccessible to others is an alien concept to them.

I did suggest shadowing the value in a tmp file as a workaround, but that
is considered far too bodgy :-).

The solution there is the dbus daemon. That would allow them to perform
random sets and gets on arbitrary lines on a whim, just like they do now.
They seem very open to that option, both the Pi devs and end users, so the
sooner the daemon can be available the better.

That covers the main points.  For the most part they are coming
around to the conclusion that libgpiod can do the job for them after
all, despite their long held biases.

Cheers,
Kent.

[1] https://git.kernel.org/pub/scm/libs/libgpiod/libgpiod.git/commit/?id=3a912fbc1e2697f4479396fd11557b943b31d217

Re: [libgpiod] Feedback from the Raspberry Pi community

[Bartosz Golaszewski]

On Fri, Jan 12, 2024 at 7:32 AM Kent Gibson <warthog618@gmail.com> wrote:
>
> I've been poking around various forums looking for libgpiod related
> problems recently, and most recently stuck my nose into
> forums.raspberrypi.com.  What was I thinking??
>

Thank you for doing this Kent. This email is very useful.

> Turns out the Raspberry Pi community is in a bit of turmoil.
> Historically they have had three options for userspace GPIO; sysfs,
> gpiomem (which exposes the hardware registers to userspace) and
> libgpiod.  The Pi devs have finally seen the light and are moving away from
> the sysfs interface.  That only took what, eight years?
> At the same time the recently released RPi5 is incompatible with the existing
> gpiomem approaches as their new architecture has a very different register map.
> Rather than continuing down that path, the Pi devs have realised
> the problems inherent in that approach and are now endorsing the
> standard Linux interfaces as the correct way.  So libgpiod has gone from
> being their last resort GPIO interface, since sysfs and poking at
> registers is so much easier, to now essentially being the only choice.
>
> Of course this means libgpiod is getting a lot more of their attention
> and, as it doesn't provide a seamless plug-in replacement for their
> existing solutions, they are not happy and have taken to blaming libgpiod
> for their wows.
>

Of course they have, the world revolves around RPi after all, doesn't it? :)

> Here is a short list of their grievances:
>  - forum
>  - packaging v2
>  - persistence
>  - ownership
>
> I'm not claiming that to be exhaustive, but those are the major issues.
>
> In more detail:
>
> Forum:
>
> While I've gotten them to understand (fingers crossed) that the linux-gpio
> mailing list is the contact point for libgpiod, that isn't very
> approachable for the average user who would be more comfortable with a
> github-type experience with an issue tracker, wiki, etc.
>
> I agree with that.  I've raised it before and again now.
> Having an interactive website of some form would improve community
> engagement enormously.
> I'm not sure what form that should take, and I'm not suggesting moving
> the primary repo from kernel.org or changing the development process,
> but I don't see how, say, having a github mirror that could double as
> the community engagement hub could hurt.
> (ok, on second thought after re-reading that, I can see how that
> **could** hurt, but I was thinking in terms of increasing engagement, not
> having to deal with some of the cr*p that would inevitibly land there.)
>
> Anything that would help address the misunderstandings,
> misinformation, and outright propaganda I've seen out there can only be
> a good thing, right :-| ?
>

I see. While I prefer using the mailing list as the single point for
discussion and development, I understand that this opinion is not
shared by the majority of user-space developers out there. I will
reopen the original github repo[1] for reporting issues. Possibly for
sending PRs as well for initial discussion but I'd prefer to pass all
patches going into libgpiod by the mailing list anyway in the end.

> Packaging:
>
> Raspberry Pi OS is based on Debian, currently bookworm.
> The problem there is that libgpiod v2 didn't make it into bookworm so they
> are still on 1.6.3.  The problem I see there is that their user base will
> migrate onto v1, and then have to face another migration to v2 somewhere down
> the track, when we would obviously prefer they migrate direct to v2.
>
> The problem as they see it is that they like the gpioget --as-is option
> in v2, as it works with the driver/persistence solution that I will get
> to later, but that option is not available in v1.
>
> Either way, the Raspberry Pi devs appreciate that v2 would be preferable
> and appear interested in packaging libgpiod v2 themselves, rather
> than waiting for Debian, and IIUC are looking into doing that.
> They currently package libgpiod2 (libgpiod v1.6.3) and gpiod (libgpiod
> tools v1.6.3).  I figure they can add a libgpiod3 package (libgpiod
> v2.1) so they can install both library versions at once.  Wrt the tools,
> update gpiod to contain the new tools and depend on libgpiod3, and allow
> the user to pick which rev of the gpiod package they want to install, if
> they want to support v1 or v2.
> Requiring both versions of the tools seems like an extreme corner case
> to me, and could be handled by having the user download, build and install
> v1.6.3 into a non-standard location.  Alternatively they can package
> them independently and rename the binaries from one...
>
> I don't think there is anything we need to do here, and with any luck
> this will be resolved in the near future.  Maybe just keep an eye on it.
>

I am very bad with distros. I have no idea how debian or red hat
packaging works (other than being a somewhat "advanced" user) and I
just let distro maintainers handle packaging of libgpiod. For my own
work I rely 100% on yocto and so keep the relevant recipes in
meta-openembedded up to date but that's all I have time for. So there
will be no help from my side on the debian packages. I just don't care
enough.

> Persistance:
>
> Most Raspberry Pi users want to be able to call something from shell to
> set an output line and have it stick after it returns.
> This means they absolutely hate gpioset as it either doesn't stick
> (depending on the driver) or doesn't return.  The Raspberry Pi devs have
> addressed this by patching their GPIO driver to NOT revert GPIO lines to
> their default state when released.  This solves most of their use cases,
> other than the gpioget --as-is.  Raspberry Pi users are accustomed to being
> able to read back the last value written to an output line, and insist
> vociferously that any other solution is a bodge. :-).
>
> The two solutions I see are the --as-is option and the daemon.  Sadly
> neither is currently available to them via standard packages.
>
> So two things we could do; the first is backport the --as-is to v1.
> Ironically there is a patch[1] for this early in the v2 development that
> I had thought was also in the v1 tree, but apparently not.
> Perhaps backport that to v1, possibly making it closer to the v2 option
> if possible?
> Not at all sure this would be any faster than the Pi devs packaging v2
> and it might even reduce their incentive to package v2, so maybe just
> focus on that....
>
> The second solution is the dbus daemon.  Which beings us to...
>
> Ownership:
>
> Pi users are used to shared access to GPIO lines and are content to use
> GPIO output lines to share state.  A line being held by one app being
> inaccessible to others is an alien concept to them.
>
> I did suggest shadowing the value in a tmp file as a workaround, but that
> is considered far too bodgy :-).
>
> The solution there is the dbus daemon. That would allow them to perform
> random sets and gets on arbitrary lines on a whim, just like they do now.
> They seem very open to that option, both the Pi devs and end users, so the
> sooner the daemon can be available the better.
>

I know. I'm half-way done with the locking rework for GPIOLIB and the
daemon is next on my TODO list. I'm estimating I'm 2/3 done with the
needed functionality.

> That covers the main points.  For the most part they are coming
> around to the conclusion that libgpiod can do the job for them after
> all, despite their long held biases.
>

That's good to hear. Thanks again.

Bartosz

[1] https://github.com/brgl/libgpiod

Re: [libgpiod] Feedback from the Raspberry Pi community

[Kent Gibson]

On Fri, Jan 12, 2024 at 11:22:17AM +0100, Bartosz Golaszewski wrote:
> On Fri, Jan 12, 2024 at 7:32 AM Kent Gibson <warthog618@gmail.com> wrote:
> >
> >
> >
> > I agree with that.  I've raised it before and again now.
> > Having an interactive website of some form would improve community
> > engagement enormously.
> > I'm not sure what form that should take, and I'm not suggesting moving
> > the primary repo from kernel.org or changing the development process,
> > but I don't see how, say, having a github mirror that could double as
> > the community engagement hub could hurt.
> > (ok, on second thought after re-reading that, I can see how that
> > **could** hurt, but I was thinking in terms of increasing engagement, not
> > having to deal with some of the cr*p that would inevitibly land there.)
> >
> > Anything that would help address the misunderstandings,
> > misinformation, and outright propaganda I've seen out there can only be
> > a good thing, right :-| ?
> >
>
> I see. While I prefer using the mailing list as the single point for
> discussion and development, I understand that this opinion is not
> shared by the majority of user-space developers out there. I will
> reopen the original github repo[1] for reporting issues. Possibly for
> sending PRs as well for initial discussion but I'd prefer to pass all
> patches going into libgpiod by the mailing list anyway in the end.
>

Great. And that is what I meant by maintaining the development process -
patches still go via the list, not pull-requests.

Can we update the README to reference that?

> > Either way, the Raspberry Pi devs appreciate that v2 would be preferable
> > and appear interested in packaging libgpiod v2 themselves, rather
> > than waiting for Debian, and IIUC are looking into doing that.
> > They currently package libgpiod2 (libgpiod v1.6.3) and gpiod (libgpiod
> > tools v1.6.3).  I figure they can add a libgpiod3 package (libgpiod
> > v2.1) so they can install both library versions at once.  Wrt the tools,
> > update gpiod to contain the new tools and depend on libgpiod3, and allow
> > the user to pick which rev of the gpiod package they want to install, if
> > they want to support v1 or v2.
> > Requiring both versions of the tools seems like an extreme corner case
> > to me, and could be handled by having the user download, build and install
> > v1.6.3 into a non-standard location.  Alternatively they can package
> > them independently and rename the binaries from one...
> >
> > I don't think there is anything we need to do here, and with any luck
> > this will be resolved in the near future.  Maybe just keep an eye on it.
> >
>
> I am very bad with distros. I have no idea how debian or red hat
> packaging works (other than being a somewhat "advanced" user) and I
> just let distro maintainers handle packaging of libgpiod. For my own
> work I rely 100% on yocto and so keep the relevant recipes in
> meta-openembedded up to date but that's all I have time for. So there
> will be no help from my side on the debian packages. I just don't care
> enough.
>

I've created packages for both at some point, but never as part of
a distro, so I've got a rough idea how the packages themselves work but
no idea how the distro packagers select what goes where or when.
As with you - not something I've ever needed to have a concern for.

> >
> > The solution there is the dbus daemon. That would allow them to perform
> > random sets and gets on arbitrary lines on a whim, just like they do now.
> > They seem very open to that option, both the Pi devs and end users, so the
> > sooner the daemon can be available the better.
> >
>
> I know. I'm half-way done with the locking rework for GPIOLIB and the
> daemon is next on my TODO list. I'm estimating I'm 2/3 done with the
> needed functionality.
>

Let me know if there is anything I can do to help out.

Cheers,
Kent.