[cocci] Flatpak package for Coccinelle

[cocci] Flatpak package for Coccinelle

[Jan Tojnar]

Hello Julia and everyone!

To make Coccinelle more easily available to a wider range of people, I
have decided to package it using Flatpak [1], a distribution-
independent package manager backed by Freedesktop.org and preferred by
many developers of Linux desktop environments like GNOME, KDE or
Pantheon. This will be useful for situations where Flatpak is more
available or familiar than OCaml toolchain, such as for most GNOME
developers.

Eventually, the goal is to have it available in the Flathub repository
but there are several (numbered) questions to resolve before that can
happen.

First, each app requires an ID [2] in reverse-DNS format. I think
`fr.inria.Coccinelle` would be nicest. But I have also seen
`fr.lip6.Coccinelle` floating around but that might be older domain
name. And following the guidelines literally (you should have control
over the domain), it would probably have to be
`fr.inria.gitlab.coccinelle.Coccinelle` or
`fr.inria.gitlabpages.coccinelle.Coccinelle` but that feels over the
top.

    (1) Is `fr.inria.Coccinelle` acceptable?

Second, uploading a package to Flathub requires an AppStream [3]
manifest, which contains items such as textual description of the
project, categories, authors, links and information about releases. I
took the description from the Coccinelle website but I am not sure what
the copyright status is:

> Coccinelle is a program matching and transformation engine which
provides the language SmPL (Semantic Patch Language) for specifying
desired matches and transformations in C code. Coccinelle was initially
targeted towards performing collateral evolutions in Linux. Such
evolutions comprise the changes that are needed in client code in
response to evolutions in library APIs, and may include modifications
such as renaming a function, adding a function argument whose value is
somehow context-dependent, and reorganizing a data structure. Beyond
collateral evolutions, Coccinelle is successfully used (by us and
others) for finding and fixing bugs in systems code.

Flathub and distro tooling combine metadata of software projects in
their repositories into indexes and other derivative databases so they
require the AppStream metadata to be distributed under one of several
licenses [4] vetted for mutual compatibility (e.g. Creative Commons,
Public Domain, or GNU Free Documentation License).

    (2) Could you clarify who the copyright holder of the above
description is, and if it can be shared under one for the licenses
listed in [4]?

Third, it is preferred [5] to ship the AppStream metadata upstream so
that other distributions can use it. Shipping the metainfo in the
Coccinelle repository would require updating the file with a new
version and date before each release but that can be integrated into
the existing maintenance version script. No problem if you do not want
that, we can maintain it locally in a Flathub repository.

    (3) Do you wish to include the AppStream metadata in the
repository?

Fourth, the Flatpak manifest can be also included in the Git repository
to allow building Coccinelle from the repo in a Flatpak environment.
This would improve access to the development versions of Coccinelle
but, since people who need those are probably more likely to be able to
obtain OCaml toolchain themselves, it might not be worth the
maintenance burden. I can help but I understand if this is not
something you want to bother with.

    (4) Do you wish to include Flatpak manifest in the repository?

On that note, having the Flatpak manifest in the repo would also allow
us to have GitHub/GitLab CI (Continuous Integration) environment
automatically build latest version of Coccinelle and allow users to
download it without having to build it, just like Flathub will enable
that for stable versions of Coccinelle.

    (5) Are you interested in such integration?

Lastly, one of the goals of Flatpak is to have upstream developers
publish their software using Flatpak themselves, essentially creating a
singular distribution platform for Linux systems. It is recommended [6]
that I ask you if you want to submit Coccinelle to Flathub yourself.
But I understand that you are busy and I can maintain the package if
you will not.

    (6) Do you want to maintain Coccinelle package on Flathub yourself?


I have opened a draft PR on GitHub so that external reviewers can give
input on the patches before submitting them here (if desired):

https://github.com/coccinelle/coccinelle/pull/370

I have also created a PR to add Coccinelle to Flathub:

https://github.com/flathub/flathub/pull/5354


Cheers,

Jan

[1]: https://en.wikipedia.org/wiki/Flatpak
[2]:
https://docs.flathub.org/docs/for-app-authors/requirements#application-id
[3]: https://en.wikipedia.org/wiki/AppStream
[4]:
https://www.freedesktop.org/software/appstream/docs/chap-Metadata.html#tag-metadata_license
[5]:
https://docs.flathub.org/docs/for-app-authors/requirements#acceptable-but-should-be-submitted-upstream
[6]:
https://docs.flathub.org/docs/for-app-authors/submission/#theres-an-app-that-id-like-to-see-on-flathub-but-im-not-the-developer

Re: [cocci] Flatpak package for Coccinelle

[Julia Lawall]

On Sun, 23 Jun 2024, Jan Tojnar wrote:

> Hello Julia and everyone!
>
> To make Coccinelle more easily available to a wider range of people, I
> have decided to package it using Flatpak [1], a distribution-
> independent package manager backed by Freedesktop.org and preferred by
> many developers of Linux desktop environments like GNOME, KDE or
> Pantheon. This will be useful for situations where Flatpak is more
> available or familiar than OCaml toolchain, such as for most GNOME
> developers.
>
> Eventually, the goal is to have it available in the Flathub repository
> but there are several (numbered) questions to resolve before that can
> happen.
>
> First, each app requires an ID [2] in reverse-DNS format. I think
> `fr.inria.Coccinelle` would be nicest. But I have also seen
> `fr.lip6.Coccinelle` floating around but that might be older domain
> name. And following the guidelines literally (you should have control
> over the domain), it would probably have to be
> `fr.inria.gitlab.coccinelle.Coccinelle` or
> `fr.inria.gitlabpages.coccinelle.Coccinelle` but that feels over the
> top.
>
>     (1) Is `fr.inria.Coccinelle` acceptable?

Yes.  The lip6 association is indeed out of date.

>
> Second, uploading a package to Flathub requires an AppStream [3]
> manifest, which contains items such as textual description of the
> project, categories, authors, links and information about releases. I
> took the description from the Coccinelle website but I am not sure what
> the copyright status is:
>
> > Coccinelle is a program matching and transformation engine which
> provides the language SmPL (Semantic Patch Language) for specifying
> desired matches and transformations in C code. Coccinelle was initially
> targeted towards performing collateral evolutions in Linux. Such
> evolutions comprise the changes that are needed in client code in
> response to evolutions in library APIs, and may include modifications
> such as renaming a function, adding a function argument whose value is
> somehow context-dependent, and reorganizing a data structure. Beyond
> collateral evolutions, Coccinelle is successfully used (by us and
> others) for finding and fixing bugs in systems code.
>
> Flathub and distro tooling combine metadata of software projects in
> their repositories into indexes and other derivative databases so they
> require the AppStream metadata to be distributed under one of several
> licenses [4] vetted for mutual compatibility (e.g. Creative Commons,
> Public Domain, or GNU Free Documentation License).
>
>     (2) Could you clarify who the copyright holder of the above
> description is, and if it can be shared under one for the licenses
> listed in [4]?

I believe that I wrote the text.  Unfortunately, it predates the websute
git repository.  Do you have any suggestion about the licenses?


>
> Third, it is preferred [5] to ship the AppStream metadata upstream so
> that other distributions can use it. Shipping the metainfo in the
> Coccinelle repository would require updating the file with a new
> version and date before each release but that can be integrated into
> the existing maintenance version script. No problem if you do not want
> that, we can maintain it locally in a Flathub repository.
>
>     (3) Do you wish to include the AppStream metadata in the
> repository?

I would hesitate at the moment to touch the version release script if it
is not necessary.  Currently, we have no maintainer for this code.
Someone new will start working on Coccinelle in the fall, and perhaps this
person could help with this.  But for now it seems safest not to
complicate the release process.

>
> Fourth, the Flatpak manifest can be also included in the Git repository
> to allow building Coccinelle from the repo in a Flatpak environment.
> This would improve access to the development versions of Coccinelle
> but, since people who need those are probably more likely to be able to
> obtain OCaml toolchain themselves, it might not be worth the
> maintenance burden. I can help but I understand if this is not
> something you want to bother with.
>
>     (4) Do you wish to include Flatpak manifest in the repository?

What is the maintenance effort involved here?  Because it is often the
case that if someone reports a bug, then we fix it and hope that the
person will be able to use the source code from github.

> On that note, having the Flatpak manifest in the repo would also allow
> us to have GitHub/GitLab CI (Continuous Integration) environment
> automatically build latest version of Coccinelle and allow users to
> download it without having to build it, just like Flathub will enable
> that for stable versions of Coccinelle.
>
>     (5) Are you interested in such integration?

Sorry, I don't understand the relation between 4 and 5.  Is there one
thing to do with two benefits?

> Lastly, one of the goals of Flatpak is to have upstream developers
> publish their software using Flatpak themselves, essentially creating a
> singular distribution platform for Linux systems. It is recommended [6]
> that I ask you if you want to submit Coccinelle to Flathub yourself.
> But I understand that you are busy and I can maintain the package if
> you will not.
>
>     (6) Do you want to maintain Coccinelle package on Flathub yourself?

I would prefer not to.

>
> I have opened a draft PR on GitHub so that external reviewers can give
> input on the patches before submitting them here (if desired):
>
> https://github.com/coccinelle/coccinelle/pull/370

Thanks.  I don't know anything about Flatpak, but I hope that if others
use it they will give feedback.

Thanks for this effort!

julia

> I have also created a PR to add Coccinelle to Flathub:
>
> https://github.com/flathub/flathub/pull/5354
>
>
> Cheers,
>
> Jan
>
> [1]: https://en.wikipedia.org/wiki/Flatpak
> [2]:
> https://docs.flathub.org/docs/for-app-authors/requirements#application-id
> [3]: https://en.wikipedia.org/wiki/AppStream
> [4]:
> https://www.freedesktop.org/software/appstream/docs/chap-Metadata.html#tag-metadata_license
> [5]:
> https://docs.flathub.org/docs/for-app-authors/requirements#acceptable-but-should-be-submitted-upstream
> [6]:
> https://docs.flathub.org/docs/for-app-authors/submission/#theres-an-app-that-id-like-to-see-on-flathub-but-im-not-the-developer
>
>

Re: [cocci] Flatpak package for Coccinelle

[Jan Tojnar]

On Tue, 2024-06-25 at 17:46 +1000, Julia Lawall wrote:
> I believe that I wrote the text.  Unfortunately, it predates the
> websute
> git repository.  Do you have any suggestion about the licenses?

Any of the licenses acceptable for the AppStream metadata is fine:

https://www.freedesktop.org/software/appstream/docs/chap-Metadata.html#tag-metadata_license

Personally, I am fond of Creative Commons Attribution-ShareAlike for
websites.

Just like GNU GPL, it requires derived works to use the same license,
preventing the re-licensing to non-open-source license without the
consent of all contributors/copyright holders (though this is currently
trivial, if you are the sole author of the website).

https://creativecommons.org/licenses/by-sa/4.0/deed.en

CC licenses also have more approachable websites and I find the terms
more readable than GNU licenses – compare GNU Free Documentaion License
with the CC-BY-SA text:

https://www.gnu.org/licenses/fdl-1.3.html
https://creativecommons.org/licenses/by-sa/4.0/legalcode.en

Also it might depend on the agreement you have with your employer. I am
not a lawyer but I believe that typically, the company/university owns
the copyright (well economic rights, at least) to works you produced
“on their time” so they might need to agree to any relicensing.

It might be easier to write a new description if you are not sure about
the legal status.

> I would hesitate at the moment to touch the version release script if
> it
> is not necessary.  Currently, we have no maintainer for this code.
> Someone new will start working on Coccinelle in the fall, and perhaps
> this
> person could help with this.  But for now it seems safest not to
> complicate the release process.

If the script were not modified and we included the metainfo file in
the repository, we would need to update the file manually before each
release.

The procedure consists of updating the version and release date in the
metainfo file (the following line):

    <release version="1.2" date="2024-03-28" type="stable"/>

I would say it is easy to forget and pretty amenable to automation.

> 
> What is the maintenance effort involved here?  Because it is often
> the
> case that if someone reports a bug, then we fix it and hope that the
> person will be able to use the source code from github.

It is pretty much like maintaining a package for any other package
management system/Linux distribution.

- It can break when you break something in the code. This will likely
be also a problem on other platforms so you should fix it.
- It can break when you use a dependency that is not available in the
environment (e.g. require too new version of OCaml) or forget to bundle
a dependency (for simplicity, I made the manifest use bundled
dependencies instead of downloading them from opam). This can be a
problem if you e.g. wanted to use latest OCaml version. It would
require adjusting the manifest.
- It can break when you change the build system API. Currently, the
Flatpak manifest expects standard Autotools build and would need to be
adjusted if it changes.
- The Flatpak SDK or the OCaml extension can break. Hopefully, this
would be rare. In that case, the cause would need to be investigated
and reported.
- Once a year, one should bump Flatpak SDK version to get newer
versions of packages.

I can do the Flatpak-related maintenance for the foreseeable future but
I cannot guarantee timeliness. So I can imagine a situation when you
make a fix, start to rely on sending people Flatpak links to test
changes, only to find out that it is suddenly broken, and I am on
holidays in the Australian outback for a month.


> Sorry, I don't understand the relation between 4 and 5.  Is there one
> thing to do with two benefits?

(4) is prerequisite for (5).

(4) is already implemented in
https://github.com/coccinelle/coccinelle/pull/370

(4) requires Flatpak, flatpak-builder, cloning the Coccinelle
repository and then running a command to have flatpak-builder build
Coccinelle from the manifest.

(5) would just require Flatpak, the other steps would be done by
GitHub/GitLab CI, so user would run command like the following to
install a pre-built package:
https://github.com/flathub/flathub/pull/5354#issuecomment-2184988422

(5) would require introduction of GitHub/GitLab CI configuration file
into the repository and perhaps some configuration changes to
GitHub/GitLab repository (regarding storage of built assets).


Cheers,
Jan

Re: [cocci] Flatpak package for Coccinelle

[Victor Gambier]

Hi everyone,

As Julia mentioned above, I started working on Coccinelle earlier this 
fall. I'd love to help package Coccinelle into Flatpak! I've read the 
discussion above, but some time has passed since then and things might 
have changed, so, in short: how can I help? Are you still interested? 
What questions remain?

On 6/25/24 11:38, Jan Tojnar wrote:
> Personally, I am fond of Creative Commons Attribution-ShareAlike for
> websites.
>
> (...)
>
> Also it might depend on the agreement you have with your employer. I am
> not a lawyer but I believe that typically, the company/university owns
> the copyright (well economic rights, at least) to works you produced
> “on their time” so they might need to agree to any relicensing.
>
> It might be easier to write a new description if you are not sure about
> the legal status.
(2) Regarding the license, CC should be fine (in my opinion). You're 
right that this depends on the employer. From what I understand, at 
INRIA this question has occasionally come up in the last 2 years and we 
still don't have a clear answer. In the mean time, using the text as-is 
with the CC is fine.
> The procedure consists of updating the version and release date in the
> metainfo file (the following line):
>
>      <release version="1.2" date="2024-03-28" type="stable"/>
>
> I would say it is easy to forget and pretty amenable to automation.
(3) I can update the release script to update the version of the 
AppStream metadata, that does seem like the best solution.
> (4) is already implemented in
> https://github.com/coccinelle/coccinelle/pull/370
(4) If I understand correctly, you've already written the manifest, and 
this is something we only have to do once, right? If the license was the 
last thing you needed, feel free to mark your PR as ready and I'll 
review/merge it.
> (5) would require introduction of GitHub/GitLab CI configuration file
> into the repository and perhaps some configuration changes to
> GitHub/GitLab repository (regarding storage of built assets).
(5) That integration seems nice. I'd like to improve the way we do CI 
and releases, and this seems like a nice addition. We might have to 
think about whether or not master is, as it stands, the best branch for 
this. What would this look like in practice from the point of view of 
the user? Would they just download a .flatpak file from the 
GitHub/GitLab releases page?
> Cheers,
> Jan

Re: [cocci] Flatpak package for Coccinelle

[Jan Tojnar]

On Tue, 2024-11-05 at 15:34 +0100, Victor Gambier wrote:
> Hi everyone,
> 
> As Julia mentioned above, I started working on Coccinelle earlier
> this 
> fall.

Hi Victor, love to see more people working on Coccinelle.

>  I'd love to help package Coccinelle into Flatpak! I've read the
> discussion above, but some time has passed since then and things
> might 
> have changed, so, in short: how can I help? Are you still interested?
> What questions remain?

Unfortunately, I will not be able to drive this forward in the near
future. Feel free to take over.

The primary thing that was missing is the decision on the degree of
integration we want to have, i.e. points (3) through (5) in the opening
message.

Depending on that, we will want to
- merge https://github.com/coccinelle/coccinelle/pull/370
- update the release script to bump the version in the metainfo file
- add CI for building the flatpak image
- push stable version to Flathub – based on the previous PR
https://github.com/flathub/flathub/pull/5354

> > The procedure consists of updating the version and release date in
> > the
> > metainfo file (the following line):
> > 
> >      <release version="1.2" date="2024-03-28" type="stable"/>
> > 
> > I would say it is easy to forget and pretty amenable to automation.
> (3) I can update the release script to update the version of the 
> AppStream metadata, that does seem like the best solution.

👍

> > (4) is already implemented in
> > https://github.com/coccinelle/coccinelle/pull/370
> (4) If I understand correctly, you've already written the manifest,
> and 
> this is something we only have to do once, right?

Yes, the manifest is in the PR. It might need some minor changes in the
future, e.g. bumping the runtime version once a year:
https://freedesktop-sdk.gitlab.io/documentation/updating-sdk/release-notes/
But otherwise it should be pretty stable, as there are no other
dependencies tracked in Flatpak other than the OCaml runtime extension.

> If the license was the 
> last thing you needed, feel free to mark your PR as ready and I'll 
> review/merge it.

I have undrafted it, feel free to take a look.

> > (5) would require introduction of GitHub/GitLab CI configuration
> > file
> > into the repository and perhaps some configuration changes to
> > GitHub/GitLab repository (regarding storage of built assets).
> (5) That integration seems nice. I'd like to improve the way we do CI
> and releases, and this seems like a nice addition. We might have to 
> think about whether or not master is, as it stands, the best branch
> for 
> this. What would this look like in practice from the point of view of
> the user? Would they just download a .flatpak file from the 
> GitHub/GitLab releases page?
> 

Yes, CI would build a Flatpak package, user would download a .flatpak
file from the artifact list on the CI execution page, and install it
directly.

I imagine having it on master (or even temporary development branches)
would be useful for users to test fixes for issues they reported,
without needing the OCaml development environment.

For stable releases, other installation sources (e.g. from Flathub or
Linux distro repository) would probably be more suitable, since if I
understand it correctly, the .flatpak file is more akin to a .exe file
(no automatic updates).

Cheers,

Jan