[cocci] the future of bundles in coccinelle.git

[cocci] the future of bundles in coccinelle.git

[Olaf Hering]

[-- Attachment #1: Type: text/plain, Size: 484 bytes --]

While looking for the proper way to import pcre2, I realized that the
way configure.ac looks for required OCaml libraries is ... weird.
There is no way to force the usage of the variants in bundles/, the
version provided by ocamlfind is always preferred.

I think since a while the required build environment is easy to create
with either 'opam install $ALL_BUNDLES', or 'zypper install $packages'.

Hence I suggest to remove bundles/ and all code referring to it.


Olaf

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [cocci] the future of bundles in coccinelle.git

[Julia Lawall]

On Tue, 18 Mar 2025, Olaf Hering wrote:

> While looking for the proper way to import pcre2, I realized that the
> way configure.ac looks for required OCaml libraries is ... weird.
> There is no way to force the usage of the variants in bundles/, the
> version provided by ocamlfind is always preferred.
>
> I think since a while the required build environment is easy to create
> with either 'opam install $ALL_BUNDLES', or 'zypper install $packages'.
>
> Hence I suggest to remove bundles/ and all code referring to it.

No, we don't want to do that.  The idea of bundles is to limit the
dependencies.  We don't want users to have the be highly integrated into
the OCaml ecosystem.  Typical Coccinelle users don't care about OCaml at
all.

Perhaps it is just a personal preference, but when faced with some
software that eg asks me to install 20 python libraries, I choose some
other software.  I guess others may have the same reaction to software
that requires installing opam etc.

julia

Re: [cocci] the future of bundles in coccinelle.git

[Olaf Hering]

[-- Attachment #1: Type: text/plain, Size: 570 bytes --]

Tue, 18 Mar 2025 11:19:44 +0100 (CET) Julia Lawall <julia.lawall@inria.fr>:

> No, we don't want to do that.  The idea of bundles is to limit the
> dependencies.  We don't want users to have the be highly integrated into
> the OCaml ecosystem.  Typical Coccinelle users don't care about OCaml at
> all.

I guess they do not build from git, but do 'zypper install coccinelle'
and be done with it.

Everyone who is thinks the tools needs to be build from source has to make
sure everything required is available. Both zypper and opam make that easy.


Olaf

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [cocci] the future of bundles in coccinelle.git

[Jacob Keller]

On 3/18/2025 3:19 AM, Julia Lawall wrote:
> 
> 
> On Tue, 18 Mar 2025, Olaf Hering wrote:
> 
>> While looking for the proper way to import pcre2, I realized that the
>> way configure.ac looks for required OCaml libraries is ... weird.
>> There is no way to force the usage of the variants in bundles/, the
>> version provided by ocamlfind is always preferred.
>>
>> I think since a while the required build environment is easy to create
>> with either 'opam install $ALL_BUNDLES', or 'zypper install $packages'.
>>

>> Hence I suggest to remove bundles/ and all code referring to it.

Why not go the other route and optionally use the bundles?


> 
> No, we don't want to do that.  The idea of bundles is to limit the
> dependencies.  We don't want users to have the be highly integrated into
> the OCaml ecosystem.  Typical Coccinelle users don't care about OCaml at
> all.
> 
> Perhaps it is just a personal preference, but when faced with some
> software that eg asks me to install 20 python libraries, I choose some
> other software.  I guess others may have the same reaction to software
> that requires installing opam etc.
> 
> julia

It can be tricky if asked to install dozens of libraries which aren't
packaged within the distro I'm using. If they do come with the distro
repositories, then I am quite less concerned.

On the other hand I get a bit concerned if I see a couple dozen bundles
which I have no way to know if they stayed up to date with any recent
regressions or vulnerabilities.

That being said, I have had numerous issues in the past when building
coccinelle from source where things failed to compile because of changes
in the versions I had installed by distro vs what was expected. I am not
certain if these were real conflicts or just artifacts of an incorrect
build setup, but I remember my experience being frustrated quite often
any time I had to rebuild from scratch.

Re: [cocci] the future of bundles in coccinelle.git

[Julia Lawall]

The bundles are used if you don’t have the software.
Sent from my iPhone

> On 18 Mar 2025, at 22:53, Jacob Keller <jacob.e.keller@intel.com> wrote:
> 
> 
> 
>> On 3/18/2025 3:19 AM, Julia Lawall wrote:
>> 
>> 
>>> On Tue, 18 Mar 2025, Olaf Hering wrote:
>>> 
>>> While looking for the proper way to import pcre2, I realized that the
>>> way configure.ac looks for required OCaml libraries is ... weird.
>>> There is no way to force the usage of the variants in bundles/, the
>>> version provided by ocamlfind is always preferred.
>>> 
>>> I think since a while the required build environment is easy to create
>>> with either 'opam install $ALL_BUNDLES', or 'zypper install $packages'.
>>> 
> 
>>> Hence I suggest to remove bundles/ and all code referring to it.
> 
> Why not go the other route and optionally use the bundles?
> 
> 
>> 
>> No, we don't want to do that.  The idea of bundles is to limit the
>> dependencies.  We don't want users to have the be highly integrated into
>> the OCaml ecosystem.  Typical Coccinelle users don't care about OCaml at
>> all.
>> 
>> Perhaps it is just a personal preference, but when faced with some
>> software that eg asks me to install 20 python libraries, I choose some
>> other software.  I guess others may have the same reaction to software
>> that requires installing opam etc.
>> 
>> julia
> 
> It can be tricky if asked to install dozens of libraries which aren't
> packaged within the distro I'm using. If they do come with the distro
> repositories, then I am quite less concerned.
> 
> On the other hand I get a bit concerned if I see a couple dozen bundles
> which I have no way to know if they stayed up to date with any recent
> regressions or vulnerabilities.
> 
> That being said, I have had numerous issues in the past when building
> coccinelle from source where things failed to compile because of changes
> in the versions I had installed by distro vs what was expected. I am not
> certain if these were real conflicts or just artifacts of an incorrect
> build setup, but I remember my experience being frustrated quite often
> any time I had to rebuild from scratch.

Re: [cocci] the future of bundles in coccinelle.git

[Olaf Hering]

[-- Attachment #1: Type: text/plain, Size: 1029 bytes --]

Tue, 18 Mar 2025 14:52:42 -0700 Jacob Keller <jacob.e.keller@intel.com>:

> Why not go the other route and optionally use the bundles?

What is there actual purpose, what does (any) bundle actually fix?

As you said, every bundle becomes stale. The way it is implemented
means one can not even test them because the one from the environment
takes precedence.

If one wants coccinelle, use the package manager to install it.
In case it is unavailable there, get a copy of opam and install it.
In case one has to actually work on coccinelle.git for whatever reason,
there can not be any convenience, except 'opam install coccinelle'
which will install all required development packages.

In the context of Packman I had to deal with way too many packages
that bundle something which has to be provided by the system.
Every single one was and is a mess to work with. Today my first reaction
when I see any bundle is: purge that repo from disk and do something else.
Which is what I'm going to do now.


Olaf

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [cocci] the future of bundles in coccinelle.git

[Markus Elfring]

> In case one has to actually work on coccinelle.git for whatever reason,
> there can not be any convenience, except 'opam install coccinelle'
> which will install all required development packages.

Please reconsider such a view once more.

I can occasionally prefer other software development requirements.

Regards,
Markus

Re: [cocci] the future of bundles in coccinelle.git

[Julia Lawall]

On Wed, 19 Mar 2025, Markus Elfring wrote:

> > In case one has to actually work on coccinelle.git for whatever reason,
> > there can not be any convenience, except 'opam install coccinelle'
> > which will install all required development packages.
>
> Please reconsider such a view once more.
>
> I can occasionally prefer other software development requirements.

Opam is ocaml culture, just like pip is python culture, cargo is rust
culture, etc.  Fine if you want to be part of that culture.  Annoying if
you just want to use a tool.

julia

Re: [cocci] the future of bundles in coccinelle.git

[Victor Gambier]

For what it's worth, I am also of the opinion that we should get rid of 
the bundles.
They create issues, both on the developer side and the user side, and I 
don't think the problem they're supposed to solve is a real problem.
I don't like them very much. I also believe that building the Flatpak 
locally, which is now possible, is a better way of building Coccinelle 
without worrying about dependencies.

Victor

On 19/03/2025 13:48, Olaf Hering wrote:
> Tue, 18 Mar 2025 14:52:42 -0700 Jacob Keller <jacob.e.keller@intel.com>:
>
>> Why not go the other route and optionally use the bundles?
> What is there actual purpose, what does (any) bundle actually fix?
>
> As you said, every bundle becomes stale. The way it is implemented
> means one can not even test them because the one from the environment
> takes precedence.
>
> If one wants coccinelle, use the package manager to install it.
> In case it is unavailable there, get a copy of opam and install it.
> In case one has to actually work on coccinelle.git for whatever reason,
> there can not be any convenience, except 'opam install coccinelle'
> which will install all required development packages.
>
> In the context of Packman I had to deal with way too many packages
> that bundle something which has to be provided by the system.
> Every single one was and is a mess to work with. Today my first reaction
> when I see any bundle is: purge that repo from disk and do something else.
> Which is what I'm going to do now.
>
>
> Olaf

Re: [cocci] the future of bundles in coccinelle.git

[Julia Lawall]

On Wed, 19 Mar 2025, Victor Gambier wrote:

> For what it's worth, I am also of the opinion that we should get rid of the
> bundles.
> They create issues, both on the developer side and the user side, and I don't
> think the problem they're supposed to solve is a real problem.
> I don't like them very much. I also believe that building the Flatpak locally,
> which is now possible, is a better way of building Coccinelle without worrying
> about dependencies.

As much as I don't want to figure out how which version of pip is required
to install python packages, I absolutely do not want to learn flatpak.  I
don't want to learn anything.  I just want to download the tool, compile
it, and run it.  Avoiding cultural barriers to entry is really important
to getting people to use a tool.

julia

Re: [cocci] the future of bundles in coccinelle.git

[Victor Gambier]

On 19/03/2025 14:18, Julia Lawall wrote:
> As much as I don't want to figure out how which version of pip is required
> to install python packages, I absolutely do not want to learn flatpak.  I
> don't want to learn anything.  I just want to download the tool, compile
> it, and run it.  Avoiding cultural barriers to entry is really important
> to getting people to use a tool.

Understandable. I'll just point out that the bundles aren't a magic fix 
for dependencies - as I discovered (and since documented) in the past, 
on Debian, the pcre bundle only works if you have the libpcre3-dev 
package installed - this was not documented anywhere. So even with the 
bundles failsafe, you still need to install some (undocumented) packages 
in at least some instances, which may or may not defeat the purpose. 
Fully documenting which packages need to be installed and how to easily 
install them might be a more robust alternative in most cases :)

Victor

Re: [cocci] the future of bundles in coccinelle.git

[Julia Lawall]

On Wed, 19 Mar 2025, Victor Gambier wrote:

>
> On 19/03/2025 14:18, Julia Lawall wrote:
> > As much as I don't want to figure out how which version of pip is required
> > to install python packages, I absolutely do not want to learn flatpak.  I
> > don't want to learn anything.  I just want to download the tool, compile
> > it, and run it.  Avoiding cultural barriers to entry is really important
> > to getting people to use a tool.
>
> Understandable. I'll just point out that the bundles aren't a magic fix for
> dependencies - as I discovered (and since documented) in the past, on Debian,
> the pcre bundle only works if you have the libpcre3-dev package installed -
> this was not documented anywhere. So even with the bundles failsafe, you still
> need to install some (undocumented) packages in at least some instances, which
> may or may not defeat the purpose. Fully documenting which packages need to be
> installed and how to easily install them might be a more robust alternative in
> most cases :)

If installing a package is essential that should be documented,

But the goal is that the minimum number of packages should be required.

julia

Re: [cocci] the future of bundles in coccinelle.git

[Victor Gambier]

On 19/03/2025 14:33, Julia Lawall wrote:
> If installing a package is essential that should be documented,
>
> But the goal is that the minimum number of packages should be required.

I understand your point of view. If we don't want to rely on opam, and 
since we can't assume every distribution / package manager will package 
relatively obscure software such as stdcompat, our only options do 
indeed seem to be bundles or instructions on how to manually install the 
software first. So I do get it, even if the many distributions that 
package Coccinelle seem to manage it ok.

Victor

Re: [cocci] the future of bundles in coccinelle.git

[Julia Lawall]

On Wed, 19 Mar 2025, Olaf Hering wrote:

> Tue, 18 Mar 2025 14:52:42 -0700 Jacob Keller <jacob.e.keller@intel.com>:
>
> > Why not go the other route and optionally use the bundles?
>
> What is there actual purpose, what does (any) bundle actually fix?
>
> As you said, every bundle becomes stale. The way it is implemented
> means one can not even test them because the one from the environment
> takes precedence.
>
> If one wants coccinelle, use the package manager to install it.
> In case it is unavailable there, get a copy of opam and install it.
> In case one has to actually work on coccinelle.git for whatever reason,
> there can not be any convenience, except 'opam install coccinelle'
> which will install all required development packages.
>
> In the context of Packman I had to deal with way too many packages
> that bundle something which has to be provided by the system.
> Every single one was and is a mess to work with. Today my first reaction
> when I see any bundle is: purge that repo from disk and do something else.
> Which is what I'm going to do now.

OK. Thank you for your contribution.

julia

Re: [cocci] the future of bundles in coccinelle.git

[Markus Elfring]

> While looking for the proper way to import pcre2, I realized that the
> way configure.ac looks for required OCaml libraries is ... weird.

I find such a view interesting somehow.


> There is no way to force the usage of the variants in bundles/,
> the version provided by ocamlfind is always preferred.

There are remaining system configuration challenges to consider.
https://en.wikipedia.org/wiki/Product_bundling#Software


> I think since a while the required build environment is easy to create
> with either 'opam install $ALL_BUNDLES', or 'zypper install $packages'.

Corresponding software dependencies might trigger further concerns.


> Hence I suggest to remove bundles/ and all code referring to it.
I propose to take further case distinctions better into account
so that the involved software build system can become safer and more convenient.

Regards,
Markus