[cocci] spatch #include statements and include search path

[cocci] spatch #include statements and include search path

[Jani Nikula]

I want to use a non-trivial match rule in multiple other rules, in
multiple .cocci files. I'm trying put the match rule in a separate file,
and #include that in the other files, to avoid duplication. (I'm
referring to the "include_cocci ::= #include string" usage of #include,
not transformations.)

It works when the current working directory is the same as the file to
be included, but I seem to be unable to make it work from another
directory.

It doesn't matter if the .cocci files are all in the same directory or
not, it doesn't seem to matter what I pass as -I to spatch on the
command-line. I see #include mentioned in the SmPL Grammar as well as
some tutorials, but I can't find any proper, working examples for it.

How is the #include and the search path supposed to work? There seems to
be some conflation in the documentation about include_cocci in .cocci
and #includes in the .c files.

Specifically, I'm running spatch in my kernel repo, and I keep the
.cocci files in a separate directory outside the kernel repo.

TIA,
Jani.

-- 
Jani Nikula, Intel

Re: [cocci] spatch #include statements and include search path

[Julia Lawall]

On Mon, 13 Jan 2025, Jani Nikula wrote:

>
> I want to use a non-trivial match rule in multiple other rules, in
> multiple .cocci files. I'm trying put the match rule in a separate file,
> and #include that in the other files, to avoid duplication. (I'm
> referring to the "include_cocci ::= #include string" usage of #include,
> not transformations.)
>
> It works when the current working directory is the same as the file to
> be included, but I seem to be unable to make it work from another
> directory.
>
> It doesn't matter if the .cocci files are all in the same directory or
> not, it doesn't seem to matter what I pass as -I to spatch on the
> command-line. I see #include mentioned in the SmPL Grammar as well as
> some tutorials, but I can't find any proper, working examples for it.
>
> How is the #include and the search path supposed to work? There seems to
> be some conflation in the documentation about include_cocci in .cocci
> and #includes in the .c files.
>
> Specifically, I'm running spatch in my kernel repo, and I keep the
> .cocci files in a separate directory outside the kernel repo.

I don't think there is any search path for .cocci files.

I see an example in tests/gadget_soa_defs.cocci that contains

#include "tests/gadget_match.cocci"

There was a problem about the files coming in the wrong order in the case
of multiple included files.  I've pushed the recent changes out to github
just now, so that fix should be there.

julia

Re: [cocci] spatch #include statements and include search path

[Jani Nikula]

On Mon, 13 Jan 2025, Julia Lawall <julia.lawall@inria.fr> wrote:
> On Mon, 13 Jan 2025, Jani Nikula wrote:
>
>>
>> I want to use a non-trivial match rule in multiple other rules, in
>> multiple .cocci files. I'm trying put the match rule in a separate file,
>> and #include that in the other files, to avoid duplication. (I'm
>> referring to the "include_cocci ::= #include string" usage of #include,
>> not transformations.)
>>
>> It works when the current working directory is the same as the file to
>> be included, but I seem to be unable to make it work from another
>> directory.
>>
>> It doesn't matter if the .cocci files are all in the same directory or
>> not, it doesn't seem to matter what I pass as -I to spatch on the
>> command-line. I see #include mentioned in the SmPL Grammar as well as
>> some tutorials, but I can't find any proper, working examples for it.
>>
>> How is the #include and the search path supposed to work? There seems to
>> be some conflation in the documentation about include_cocci in .cocci
>> and #includes in the .c files.
>>
>> Specifically, I'm running spatch in my kernel repo, and I keep the
>> .cocci files in a separate directory outside the kernel repo.
>
> I don't think there is any search path for .cocci files.

Oh, so it's always relative to the current working directory? That's a
bit of a bummer.

Thanks, though.

BR,
Jani.


>
> I see an example in tests/gadget_soa_defs.cocci that contains
>
> #include "tests/gadget_match.cocci"
>
> There was a problem about the files coming in the wrong order in the case
> of multiple included files.  I've pushed the recent changes out to github
> just now, so that fix should be there.
>
> julia

-- 
Jani Nikula, Intel

Re: [cocci] spatch #include statements and include search path

[Julia Lawall]

On Mon, 13 Jan 2025, Jani Nikula wrote:

> On Mon, 13 Jan 2025, Julia Lawall <julia.lawall@inria.fr> wrote:
> > On Mon, 13 Jan 2025, Jani Nikula wrote:
> >
> >>
> >> I want to use a non-trivial match rule in multiple other rules, in
> >> multiple .cocci files. I'm trying put the match rule in a separate file,
> >> and #include that in the other files, to avoid duplication. (I'm
> >> referring to the "include_cocci ::= #include string" usage of #include,
> >> not transformations.)
> >>
> >> It works when the current working directory is the same as the file to
> >> be included, but I seem to be unable to make it work from another
> >> directory.
> >>
> >> It doesn't matter if the .cocci files are all in the same directory or
> >> not, it doesn't seem to matter what I pass as -I to spatch on the
> >> command-line. I see #include mentioned in the SmPL Grammar as well as
> >> some tutorials, but I can't find any proper, working examples for it.
> >>
> >> How is the #include and the search path supposed to work? There seems to
> >> be some conflation in the documentation about include_cocci in .cocci
> >> and #includes in the .c files.
> >>
> >> Specifically, I'm running spatch in my kernel repo, and I keep the
> >> .cocci files in a separate directory outside the kernel repo.
> >
> > I don't think there is any search path for .cocci files.
>
> Oh, so it's always relative to the current working directory? That's a
> bit of a bummer.

It's not a well developed feature.  If an include path for it would be
useful, we can add that.

julia

>
> Thanks, though.
>
> BR,
> Jani.
>
>
> >
> > I see an example in tests/gadget_soa_defs.cocci that contains
> >
> > #include "tests/gadget_match.cocci"
> >
> > There was a problem about the files coming in the wrong order in the case
> > of multiple included files.  I've pushed the recent changes out to github
> > just now, so that fix should be there.
> >
> > julia
>
> --
> Jani Nikula, Intel
>

Re: [cocci] spatch #include statements and include search path

[Jani Nikula]

On Mon, 13 Jan 2025, Julia Lawall <julia.lawall@inria.fr> wrote:
> On Mon, 13 Jan 2025, Jani Nikula wrote:
>
>> On Mon, 13 Jan 2025, Julia Lawall <julia.lawall@inria.fr> wrote:
>> > I don't think there is any search path for .cocci files.
>>
>> Oh, so it's always relative to the current working directory? That's a
>> bit of a bummer.
>
> It's not a well developed feature.  If an include path for it would be
> useful, we can add that.

It might be. Or do you have any other ideas for reusing match rules or,
say, "iterator name" lists, without duplicating them into all .cocci
files?

So I guess I can try working around not having include search path by
running spatch outside of my kernel repo. Does spatch otherwise handle
that scenario well, or any caveats there?


BR,
Jani.


-- 
Jani Nikula, Intel

Re: [cocci] spatch #include statements and include search path

[Julia Lawall]

On Mon, 13 Jan 2025, Jani Nikula wrote:

> On Mon, 13 Jan 2025, Julia Lawall <julia.lawall@inria.fr> wrote:
> > On Mon, 13 Jan 2025, Jani Nikula wrote:
> >
> >> On Mon, 13 Jan 2025, Julia Lawall <julia.lawall@inria.fr> wrote:
> >> > I don't think there is any search path for .cocci files.
> >>
> >> Oh, so it's always relative to the current working directory? That's a
> >> bit of a bummer.
> >
> > It's not a well developed feature.  If an include path for it would be
> > useful, we can add that.
>
> It might be. Or do you have any other ideas for reusing match rules or,
> say, "iterator name" lists, without duplicating them into all .cocci
> files?
>
> So I guess I can try working around not having include search path by
> running spatch outside of my kernel repo. Does spatch otherwise handle
> that scenario well, or any caveats there?

It should be fine.  I always have my semantic patches in another place.

spatch sp.cocci linuxpath

Not sure about make coccicheck though.

julia

>
>
> BR,
> Jani.
>
>
> --
> Jani Nikula, Intel
>

Re: [cocci] spatch #include statements and include search path

[Victor Gambier]

Hi Jani,

Could you give me a precise example of what you're trying to do? The 
following works on my end (commit 8aa44ca):

.
├── A
│   ├── cocciincludemain.c
│   └── cocciincludemain.cocci
└── B
     └── cocciinclude1.cocci

(...)/A $ spatch --sp-file ./cocciincludemain.cocci ./cocciincludemain.c

Absolute and relative includes in cocciincludemain.cocci both work:

#include 
"/home/vgambier/Documents/git/coccinelle/playground/B/cocciinclude1.cocci"

#include "../B/cocciinclude1.cocci"

Thanks,
Victor

On 13/01/2025 17:59, Jani Nikula wrote:
> On Mon, 13 Jan 2025, Julia Lawall <julia.lawall@inria.fr> wrote:
>> On Mon, 13 Jan 2025, Jani Nikula wrote:
>>
>>> On Mon, 13 Jan 2025, Julia Lawall <julia.lawall@inria.fr> wrote:
>>>> I don't think there is any search path for .cocci files.
>>> Oh, so it's always relative to the current working directory? That's a
>>> bit of a bummer.
>> It's not a well developed feature.  If an include path for it would be
>> useful, we can add that.
> It might be. Or do you have any other ideas for reusing match rules or,
> say, "iterator name" lists, without duplicating them into all .cocci
> files?
>
> So I guess I can try working around not having include search path by
> running spatch outside of my kernel repo. Does spatch otherwise handle
> that scenario well, or any caveats there?
>
>
> BR,
> Jani.
>
>

Re: [cocci] spatch #include statements and include search path

[Jani Nikula]

On Wed, 22 Jan 2025, Victor Gambier <victor.gambier@inria.fr> wrote:
> Hi Jani,
>
> Could you give me a precise example of what you're trying to do? The 
> following works on my end (commit 8aa44ca):
>
> .
> ├── A
> │   ├── cocciincludemain.c
> │   └── cocciincludemain.cocci
> └── B
>      └── cocciinclude1.cocci
>
> (...)/A $ spatch --sp-file ./cocciincludemain.cocci ./cocciincludemain.c
>
> Absolute and relative includes in cocciincludemain.cocci both work:
>
> #include 
> "/home/vgambier/Documents/git/coccinelle/playground/B/cocciinclude1.cocci"
>
> #include "../B/cocciinclude1.cocci"

This is what I'm trying to do:

+- A
|  +- main.c
+- B
   +- main.cocci
   +- include.cocci

$ cd A
$ spatch --sp-file ../B/main.cocci main.c

With main.cocci having:

#include "include.cocci"

Currently spatch looks for the include in the CWD only, not next to the
.cocci being processed, nor does -I help in specifying where to look for
the include.

BR,
Jani.

>
> Thanks,
> Victor
>
> On 13/01/2025 17:59, Jani Nikula wrote:
>> On Mon, 13 Jan 2025, Julia Lawall <julia.lawall@inria.fr> wrote:
>>> On Mon, 13 Jan 2025, Jani Nikula wrote:
>>>
>>>> On Mon, 13 Jan 2025, Julia Lawall <julia.lawall@inria.fr> wrote:
>>>>> I don't think there is any search path for .cocci files.
>>>> Oh, so it's always relative to the current working directory? That's a
>>>> bit of a bummer.
>>> It's not a well developed feature.  If an include path for it would be
>>> useful, we can add that.
>> It might be. Or do you have any other ideas for reusing match rules or,
>> say, "iterator name" lists, without duplicating them into all .cocci
>> files?
>>
>> So I guess I can try working around not having include search path by
>> running spatch outside of my kernel repo. Does spatch otherwise handle
>> that scenario well, or any caveats there?
>>
>>
>> BR,
>> Jani.
>>
>>

-- 
Jani Nikula, Intel

Re: [cocci] [cocci-mirror] spatch #include statements and include search path

[Victor Gambier]

The issue should be fixed in commit fb9a33. The idea is if we can't find 
the included cocci file in the C directory, then we start looking for it 
in the cocci directory.

Thanks for the report!
Victor