[cocci] Is there a --no-includes --but-includet-this foo.h --but-in[...] mode?

[cocci] Is there a --no-includes --but-includet-this foo.h --but-in[...] mode?

[Ævar Arnfjörð Bjarmason]

[First time poster, long time spatch user, thanks a lot for providing
it!]

I'm trying to optimize the runtime of spatch in git.git by leveraging
information that's already in the Makefile. Both GCC and Clang know how
to spew out information about what headers a given file depends on in
"Makefile" format, e.g. for a "hook.c" in git.git:
    
    $ cat .depend/hook.o.d 
    hook.o: hook.c cache.h git-compat-util.h compat/bswap.h wildmatch.h \
     banned.h strbuf.h hashmap.h hash.h repository.h path.h sha1dc_git.h \
     sha1collisiondetection/lib/sha1.h sha256/block/sha256.h list.h advice.h \
     gettext.h convert.h string-list.h trace.h trace2.h pack-revindex.h \
     oid-array.h mem-pool.h hook.h run-command.h thread-utils.h strvec.h
    [...]

I wanted to feed that into spatch somehow, but with e.g. --include it
will not include anything:

    ~/g/coccinelle/spatch.opt --sp-file contrib/coccinelle/strbuf.cocci --verbose-includes --no-includes --include cache.h --include path.h hook.c
    init_defs_builtins: /home/avar/g/coccinelle/standard.h
    HANDLING: hook.c

It seems that there's --recursive-includes, --all-includes,
--no-includes etc, but those all tweak different aspects of
auto-discover, or "seed" the auto-discovery via --include.

But I'd like a way to say "here, exhaustively, are my includes", does it
have such a mode? If not consider this a feature request.

Even better on top would be to further optionally tell it due die if it
ever encounters an include file that's not explicitly provided on the
command-line.

That would serve as a nice sanity check on the *.d files we get from
gcc/clang, and make sure that this way of optimizing its runtime works
as intended. I was planning to do that by parsing its output, but having
it as a native option would be easier.

Note: I probably don't *really* need this, since I can look at the *.d
file myself and have "make" determine if spatch needs to run at all, and
I trust the gcc/clang information well enough.

But being able to have spatch use that information exclusively would
also be nice, especially for the above mentioned sanity check.

Thanks!

Re: [cocci] Is there a --no-includes --but-includet-this foo.h --but-in[...] mode?

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 2604 bytes --]



On Tue, 16 Nov 2021, Ævar Arnfjörð Bjarmason wrote:

> [First time poster, long time spatch user, thanks a lot for providing
> it!]
>
> I'm trying to optimize the runtime of spatch in git.git by leveraging
> information that's already in the Makefile. Both GCC and Clang know how
> to spew out information about what headers a given file depends on in
> "Makefile" format, e.g. for a "hook.c" in git.git:
>
>     $ cat .depend/hook.o.d
>     hook.o: hook.c cache.h git-compat-util.h compat/bswap.h wildmatch.h \
>      banned.h strbuf.h hashmap.h hash.h repository.h path.h sha1dc_git.h \
>      sha1collisiondetection/lib/sha1.h sha256/block/sha256.h list.h advice.h \
>      gettext.h convert.h string-list.h trace.h trace2.h pack-revindex.h \
>      oid-array.h mem-pool.h hook.h run-command.h thread-utils.h strvec.h
>     [...]
>
> I wanted to feed that into spatch somehow, but with e.g. --include it
> will not include anything:

How did you identify that nothing was included with --include?  --include
is supposed to allow you to specify files that should be included
regardless of the #includes found in the file.  On the other hand, there
is nothing to make a list of includes be considered to be the only
includes that are allowed as you suggest.

julia

>
>     ~/g/coccinelle/spatch.opt --sp-file contrib/coccinelle/strbuf.cocci --verbose-includes --no-includes --include cache.h --include path.h hook.c
>     init_defs_builtins: /home/avar/g/coccinelle/standard.h
>     HANDLING: hook.c
>
> It seems that there's --recursive-includes, --all-includes,
> --no-includes etc, but those all tweak different aspects of
> auto-discover, or "seed" the auto-discovery via --include.
>
> But I'd like a way to say "here, exhaustively, are my includes", does it
> have such a mode? If not consider this a feature request.
>
> Even better on top would be to further optionally tell it due die if it
> ever encounters an include file that's not explicitly provided on the
> command-line.
>
> That would serve as a nice sanity check on the *.d files we get from
> gcc/clang, and make sure that this way of optimizing its runtime works
> as intended. I was planning to do that by parsing its output, but having
> it as a native option would be easier.
>
> Note: I probably don't *really* need this, since I can look at the *.d
> file myself and have "make" determine if spatch needs to run at all, and
> I trust the gcc/clang information well enough.
>
> But being able to have spatch use that information exclusively would
> also be nice, especially for the above mentioned sanity check.
>
> Thanks!
>

Re: [cocci] Is there a --no-includes --but-includet-this foo.h --but-in[...] mode?

[Ævar Arnfjörð Bjarmason]

On Tue, Nov 16 2021, Julia Lawall wrote:

> On Tue, 16 Nov 2021, Ævar Arnfjörð Bjarmason wrote:
>
>> [First time poster, long time spatch user, thanks a lot for providing
>> it!]
>>
>> I'm trying to optimize the runtime of spatch in git.git by leveraging
>> information that's already in the Makefile. Both GCC and Clang know how
>> to spew out information about what headers a given file depends on in
>> "Makefile" format, e.g. for a "hook.c" in git.git:
>>
>>     $ cat .depend/hook.o.d
>>     hook.o: hook.c cache.h git-compat-util.h compat/bswap.h wildmatch.h \
>>      banned.h strbuf.h hashmap.h hash.h repository.h path.h sha1dc_git.h \
>>      sha1collisiondetection/lib/sha1.h sha256/block/sha256.h list.h advice.h \
>>      gettext.h convert.h string-list.h trace.h trace2.h pack-revindex.h \
>>      oid-array.h mem-pool.h hook.h run-command.h thread-utils.h strvec.h
>>     [...]
>>
>> I wanted to feed that into spatch somehow, but with e.g. --include it
>> will not include anything:
>
> How did you identify that nothing was included with --include?  --include
> is supposed to allow you to specify files that should be included
> regardless of the #includes found in the file.

From strace-ing it it looks like it is including those files I asked for
with just --include, but it doesn't print out an "including" message
like in the other modes, so I assumed it was simply ignoring them.

But it looks like it's working!

> On the other hand, there is nothing to make a list of includes be
> considered to be the only includes that are allowed as you suggest.

For what it's worth I had in mind (presumably simpler)
--check-include-exhaustiveness or whatever, i.e. it wouldn't try to
generate a full list of includes, but just error out if it ever ran
across an "#include" that I hadn't specifiec.

But perhaps for that to be truly useful it would need to at least go far
enough to check if that not-listed include was a local or a non-local
one.

In any case, I could just do that myself by running it twice in some
assertion mode & parsing its output, thanks!

>>
>>     ~/g/coccinelle/spatch.opt --sp-file contrib/coccinelle/strbuf.cocci --verbose-includes --no-includes --include cache.h --include path.h hook.c
>>     init_defs_builtins: /home/avar/g/coccinelle/standard.h
>>     HANDLING: hook.c
>>
>> It seems that there's --recursive-includes, --all-includes,
>> --no-includes etc, but those all tweak different aspects of
>> auto-discover, or "seed" the auto-discovery via --include.
>>
>> But I'd like a way to say "here, exhaustively, are my includes", does it
>> have such a mode? If not consider this a feature request.
>>
>> Even better on top would be to further optionally tell it due die if it
>> ever encounters an include file that's not explicitly provided on the
>> command-line.
>>
>> That would serve as a nice sanity check on the *.d files we get from
>> gcc/clang, and make sure that this way of optimizing its runtime works
>> as intended. I was planning to do that by parsing its output, but having
>> it as a native option would be easier.
>>
>> Note: I probably don't *really* need this, since I can look at the *.d
>> file myself and have "make" determine if spatch needs to run at all, and
>> I trust the gcc/clang information well enough.
>>
>> But being able to have spatch use that information exclusively would
>> also be nice, especially for the above mentioned sanity check.
>>
>> Thanks!
>>

Re: [cocci] Is there a --no-includes --but-includet-this foo.h --but-in[...] mode?

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 3858 bytes --]



On Tue, 16 Nov 2021, Ævar Arnfjörð Bjarmason wrote:

>
> On Tue, Nov 16 2021, Julia Lawall wrote:
>
> > On Tue, 16 Nov 2021, Ævar Arnfjörð Bjarmason wrote:
> >
> >> [First time poster, long time spatch user, thanks a lot for providing
> >> it!]
> >>
> >> I'm trying to optimize the runtime of spatch in git.git by leveraging
> >> information that's already in the Makefile. Both GCC and Clang know how
> >> to spew out information about what headers a given file depends on in
> >> "Makefile" format, e.g. for a "hook.c" in git.git:
> >>
> >>     $ cat .depend/hook.o.d
> >>     hook.o: hook.c cache.h git-compat-util.h compat/bswap.h wildmatch.h \
> >>      banned.h strbuf.h hashmap.h hash.h repository.h path.h sha1dc_git.h \
> >>      sha1collisiondetection/lib/sha1.h sha256/block/sha256.h list.h advice.h \
> >>      gettext.h convert.h string-list.h trace.h trace2.h pack-revindex.h \
> >>      oid-array.h mem-pool.h hook.h run-command.h thread-utils.h strvec.h
> >>     [...]
> >>
> >> I wanted to feed that into spatch somehow, but with e.g. --include it
> >> will not include anything:
> >
> > How did you identify that nothing was included with --include?  --include
> > is supposed to allow you to specify files that should be included
> > regardless of the #includes found in the file.
>
> From strace-ing it it looks like it is including those files I asked for
> with just --include, but it doesn't print out an "including" message
> like in the other modes, so I assumed it was simply ignoring them.
>
> But it looks like it's working!

I think that the message is part of the inference process.

>
> > On the other hand, there is nothing to make a list of includes be
> > considered to be the only includes that are allowed as you suggest.
>
> For what it's worth I had in mind (presumably simpler)
> --check-include-exhaustiveness or whatever, i.e. it wouldn't try to
> generate a full list of includes, but just error out if it ever ran
> across an "#include" that I hadn't specifiec.
>
> But perhaps for that to be truly useful it would need to at least go far
> enough to check if that not-listed include was a local or a non-local
> one.
>
> In any case, I could just do that myself by running it twice in some
> assertion mode & parsing its output, thanks!

I'm not really sure that this is Coccinelle's problem.  You have the list
of inferred includes with the --verbose-includes option.

julia

> >>
> >>     ~/g/coccinelle/spatch.opt --sp-file contrib/coccinelle/strbuf.cocci --verbose-includes --no-includes --include cache.h --include path.h hook.c
> >>     init_defs_builtins: /home/avar/g/coccinelle/standard.h
> >>     HANDLING: hook.c
> >>
> >> It seems that there's --recursive-includes, --all-includes,
> >> --no-includes etc, but those all tweak different aspects of
> >> auto-discover, or "seed" the auto-discovery via --include.
> >>
> >> But I'd like a way to say "here, exhaustively, are my includes", does it
> >> have such a mode? If not consider this a feature request.
> >>
> >> Even better on top would be to further optionally tell it due die if it
> >> ever encounters an include file that's not explicitly provided on the
> >> command-line.
> >>
> >> That would serve as a nice sanity check on the *.d files we get from
> >> gcc/clang, and make sure that this way of optimizing its runtime works
> >> as intended. I was planning to do that by parsing its output, but having
> >> it as a native option would be easier.
> >>
> >> Note: I probably don't *really* need this, since I can look at the *.d
> >> file myself and have "make" determine if spatch needs to run at all, and
> >> I trust the gcc/clang information well enough.
> >>
> >> But being able to have spatch use that information exclusively would
> >> also be nice, especially for the above mentioned sanity check.
> >>
> >> Thanks!
> >>
>
>