[Cocci] Improvements for source code analysis with inclusion of header files

[Cocci] Improvements for source code analysis with inclusion of header files

[SF Markus Elfring]

Hello,

Some filter patterns can be specified in the semantic patch language so that
they will also work without inclusion of header files. Some other source code
analysis approaches will need all relevant data (e. g. macros and type
declarations) from them.

Various compilers provide a functionality to store processed information from
such headers in an alternative data format.
http://en.wikipedia.org/wiki/Precompiled_header

Would another intermediate form help to reduce analysis and transformation time
by the Coccinelle software here?

How do you think about to store the results from a step "precompilation" like
corresponding abstract syntax graphs in a kind of dedicated data base?
http://en.wikipedia.org/wiki/Graph_database#Graph_database_projects

Regards,
Markus

[Cocci] Improvements for source code analysis with inclusion of header files

[Julia Lawall]

On Sun, 30 Mar 2014, SF Markus Elfring wrote:

> Hello,
>
> Some filter patterns can be specified in the semantic patch language so that
> they will also work without inclusion of header files. Some other source code
> analysis approaches will need all relevant data (e. g. macros and type
> declarations) from them.
>
> Various compilers provide a functionality to store processed information from
> such headers in an alternative data format.
> http://en.wikipedia.org/wiki/Precompiled_header
>
> Would another intermediate form help to reduce analysis and transformation time
> by the Coccinelle software here?
>
> How do you think about to store the results from a step "precompilation" like
> corresponding abstract syntax graphs in a kind of dedicated data base?
> http://en.wikipedia.org/wiki/Graph_database#Graph_database_projects

Coccinelle already allows you to cache abstract syntax trees on disk.
Relevant command line arguments are --use-cache and --cache-limit.

julia

[Cocci] Improvements for source code analysis with inclusion of header files

[SF Markus Elfring]

> Coccinelle already allows you to cache abstract syntax trees on disk.
> Relevant command line arguments are --use-cache and --cache-limit.

Will any more documentation become available than a terse option description
like "Use preparsed versions of the C files that are stored in a cache."?

Regards,
Markus

[Cocci] Improvements for source code analysis with inclusion of header files

[Julia Lawall]

On Sun, 30 Mar 2014, SF Markus Elfring wrote:

> > Coccinelle already allows you to cache abstract syntax trees on disk.
> > Relevant command line arguments are --use-cache and --cache-limit.
>
> Will any more documentation become available than a terse option description
> like "Use preparsed versions of the C files that are stored in a cache."?

Not sure what more documentation one would want.

julia

[Cocci] Improvements for source code analysis with inclusion of header files

[SF Markus Elfring]

> Not sure what more documentation one would want.

I would find a clarification nice for details like the following.

1. Which "C files" are referenced here?

2. File formats for the cache

3. Explanation for abstract syntax trees in this context

Regards,
Markus

[Cocci] Improvements for source code analysis with inclusion of header files

[Julia Lawall]

On Sun, 30 Mar 2014, SF Markus Elfring wrote:

> > Not sure what more documentation one would want.
>
> I would find a clarification nice for details like the following.
>
> 1. Which "C files" are referenced here?

The ones Coccinelle was asked to work on.

> 2. File formats for the cache

Doesn't matter to the user.

> 3. Explanation for abstract syntax trees in this context

Likewise.

The cache is something that Coccinelle writes into and reads out of.  Only
Coccinelle cares what that format is.

julia

[Cocci] Improvements for source code analysis with inclusion of header files

[SF Markus Elfring]

>> 1. Which "C files" are referenced here?
> 
> The ones Coccinelle was asked to work on.

Do you distinguish headers from source files in this use case?


>> 2. File formats for the cache
> 
> Doesn't matter to the user.

I guess that such information will become more interesting for other software
developers, won't it?


>> 3. Explanation for abstract syntax trees in this context
> 
> Likewise.
> 
> The cache is something that Coccinelle writes into and reads out of.  Only
> Coccinelle cares what that format is.

I would appreciate if more details could be eventually documented in an other
format than the original OCaml source code.

Regards,
Markus

[Cocci] Improvements for source code analysis with inclusion of header files

[Julia Lawall]

On Sun, 30 Mar 2014, SF Markus Elfring wrote:

> >> 1. Which "C files" are referenced here?
> >
> > The ones Coccinelle was asked to work on.
>
> Do you distinguish headers from source files in this use case?

No that I recall.

> >> 2. File formats for the cache
> >
> > Doesn't matter to the user.
>
> I guess that such information will become more interesting for other software
> developers, won't it?

I don't think so.  No more than any ordinary user is interested in the
format that a web browser uses for its cache.

In general, a cache has a short lifetime.  The software changes, or one
wants to use Coccinelle with other options, and then one would rather
start over with a new cache.  The cache is mostly useful when one does
iteration.  In that case, I usually limit the size of the cache as well,
because the format is not particularly concise.  This further implies that
it is not useful to anything other than the current run.

julia

[Cocci] Improvements for source code analysis with inclusion of header files

[SF Markus Elfring]

>> Do you distinguish headers from source files in this use case?
> 
> No that I recall.

Would you like to support any variant of precompiled header files?


> The software changes, or one wants to use Coccinelle with other options,
> and then one would rather start over with a new cache.

I imagine that caches can also be generated for data representations with a
longer "life time".


> In that case, I usually limit the size of the cache as well, because the format
> is not particularly concise.  This further implies that it is not useful
> to anything other than the current run.

I am becoming a bit more interested in conciseness for relevant data structures.
How can it be achieved to make the intermediate data storage useful for several
different source code analysis runs?

Regards,
Markus

[Cocci] Improvements for source code analysis with inclusion of header files

[Derek M Jones]

Julia,

> Not sure what more documentation one would want.

Coccinelle is getting to be widely used.

When is the O'Reilly book coming out :-)

-- 
Derek M. Jones                  tel: +44 (0) 1252 520 667
Knowledge Software Ltd          blog:shape-of-code.coding-guidelines.com
Software analysis               http://www.knosof.co.uk

[Cocci] Improvements for source code analysis with inclusion of header files

[Julia Lawall]

On Sun, 30 Mar 2014, Derek M Jones wrote:

> Julia,
>
> > Not sure what more documentation one would want.
>
> Coccinelle is getting to be widely used.
>
> When is the O'Reilly book coming out :-)

I think it will coincide with the release of 1.0.0 :)

julia