[Cocci] version 1.0.3 released

[Cocci] version 1.0.3 released

[Julia Lawall]

Highlights:

* Support for people who don't like tabs
* .cocciconfig configuration file
* Statement list metavariables
* Default values for script metavariables
* Conjunctions

------------------------------
Details:

* OCaml 3.11 is no longer supported

* A new option --indent is added, giving the default number of spaces to 
indent when Coccinelle is no able to infer anything from the context.  
This is for use with software that does not use tabs for indenting.

* It is now possible to make a .cocciconfig configuration file.  This is 
quite minimalistic.  There should be a line containing [spatch] followed 
by any number of lines that contain, eg

options = --some --options

The options accumulate.  They should have the same form as they do on the 
command line.  .cocciconfig files can be placed in the user's home 
directory, the directory from which spatch is called, and the directory 
provided with the --dir option.  The .cocciconfig file in the user's home 
directory is processed first, the .cocciconfig file in the directory from 
which spatch is called is processed next, and the .cocciconfig file in the 
directory provided with the --dir option is processed last.  In each case, 
the read options extend/override the previously read ones.  In all cases, 
the user can extend/override the options found in the .cocciconfig files 
on the command line.

* --timeout 0 means no timeout.  This can be useful for overriding a timeout 
in a .cocciconfig file.

* The option --print-options-only makes it possible to see the accumulated 
list of options, including the options obtained from the .cocciconfig 
file.

* It is now possible to use a statement list metavariable to match the 
complete series of statements from the beginning to the end of a block.  
This can be helpful in moving a function definition from one place to 
another.

* It is now possible to provide default value for script metavariables, as 
illustrated by the following example:

f << r.f = "no function";

For a position variable, the only default value is the empty list, ie:

p << r.p = [];

Previously a script would only be executed if values for all of the 
metavariables were provided.  Now it is only the metavariables that do not 
have default values that are needed.

This is illustrated in demos/defaultscript.cocci

* There is now support for conjunctions ( & ... & ).  As for a 
disjunction, the () and & should be in column 0.  A conjunction requires 
two patterns to match against a given piece of code in a consistent way.  
It may help reduce the need for using position variables to rematch things 
in multiple rules.  An example is in demos/conjunction.cocci.

* The dependency on ocamlp4 is removed, except when support for python is 
included.

julia

[Cocci] version 1.0.3 released

[Sébastien Hinderer]

Hello,

Julia Lawall (2015/10/26 22:12 +0100):
> * The dependency on ocamlp4 is removed, except when support for python is 
> included.

Read camlp4 rather than ocamlp4.
To be more precise, Coccinelle relies on a library called pycaml to
support Python scripts. camlp4 is required only to compile this library.
So if the library has been installed independently, e.g. through the
distribution's packaging system (as can be done on Debian since pycaml
is packaed), then camlp4 is not required. If the library has not been
installed and Python scripting support is requested, then Coccinelle's
build systemwill attempt to compile the bundled version of the camlp4
library, which will require camlp4.

S?bastien.

[Cocci] version 1.0.3 released

[Richard W.M. Jones]

This version seems to be incompatible with the latest menhir.  In
Fedora we have ocaml-menhir-20151012-1.fc24.x86_64, and I get
this error when compiling:

/usr/bin/ocamlc.opt -unsafe -I ../commons -I ../commons/ocamlextra -I ../globals -I /usr/lib64/ocaml/menhirLib -c parser_cocci_menhir.mli
/usr/bin/ocamlopt.opt -unsafe -I ../commons -I ../commons/ocamlextra -I ../globals -I /usr/lib64/ocaml/menhirLib -c parser_cocci_menhir.ml
File "parser_cocci_menhir.ml", line 1855, characters 15-48:
Error: Unbound constructor MenhirLib.TableInterpreter.Accept
Makefile:90: recipe for target 'parser_cocci_menhir.cmx' failed
make[6]: *** [parser_cocci_menhir.cmx] Error 2

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

[Cocci] version 1.0.3 released

[Sébastien Hinderer]

Dear Richard,

Many thanks for your report!

In coccinelle's source directory, could you please try the following:

make distclean
./autogen
./configure
...

hth,

S?bastien.

[Cocci] version 1.0.3 released

[Richard W.M. Jones]

On Tue, Oct 27, 2015 at 11:25:17AM +0100, S?bastien Hinderer wrote:
> Dear Richard,
> 
> Many thanks for your report!
> 
> In coccinelle's source directory, could you please try the following:
> 
> make distclean
> ./autogen
> ./configure
> ...

Note this is an RPM build, so it always starts from the pristine
unpacked tarball.

I finally worked out how to fix the problem:

  rm parsing_cocci/parser_cocci_menhir.ml parsing_cocci/parser_cocci_menhir.mli
  ./configure --with-menhir=/usr/bin/menhir

so that the intermediate files are regenerated.

BTW the 'spgen' program is installed, but it's broken.  It's built as
a bytecode program using 'ocamlc -custom', RPM strips it (removing the
bytecode), and consequently the binary won't function on Fedora.  It
should have been built as a native program since I requested 'make world'.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/

[Cocci] version 1.0.3 released

[Sébastien Hinderer]

Hi Richard, thanks a lot or your comments.

Richard W.M. Jones (2015/10/27 11:57 +0000):
> On Tue, Oct 27, 2015 at 11:25:17AM +0100, S?bastien Hinderer wrote:
> > Dear Richard,
> > 
> > Many thanks for your report!
> > 
> > In coccinelle's source directory, could you please try the following:
> > 
> > make distclean
> > ./autogen
> > ./configure
> > ...
> 
> Note this is an RPM build, so it always starts from the pristine
> unpacked tarball.

Yes, I realise this. It's because the coccinelle distribution comes with
iles generated by menhir to avoid the user having to generate them but
it does not work well because the menhir used to produce the distributed
files is too old compared to the menhir you have, so the code that is
generated does not compile with newer menhirs.

> I finally worked out how to fix the problem:
> 
>   rm parsing_cocci/parser_cocci_menhir.ml parsing_cocci/parser_cocci_menhir.mli
>   ./configure --with-menhir=/usr/bin/menhir

You're right, the rm command you use is enough.

> so that the intermediate files are regenerated.
> 
> BTW the 'spgen' program is installed, but it's broken.  It's built as
> a bytecode program using 'ocamlc -custom', RPM strips it (removing the
> bytecode), and consequently the binary won't function on Fedora.  It
> should have been built as a native program since I requested 'make
> world'.

I'm aware that spgen does not build as a native-code program yet.However
I'm surprised that it got built at all, since I disabled it from the
build system just beofre the release.

Will have a look into it.

S?bastien.

[Cocci] version 1.0.3 released

[Sébastien Hinderer]

Hi again Richard and all,

Richard W.M. Jones (2015/10/27 11:57 +0000):
> BTW the 'spgen' program is installed, but it's broken.  It's built as
> a bytecode program using 'ocamlc -custom', RPM strips it (removing the
> bytecode), and consequently the binary won't function on Fedora.  It
> should have been built as a native program since I requested 'make
> world'.

A new release of coccinelle fixing this issue will be published soon
(hopefully tomorrow).

S?bastien.

[Cocci] version 1.0.3 released

[Richard W.M. Jones]

On Tue, Oct 27, 2015 at 01:37:48PM +0100, S?bastien Hinderer wrote:
> Hi again Richard and all,
> 
> Richard W.M. Jones (2015/10/27 11:57 +0000):
> > BTW the 'spgen' program is installed, but it's broken.  It's built as
> > a bytecode program using 'ocamlc -custom', RPM strips it (removing the
> > bytecode), and consequently the binary won't function on Fedora.  It
> > should have been built as a native program since I requested 'make
> > world'.
> 
> A new release of coccinelle fixing this issue will be published soon
> (hopefully tomorrow).

So this means I shouldn't be distributing spgen in our binary package?

I was a bit confused about what this program does - there didn't seem
to be much documentation for it :-(

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v

[Cocci] version 1.0.3 released

[Julia Lawall]

On Tue, 27 Oct 2015, Richard W.M. Jones wrote:

> On Tue, Oct 27, 2015 at 01:37:48PM +0100, S?bastien Hinderer wrote:
> > Hi again Richard and all,
> > 
> > Richard W.M. Jones (2015/10/27 11:57 +0000):
> > > BTW the 'spgen' program is installed, but it's broken.  It's built as
> > > a bytecode program using 'ocamlc -custom', RPM strips it (removing the
> > > bytecode), and consequently the binary won't function on Fedora.  It
> > > should have been built as a native program since I requested 'make
> > > world'.
> > 
> > A new release of coccinelle fixing this issue will be published soon
> > (hopefully tomorrow).
> 
> So this means I shouldn't be distributing spgen in our binary package?
> 
> I was a bit confused about what this program does - there didn't seem
> to be much documentation for it :-(

It is used to make a semntic patche acceptable for inclusion in the Linux 
kernel.  There is actually a very nice documentation for it in the 
spgen/documentation directory.  But don't worry about distributing it now.  
We will try to make it available in a better manner for the next release 
(not the one planned for today, but after that).

julia

[Cocci] version 1.0.3 released

[Julia Lawall]

> It is used to make a semntic patche acceptable for inclusion in the Linux 
> kernel.  There is actually a very nice documentation for it in the 
> spgen/documentation directory.  But don't worry about distributing it now.  
> We will try to make it available in a better manner for the next release 
> (not the one planned for today, but after that).

spgen should have a man page too.  Not sure what is the status of that, 
though.

julia

[Cocci] version 1.0.3 released

[Sébastien Hinderer]

Julia Lawall (2015/10/28 08:33 +0100):
> spgen should have a man page too.  Not sure what is the status of that, 
> though.

There is one in docs/spgen.1.

S?bastien.

[Cocci] version 1.0.3 released

[Richard W.M. Jones]

On Wed, Oct 28, 2015 at 08:08:15AM +0100, Julia Lawall wrote:
> On Tue, 27 Oct 2015, Richard W.M. Jones wrote:
> 
> > On Tue, Oct 27, 2015 at 01:37:48PM +0100, S?bastien Hinderer wrote:
> > > Hi again Richard and all,
> > > 
> > > Richard W.M. Jones (2015/10/27 11:57 +0000):
> > > > BTW the 'spgen' program is installed, but it's broken.  It's built as
> > > > a bytecode program using 'ocamlc -custom', RPM strips it (removing the
> > > > bytecode), and consequently the binary won't function on Fedora.  It
> > > > should have been built as a native program since I requested 'make
> > > > world'.
> > > 
> > > A new release of coccinelle fixing this issue will be published soon
> > > (hopefully tomorrow).
> > 
> > So this means I shouldn't be distributing spgen in our binary package?
> > 
> > I was a bit confused about what this program does - there didn't seem
> > to be much documentation for it :-(
> 
> It is used to make a semntic patche acceptable for inclusion in the Linux 
> kernel.  There is actually a very nice documentation for it in the 
> spgen/documentation directory.  But don't worry about distributing it now.  
> We will try to make it available in a better manner for the next release 
> (not the one planned for today, but after that).

Ah got it.  For some reason I was only looking in the
tools/spgen/source directory and overlooked the documentation and
examples directories completely.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

[Cocci] version 1.0.4 released

[Julia Lawall]

Version 1.0.4 is released.  This removes the make of spgen when making 
spatch, removes the pre-generated menhir file when menhir is available, 
and adds the ability to reason about the lengths of statement lists.

julia

[Cocci] version 1.0.4 released

[Richard W.M. Jones]

On Wed, Oct 28, 2015 at 11:06:34PM +0100, Julia Lawall wrote:
> Version 1.0.4 is released.  This removes the make of spgen when making 
> spatch, removes the pre-generated menhir file when menhir is available, 
> and adds the ability to reason about the lengths of statement lists.

This problem has reoccured in 1.0.5.  The generated files must
be removed first, if compiling with external menhir.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html

[Cocci] version 1.0.4 released

[Julia Lawall]

On Tue, 19 Jul 2016, Richard W.M. Jones wrote:

> On Wed, Oct 28, 2015 at 11:06:34PM +0100, Julia Lawall wrote:
> > Version 1.0.4 is released.  This removes the make of spgen when making
> > spatch, removes the pre-generated menhir file when menhir is available,
> > and adds the ability to reason about the lengths of statement lists.
>
> This problem has reoccured in 1.0.5.  The generated files must
> be removed first, if compiling with external menhir.

OK.  I think this problem has been fixed, but we need to make a new
release.  Maybe you should just wait for 1.0.6, if it is causing a
problem.  Thanks for the report.

julia

[Cocci] version 1.0.4 released

[Richard W.M. Jones]

On Tue, Jul 19, 2016 at 12:07:42PM +0200, Julia Lawall wrote:
> 
> 
> On Tue, 19 Jul 2016, Richard W.M. Jones wrote:
> 
> > On Wed, Oct 28, 2015 at 11:06:34PM +0100, Julia Lawall wrote:
> > > Version 1.0.4 is released.  This removes the make of spgen when making
> > > spatch, removes the pre-generated menhir file when menhir is available,
> > > and adds the ability to reason about the lengths of statement lists.
> >
> > This problem has reoccured in 1.0.5.  The generated files must
> > be removed first, if compiling with external menhir.
> 
> OK.  I think this problem has been fixed, but we need to make a new
> release.  Maybe you should just wait for 1.0.6, if it is causing a
> problem.  Thanks for the report.

It's no problem - I have added a workaround in the RPM build.

I was just noting that the workaround was necessary.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html

[Cocci] Determination of statement list lengths with SmPL

[SF Markus Elfring]

> Version 1.0.4 is released.  This removes the make of spgen when making 
> spatch, removes the pre-generated menhir file when menhir is available, 
> and adds the ability to reason about the lengths of statement lists.

I am becoming interested in this functionality a bit more.


The documentation contains information like the following so far.
https://github.com/coccinelle/coccinelle/blob/3b52bd3c6960dc996186e1c58c96355db288825c/docs/manual/cocci_syntax.tex#L181

"?
metadecl 	  ::=
?
|	parameter list [ id ] ids ;
|	parameter list [ const ] ids ;
?
|	statement [list] ids ;
?
|	expression list [ id ] ids ;
	expression list [ const ] ids ;
?"


Which programming interfaces (for OCaml and Python) will make it possible
to determine the length of statement sequences

Would it make sense to extend the support for statement lists
in a way which is similar to the other list types in the semantic
patch language?

Regards,
Markus

[Cocci] Determination of statement list lengths with SmPL

[Julia Lawall]

On Thu, 28 Jul 2016, SF Markus Elfring wrote:

> > Version 1.0.4 is released.  This removes the make of spgen when making
> > spatch, removes the pre-generated menhir file when menhir is available,
> > and adds the ability to reason about the lengths of statement lists.
>
> I am becoming interested in this functionality a bit more.
>
>
> The documentation contains information like the following so far.
> https://github.com/coccinelle/coccinelle/blob/3b52bd3c6960dc996186e1c58c96355db288825c/docs/manual/cocci_syntax.tex#L181
>
> "?
> metadecl 	  ::=
> ?
> |	parameter list [ id ] ids ;
> |	parameter list [ const ] ids ;
> ?
> |	statement [list] ids ;
> ?
> |	expression list [ id ] ids ;
> 	expression list [ const ] ids ;
> ?"
>
>
> Which programming interfaces (for OCaml and Python) will make it possible
> to determine the length of statement sequences
>
> Would it make sense to extend the support for statement lists
> in a way which is similar to the other list types in the semantic
> patch language?

I believe that it is already possible.

julia

[Cocci] Determination of statement list lengths with SmPL

[SF Markus Elfring]

>> Would it make sense to extend the support for statement lists
>> in a way which is similar to the other list types in the semantic
>> patch language?
> 
> I believe that it is already possible.

Do I miss the corresponding description only in the documentation then?

"?
metadecl 	  ::=
?
|	statement list [ id ] ids ;
|	statement list [ const ] ids ;
?"


Regards,
Markus

[Cocci] Determination of statement list lengths with SmPL

[SF Markus Elfring]

> I believe that it is already possible.

Should the following approach work together with the semantic patch language?


@lengths@
statement list [if_count] is;
statement list [else_count] es;
@@
 if (...)
    is
 else
    es

@script:python display@
if_n << lengths.if_count;
else_n << lengths.else_count;
@@
print("{0}|{1}".format(if_n, else_n))


Regards,
Markus

[Cocci] Determination of statement list lengths with SmPL

[Julia Lawall]

On Thu, 28 Jul 2016, SF Markus Elfring wrote:

> > I believe that it is already possible.
>
> Should the following approach work together with the semantic patch language?
>
>
> @lengths@
> statement list [if_count] is;
> statement list [else_count] es;
> @@
>  if (...)
>     is
>  else
>     es
>
> @script:python display@
> if_n << lengths.if_count;
> else_n << lengths.else_count;
> @@
> print("{0}|{1}".format(if_n, else_n))

Did you try it?  I think you would need braces around is and es.
Otherwise, the result would always be 1, because there can only be one
statement in each branch of an if.

julia

[Cocci] Determination of statement list lengths with SmPL

[SF Markus Elfring]

> Did you try it?

Yes.

elfring at Sonne:~/Projekte/Linux/next-patched> spatch.opt -sp-file ~/Projekte/Coccinelle/janitor/show_lengths_for_if_statements2.cocci drivers/staging/ks7010/ks7010_sdio.c
init_defs_builtins: /usr/local/lib64/coccinelle/standard.h
minus: parse error: 
  File "/home/elfring/Projekte/Coccinelle/janitor/show_lengths_for_if_statements2.cocci", line 6, column 4, charpos = 89
  around = 'is',
  whole content =     is


> I think you would need braces around is and es.

Another SmPL approach:

@lengths@
statement list [if_count] isl;
statement list [else_count] esl;
@@
 if (...)
    { isl }
 else
    { esl }

@script:python display@
if_n << lengths.if_count;
else_n << lengths.else_count;
@@
print("{0}|{1}".format(if_n, else_n))


elfring@Sonne:~/Projekte/Linux/next-patched> spatch.opt -sp-file ~/Projekte/Coccinelle/janitor/show_lengths_for_if_statements3.cocci drivers/staging/ks7010/ks7010_sdio.c
init_defs_builtins: /usr/local/lib64/coccinelle/standard.h
Cannot resolve PyUnicodeUCS2_AsEncodedString.


How should this surprise be fixed?


> Otherwise, the result would always be 1, because there can only be one
> statement in each branch of an if.

Can the number of "steps" that were stored by a metavariable of the
type "statement" also be determined anyhow?

Regards,
Markus

[Cocci] Determination of statement list lengths with SmPL

[Julia Lawall]

On Thu, 28 Jul 2016, SF Markus Elfring wrote:

> > Did you try it?
>
> Yes.
>
> elfring at Sonne:~/Projekte/Linux/next-patched> spatch.opt -sp-file ~/Projekte/Coccinelle/janitor/show_lengths_for_if_statements2.cocci drivers/staging/ks7010/ks7010_sdio.c
> init_defs_builtins: /usr/local/lib64/coccinelle/standard.h
> minus: parse error:
>   File "/home/elfring/Projekte/Coccinelle/janitor/show_lengths_for_if_statements2.cocci", line 6, column 4, charpos = 89
>   around = 'is',
>   whole content =     is
>
>
> > I think you would need braces around is and es.
>
> Another SmPL approach:
>
> @lengths@
> statement list [if_count] isl;
> statement list [else_count] esl;
> @@
>  if (...)
>     { isl }
>  else
>     { esl }
>
> @script:python display@
> if_n << lengths.if_count;
> else_n << lengths.else_count;
> @@
> print("{0}|{1}".format(if_n, else_n))
>
>
> elfring at Sonne:~/Projekte/Linux/next-patched> spatch.opt -sp-file ~/Projekte/Coccinelle/janitor/show_lengths_for_if_statements3.cocci drivers/staging/ks7010/ks7010_sdio.c
> init_defs_builtins: /usr/local/lib64/coccinelle/standard.h
> Cannot resolve PyUnicodeUCS2_AsEncodedString.
>
>
> How should this surprise be fixed?

Make clean, or maybe make distclean.

>
>
> > Otherwise, the result would always be 1, because there can only be one
> > statement in each branch of an if.
>
> Can the number of "steps" that were stored by a metavariable of the
> type "statement" also be determined anyhow?

No idea what you mean by steps.  A statement metavariable only matches a
single statement.

julia

[Cocci] Determination of statement list lengths with SmPL

[SF Markus Elfring]

>> Cannot resolve PyUnicodeUCS2_AsEncodedString.
>>
>>
>> How should this surprise be fixed?
> 
> Make clean, or maybe make distclean.

This suggestion does not help me with the software combination
"spatch version 1.0.5-00058-g36edc2b-dirty compiled with OCaml version 4.03.0"
at the moment.
Should I try any other data type conversion out for my small
SmPL script example?

Regards,
Markus

[Cocci] Determination of statement list lengths with SmPL

[Julia Lawall]

On Thu, 28 Jul 2016, SF Markus Elfring wrote:

> >> Cannot resolve PyUnicodeUCS2_AsEncodedString.
> >>
> >>
> >> How should this surprise be fixed?
> >
> > Make clean, or maybe make distclean.
>
> This suggestion does not help me with the software combination
> "spatch version 1.0.5-00058-g36edc2b-dirty compiled with OCaml version 4.03.0"
> at the moment.
> Should I try any other data type conversion out for my small
> SmPL script example?

Please try pulling from github again and restart with make distclean.

julia

[Cocci] Determination of statement list lengths with SmPL

[SF Markus Elfring]

>>>> Cannot resolve PyUnicodeUCS2_AsEncodedString.

I can circumvent this error message also by changing
the scripting language.


@lengths@
statement list [if_count] isl;
statement list [else_count] esl;
@@
 if (...)
    { isl }
 else
    { esl }

@script:ocaml display@
if_n << lengths.if_count;
else_n << lengths.else_count;
@@
Printf.printf "%u|%u\n" if_n else_n


>> Should I try any other data type conversion out for my small
>> SmPL script example?
> 
> Please try pulling from github again and restart with make distclean.

In which development branch did you publish any further commits
for me to try out?

Regards,
Markus

[Cocci] Determination of statement list lengths with SmPL

[Julia Lawall]

On Thu, 28 Jul 2016, SF Markus Elfring wrote:

> >>>> Cannot resolve PyUnicodeUCS2_AsEncodedString.
>
> I can circumvent this error message also by changing
> the scripting language.
>
>
> @lengths@
> statement list [if_count] isl;
> statement list [else_count] esl;
> @@
>  if (...)
>     { isl }
>  else
>     { esl }
>
> @script:ocaml display@
> if_n << lengths.if_count;
> else_n << lengths.else_count;
> @@
> Printf.printf "%u|%u\n" if_n else_n
>
>
> >> Should I try any other data type conversion out for my small
> >> SmPL script example?
> >
> > Please try pulling from github again and restart with make distclean.
>
> In which development branch did you publish any further commits
> for me to try out?

As far as I can see, the only branch available on github is master.

julia

[Cocci] Determination of statement list lengths with SmPL

[SF Markus Elfring]

> As far as I can see, the only branch available on github is master.

The current software (together with the commit 69ba50121a5c80ae91bba16210bd8b281ab8595e
from 2016-07-17) seems to work better also for my small SmPL Python script.

Thanks for your improvements.

Regards,
Markus

[Cocci] Determination of statement list lengths with SmPL

[SF Markus Elfring]

>> Cannot resolve PyUnicodeUCS2_AsEncodedString.

Was this issue be fixed by the commit "Updated py.ml for UCS4 support"
(fdbafd49c0f2b9267229b105beb870cfc2bd0d06 from 2016-07-16)?


>> Can the number of "steps" that were stored by a metavariable of the
>> type "statement" also be determined anyhow?
> 
> No idea what you mean by steps.  A statement metavariable only matches a
> single statement.

* Are "C statements" mapped to entries in a statement list metavariable
  by the semantic patch language?
  How do you think about to drill down from a single compound statement
  to several subitems?

* Will it matter if curly brackets are occasionally missing?

Regards,
Markus

[Cocci] Exclusion of else branches from if statements with SmPL

[SF Markus Elfring]

> Did you try it?  I think you would need braces around is and es.

The software combination "spatch version 1.0.5-00075-g69ba501 compiled
with OCaml version 4.03.0" is working to some degree as expected with
the following tiny SmPL script.


@single_entry@
statement list [1] isl;
@@
*if (...) { isl }


This source code search approach finds also if statements which have got
an else branch. Which techniques would you like to recommend to exclude
this case here?

Regards,
Markus

[Cocci] Release 1.0.3: Clarification around SmPL conjunctions

[SF Markus Elfring]

> * There is now support for conjunctions ( & ... & ).  As for a 
> disjunction, the () and & should be in column 0.  A conjunction requires 
> two patterns to match against a given piece of code in a consistent way.

When will corresponding descriptions appear also in the manual?

How does this software extension differ in the handling of several changes
in patch scripts together without using an ampersand?

Regards,
Markus

[Cocci] Release 1.0.3: Clarification around SmPL conjunctions

[Julia Lawall]

On Thu, 29 Oct 2015, SF Markus Elfring wrote:

> > * There is now support for conjunctions ( & ... & ).  As for a
> > disjunction, the () and & should be in column 0.  A conjunction requires
> > two patterns to match against a given piece of code in a consistent way.
>
> When will corresponding descriptions appear also in the manual?

I have added something.  It will be available in the next release.

> How does this software extension differ in the handling of several changes
> in patch scripts together without using an ampersand?

The & combines patterns that must match at the same place, with a
consistent metavariable environment.  If you have two separate rules, they
can match anywhere.  Also, if you put matches in multiple branches of a
conjunction, you may run the risk of an already tagged token error, if the
two branches make changes to the same tokens. Two different rules can make
changes the same tokens with no problem.

julia

[Cocci] Release 1.0.3: Clarification around SmPL conjunctions

[SF Markus Elfring]

> If you have two separate rules, they can match anywhere.

I became interested to compare the effects from the added functionality
to the use case where several changes would be applied in a specific SmPL rule
without an ampersand specification as in previous versions.

Regards,
Markus

[Cocci] Release 1.0.3: Clarification around SmPL conjunctions

[Julia Lawall]

On Fri, 30 Oct 2015, SF Markus Elfring wrote:

> > If you have two separate rules, they can match anywhere.
>
> I became interested to compare the effects from the added functionality
> to the use case where several changes would be applied in a specific SmPL rule
> without an ampersand specification as in previous versions.

No idea what you mean.  You would have to give an example.

julia

[Cocci] Release 1.0.3: Clarification around SmPL conjunctions

[SF Markus Elfring]

> You would have to give an example.

Should an approach like the following which is derived from
the script "demos/conjunction.cocci" also work?


@@
identifier f;
@@
-f
+m
  (...,3,...);
 ...
 g(...,3,...);
 ...
 f(...,
-  8
+  80
   ,...);
 ...
 g(...,8,...);


Regards,
Markus

[Cocci] Release 1.0.3: Clarification around SmPL conjunctions

[Julia Lawall]

On Fri, 30 Oct 2015, SF Markus Elfring wrote:

> > You would have to give an example.
>
> Should an approach like the following which is derived from
> the script "demos/conjunction.cocci" also work?
>
>
> @@
> identifier f;
> @@
> -f
> +m
>   (...,3,...);
>  ...
>  g(...,3,...);
>  ...
>  f(...,
> -  8
> +  80
>    ,...);
>  ...
>  g(...,8,...);

When you have ..., the thing after the ... has to come after (in the
control-flow graph) the thing before the ...  So this would only match
code that has a call to f followed by a call to g followed by a call to f
followed by a call to g.  In particular, if the C code has no loops, then
it must contain two calls to g.  If you look a demos/conjunction.c, both
of the C functions contain only one call to g, so neither would match your
pattern.

demos/conjunction.cocci will only match code that contains a call to f
that has 3 as one of its arguments and that has 8 as another.  The 8 may
appear before or after the 3.

julia

[Cocci] Release 1.0.3: Clarification around SmPL conjunctions

[SF Markus Elfring]

> When you have ..., the thing after the ... has to come after (in the
> control-flow graph) the thing before the ...  So this would only match
> code that has a call to f followed by a call to g followed by a call to f
> followed by a call to g.  In particular, if the C code has no loops, then
> it must contain two calls to g.

I would find such a situation also occasionally interesting for specific
function implementations.


> If you look a demos/conjunction.c, both of the C functions contain only
> one call to g, so neither would match your pattern.

I find it strange that the function name "main" is repeated without
conditional compilation statements in this source code example.

Regards,
Markus

[Cocci] Release 1.0.3: Clarification around SmPL conjunctions

[Julia Lawall]

On Fri, 30 Oct 2015, SF Markus Elfring wrote:

> > When you have ..., the thing after the ... has to come after (in the
> > control-flow graph) the thing before the ...  So this would only match
> > code that has a call to f followed by a call to g followed by a call to f
> > followed by a call to g.  In particular, if the C code has no loops, then
> > it must contain two calls to g.
>
> I would find such a situation also occasionally interesting for specific
> function implementations.
>
>
> > If you look a demos/conjunction.c, both of the C functions contain only
> > one call to g, so neither would match your pattern.
>
> I find it strange that the function name "main" is repeated without
> conditional compilation statements in this source code example.

The code is not intended to be run.

julia

[Cocci] Release 1.0.3: Clarification around SmPL conjunctions

[SF Markus Elfring]

> The code is not intended to be run.

I guess that a duplicate function name can result in unwanted data
from the analysis for a source file like "demos/conjunction.c", can't it?

Regards,
Markus

[Cocci] Release 1.0.3: Clarification around SmPL conjunctions

[Julia Lawall]

On Fri, 30 Oct 2015, SF Markus Elfring wrote:

> > The code is not intended to be run.
>
> I guess that a duplicate function name can result in unwanted data
> from the analysis for a source file like "demos/conjunction.c", can't it?

No idea what you mean by that.  demos/conjunction.c is only intended for
use with demos/conjunction.cocci, which doesn't care at all about the name
of the containing function.

julia

[Cocci] Release 1.0.3: Clarification around SmPL conjunctions

[SF Markus Elfring]

> demos/conjunction.c is only intended for use with demos/conjunction.cocci,
> which doesn't care at all about the name of the containing function.

Will this source code example become a bit easier to understand
if one of the test functions will be renamed so that the names are unique?

Regards,
Markus