[Cocci] script code on metavariables

[Cocci] script code on metavariables

[Julia Lawall]

Some time ago, the idea of putting constraints expressed as scripts was 
discussed 
(http://article.gmane.org/gmane.comp.version-control.coccinelle/1928).

This has now been implemented, only for position variables and only for 
ocaml scripting:

The basic notation is:

position p : script:ocaml(params) { expression };

Params can only be inherited metavariables.  Due to parsing constraints, 
the expression has to be a C expression, even though the script language 
is OCaml.  Fortunately, it is possible to make an OCaml function call that 
looks just like a C function call.  The expression can also refer to the 
metavariable p being defined.  The expression should return true or false, 
ie true if the proposed value of p is acceptable as a match, and false if 
it is not.

A real-life example is:
position jp : script:ocaml() { check_in_a_hunk("P",plus_hunks,jp) };

Any thoughts about the syntax?  For example, I don't think that the 
script: and the { } are strictly necessary from a parsing point of view.

This could be extended to other metavariable types, and eventully to 
Python.

thanks,
julia

[Cocci] script code on metavariables

[SF Markus Elfring]

> Some time ago, the idea of putting constraints expressed as scripts was discussed
> (http://article.gmane.org/gmane.comp.version-control.coccinelle/1928).

I am curious if the discussion (from December 2011) will be continued
around a topic like "assignments and support for SmPL rule extensions".


> This has now been implemented,

Interesting ?


> only for position variables and only for ocaml scripting:

I hope that this software limitation can be changed anyhow
in the future.


> The basic notation is:
> 
> position p : script:ocaml(params) { expression };
> 
> Params can only be inherited metavariables.

Would you like to clarify the specification of such parameters
a bit more?


> Due to parsing constraints, the expression has to be a C expression,
> even though the script language is OCaml.

How do you think about to explain this restriction better?


> Fortunately, it is possible to make an OCaml function call that 
> looks just like a C function call.

I am unsure if I understand the consequences from this information
good enough.


> The expression can also refer to the metavariable p being defined.

Nice ?


> The expression should return true or false, ie true if the proposed value
> of p is acceptable as a match, and false if it is not.

Do you describe the introduction of generic predicate functions here?


> A real-life example is:
> position jp : script:ocaml() { check_in_a_hunk("P",plus_hunks,jp) };
> 
> Any thoughts about the syntax?  For example, I don't think that the 
> script: and the { } are strictly necessary from a parsing point of view.

I find that extra delimiter characters will help to make such
source code a bit easier to read.


> This could be extended to other metavariable types, and eventully to Python.

I am curious on corresponding software evolution.

Regards,
Markus

[Cocci] script code on metavariables

[Julia Lawall]

> > The basic notation is:
> >
> > position p : script:ocaml(params) { expression };
> >
> > Params can only be inherited metavariables.
>
> Would you like to clarify the specification of such parameters
> a bit more?

r.x

Unlike the case of an ordinary python/ocaml script, you are stuck with the
name x in the script code.

> > The expression should return true or false, ie true if the proposed value
> > of p is acceptable as a match, and false if it is not.
>
> Do you describe the introduction of generic predicate functions here?

I don't understand the question.  In any case, the script doesn't define
functions, due to the very limited syntax.  The normal case woudl be to
define some functions in the initialize rule of the semantic patch, and
just call them here.

thanks,
julia

[Cocci] script code on metavariables

[SF Markus Elfring]

>>> The expression should return true or false, ie true if the proposed value
>>> of p is acceptable as a match, and false if it is not.
>>
>> Do you describe the introduction of generic predicate functions here?
> 
> I don't understand the question.

I try another wording ?


> In any case, the script doesn't define functions, due to the very limited syntax.

Do you exclude function definitions there so far?

Does the described SmPL extension correspond to predicate functionality?


> The normal case woudl be to define some functions in the initialize rule
> of the semantic patch, and just call them here.

I like such source code organisation in principle for some use cases.
Will this aspect need further considerations because of the evolving
parallelisation support?
https://github.com/coccinelle/coccinelle/issues/50

Regards,
Markus

[Cocci] script code on metavariables

[Julia Lawall]

On Mon, 18 Jan 2016, SF Markus Elfring wrote:

> >>> The expression should return true or false, ie true if the proposed value
> >>> of p is acceptable as a match, and false if it is not.
> >>
> >> Do you describe the introduction of generic predicate functions here?
> >
> > I don't understand the question.
>
> I try another wording ?
>
>
> > In any case, the script doesn't define functions, due to the very limited syntax.
>
> Do you exclude function definitions there so far?

It is not possible to write an OCaml function definition that has the form
of a C expression.  In practice, it is likely that one would only make a
function call, like I showed in my example.

> Does the described SmPL extension correspond to predicate functionality?

I guess.  The script code should return true or false.  If the script
returns true, the match succeeds.  If the script returns false, the match
fails.

> > The normal case woudl be to define some functions in the initialize rule
> > of the semantic patch, and just call them here.
>
> I like such source code organisation in principle for some use cases.
> Will this aspect need further considerations because of the evolving
> parallelisation support?
> https://github.com/coccinelle/coccinelle/issues/50

I believe that parallelism with -j only fails if there is a finalize.  An
initialize by itself should be OK.

julia

[Cocci] script code on metavariables

[SF Markus Elfring]

> It is not possible to write an OCaml function definition that has the form
> of a C expression.  In practice, it is likely that one would only make a
> function call, like I showed in my example.

Are method calls are also supported there depending on the reused
programming language?


>> Does the described SmPL extension correspond to predicate functionality?
> 
> I guess.  The script code should return true or false.  If the script
> returns true, the match succeeds.

http://c2.com/cgi/wiki?PredicateFunction
http://stackoverflow.com/questions/5921609/what-is-predicate-in-c
https://en.wikipedia.org/wiki/Filter_%28higher-order_function%29

Will this kind of information matter for the outline in the evolving
documentation of the semantic patch language?



>> Will this aspect need further considerations because of the evolving
>> parallelisation support?
>> https://github.com/coccinelle/coccinelle/issues/50
> 
> I believe that parallelism with -j only fails if there is a finalize.

Would you like to clarify such an implementation detail a bit more?


> An initialize by itself should be OK.

How often will you need a corresponding finalisation action
(or destructor)?

Can the predicate function be eventually executed in parallel
(by a dedicated thread or another background process)?

Regards,
Markus

[Cocci] script code on metavariables

[Julia Lawall]

On Mon, 18 Jan 2016, SF Markus Elfring wrote:

> > It is not possible to write an OCaml function definition that has the form
> > of a C expression.  In practice, it is likely that one would only make a
> > function call, like I showed in my example.
>
> Are method calls are also supported there depending on the reused
> programming language?

I don't know what is meant exactly by a method call, but anything that
looks like a valid C expression is fine.  The code will not be interpreted
by Coccinelle, only by the relevant scripting language, which is currently
only OCaml.  The restriction to being a C expression is only a parsing
issue.

> >> Will this aspect need further considerations because of the evolving
> >> parallelisation support?
> >> https://github.com/coccinelle/coccinelle/issues/50
> >
> > I believe that parallelism with -j only fails if there is a finalize.
>
> Would you like to clarify such an implementation detail a bit more?

if Cocci.has_finalize cocci_infos
            then
              begin
                pr2 "warning: parallel mode is disabled due to a finalize";
                (seq_fold, false)
              end

> Can the predicate function be eventually executed in parallel
> (by a dedicated thread or another background process)?

No, it runs in the same thread as the rest of the processing.

julia

[Cocci] script code on metavariables

[SF Markus Elfring]

> I don't know what is meant exactly by a method call, but anything that
> looks like a valid C expression is fine.  The code will not be interpreted
> by Coccinelle, only by the relevant scripting language, which is currently
> only OCaml.

Would an other wording fit also?

The returning of a boolean value should just work. Can the reference to
the C programming language be lifted already if an external script processor
will finally take care for the embedded source code?


> The restriction to being a C expression is only a parsing issue.

I imagine that there will be more dependencies to consider, won't it?


> No, it runs in the same thread as the rest of the processing.

Would it make sense occasionally to execute the predicate function
in parallel (by a dedicated thread or another background process)?

Regards,
Markus

[Cocci] Documentation check for parallelisation support?

[SF Markus Elfring]

>> Will this aspect need further considerations because of the evolving
>> parallelisation support?
>> https://github.com/coccinelle/coccinelle/issues/50
> 
> I believe that parallelism with -j only fails if there is a finalize.
> An initialize by itself should be OK.

Does the corresponding wording need another update in the SmPL manual?
https://github.com/coccinelle/coccinelle/blob/246fb87e74e448b46749601a25fa019925b07e43/docs/manual/spatch_options.tex#L678

Regards,
Markus