[Cocci] Searching for pass-through functions

[Cocci] Searching for pass-through functions

[SF Markus Elfring]

Hello,

I have retried a source code analysis approach which was discussed a while ago.
http://article.gmane.org/gmane.comp.version-control.coccinelle/1900
https://www.mail-archive.com/cocci%40diku.dk/msg01834.html


elfring at Sonne:~/Projekte/Coccinelle/lokal/demos/pass-through> spatch.opt
-sp_file list_pass-through_functions.cocci -dir /usr/src/linux-stable
--include-headers-for-types >list_pass-through_functions-Linux-3_16_3.txt
2>list_pass-through_functions-Linux-3_16_3-errors.txt


I have stopped this run after its execution took over 4,8 hours with the
software "Coccinelle 1.0.0-rc22" on my openSUSE system. A log file contains the
following information.

?
HANDLING: /usr/src/linux-stable/drivers/media/pci/ttpci/av7110_ipack.c
     C-c intercepted, will do some cleaning before exiting


Can it be that the analysis tool became stuck on this source file after 4054
files were processed before?

How are the chances to complete the desired data extraction for all 23753 C
files in a shorter time frame?

Regards,
Markus

[Cocci] Searching for pass-through functions

[Julia Lawall]

On Sat, 4 Oct 2014, SF Markus Elfring wrote:

> Hello,
> 
> I have retried a source code analysis approach which was discussed a while ago.
> http://article.gmane.org/gmane.comp.version-control.coccinelle/1900
> https://www.mail-archive.com/cocci%40diku.dk/msg01834.html
> 
> 
> elfring at Sonne:~/Projekte/Coccinelle/lokal/demos/pass-through> spatch.opt
> -sp_file list_pass-through_functions.cocci -dir /usr/src/linux-stable
> --include-headers-for-types >list_pass-through_functions-Linux-3_16_3.txt
> 2>list_pass-through_functions-Linux-3_16_3-errors.txt
> 
> 
> I have stopped this run after its execution took over 4,8 hours with the
> software "Coccinelle 1.0.0-rc22" on my openSUSE system. A log file contains the
> following information.
> 
> ?
> HANDLING: /usr/src/linux-stable/drivers/media/pci/ttpci/av7110_ipack.c
>      C-c intercepted, will do some cleaning before exiting
> 
> 
> Can it be that the analysis tool became stuck on this source file after 4054
> files were processed before?
> 
> How are the chances to complete the desired data extraction for all 23753 C
> files in a shorter time frame?

Use the timeout option.  Then hopefully you can get a result for 23753-1 
files.

julia

[Cocci] Searching for pass-through functions

[SF Markus Elfring]

>> How are the chances to complete the desired data extraction for all 23753 C
>> files in a shorter time frame?
> 
> Use the timeout option.

I would like to avoid it because I do not really have got an overview for source
file sizes and corresponding source code complexity.


> Then hopefully you can get a result for 23753-1 files.

Would you like to try out the discussed source code analysis approach also on
your software development system?

elfring at Sonne:~/Projekte/Coccinelle/lokal/demos/pass-through> spatch.opt -debug
--include-headers-for-types -I /usr/src/linux-stable/include -sp_file
list_pass-through_functions.cocci
/usr/src/linux-stable/drivers/media/pci/ttpci/av7110_ipack.c



init_defs_builtins: /usr/local/share/coccinelle/standard.h


-----------------------------------------------------------------------


processing semantic patch file: list_pass-through_functions.cocci


with isos from: /usr/local/share/coccinelle/standard.iso


-----------------------------------------------------------------------


@initialize:python@


@@


import sys


result = []


mark = ['"', '', '"']
delimiter = '|'

def store_positions(fun, typ, pt_param, places, data_structure = ""):
    """Add source code positions to an internal list."""
    for place in places:
        fields = []
        fields.append(fun)

        mark[1] = typ
        fields.append(''.join(mark))

        fields.append(pt_param)
        fields.append(data_structure)

        mark[1] = place.file.replace('"', '""')
        fields.append(''.join(mark))

        fields.append(place.line)
        fields.append(str(int(place.column) + 1))
        result.append(delimiter.join(fields))

@is_pass_through@
identifier pt_function, input;
type data_type;
position pos;
@@
 data_type at pos pt_function(...,data_type input,...)
 {
  <+...
  return input;
  ...+>
 }

@is_pass_through_member_candidate@
identifier pt_function, input, tag;
type data_type;
position pos;
@@
 struct tag
 {
  ...
  data_type at pos (*pt_function)(...,data_type input,...);
  ...
 }

@script:python collection1 depends on is_pass_through@
typ << is_pass_through.data_type;
fun << is_pass_through.pt_function;
pt_param << is_pass_through.input;
places << is_pass_through.pos;
@@
store_positions(fun, typ, pt_param, places)

@script:python collection2 depends on is_pass_through_member_candidate@
tag << is_pass_through_member_candidate.tag;
typ << is_pass_through_member_candidate.data_type;
fun << is_pass_through_member_candidate.pt_function;
pt_param << is_pass_through_member_candidate.input;
places << is_pass_through_member_candidate.pos;
@@
store_positions(fun, typ, pt_param, places, tag)

@finalize:python@
@@
if result:
   result.insert(0, delimiter.join(("function", '"data type"', "parameter",
'"contained in"', '"source file"', "line", "column")))
   print("\r\n".join(result))
else:
   sys.stderr.write("No result for this analysis!\n")

-----------------------------------------------------------------------
script rule 1 =
-----------------------------------------------------------------------
dependencies for script satisfied:
binding in = []
HANDLING: /usr/src/linux-stable/drivers/media/pci/ttpci/av7110_ipack.c
-----------------------------------------------------------------------
let's go
-----------------------------------------------------------------------
-----------------------------------------------------------------------
-----------------------------------------------------------------------
is_pass_through =
-----------------------------------------------------------------------
dependencies for rule is_pass_through satisfied:
binding in = []
binding relevant in = []
     (ONCE) USING optional_storage builtin isomorphism
^C     C-c intercepted, will do some cleaning before exiting


In which time frame should such a SmPL script be usually executed?

Regards,
Markus

[Cocci] Searching for pass-through functions

[Julia Lawall]

On Sat, 4 Oct 2014, SF Markus Elfring wrote:

> >> How are the chances to complete the desired data extraction for all 23753 C
> >> files in a shorter time frame?
> > 
> > Use the timeout option.
> 
> I would like to avoid it because I do not really have got an overview for source
> file sizes and corresponding source code complexity.
> 
> 
> > Then hopefully you can get a result for 23753-1 files.
> 
> Would you like to try out the discussed source code analysis approach also on
> your software development system?
> 
> elfring at Sonne:~/Projekte/Coccinelle/lokal/demos/pass-through> spatch.opt -debug
> --include-headers-for-types -I /usr/src/linux-stable/include -sp_file
> list_pass-through_functions.cocci
> /usr/src/linux-stable/drivers/media/pci/ttpci/av7110_ipack.c
> 
> 
> 
> init_defs_builtins: /usr/local/share/coccinelle/standard.h
> 
> 
> -----------------------------------------------------------------------
> 
> 
> processing semantic patch file: list_pass-through_functions.cocci
> 
> 
> with isos from: /usr/local/share/coccinelle/standard.iso
> 
> 
> -----------------------------------------------------------------------
> 
> 
> @initialize:python@
> 
> 
> @@
> 
> 
> import sys
> 
> 
> result = []
> 
> 
> mark = ['"', '', '"']
> delimiter = '|'
> 
> def store_positions(fun, typ, pt_param, places, data_structure = ""):
>     """Add source code positions to an internal list."""
>     for place in places:
>         fields = []
>         fields.append(fun)
> 
>         mark[1] = typ
>         fields.append(''.join(mark))
> 
>         fields.append(pt_param)
>         fields.append(data_structure)
> 
>         mark[1] = place.file.replace('"', '""')
>         fields.append(''.join(mark))
> 
>         fields.append(place.line)
>         fields.append(str(int(place.column) + 1))
>         result.append(delimiter.join(fields))
> 
> @is_pass_through@
> identifier pt_function, input;
> type data_type;
> position pos;
> @@
>  data_type at pos pt_function(...,data_type input,...)
>  {
>   <+...
>   return input;
>   ...+>
>  }
> 
> @is_pass_through_member_candidate@
> identifier pt_function, input, tag;
> type data_type;
> position pos;
> @@
>  struct tag
>  {
>   ...
>   data_type at pos (*pt_function)(...,data_type input,...);
>   ...
>  }
> 
> @script:python collection1 depends on is_pass_through@
> typ << is_pass_through.data_type;
> fun << is_pass_through.pt_function;
> pt_param << is_pass_through.input;
> places << is_pass_through.pos;
> @@
> store_positions(fun, typ, pt_param, places)
> 
> @script:python collection2 depends on is_pass_through_member_candidate@
> tag << is_pass_through_member_candidate.tag;
> typ << is_pass_through_member_candidate.data_type;
> fun << is_pass_through_member_candidate.pt_function;
> pt_param << is_pass_through_member_candidate.input;
> places << is_pass_through_member_candidate.pos;
> @@
> store_positions(fun, typ, pt_param, places, tag)
> 
> @finalize:python@
> @@
> if result:
>    result.insert(0, delimiter.join(("function", '"data type"', "parameter",
> '"contained in"', '"source file"', "line", "column")))
>    print("\r\n".join(result))
> else:
>    sys.stderr.write("No result for this analysis!\n")
> 
> -----------------------------------------------------------------------
> script rule 1 =
> -----------------------------------------------------------------------
> dependencies for script satisfied:
> binding in = []
> HANDLING: /usr/src/linux-stable/drivers/media/pci/ttpci/av7110_ipack.c
> -----------------------------------------------------------------------
> let's go
> -----------------------------------------------------------------------
> -----------------------------------------------------------------------
> -----------------------------------------------------------------------
> is_pass_through =
> -----------------------------------------------------------------------
> dependencies for rule is_pass_through satisfied:
> binding in = []
> binding relevant in = []
>      (ONCE) USING optional_storage builtin isomorphism
> ^C     C-c intercepted, will do some cleaning before exiting
> 
> 
> In which time frame should such a SmPL script be usually executed?

The rule is_pass_through asks it to analyze every input of every function.  
Perhaps it is not surprising that it takes some time.

julia

> 
> Regards,
> Markus
> 

[Cocci] Searching for pass-through functions

[SF Markus Elfring]

>> In which time frame should such a SmPL script be usually executed?
> 
> The rule is_pass_through asks it to analyze every input of every function.  
> Perhaps it is not surprising that it takes some time.

How are the chances to determine which implementation details from a source file
like "/usr/src/linux-stable/drivers/media/pci/ttpci/av7110_ipack.c" cause an
especially long processing time?

Can the execution durations be measured for the other files in an efficient way
where the source code analysis was performed earlier?

The Coccinelle software manages timeouts already. Would you like to print
execution durations in a specific data format so that they can be better
analysed afterwards?

Regards,
Markus

[Cocci] Searching for pass-through functions

[Julia Lawall]

On Sun, 5 Oct 2014, SF Markus Elfring wrote:

> >> In which time frame should such a SmPL script be usually executed?
> > 
> > The rule is_pass_through asks it to analyze every input of every function.  
> > Perhaps it is not surprising that it takes some time.
> 
> How are the chances to determine which implementation details from a source file
> like "/usr/src/linux-stable/drivers/media/pci/ttpci/av7110_ipack.c" cause an
> especially long processing time?
> 
> Can the execution durations be measured for the other files in an efficient way
> where the source code analysis was performed earlier?
> 
> The Coccinelle software manages timeouts already. Would you like to print
> execution durations in a specific data format so that they can be better
> analysed afterwards?

If you give the option --profile, it will collect information about what 
it is doing.  In particular, you can see which rules take a long time.

Probably the problem is that there are loops and nested conditionals, 
requiring it to consider many possible execution paths, but I haven't 
looked at the code in detail.

julia

[Cocci] Searching for pass-through functions

[SF Markus Elfring]

> If you give the option --profile, it will collect information about what 
> it is doing.  In particular, you can see which rules take a long time.

Would you like to clarify corresponding profiling results in more detail?

Regards,
Markus

[Cocci] Searching for pass-through functions

[SF Markus Elfring]

> The rule is_pass_through asks it to analyze every input of every function.  
> Perhaps it is not surprising that it takes some time.

I tried out another command variant again.

elfring at Sonne:~/Projekte/Coccinelle/lokal/demos/pass-through> date && spatch.opt
--timeout 30 --include-headers-for-types -I /usr/src/linux-stable/include
-sp_file list_pass-through_functions.cocci -dir /usr/src/linux-stable >
list_pass-through_functions-Linux-3_16_3-B.txt 2>
list_pass-through_functions-Linux-3_16_3-B-errors.txt && date
So 5. Okt 09:47:03 CEST 2014
So 5. Okt 10:15:39 CEST 2014


Now it seems that 2924 pass-through functions were found by this source code
analysis approach at least.

Excerpt from log file:
?
HANDLING: /usr/src/linux-stable/drivers/media/pci/ttpci/av7110_ipack.c
EXN:Common.Timeout
?
HANDLING: /usr/src/linux-stable/drivers/net/fddi/skfp/ess.c
EXN:Common.Timeout
?
HANDLING: /usr/src/linux-stable/crypto/fcrypt.c
EXN:Failure("lexing: empty token")
?
HANDLING: /usr/src/linux-stable/kernel/debug/kdb/kdb_io.c
EXN:Common.Timeout
?
HANDLING: /usr/src/linux-stable/net/ipv6/ndisc.c
EXN:Failure("is_pass_through: node 121: return ...[1,2,47] in
ndisc_parse_options reachable by inconsistent control-flow paths")
?


Would you like to look at these source files in more detail?

Regards,
Markus

[Cocci] Searching for pass-through functions

[Julia Lawall]

On Sun, 5 Oct 2014, SF Markus Elfring wrote:

> > The rule is_pass_through asks it to analyze every input of every function.  
> > Perhaps it is not surprising that it takes some time.
> 
> I tried out another command variant again.
> 
> elfring at Sonne:~/Projekte/Coccinelle/lokal/demos/pass-through> date && spatch.opt
> --timeout 30 --include-headers-for-types -I /usr/src/linux-stable/include
> -sp_file list_pass-through_functions.cocci -dir /usr/src/linux-stable >
> list_pass-through_functions-Linux-3_16_3-B.txt 2>
> list_pass-through_functions-Linux-3_16_3-B-errors.txt && date
> So 5. Okt 09:47:03 CEST 2014
> So 5. Okt 10:15:39 CEST 2014
> 
> 
> Now it seems that 2924 pass-through functions were found by this source code
> analysis approach at least.
> 
> Excerpt from log file:
> ?
> HANDLING: /usr/src/linux-stable/drivers/media/pci/ttpci/av7110_ipack.c
> EXN:Common.Timeout
> ?
> HANDLING: /usr/src/linux-stable/drivers/net/fddi/skfp/ess.c
> EXN:Common.Timeout
> ?
> HANDLING: /usr/src/linux-stable/crypto/fcrypt.c
> EXN:Failure("lexing: empty token")
> ?
> HANDLING: /usr/src/linux-stable/kernel/debug/kdb/kdb_io.c
> EXN:Common.Timeout
> ?
> HANDLING: /usr/src/linux-stable/net/ipv6/ndisc.c
> EXN:Failure("is_pass_through: node 121: return ...[1,2,47] in
> ndisc_parse_options reachable by inconsistent control-flow paths")
> ?
> 
> 
> Would you like to look at these source files in more detail?

I would suggest that you do so.  You can use --show-trying to find on what 
function it gets stuck.

julia

[Cocci] Searching for pass-through functions

[SF Markus Elfring]

> You can use --show-trying to find on what function it gets stuck.

1. /usr/src/linux-stable/drivers/media/pci/ttpci/av7110_ipack.c
?
     trying function: av7110_ipack_instant_repack
timeout (we abort)
?

2. /usr/src/linux-stable/drivers/net/fddi/skfp/ess.c
?
     trying function: ess_raf_received_pack
timeout (we abort)
?


3. /usr/src/linux-stable/crypto/fcrypt.c
?
     trying function: kdb_read
timeout (we abort)
?

4. /usr/src/linux-stable/crypto/fcrypt.c
?
exn while in timeout_function
Fatal error: exception Failure("lexing: empty token")

5. /usr/src/linux-stable/net/ipv6/ndisc.c
?
     trying function: ndisc_parse_options
exn while in timeout_function
Fatal error: exception Failure("is_pass_through: node 121: return ...[1,2,47] in
ndisc_parse_options reachable by inconsistent control-flow paths")


Should we reconsider more implementation details here?

Regards,
Markus

[Cocci] Searching for pass-through functions

[Julia Lawall]

I'm not sure what is the goal of your semantic patch.  For example are you 
only interested in pointers?  If so, it could be helpful to specify it in 
your semantic patch.  The first two I cases I looked at return int.

Another thing that could be helpful is to check if there exists the 
pattern you are looking for, before checking whether it is found on all 
execution paths.  You can use the same rule you have, but with exists in 
the header, and then have a second rule that checks all paths and depends 
on the first one.

Also, since you just want to find returns, but don't care about what 
happens in between, you can use arguments like --no-loops and --no-gotos.

julia

On Sun, 5 Oct 2014, SF Markus Elfring wrote:

> > You can use --show-trying to find on what function it gets stuck.
> 
> 1. /usr/src/linux-stable/drivers/media/pci/ttpci/av7110_ipack.c
> ?
>      trying function: av7110_ipack_instant_repack
> timeout (we abort)
> ?
> 
> 2. /usr/src/linux-stable/drivers/net/fddi/skfp/ess.c
> ?
>      trying function: ess_raf_received_pack
> timeout (we abort)
> ?
> 
> 
> 3. /usr/src/linux-stable/crypto/fcrypt.c
> ?
>      trying function: kdb_read
> timeout (we abort)
> ?
> 
> 4. /usr/src/linux-stable/crypto/fcrypt.c
> ?
> exn while in timeout_function
> Fatal error: exception Failure("lexing: empty token")
> 
> 5. /usr/src/linux-stable/net/ipv6/ndisc.c
> ?
>      trying function: ndisc_parse_options
> exn while in timeout_function
> Fatal error: exception Failure("is_pass_through: node 121: return ...[1,2,47] in
> ndisc_parse_options reachable by inconsistent control-flow paths")
> 
> 
> Should we reconsider more implementation details here?
> 
> Regards,
> Markus
> 

[Cocci] Searching for pass-through functions

[SF Markus Elfring]

> I'm not sure what is the goal of your semantic patch.

The shown pattern should not be a patch at this time.   ;-)

I try to find functions by this approach that do not perform validation for a
specific input parameter usually because this parameter will be forwarded as the
return value.


> For example are you only interested in pointers?

No. - But I imagine that pointers will be specified more often in corresponding
function signatures than integral data types.


> Another thing that could be helpful is to check if there exists the 
> pattern you are looking for, before checking whether it is found on all 
> execution paths.  You can use the same rule you have, but with exists in 
> the header, and then have a second rule that checks all paths and depends 
> on the first one.

Would you like to show your fine-tuning suggestion in the affected SmPL script?


> Also, since you just want to find returns, but don't care about what 
> happens in between, you can use arguments like --no-loops and --no-gotos.

Will it matter if the relevant return statement will be executed within a loop
or is also marked with a jump label?


Can the five issues on which I stumbled during the source code analysis be fixed
anyhow (eventually without the addtion of "--no-?" command options)?

Regards,
Markus

[Cocci] Searching for pass-through functions

[Julia Lawall]

On Sun, 5 Oct 2014, SF Markus Elfring wrote:

> > I'm not sure what is the goal of your semantic patch.
> 
> The shown pattern should not be a patch at this time.   ;-)
> 
> I try to find functions by this approach that do not perform validation for a
> specific input parameter usually because this parameter will be forwarded as the
> return value.
> 
> 
> > For example are you only interested in pointers?
> 
> No. - But I imagine that pointers will be specified more often in corresponding
> function signatures than integral data types.
> 
> 
> > Another thing that could be helpful is to check if there exists the 
> > pattern you are looking for, before checking whether it is found on all 
> > execution paths.  You can use the same rule you have, but with exists in 
> > the header, and then have a second rule that checks all paths and depends 
> > on the first one.
> 
> Would you like to show your fine-tuning suggestion in the affected SmPL script?
> 
> 
> > Also, since you just want to find returns, but don't care about what 
> > happens in between, you can use arguments like --no-loops and --no-gotos.
> 
> Will it matter if the relevant return statement will be executed within a loop
> or is also marked with a jump label?
> 
> 
> Can the five issues on which I stumbled during the source code analysis be fixed
> anyhow (eventually without the addtion of "--no-?" command options)?

For the timeouts, no they cannot.

julia

[Cocci] Searching for pass-through functions

[SF Markus Elfring]

>> Can the five issues on which I stumbled during the source code analysis be fixed
>> anyhow (eventually without the addtion of "--no-?" command options)?
> 
> For the timeouts, no they cannot.

I am surprised by this answer.
- How many processing hours will the desired analysis need for completion with
these "special" three source files?
- Can the software situation be improved for the other two issues?

Regards,
Markus

[Cocci] Searching for pass-through functions

[Julia Lawall]

On Sun, 5 Oct 2014, SF Markus Elfring wrote:

> >> Can the five issues on which I stumbled during the source code analysis be fixed
> >> anyhow (eventually without the addtion of "--no-?" command options)?
> > 
> > For the timeouts, no they cannot.
> 
> I am surprised by this answer.
> - How many processing hours will the desired analysis need for completion with
> these "special" three source files?

Who knows.  At least the first one has a lot of nested loops and 
conditionals.  This is a case that Coccinelle does not handle well.

> - Can the software situation be improved for the other two issues?

Probably for the lexer.  Perhaps not for the case of inconsistent paths.  
Again that relates to a particular kind of code structure.

julia

[Cocci] Searching for pass-through functions

[SF Markus Elfring]

> At least the first one has a lot of nested loops and conditionals.
> This is a case that Coccinelle does not handle well.

I am curious if the chances will increase to improve this detail in the future.


>> - Can the software situation be improved for the other two issues?
> 
> Probably for the lexer.  Perhaps not for the case of inconsistent paths.  
> Again that relates to a particular kind of code structure.

I am also curious about how the software development will be continued for these
update candidates.

Regards,
Markus

[Cocci] Searching for pass-through functions

[SF Markus Elfring]

> For example are you only interested in pointers?

I get the impression from the source files for the operating system "Linux
3.16.3" that pointers are specified for input parameters (which will be also
forwarded as return values) at 1280 source code places while integral data types
are used 1913 times for such a specific parameter in corresponding function
signatures.
Are such facts worth for further considerations?

Regards,
Markus

[Cocci] Searching for pass-through functions

[SF Markus Elfring]

> For example are you only interested in pointers?

I get the impression from the source files within the software repository for
the operating system "MINIX 3.3.0" that pointers are specified for input
parameters (which will be also forwarded as return values) at 177 source code
places while non-pointer data types are used 301 times for such a specific
parameter in corresponding function signatures.
Are such facts worth for further considerations?

Regards,
Markus

[Cocci] Searching for pass-through functions

[Julia Lawall]

On Wed, 8 Oct 2014, SF Markus Elfring wrote:

> > For example are you only interested in pointers?
>
> I get the impression from the source files within the software repository for
> the operating system "MINIX 3.3.0" that pointers are specified for input
> parameters (which will be also forwarded as return values) at 177 source code
> places while non-pointer data types are used 301 times for such a specific
> parameter in corresponding function signatures.
> Are such facts worth for further considerations?

I don't know.  It depends on what you specifically want to do with the
information.

julia

[Cocci] Searching for pass-through functions

[SF Markus Elfring]

> It depends on what you specifically want to do with the information.

I would like to reuse the extracted data to convert a function name list to a
corresponding SmPL constraint. It might become interesting again if a generated
long regular expression will work for further source code analysis.

Regards,
Markus

[Cocci] Searching for pass-through functions

[Julia Lawall]

On Wed, 8 Oct 2014, SF Markus Elfring wrote:

> > It depends on what you specifically want to do with the information.
>
> I would like to reuse the extracted data to convert a function name list to a
> corresponding SmPL constraint. It might become interesting again if a generated
> long regular expression will work for further source code analysis.

I see no reason why a regular expression would be more usefu than a simple
list of names.  That is, I would expect that the names have nothing to do
with each other.

julia

[Cocci] Searching for pass-through functions

[SF Markus Elfring]

> I see no reason why a regular expression would be more usefu than a simple
> list of names.

Would you like to suggest an approach for a search pattern that will work better
to exclude (!) some functions with a SmPL disjunction instead of a construction
with a long regular expression in a constraint?


> That is, I would expect that the names have nothing to do with each other.

Their relationship was determined by the shown analysis.

Regards,
Markus

[Cocci] Searching for pass-through functions

[Julia Lawall]

On Wed, 8 Oct 2014, SF Markus Elfring wrote:

> > I see no reason why a regular expression would be more usefu than a simple
> > list of names.
>
> Would you like to suggest an approach for a search pattern that will work better
> to exclude (!) some functions with a SmPL disjunction instead of a construction
> with a long regular expression in a constraint?

@@
identifier other_functions;
@@

(
 \(function\|names\|that\|are\|of\|no\|interest\)
|
* other_functions
)
  (...)

> > That is, I would expect that the names have nothing to do with each other.
>
> Their relationship was determined by the shown analysis.

Sure, but the only interest of a regular expression is to express, eg all
of the functions that have a name that ends in "alloc" in a concise way.

julia

[Cocci] Searching for pass-through functions

[SF Markus Elfring]

>> Their relationship was determined by the shown analysis.
> 
> Sure, but the only interest of a regular expression is to express, eg all
> of the functions that have a name that ends in "alloc" in a concise way.

I have got a different opinion. Now I am trying to refresh interest for the safe
determination of a well-known property (like from functions "strcat" and
"strcpy") and its consequence for return value checking.

Regular expressions can provide some functionality (in constraints) that are not
supported at other places in SmPL scripts so far.

Regards,
Markus

[Cocci] Searching for pass-through functions

[SF Markus Elfring]

> @@
> identifier other_functions;
> @@
> 
> (
>  \(function\|names\|that\|are\|of\|no\|interest\)
> |
> * other_functions
> )
>   (...)

Thanks for another example of a semantic patch rule.

Would any more fine-tuning make sense for the shown SmPL disjunction when it
gets longer?
Does the current Coccinelle software support to organise entries in such a name
list into subgroups where common prefixes or suffixes will be used?

Your suggestion comes close to my idea to work with the functionality "negative
lookaround" (in constraints) for some use cases.
http://www.regular-expressions.info/lookaround.html

Regards,
Markus

[Cocci] Searching for pass-through functions

[Julia Lawall]

On Wed, 8 Oct 2014, SF Markus Elfring wrote:

> > @@
> > identifier other_functions;
> > @@
> > 
> > (
> >  \(function\|names\|that\|are\|of\|no\|interest\)
> > |
> > * other_functions
> > )
> >   (...)
> 
> Thanks for another example of a semantic patch rule.
> 
> Would any more fine-tuning make sense for the shown SmPL disjunction when it
> gets longer?
> Does the current Coccinelle software support to organise entries in such a name
> list into subgroups where common prefixes or suffixes will be used?

No.

julia

> Your suggestion comes close to my idea to work with the functionality "negative
> lookaround" (in constraints) for some use cases.
> http://www.regular-expressions.info/lookaround.html
> 
> Regards,
> Markus
> 

[Cocci] Searching for pass-through functions

[SF Markus Elfring]

> I have retried a source code analysis approach which was discussed a while ago.

http://git.minix3.org/index.cgi?p=minix.git;a=summary

elfring at Sonne:~/Projekte/MINIX/lokal> date && spatch.opt -timeout 30 -sp-file
~/Projekte/Coccinelle/lokal/demos/pass-through/list_pass-through_functions.cocci
-dir . > list_pass-through_functions.txt 2>
list_pass-through_functions-errors.txt && date
Di 7. Okt 19:51:59 CEST 2014
Di 7. Okt 19:58:10 CEST 2014


The exceptions in the mentioned log file can be transformed into a small statistic.

Nr. | Message | Incidence
 1  | Failure("lexing: empty token") | 24
 2  | File "./ctlcocci_integration.ml", line 246, characters 4-10: Assertion
failed | 3
 3  | Common.Timeout | 3
 4  | Failure("is_pass_through: node 612: return ...[1,2,259] in getanswer
reachable by inconsistent control-flow paths") | 1
 5  | Failure("internal error: there is no noconst or novolatile keyword") | 1


How do you think about such a result?
Would you like to clarify corresponding details?

Regards,
Markus

[Cocci] Searching for pass-through functions

[Julia Lawall]

On Tue, 7 Oct 2014, SF Markus Elfring wrote:

> > I have retried a source code analysis approach which was discussed a while ago.
> 
> http://git.minix3.org/index.cgi?p=minix.git;a=summary
> 
> elfring at Sonne:~/Projekte/MINIX/lokal> date && spatch.opt -timeout 30 -sp-file
> ~/Projekte/Coccinelle/lokal/demos/pass-through/list_pass-through_functions.cocci
> -dir . > list_pass-through_functions.txt 2>
> list_pass-through_functions-errors.txt && date
> Di 7. Okt 19:51:59 CEST 2014
> Di 7. Okt 19:58:10 CEST 2014
> 
> 
> The exceptions in the mentioned log file can be transformed into a small statistic.
> 
> Nr. | Message | Incidence
>  1  | Failure("lexing: empty token") | 24
>  2  | File "./ctlcocci_integration.ml", line 246, characters 4-10: Assertion
> failed | 3
>  3  | Common.Timeout | 3
>  4  | Failure("is_pass_through: node 612: return ...[1,2,259] in getanswer
> reachable by inconsistent control-flow paths") | 1
>  5  | Failure("internal error: there is no noconst or novolatile keyword") | 1
> 
> 
> How do you think about such a result?
> Would you like to clarify corresponding details?

It is completely useless to me without knowing what C files are affected.

julia

[Cocci] Searching for pass-through functions

[SF Markus Elfring]

>> Would you like to clarify corresponding details?
> 
> It is completely useless to me without knowing what C files are affected.

They are stored in a public repository.
http://git.minix3.org/index.cgi?p=minix.git;a=tree

I guess that the 24 occurrences of the message 'Failure("lexing: empty token")'
can be discussed a bit later.


I got the message 'EXN:File "./ctlcocci_integration.ml", line 246, characters
4-10: Assertion failed' for the following source files.
1. external/bsd/llvm/dist/clang/test/Sema/statements.c
2. external/bsd/llvm/dist/clang/test/Sema/vla.c
3. external/bsd/llvm/dist/clang/test/Sema/scope-check.c


I got the message 'EXN:Common.Timeout' for the following source files.
1. sys/external/bsd/compiler_rt/dist/test/Unit/udivmodti4_test.c
2. external/bsd/llvm/dist/clang/test/Parser/parser_overflow.c
3. external/public-domain/sqlite/dist/sqlite3.c

The command-line parameter "show-trying" does not display more useful
information in these cases. Is this a bit strange?


I got the message 'EXN:Failure("is_pass_through: node 612: return ...[1,2,259]
in getanswer reachable by inconsistent control-flow paths")' for the source file
'lib/libc/net/gethnamaddr.c'.


I got the message 'EXN:Failure("internal error: there is no noconst or
novolatile keyword")' for the source file
'external/bsd/llvm/dist/clang/test/Sema/declspec.c'.

Regards,
Markus

[Cocci] Searching for pass-through functions

[Julia Lawall]

On Tue, 7 Oct 2014, SF Markus Elfring wrote:

> >> Would you like to clarify corresponding details?
> > 
> > It is completely useless to me without knowing what C files are affected.
> 
> They are stored in a public repository.
> http://git.minix3.org/index.cgi?p=minix.git;a=tree
> 
> I guess that the 24 occurrences of the message 'Failure("lexing: empty token")'
> can be discussed a bit later.
> 
> 
> I got the message 'EXN:File "./ctlcocci_integration.ml", line 246, characters
> 4-10: Assertion failed' for the following source files.
> 1. external/bsd/llvm/dist/clang/test/Sema/statements.c
> 2. external/bsd/llvm/dist/clang/test/Sema/vla.c
> 3. external/bsd/llvm/dist/clang/test/Sema/scope-check.c
> 
> 
> I got the message 'EXN:Common.Timeout' for the following source files.
> 1. sys/external/bsd/compiler_rt/dist/test/Unit/udivmodti4_test.c
> 2. external/bsd/llvm/dist/clang/test/Parser/parser_overflow.c
> 3. external/public-domain/sqlite/dist/sqlite3.c
> 
> The command-line parameter "show-trying" does not display more useful
> information in these cases. Is this a bit strange?
> 
> 
> I got the message 'EXN:Failure("is_pass_through: node 612: return ...[1,2,259]
> in getanswer reachable by inconsistent control-flow paths")' for the source file
> 'lib/libc/net/gethnamaddr.c'.
> 
> 
> I got the message 'EXN:Failure("internal error: there is no noconst or
> novolatile keyword")' for the source file
> 'external/bsd/llvm/dist/clang/test/Sema/declspec.c'.

OK, thanks.  I will take a look at the cases other than the timeouts.  
Timeouts seem inevitable for this rule.

julia