[Cocci] Q: stdint format specifiers

[Cocci] Q: stdint format specifiers

[Johann Klammer]

Does anyone have a coccinelle script to fixup *printf format specifiers to 
those pesky PRI*** ones? I could not find one on coccinellery <http://coccinellery.org/>.

[Cocci] Q: stdint format specifiers

[Julia Lawall]

On Thu, 4 Aug 2016, Johann Klammer wrote:

> Does anyone have a coccinelle script to fixup *printf format specifiers to
> those pesky PRI*** ones? I could not find one on coccinellery <http://coccinellery.org/>.

Could you give a more precise example of what you want to do?

thanks,
julia

>
> _______________________________________________
> Cocci mailing list
> Cocci at systeme.lip6.fr
> https://systeme.lip6.fr/mailman/listinfo/cocci
>

[Cocci] Q: stdint format specifiers

[Johann Klammer]

On 08/04/2016 06:33 PM, Julia Lawall wrote:
> On Thu, 4 Aug 2016, Johann Klammer wrote:
> 
>> Does anyone have a coccinelle script to fixup *printf format specifiers to
>> those pesky PRI*** ones? I could not find one on coccinellery <http://coccinellery.org/>.
> 
> Could you give a more precise example of what you want to do?
> 
> thanks,
> julia

I was getting a gcc warning:

> make[1]: Entering directory '/home/klammerj/projects/FPGA/src/fpgatools/libs'
>   CC        floorplan.o
> floorplan.c: In function ?printf_LOGIC?:
> floorplan.c:369:17: warning: format ?%lX? expects argument of type ?long unsigned int?, but argument 6 has type ?uint64_t {aka long long unsigned int}? [-Wformat=]
>       fprintf(f, "%s%s\"%c6_lut_val\" : \"0x%016lX\" }",

I've fixed those by hand, but I think not all cases will give 
warnings, because int etc are often aliases for those stdint things. 
..same width signedness..
Thus I'd like to run coccinelle on the .c files, to look for printfs 
which have uintxx intxx type arguments and replace the corresponding
specifiers by the ones suggested in the C standard:



> 7.8.1 Macros for format specifiers
> 1 Each of the following object-like macros expands to a character string literal containing a
> conversion specifier, possibly modified by a length modifier, suitable for use within the
> format argument of a formatted input/output function when converting the corresponding
> integer type. These macro names have the general form of PRI (character string literals
> for the fprintf and fwprintf family) or SCN (character string literals for the
> fscanf and fwscanf family), 220) followed by the conversion specifier, followed by a
> name corresponding to a similar type name in 7.20.1. In these names, N represents the
> width of the type as described in 7.20.1. For example, PRIdFAST32 can be used in a
> format string to print the value of an integer of type int_fast32_t.
> 2 The fprintf macros for signed integers are:
> PRIdN
> PRIiN
> 3
> PRIdFASTN
> PRIiFASTN
> PRIdMAX
> PRIiMAX PRIdPTR
> PRIiPTR
> PRIoMAX
> PRIuMAX
> PRIxMAX
> PRIXMAX PRIoPTR
> PRIuPTR
> PRIxPTR
> PRIXPTR
> The fprintf macros for unsigned integers are:
> PRIoN
> PRIuN
> PRIxN
> PRIXN
> 4
> PRIdLEASTN
> PRIiLEASTN
> PRIoLEASTN
> PRIuLEASTN
> PRIxLEASTN
> PRIXLEASTN
> PRIoFASTN
> PRIuFASTN
> PRIxFASTN
> PRIXFASTN
> The fscanf macros for signed integers are:
[...]

result something along the lines of:

> diff --git a/libs/floorplan.c b/libs/floorplan.c
> index 2b9b27a..fb27edb 100644
> --- a/libs/floorplan.c
> +++ b/libs/floorplan.c
> @@ -366,7 +366,7 @@ int printf_LOGIC(FILE* f, struct fpga_model* model,
>                         if (cfg->a2d[j].flags & LUT6VAL_SET) {
>                                 RC_ASSERT(model, !ULL_HIGH32(cfg->a2d[j].lut6_val));
>                                 if (print_hex_vals)
> -                                       fprintf(f, "%s%s\"%c6_lut_val\" : \"0x%016lX\" }",
> +                                       fprintf(f, "%s%s\"%c6_lut_val\" : \"0x%016"PRIX64"\" }",
>                                                 first_line ? "" : ",\n", pref, 'A'+j, cfg->a2d[j].lut6_val);
>                                 else {
>                                         str = bool_bits2str(cfg->a2d[j].lut6_val, 32);
> @@ -389,7 +389,7 @@ int printf_LOGIC(FILE* f, struct fpga_model* model,
>                 } else {
>                         if (cfg->a2d[j].flags & LUT6VAL_SET) {
>                                 if (print_hex_vals)
> -                                       fprintf(f, "%s%s\"%c6_lut_val\" : \"0x%016lX\" }",
> +                                       fprintf(f, "%s%s\"%c6_lut_val\" : \"0x%016"PRIX64"\" }",
>                                                 first_line ? "" : ",\n", pref, 'A'+j, cfg->a2d[j].lut6_val);
>                                 else {
>                                         str = bool_bits2str(cfg->a2d[j].lut6_val, 64);

[Cocci] Q: stdint format specifiers

[Julia Lawall]

On Thu, 4 Aug 2016, Johann Klammer wrote:

> On 08/04/2016 06:33 PM, Julia Lawall wrote:
> > On Thu, 4 Aug 2016, Johann Klammer wrote:
> >
> >> Does anyone have a coccinelle script to fixup *printf format specifiers to
> >> those pesky PRI*** ones? I could not find one on coccinellery <http://coccinellery.org/>.
> >
> > Could you give a more precise example of what you want to do?
> >
> > thanks,
> > julia
>
> I was getting a gcc warning:
>
> > make[1]: Entering directory '/home/klammerj/projects/FPGA/src/fpgatools/libs'
> >   CC        floorplan.o
> > floorplan.c: In function ?printf_LOGIC?:
> > floorplan.c:369:17: warning: format ?%lX? expects argument of type ?long unsigned int?, but argument 6 has type ?uint64_t {aka long long unsigned int}? [-Wformat=]
> >       fprintf(f, "%s%s\"%c6_lut_val\" : \"0x%016lX\" }",
>
> I've fixed those by hand, but I think not all cases will give
> warnings, because int etc are often aliases for those stdint things.
> ..same width signedness..
> Thus I'd like to run coccinelle on the .c files, to look for printfs
> which have uintxx intxx type arguments and replace the corresponding
> specifiers by the ones suggested in the C standard:

Coccinelle makes a small effort to help with parsing format strings, but
it doesn't support the PRI things you mention.  What you could do is
something like this:

@r@
{suspicious_type1,suspicious_type2,...} x;
type T;
T y;
expression list[n] es;
constant char[] c;
identifier f;
position p;
@@

f(...,c at p,es,x at y,...)

@script:python s@
T << r.T;  // the type of the argument of interest
c << r.c;  // the format string
n << r.n;  // the offset of the argument of interest, counted from 0
c2;
@@

Now use python to compute a new format string for your type.
coccinelle.c2 = the format string with extra quotes around it

@@
expression e;
position r.p;
identifier s.c2;
@@

- e at p
+ c2

Unfortunately, however, this will not work if you have a print with two
suspicious arguments.  Indeed, Coccinelle will completely crash on that
file, because it has no way to merge two changes made for independent
reasons.

If the problem arises frequently and you are willing to do some work by
hand, then you could just let the format strings accumulate before the
calls and replace/merge them by hand.  The third rule would become:

@@
expression e;
position r.p;
identifier r.f,s.c2;
@@

++c2;
  f(...,e at p,...);

If that is not acceptable, you can use a hash table to collect all the
changes that are needed in a particular format string, and then work on
them all at once.  The second rule would put the T n p and c information
in a hashtable (you would need to add the declaration p << r.p; for this),
and then you would need another python rule with only the metavariable p
<< r.p; and the variale c2 that would collect the information relevant to
the position p from the hash table and make the appropriate format string.
The last rule would be just

@@
expression e;
position r.p;
identifier s.c2;
@@

- e@p
+ c2

in that case.

I can elaborate more if needed on any of the above options.

julia

>
>
>
> > 7.8.1 Macros for format specifiers
> > 1 Each of the following object-like macros expands to a character string literal containing a
> > conversion specifier, possibly modified by a length modifier, suitable for use within the
> > format argument of a formatted input/output function when converting the corresponding
> > integer type. These macro names have the general form of PRI (character string literals
> > for the fprintf and fwprintf family) or SCN (character string literals for the
> > fscanf and fwscanf family), 220) followed by the conversion specifier, followed by a
> > name corresponding to a similar type name in 7.20.1. In these names, N represents the
> > width of the type as described in 7.20.1. For example, PRIdFAST32 can be used in a
> > format string to print the value of an integer of type int_fast32_t.
> > 2 The fprintf macros for signed integers are:
> > PRIdN
> > PRIiN
> > 3
> > PRIdFASTN
> > PRIiFASTN
> > PRIdMAX
> > PRIiMAX PRIdPTR
> > PRIiPTR
> > PRIoMAX
> > PRIuMAX
> > PRIxMAX
> > PRIXMAX PRIoPTR
> > PRIuPTR
> > PRIxPTR
> > PRIXPTR
> > The fprintf macros for unsigned integers are:
> > PRIoN
> > PRIuN
> > PRIxN
> > PRIXN
> > 4
> > PRIdLEASTN
> > PRIiLEASTN
> > PRIoLEASTN
> > PRIuLEASTN
> > PRIxLEASTN
> > PRIXLEASTN
> > PRIoFASTN
> > PRIuFASTN
> > PRIxFASTN
> > PRIXFASTN
> > The fscanf macros for signed integers are:
> [...]
>
> result something along the lines of:
>
> > diff --git a/libs/floorplan.c b/libs/floorplan.c
> > index 2b9b27a..fb27edb 100644
> > --- a/libs/floorplan.c
> > +++ b/libs/floorplan.c
> > @@ -366,7 +366,7 @@ int printf_LOGIC(FILE* f, struct fpga_model* model,
> >                         if (cfg->a2d[j].flags & LUT6VAL_SET) {
> >                                 RC_ASSERT(model, !ULL_HIGH32(cfg->a2d[j].lut6_val));
> >                                 if (print_hex_vals)
> > -                                       fprintf(f, "%s%s\"%c6_lut_val\" : \"0x%016lX\" }",
> > +                                       fprintf(f, "%s%s\"%c6_lut_val\" : \"0x%016"PRIX64"\" }",
> >                                                 first_line ? "" : ",\n", pref, 'A'+j, cfg->a2d[j].lut6_val);
> >                                 else {
> >                                         str = bool_bits2str(cfg->a2d[j].lut6_val, 32);
> > @@ -389,7 +389,7 @@ int printf_LOGIC(FILE* f, struct fpga_model* model,
> >                 } else {
> >                         if (cfg->a2d[j].flags & LUT6VAL_SET) {
> >                                 if (print_hex_vals)
> > -                                       fprintf(f, "%s%s\"%c6_lut_val\" : \"0x%016lX\" }",
> > +                                       fprintf(f, "%s%s\"%c6_lut_val\" : \"0x%016"PRIX64"\" }",
> >                                                 first_line ? "" : ",\n", pref, 'A'+j, cfg->a2d[j].lut6_val);
> >                                 else {
> >                                         str = bool_bits2str(cfg->a2d[j].lut6_val, 64);
>
> _______________________________________________
> Cocci mailing list
> Cocci at systeme.lip6.fr
> https://systeme.lip6.fr/mailman/listinfo/cocci
>

[Cocci] Q: stdint format specifiers

[Johann Klammer]

Mkay...

I tried.
> @r@
> {int8_t,int16_t,int32_t,int64_t,
> uint8_t,uint16_t,uint32_t,uint64_t} x;
> type T;
> T y;
> expression list[n] es;
> constant char[] c;
> identifier f;
> position p;
> @@
> 
> f(...,c at p,es,x at y,...)
> 
> @script:python s@
> T << r.T;  // the type of the argument of interest
> c << r.c;  // the format string
> n << r.n;  // the offset of the argument of interest, counted from 0
> c2;
> @@
fails with:

> ~/projects/FPGA/src/fpgatools/libs$ spatch --sp-file ../pri.sp ./*.c
> init_defs_builtins: /usr/lib/coccinelle/standard.h
> 5 11
> Fatal error: exception Failure("meta: parse error: \n = File \"../pri.sp\", line 2, column 1,  charpos = 5\n    around = 'int8_t', whole content = {int8_t,int16_t,int32_t,int64_t,\n")

looking at the main_grammar.pdf reveals that the typedefs are resolved....
Is there a way to match the original typename? 
If someone actually wrote long int, the lX is perfectly fine. 
if it's one of the stdints it'll change on different subarchs, so it needs the PRI*.

[Cocci] Q: stdint format specifiers

[Julia Lawall]

On Thu, 4 Aug 2016, Johann Klammer wrote:

> Mkay...
>
> I tried.
> > @r@
> > {int8_t,int16_t,int32_t,int64_t,
> > uint8_t,uint16_t,uint32_t,uint64_t} x;
> > type T;
> > T y;
> > expression list[n] es;
> > constant char[] c;
> > identifier f;
> > position p;
> > @@
> >
> > f(...,c at p,es,x at y,...)
> >
> > @script:python s@
> > T << r.T;  // the type of the argument of interest
> > c << r.c;  // the format string
> > n << r.n;  // the offset of the argument of interest, counted from 0
> > c2;
> > @@
> fails with:
>
> > ~/projects/FPGA/src/fpgatools/libs$ spatch --sp-file ../pri.sp ./*.c
> > init_defs_builtins: /usr/lib/coccinelle/standard.h
> > 5 11
> > Fatal error: exception Failure("meta: parse error: \n = File \"../pri.sp\", line 2, column 1,  charpos = 5\n    around = 'int8_t', whole content = {int8_t,int16_t,int32_t,int64_t,\n")

Add typedef int8_t,int16_t,int32_t,int64_t,etc; before the definition of
x.

> looking at the main_grammar.pdf reveals that the typedefs are resolved....

I'm not sure to understand what you mean by "resolved".  If Coccinelle can
see the typedef definition, then it will keep track of both the typedef
name and the underlying type name.  But by default, Coccinelle only picks
up on header files that are in the same directory as the .c file and
header files in the include path that have the same name as the .c file.
To get more, you need to give the option --all-includes (all explicitly
mentioned header files) or --recursive-includes (also header files that
are included by other header files).  But both of these options are likely
to cost a lot in performance.  So if you are ok with the type names as
they are written in the code, you are better off.  If you want to be sure
that Coccinelle won't find a definition of a typdef in an include file,
you can use --no-includes.

But note that include files also may contain structure type definitions,
which may also be useful.

julia

> Is there a way to match the original typename?
> If someone actually wrote long int, the lX is perfectly fine.
> if it's one of the stdints it'll change on different subarchs, so it needs the PRI*.
>
>
>
> _______________________________________________
> Cocci mailing list
> Cocci at systeme.lip6.fr
> https://systeme.lip6.fr/mailman/listinfo/cocci
>

[Cocci] [Solved] Q: stdint format specifiers

[Johann Klammer]

//kinda works now ..kinda
@initialize:python@
@@
coccinelle.pd={}

@r@
typedef int8_t,int16_t,int32_t,int64_t,uint8_t,uint16_t,uint32_t,uint64_t;
{int8_t,int16_t,int32_t,int64_t,
uint8_t,uint16_t,uint32_t,uint64_t} x;
type T;
T y;
expression list[n] es;
constant char[] c;
identifier f =~ ".*printf$";
position p;
@@

f(...,c at p,es,x at y,...)

@script:python collect@
p << r.p;  // position
f << r.f;  // func name
T << r.T;  // the type of the argument of interest
c << r.c;  // the format string
n << r.n;  // the offset of the argument of interest, counted from 0
@@
import re
trex = re.compile("((uint\\S*_t)|(int\\S*_t))")//to match only typename(no qualifiers)
m=trex.search(T)
t=m.group(1)
ps=("%s:(%s,%s),(%s,%s)::" % (p[0].file,p[0].line,p[0].column,p[0].line_end,p[0].column_end))
if (ps in coccinelle.pd):
	coccinelle.pd[ps][1][n]=t
else:
	coccinelle.pd[ps]=[c,{n:t}]

@script:python fmt@
p << r.p;  // position
c2;
@@
ps=("%s:(%s,%s),(%s,%s)::" % (p[0].file,p[0].line,p[0].column,p[0].line_end,p[0].column_end))
in_str=False//inside string lit
in_pesc=False//seen a %
in_besc=False//seen a \
tps=coccinelle.pd[ps][1]//the types
import re
prex = re.compile("\s*(PRI([diouxX])(8|16|32|64))\s*")//to match existing ones...
pmap= {
"int8_t"  :{"c":"\"c\"","d":"PRId8",  "i":"PRIi8",  "o":"PRIo8",  "u":"PRIu8",  "x":"PRIx8",  "X":"PRIX8"},
"int16_t" :{"d":"PRId16", "i":"PRIi16", "o":"PRIo16", "u":"PRIu16", "x":"PRIx16", "X":"PRIX16"},
"int32_t" :{"d":"PRId32", "i":"PRIi32", "o":"PRIo32", "u":"PRIu32", "x":"PRIx32", "X":"PRIX32"},
"int64_t" :{"d":"PRId64", "i":"PRIi64", "o":"PRIo64", "u":"PRIu64", "x":"PRIx64", "X":"PRIX64"},
"uint8_t" :{"c":"\"c\"","d":"PRIu8",  "i":"PRIu8",  "o":"PRIo8",  "u":"PRIu8",  "x":"PRIx8",  "X":"PRIX8"},
"uint16_t":{"d":"PRIu16", "i":"PRIu16", "o":"PRIo16", "u":"PRIu16", "x":"PRIx16", "X":"PRIX16"},
"uint32_t":{"d":"PRIu32", "i":"PRIu32", "o":"PRIo32", "u":"PRIu32", "x":"PRIx32", "X":"PRIX32"},
"uint64_t":{"d":"PRIu64", "i":"PRIu64", "o":"PRIo64", "u":"PRIu64", "x":"PRIx64", "X":"PRIX64"},
}
fmt_ctr=0//some index
str_a=coccinelle.pd[ps][0]
coccinelle.c2=""
i=0
a=-1
while True:
	if i>=len(str_a):
		break
	c=str_a[i]
	if in_str==True:
		if in_pesc==True:
			if c=="%":
				in_pesc=False
				coccinelle.c2="%s%s" % (coccinelle.c2,c)
				i+=1
			else:
				if c in "hlLqjzt"://size modifiers. mem index of first one
					if a==-1:
						a=i
					i+=1
				elif c in "diouxXEeFfGgAacsCSP":
					in_pesc=False
					if a==-1:
						a=i
					fs="%s" % (fmt_ctr)
					if fs in tps:
						coccinelle.c2="%s\"%s\"" % (coccinelle.c2,pmap[tps[fs]][c])
					else:
						coccinelle.c2="%s%s" % (coccinelle.c2,str_a[a:(i+1)])
					fmt_ctr+=1
					i+=1
				else://just append the prec stuff...
					if c=="\"":
						in_str=False
					coccinelle.c2="%s%s" % (coccinelle.c2,c)
					i+=1
		elif in_besc==True:
			in_besc=False
			coccinelle.c2="%s%s" % (coccinelle.c2,c)
			i+=1
		else:
			if c=="\\":
				in_besc=True
			elif c=="%":
				in_pesc=True
				a=-1
			elif c=="\"":
				in_str=False
			coccinelle.c2="%s%s" % (coccinelle.c2,c)
			i+=1
	else:
		if c=="\"":
			in_str=True
			coccinelle.c2="%s%s" % (coccinelle.c2,c)
			i+=1;
		else:
			if in_pesc==True:
				m=prex.match(str_a[i:])
				fs="%s" % (fmt_ctr)
				if m and (fs in tps):
					i+=m.end();
					coccinelle.c2="%s%s" % (coccinelle.c2,pmap[tps[fs]][m.group(2)])
					fmt_ctr+=1
					in_pesc=False
				else:
					coccinelle.c2="%s%s" % (coccinelle.c2,c)
					i+=1
			else:
				coccinelle.c2="%s%s" % (coccinelle.c2,c)
				i+=1;
@q@
expression e;
position r.p;
identifier fmt.c2;
@@
- e at p
+ c2

[Cocci] [Solved] Q: stdint format specifiers

[Johann Klammer]

On 08/08/2016 07:46 AM, Julia Lawall wrote:
> On Mon, 8 Aug 2016, Johann Klammer wrote:
> 
>> //kinda works now ..kinda
> 
> It does rather lack elegance...
> 
> Does your software always use print variants for printing?  If there are
> some wrappers, you could see what happens without the constraint on f.
> 
> julia
> 
No, and I had to change that pattern a few times.
the thing is that scanf variants use different macros. 
and I can't really see what except for the func name 
would prevent me from matching those.