[PATCH V2 1/2] dtc: cpp co-existence: allow names starting with # to be escaped

[PATCH V2 1/2] dtc: cpp co-existence: allow names starting with # to be escaped

[Stephen Warren]

From: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>

The device tree language as currently defined conflicts with the C pre-
processor in one aspect - when a property or node name begins with a #
character, a pre-processor would attempt to interpret it as a directive,
fail, and most likely error out.

This change allows a property/node name to be prefixed with \. This
prevents a pre-processor from seeing # as the first non-whitespace
character on the line, and hence prevents the conflict. \ was previously
an illegal character in property/node names, so this change is
backwards compatible. The \ is stripped from the name during parsing
by dtc.

Signed-off-by: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
---
 dtc-lexer.l                |    5 +++--
 tests/.gitignore           |    1 +
 tests/Makefile.tests       |    2 +-
 tests/propname_escapes.c   |   43 +++++++++++++++++++++++++++++++++++++++++++
 tests/propname_escapes.dts |    6 ++++++
 tests/run_tests.sh         |    3 +++
 6 files changed, 57 insertions(+), 3 deletions(-)
 create mode 100644 tests/propname_escapes.c
 create mode 100644 tests/propname_escapes.dts

diff --git a/dtc-lexer.l b/dtc-lexer.l
index 91c4930..edbeb86 100644
--- a/dtc-lexer.l
+++ b/dtc-lexer.l
@@ -162,9 +162,10 @@ static int pop_input_file(void);
 			return ']';
 		}
 
-<PROPNODENAME>{PROPNODECHAR}+ {
+<PROPNODENAME>\\?{PROPNODECHAR}+ {
 			DPRINT("PropNodeName: %s\n", yytext);
-			yylval.propnodename = xstrdup(yytext);
+			yylval.propnodename = xstrdup((yytext[0] == '\\') ?
+							yytext + 1 : yytext);
 			BEGIN_DEFAULT();
 			return DT_PROPNODENAME;
 		}
diff --git a/tests/.gitignore b/tests/.gitignore
index f8e1af0..e2aa24a 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -39,6 +39,7 @@ tmp.*
 /path_offset
 /path_offset_aliases
 /phandle_format
+/propname_escapes
 /references
 /root_node
 /rw_tree1
diff --git a/tests/Makefile.tests b/tests/Makefile.tests
index 1795466..d59bff8 100644
--- a/tests/Makefile.tests
+++ b/tests/Makefile.tests
@@ -12,7 +12,7 @@ LIB_TESTS_L = get_mem_rsv \
 	sw_tree1 \
 	move_and_save mangle-layout nopulate \
 	open_pack rw_tree1 set_name setprop del_property del_node \
-	appendprop1 appendprop2 \
+	appendprop1 appendprop2 propname_escapes \
 	string_escapes references path-references phandle_format \
 	boot-cpuid incbin \
 	extra-terminating-null \
diff --git a/tests/propname_escapes.c b/tests/propname_escapes.c
new file mode 100644
index 0000000..3aec28f
--- /dev/null
+++ b/tests/propname_escapes.c
@@ -0,0 +1,43 @@
+/*
+ * libfdt - Flat Device Tree manipulation
+ *	Testcase for fdt_getprop()
+ * Copyright (C) 2006 David Gibson, IBM Corporation.
+ * Copyright (C) 2012 NVIDIA CORPORATION. All rights reserved.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public License
+ * as published by the Free Software Foundation; either version 2.1 of
+ * the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdint.h>
+
+#include <fdt.h>
+#include <libfdt.h>
+
+#include "tests.h"
+
+int main(int argc, char *argv[])
+{
+	void *fdt;
+
+	test_init(argc, argv);
+	fdt = load_blob_arg(argc, argv);
+
+	check_getprop_cell(fdt, 0, "#address-cells", 1);
+	check_getprop_cell(fdt, 0, "#gpio-cells", 2);
+
+	PASS();
+}
diff --git a/tests/propname_escapes.dts b/tests/propname_escapes.dts
new file mode 100644
index 0000000..9f70618
--- /dev/null
+++ b/tests/propname_escapes.dts
@@ -0,0 +1,6 @@
+/dts-v1/;
+
+/ {
+	#address-cells = <1>;
+	\#gpio-cells = <2>;
+};
diff --git a/tests/run_tests.sh b/tests/run_tests.sh
index e2158f7..7d7a5f7 100755
--- a/tests/run_tests.sh
+++ b/tests/run_tests.sh
@@ -254,6 +254,9 @@ dtc_tests () {
     tree1_tests_rw dtc_tree1.test.dtb
     run_test dtbs_equal_ordered dtc_tree1.test.dtb test_tree1.dtb
 
+    run_dtc_test -I dts -O dtb -o dtc_escapes.test.dtb propname_escapes.dts
+    run_test propname_escapes dtc_escapes.test.dtb
+
     run_dtc_test -I dts -O dtb -o dtc_escapes.test.dtb escapes.dts
     run_test string_escapes dtc_escapes.test.dtb
 
-- 
1.7.0.4

[PATCH V2 2/2] dtc: cpp co-existence: add support for #line directives

[Stephen Warren]

From: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>

Line control directives of the following formats are supported:
    #line LINE "FILE"
    # LINE "FILE" [FLAGS]

This allows dtc to consume the output of pre-processors, and to provide
error messages that refer to the original filename, including taking
into account any #include directives that the pre-processor may have
performed.

Signed-off-by: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
---
v2: Only match #line directives at the start of the line.
---
 dtc-lexer.l                |   21 +++++++++++++++++++++
 srcpos.c                   |    6 ++++++
 srcpos.h                   |    2 ++
 tests/propname_escapes.dts |    7 +++++++
 4 files changed, 36 insertions(+), 0 deletions(-)

diff --git a/dtc-lexer.l b/dtc-lexer.l
index edbeb86..ce7ee06 100644
--- a/dtc-lexer.l
+++ b/dtc-lexer.l
@@ -71,6 +71,27 @@ static int pop_input_file(void);
 			push_input_file(name);
 		}
 
+<*>^"#"(line{WS}|" "){WS}*[0-9]+{WS}+{STRING}({WS}+[0-9]+)? {
+			char *line, *tmp, *fn;
+			/* skip text before line # */
+			line = yytext;
+			while (!isdigit(*line))
+				line++;
+			/* skip digits in line # */
+			tmp = line;
+			while (!isspace(*tmp))
+				tmp++;
+			/* "NULL"-terminate line # */
+			*tmp = '\0';
+			/* start of filename */
+			fn = strchr(tmp + 1, '"') + 1;
+			/* strip trailing " from filename */
+			tmp = strchr(fn, '"');
+			*tmp = 0;
+			/* -1 since #line is the number of the next line */
+			srcpos_set_line(xstrdup(fn), atoi(line) - 1);
+		}
+
 <*><<EOF>>		{
 			if (!pop_input_file()) {
 				yyterminate();
diff --git a/srcpos.c b/srcpos.c
index 3ee523d..246ab4b 100644
--- a/srcpos.c
+++ b/srcpos.c
@@ -328,3 +328,9 @@ srcpos_warn(struct srcpos *pos, char const *fmt, ...)
 
 	va_end(va);
 }
+
+void srcpos_set_line(char *f, int l)
+{
+	current_srcfile->name = f;
+	current_srcfile->lineno = l;
+}
diff --git a/srcpos.h b/srcpos.h
index 5617916..93a2712 100644
--- a/srcpos.h
+++ b/srcpos.h
@@ -113,4 +113,6 @@ extern void srcpos_error(struct srcpos *pos, char const *, ...)
 extern void srcpos_warn(struct srcpos *pos, char const *, ...)
      __attribute__((format(printf, 2, 3)));
 
+extern void srcpos_set_line(char *f, int l);
+
 #endif /* _SRCPOS_H_ */
diff --git a/tests/propname_escapes.dts b/tests/propname_escapes.dts
index 9f70618..9f5793d 100644
--- a/tests/propname_escapes.dts
+++ b/tests/propname_escapes.dts
@@ -1,5 +1,12 @@
 /dts-v1/;
 
+/* common format */
+#line 3 "foo.dts"
+/* newer gcc format */
+# 6 "bar.dts"
+/* newer gcc sometimes uses */
+# 9 "baz.dts" 1
+
 / {
 	#address-cells = <1>;
 	\#gpio-cells = <2>;
-- 
1.7.0.4

Re: [PATCH V2 2/2] dtc: cpp co-existence: add support for #line directives

[David Gibson]

On Wed, Sep 26, 2012 at 10:18:05AM -0600, Stephen Warren wrote:
> From: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
> 
> Line control directives of the following formats are supported:
>     #line LINE "FILE"
>     # LINE "FILE" [FLAGS]
> 
> This allows dtc to consume the output of pre-processors, and to provide
> error messages that refer to the original filename, including taking
> into account any #include directives that the pre-processor may have
> performed.
> 
> Signed-off-by: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>

Nice.  Two small changes I'd like to see:

> +<*>^"#"(line{WS}|" "){WS}*[0-9]+{WS}+{STRING}({WS}+[0-9]+)? {

I think this can be done slightly more cleanly as:
^"#"(line)?{WS}+[0-9]+{WS}+{STRING}({WS}+[0-9]+)?

[snip]
> diff --git a/tests/propname_escapes.dts b/tests/propname_escapes.dts
> index 9f70618..9f5793d 100644
> --- a/tests/propname_escapes.dts
> +++ b/tests/propname_escapes.dts
> @@ -1,5 +1,12 @@
>  /dts-v1/;
>  
> +/* common format */
> +#line 3 "foo.dts"
> +/* newer gcc format */
> +# 6 "bar.dts"
> +/* newer gcc sometimes uses */
> +# 9 "baz.dts" 1
> +

I'd prefer a separate testcase for the line directives, rather than
piggybacking on the propnodename escaping testcase.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [PATCH V2 2/2] dtc: cpp co-existence: add support for #line directives

[Stephen Warren]

On 09/27/2012 01:07 AM, David Gibson wrote:
> On Wed, Sep 26, 2012 at 10:18:05AM -0600, Stephen Warren wrote:
>> From: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
>>
>> Line control directives of the following formats are supported:
>>     #line LINE "FILE"
>>     # LINE "FILE" [FLAGS]
>>
>> This allows dtc to consume the output of pre-processors, and to provide
>> error messages that refer to the original filename, including taking
>> into account any #include directives that the pre-processor may have
>> performed.
>>
>> Signed-off-by: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
> 
> Nice.  Two small changes I'd like to see:
> 
>> +<*>^"#"(line{WS}|" "){WS}*[0-9]+{WS}+{STRING}({WS}+[0-9]+)? {
> 
> I think this can be done slightly more cleanly as:
> ^"#"(line)?{WS}+[0-9]+{WS}+{STRING}({WS}+[0-9]+)?

Did you mean to ommit the <*> from the beginning there? I'm not sure
what the difference is between no start state tag and what I assume is a
wildcard tag.

Other than that, it looks like that should work; I'll try it out.