[PATCH 0/2] dtc: cpp co-existence

[PATCH 0/2] dtc: cpp co-existence

[Stephen Warren]

From: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>

This series is a minimal set of changes to dtc that allows one to run
cpp on a .dts file, and pass the result to dtc for final compilation.

Stephen Warren (2):
  dtc: cpp co-existence: allow names starting with # to be escaped
  dtc: cpp co-existence: add support for #line directives

 dtc-lexer.l                |   26 ++++++++++++++++++++++++--
 srcpos.c                   |    6 ++++++
 srcpos.h                   |    2 ++
 tests/.gitignore           |    1 +
 tests/Makefile.tests       |    2 +-
 tests/propname_escapes.c   |   43 +++++++++++++++++++++++++++++++++++++++++++
 tests/propname_escapes.dts |   13 +++++++++++++
 tests/run_tests.sh         |    3 +++
 8 files changed, 93 insertions(+), 3 deletions(-)
 create mode 100644 tests/propname_escapes.c
 create mode 100644 tests/propname_escapes.dts

[PATCH 1/2] dtc: cpp co-existence: allow names starting with # to be escaped

[Stephen Warren]

From: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>

The device tree language as currently defined conflicts with the C pre-
processor in one aspect - when a property or node name begins with a #
character, a pre-processor would attempt to interpret it as a directive,
fail, and most likely error out.

This change allows a property/node name to be prefixed with \. This
prevents a pre-processor from seeing # as the first non-whitespace
character on the line, and hence prevents the conflict. \ was previously
an illegal character in property/node names, so this change is
backwards compatible. The \ is stripped from the name during parsing
by dtc.

Signed-off-by: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
---
 dtc-lexer.l                |    5 +++--
 tests/.gitignore           |    1 +
 tests/Makefile.tests       |    2 +-
 tests/propname_escapes.c   |   43 +++++++++++++++++++++++++++++++++++++++++++
 tests/propname_escapes.dts |    6 ++++++
 tests/run_tests.sh         |    3 +++
 6 files changed, 57 insertions(+), 3 deletions(-)
 create mode 100644 tests/propname_escapes.c
 create mode 100644 tests/propname_escapes.dts

diff --git a/dtc-lexer.l b/dtc-lexer.l
index 91c4930..edbeb86 100644
--- a/dtc-lexer.l
+++ b/dtc-lexer.l
@@ -162,9 +162,10 @@ static int pop_input_file(void);
 			return ']';
 		}
 
-<PROPNODENAME>{PROPNODECHAR}+ {
+<PROPNODENAME>\\?{PROPNODECHAR}+ {
 			DPRINT("PropNodeName: %s\n", yytext);
-			yylval.propnodename = xstrdup(yytext);
+			yylval.propnodename = xstrdup((yytext[0] == '\\') ?
+							yytext + 1 : yytext);
 			BEGIN_DEFAULT();
 			return DT_PROPNODENAME;
 		}
diff --git a/tests/.gitignore b/tests/.gitignore
index f8e1af0..e2aa24a 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -39,6 +39,7 @@ tmp.*
 /path_offset
 /path_offset_aliases
 /phandle_format
+/propname_escapes
 /references
 /root_node
 /rw_tree1
diff --git a/tests/Makefile.tests b/tests/Makefile.tests
index 1795466..d59bff8 100644
--- a/tests/Makefile.tests
+++ b/tests/Makefile.tests
@@ -12,7 +12,7 @@ LIB_TESTS_L = get_mem_rsv \
 	sw_tree1 \
 	move_and_save mangle-layout nopulate \
 	open_pack rw_tree1 set_name setprop del_property del_node \
-	appendprop1 appendprop2 \
+	appendprop1 appendprop2 propname_escapes \
 	string_escapes references path-references phandle_format \
 	boot-cpuid incbin \
 	extra-terminating-null \
diff --git a/tests/propname_escapes.c b/tests/propname_escapes.c
new file mode 100644
index 0000000..3aec28f
--- /dev/null
+++ b/tests/propname_escapes.c
@@ -0,0 +1,43 @@
+/*
+ * libfdt - Flat Device Tree manipulation
+ *	Testcase for fdt_getprop()
+ * Copyright (C) 2006 David Gibson, IBM Corporation.
+ * Copyright (C) 2012 NVIDIA CORPORATION. All rights reserved.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public License
+ * as published by the Free Software Foundation; either version 2.1 of
+ * the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdint.h>
+
+#include <fdt.h>
+#include <libfdt.h>
+
+#include "tests.h"
+
+int main(int argc, char *argv[])
+{
+	void *fdt;
+
+	test_init(argc, argv);
+	fdt = load_blob_arg(argc, argv);
+
+	check_getprop_cell(fdt, 0, "#address-cells", 1);
+	check_getprop_cell(fdt, 0, "#gpio-cells", 2);
+
+	PASS();
+}
diff --git a/tests/propname_escapes.dts b/tests/propname_escapes.dts
new file mode 100644
index 0000000..9f70618
--- /dev/null
+++ b/tests/propname_escapes.dts
@@ -0,0 +1,6 @@
+/dts-v1/;
+
+/ {
+	#address-cells = <1>;
+	\#gpio-cells = <2>;
+};
diff --git a/tests/run_tests.sh b/tests/run_tests.sh
index e2158f7..7d7a5f7 100755
--- a/tests/run_tests.sh
+++ b/tests/run_tests.sh
@@ -254,6 +254,9 @@ dtc_tests () {
     tree1_tests_rw dtc_tree1.test.dtb
     run_test dtbs_equal_ordered dtc_tree1.test.dtb test_tree1.dtb
 
+    run_dtc_test -I dts -O dtb -o dtc_escapes.test.dtb propname_escapes.dts
+    run_test propname_escapes dtc_escapes.test.dtb
+
     run_dtc_test -I dts -O dtb -o dtc_escapes.test.dtb escapes.dts
     run_test string_escapes dtc_escapes.test.dtb
 
-- 
1.7.0.4

[PATCH 2/2] dtc: cpp co-existence: add support for #line directives

[Stephen Warren]

From: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>

Line control directives of the following formats are supported:
    #line LINE "FILE"
    # LINE FILE [FLAGS]

This allows dtc to consume the output of pre-processors, and to provide
error messages that refer to the original filename, including taking
into account any #include directives that the pre-processor may have
performed.

Signed-off-by: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
---
 dtc-lexer.l                |   21 +++++++++++++++++++++
 srcpos.c                   |    6 ++++++
 srcpos.h                   |    2 ++
 tests/propname_escapes.dts |    7 +++++++
 4 files changed, 36 insertions(+), 0 deletions(-)

diff --git a/dtc-lexer.l b/dtc-lexer.l
index edbeb86..506a1bf 100644
--- a/dtc-lexer.l
+++ b/dtc-lexer.l
@@ -71,6 +71,27 @@ static int pop_input_file(void);
 			push_input_file(name);
 		}
 
+<*>"#"(line{WS}|" "){WS}*[0-9]+{WS}+{STRING}({WS}+[0-9]+)? {
+			char *line, *tmp, *fn;
+			/* skip text before line # */
+			line = yytext;
+			while (!isdigit(*line))
+				line++;
+			/* skip digits in line # */
+			tmp = line;
+			while (!isspace(*tmp))
+				tmp++;
+			/* "NULL"-terminate line # */
+			*tmp = '\0';
+			/* start of filename */
+			fn = strchr(tmp + 1, '"') + 1;
+			/* strip trailing " from filename */
+			tmp = strchr(fn, '"');
+			*tmp = 0;
+			/* -1 since #line is the number of the next line */
+			srcpos_set_line(xstrdup(fn), atoi(line) - 1);
+		}
+
 <*><<EOF>>		{
 			if (!pop_input_file()) {
 				yyterminate();
diff --git a/srcpos.c b/srcpos.c
index 3ee523d..246ab4b 100644
--- a/srcpos.c
+++ b/srcpos.c
@@ -328,3 +328,9 @@ srcpos_warn(struct srcpos *pos, char const *fmt, ...)
 
 	va_end(va);
 }
+
+void srcpos_set_line(char *f, int l)
+{
+	current_srcfile->name = f;
+	current_srcfile->lineno = l;
+}
diff --git a/srcpos.h b/srcpos.h
index 5617916..93a2712 100644
--- a/srcpos.h
+++ b/srcpos.h
@@ -113,4 +113,6 @@ extern void srcpos_error(struct srcpos *pos, char const *, ...)
 extern void srcpos_warn(struct srcpos *pos, char const *, ...)
      __attribute__((format(printf, 2, 3)));
 
+extern void srcpos_set_line(char *f, int l);
+
 #endif /* _SRCPOS_H_ */
diff --git a/tests/propname_escapes.dts b/tests/propname_escapes.dts
index 9f70618..9f5793d 100644
--- a/tests/propname_escapes.dts
+++ b/tests/propname_escapes.dts
@@ -1,5 +1,12 @@
 /dts-v1/;
 
+/* common format */
+#line 3 "foo.dts"
+/* newer gcc format */
+# 6 "bar.dts"
+/* newer gcc sometimes uses */
+# 9 "baz.dts" 1
+
 / {
 	#address-cells = <1>;
 	\#gpio-cells = <2>;
-- 
1.7.0.4

Re: [PATCH 2/2] dtc: cpp co-existence: add support for #line directives

[David Gibson]

On Mon, Sep 24, 2012 at 04:51:17PM -0600, Stephen Warren wrote:
> From: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
> 
> Line control directives of the following formats are supported:
>     #line LINE "FILE"
>     # LINE FILE [FLAGS]
> 
> This allows dtc to consume the output of pre-processors, and to provide
> error messages that refer to the original filename, including taking
> into account any #include directives that the pre-processor may have
> performed.

I like the idea of handling line directives, whichever way we go with
the other cpp-related proposals.  The fact that these are
distinguished from a propnodename beginning with # only by the
following digits is more subtle than I'd ideally like, but I think
it's wirth it.

The only thing I would like to see change is to only recognize line
directives at the beginning of the line (in fact, preferably in column
0 only).  Unfortunately that is going to mean some mucky flex work,
including a new start state, I expect.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [PATCH 2/2] dtc: cpp co-existence: add support for #line directives

[Stephen Warren]

On 09/25/2012 10:34 PM, David Gibson wrote:
> On Mon, Sep 24, 2012 at 04:51:17PM -0600, Stephen Warren wrote:
>> From: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
>>
>> Line control directives of the following formats are supported:
>>     #line LINE "FILE"
>>     # LINE FILE [FLAGS]
>>
>> This allows dtc to consume the output of pre-processors, and to provide
>> error messages that refer to the original filename, including taking
>> into account any #include directives that the pre-processor may have
>> performed.
> 
> I like the idea of handling line directives, whichever way we go with
> the other cpp-related proposals.  The fact that these are
> distinguished from a propnodename beginning with # only by the
> following digits is more subtle than I'd ideally like, but I think
> it's wirth it.
> 
> The only thing I would like to see change is to only recognize line
> directives at the beginning of the line (in fact, preferably in column
> 0 only).  Unfortunately that is going to mean some mucky flex work,
> including a new start state, I expect.

That turns out to be exceedingly easy; just put ^ at the start of the
regex (after the start state specification), and Flex takes care of
everything:-)

Re: [PATCH 2/2] dtc: cpp co-existence: add support for #line directives

[David Gibson]

On Tue, Sep 25, 2012 at 11:58:00PM -0600, Stephen Warren wrote:
> On 09/25/2012 10:34 PM, David Gibson wrote:
> > On Mon, Sep 24, 2012 at 04:51:17PM -0600, Stephen Warren wrote:
> >> From: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
> >>
> >> Line control directives of the following formats are supported:
> >>     #line LINE "FILE"
> >>     # LINE FILE [FLAGS]
> >>
> >> This allows dtc to consume the output of pre-processors, and to provide
> >> error messages that refer to the original filename, including taking
> >> into account any #include directives that the pre-processor may have
> >> performed.
> > 
> > I like the idea of handling line directives, whichever way we go with
> > the other cpp-related proposals.  The fact that these are
> > distinguished from a propnodename beginning with # only by the
> > following digits is more subtle than I'd ideally like, but I think
> > it's wirth it.
> > 
> > The only thing I would like to see change is to only recognize line
> > directives at the beginning of the line (in fact, preferably in column
> > 0 only).  Unfortunately that is going to mean some mucky flex work,
> > including a new start state, I expect.
> 
> That turns out to be exceedingly easy; just put ^ at the start of the
> regex (after the start state specification), and Flex takes care of
> everything:-)

Oh yeah, I forgot about ^ :).  I like it when things turn out easier
than expected :).

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson