[dtc] Allow multipart property values

[dtc] Allow multipart property values

[David Gibson]

At present each property definition in a dts file must give as the
value either a string ("abc..."), a bytestring ([12abcd...]) or a cell
list (<1 2 3 ...>).  This patch allows a property value to be given as
several of these, comma-separated.  The final property value is just
the components appended together.  So a property could have a list of
cells followed by a string, or a bytestring followed by some cells.
Cells are always aligned, so if cells are given following a string or
bytestring which is not a multiple of 4 bytes long, zero bytes are
inserted to align the following cells.

The primary motivation for this feature, however, is to allow defining
a property as a list of several strings.  This is what's needed for
defining OF 'compatible' properties, and is less ugly and fiddly than
using embedded \0s in the strings.

Signed-off-by: David Gibson <dwg@au1.ibm.com>

Index: dtc/data.c
===================================================================
--- dtc.orig/data.c	2007-01-31 15:41:24.000000000 +1100
+++ dtc/data.c	2007-01-31 15:42:03.000000000 +1100
@@ -197,6 +197,33 @@ struct data data_append_data(struct data
 	return d;
 }
 
+struct data data_merge(struct data d1, struct data d2)
+{
+	struct data d;
+	struct fixup **ff;
+	struct fixup *f, *f2;
+
+	d = data_append_data(d1, d2.val, d2.len);
+
+	/* Extract d2's fixups */
+	f2 = d2.refs;
+	d2.refs = NULL;
+
+	/* Tack them onto d's list of fixups */
+	ff = &d.refs;
+	while (*ff)
+		ff = &((*ff)->next);
+	*ff = f2;
+
+	/* And correct them for their new position */
+	for (f = f2; f; f = f->next)
+		f->offset += d1.len;
+
+	data_free(d2);
+
+	return d;
+}
+
 struct data data_append_cell(struct data d, cell_t word)
 {
 	cell_t beword = cpu_to_be32(word);
Index: dtc/dtc-parser.y
===================================================================
--- dtc.orig/dtc-parser.y	2007-01-31 15:41:53.000000000 +1100
+++ dtc/dtc-parser.y	2007-01-31 15:42:03.000000000 +1100
@@ -58,6 +58,7 @@ extern struct boot_info *the_boot_info;
 %token <str> DT_REF
 
 %type <data> propdata
+%type <data> propdataprefix
 %type <re> memreserve
 %type <re> memreserves
 %type <data> celllist
@@ -121,9 +122,15 @@ propdef:	label DT_PROPNAME '=' propdata 
 		}
 	;
 
-propdata:	DT_STRING { $$ = $1; }
-	|	'<' celllist '>' { $$ = $2; }
-	|	'[' bytestring ']' { $$ = $2; }
+propdata:	propdataprefix DT_STRING { $$ = data_merge($1, $2); }
+	|	propdataprefix '<' celllist '>' {
+			$$ = data_merge(data_append_align($1, sizeof(cell_t)), $3);
+		}
+	|	propdataprefix '[' bytestring ']' { $$ = data_merge($1, $3); }
+	;
+
+propdataprefix:	propdata ',' { $$ = $1; }
+	|	/* empty */ { $$ = empty_data; }
 	;
 
 celllist:	celllist DT_CELL { $$ = data_append_cell($1, $2); }
Index: dtc/dtc.h
===================================================================
--- dtc.orig/dtc.h	2007-01-31 15:41:24.000000000 +1100
+++ dtc/dtc.h	2007-01-31 15:42:03.000000000 +1100
@@ -119,6 +119,7 @@ struct data data_copy_escape_string(char
 struct data data_copy_file(FILE *f, size_t len);
 
 struct data data_append_data(struct data d, void *p, int len);
+struct data data_merge(struct data d1, struct data d2);
 struct data data_append_cell(struct data d, cell_t word);
 struct data data_append_re(struct data d, struct reserve_entry *re);
 struct data data_append_addr(struct data d, u64 addr);
Index: dtc/dtc-lexer.l
===================================================================
--- dtc.orig/dtc-lexer.l	2007-01-31 15:41:24.000000000 +1100
+++ dtc/dtc-lexer.l	2007-01-31 15:42:03.000000000 +1100
@@ -123,6 +123,15 @@ REFCHAR		({PROPCHAR}|{UNITCHAR}|[/@])
 			return ']';
 		}
 
+,		{ /* Technically this is a valid property name,
+		     but we'd rather use it as punctuation, so detect it
+		     here in preference */
+			yylloc.first_line = yylineno;
+			DPRINT("Char (propname like): %c (\\x%02x)\n", yytext[0],
+				(unsigned)yytext[0]);
+			return yytext[0];
+		}
+
 {PROPCHAR}+	{
 			yylloc.first_line = yylineno;
 			DPRINT("PropName: %s\n", yytext);

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [dtc] Allow multipart property values

[Kumar Gala]

On Feb 6, 2007, at 9:23 PM, David Gibson wrote:

> At present each property definition in a dts file must give as the
> value either a string ("abc..."), a bytestring ([12abcd...]) or a cell
> list (<1 2 3 ...>).  This patch allows a property value to be given as
> several of these, comma-separated.  The final property value is just
> the components appended together.  So a property could have a list of
> cells followed by a string, or a bytestring followed by some cells.
> Cells are always aligned, so if cells are given following a string or
> bytestring which is not a multiple of 4 bytes long, zero bytes are
> inserted to align the following cells.
>
> The primary motivation for this feature, however, is to allow defining
> a property as a list of several strings.  This is what's needed for
> defining OF 'compatible' properties, and is less ugly and fiddly than
> using embedded \0s in the strings.
>
> Signed-off-by: David Gibson <dwg@au1.ibm.com>

Mind adding examples to the testsuite?

- k

Re: [dtc] Allow multipart property values

[David Gibson]

On Tue, Feb 06, 2007 at 10:37:25PM -0600, Kumar Gala wrote:
> 
> On Feb 6, 2007, at 9:23 PM, David Gibson wrote:
> 
> > At present each property definition in a dts file must give as the
> > value either a string ("abc..."), a bytestring ([12abcd...]) or a cell
> > list (<1 2 3 ...>).  This patch allows a property value to be given as
> > several of these, comma-separated.  The final property value is just
> > the components appended together.  So a property could have a list of
> > cells followed by a string, or a bytestring followed by some cells.
> > Cells are always aligned, so if cells are given following a string or
> > bytestring which is not a multiple of 4 bytes long, zero bytes are
> > inserted to align the following cells.
> >
> > The primary motivation for this feature, however, is to allow defining
> > a property as a list of several strings.  This is what's needed for
> > defining OF 'compatible' properties, and is less ugly and fiddly than
> > using embedded \0s in the strings.
> >
> > Signed-off-by: David Gibson <dwg@au1.ibm.com>
> 
> Mind adding examples to the testsuite?

What testsuite...?

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [dtc] Allow multipart property values

[Kumar Gala]

On Feb 6, 2007, at 10:46 PM, David Gibson wrote:

> On Tue, Feb 06, 2007 at 10:37:25PM -0600, Kumar Gala wrote:
>>
>> On Feb 6, 2007, at 9:23 PM, David Gibson wrote:
>>
>>> At present each property definition in a dts file must give as the
>>> value either a string ("abc..."), a bytestring ([12abcd...]) or a  
>>> cell
>>> list (<1 2 3 ...>).  This patch allows a property value to be  
>>> given as
>>> several of these, comma-separated.  The final property value is just
>>> the components appended together.  So a property could have a  
>>> list of
>>> cells followed by a string, or a bytestring followed by some cells.
>>> Cells are always aligned, so if cells are given following a  
>>> string or
>>> bytestring which is not a multiple of 4 bytes long, zero bytes are
>>> inserted to align the following cells.
>>>
>>> The primary motivation for this feature, however, is to allow  
>>> defining
>>> a property as a list of several strings.  This is what's needed for
>>> defining OF 'compatible' properties, and is less ugly and fiddly  
>>> than
>>> using embedded \0s in the strings.
>>>
>>> Signed-off-by: David Gibson <dwg@au1.ibm.com>
>>
>> Mind adding examples to the testsuite?
>
> What testsuite...?

Well there was a tests/ dir  and test.dts :)

How about adding something to test.dts or an example.dts that shows  
various new things you just added.

- k

Re: [dtc] Allow multipart property values

[David Gibson]

On Tue, Feb 06, 2007 at 10:50:09PM -0600, Kumar Gala wrote:
> 
> On Feb 6, 2007, at 10:46 PM, David Gibson wrote:
> 
> > On Tue, Feb 06, 2007 at 10:37:25PM -0600, Kumar Gala wrote:
> >>
> >> On Feb 6, 2007, at 9:23 PM, David Gibson wrote:
> >>
> >>> At present each property definition in a dts file must give as the
> >>> value either a string ("abc..."), a bytestring ([12abcd...]) or a  
> >>> cell
> >>> list (<1 2 3 ...>).  This patch allows a property value to be  
> >>> given as
> >>> several of these, comma-separated.  The final property value is just
> >>> the components appended together.  So a property could have a  
> >>> list of
> >>> cells followed by a string, or a bytestring followed by some cells.
> >>> Cells are always aligned, so if cells are given following a  
> >>> string or
> >>> bytestring which is not a multiple of 4 bytes long, zero bytes are
> >>> inserted to align the following cells.
> >>>
> >>> The primary motivation for this feature, however, is to allow  
> >>> defining
> >>> a property as a list of several strings.  This is what's needed for
> >>> defining OF 'compatible' properties, and is less ugly and fiddly  
> >>> than
> >>> using embedded \0s in the strings.
> >>>
> >>> Signed-off-by: David Gibson <dwg@au1.ibm.com>
> >>
> >> Mind adding examples to the testsuite?
> >
> > What testsuite...?
> 
> Well there was a tests/ dir  and test.dts :)

There's nothing in tests/ though - I always meant to make a testsuite,
but I never got around to it.  Current plan is to implement a
testsuite by merging libfdt into the dtc tree.

Here's a revised patch which adds an example usage to test.dts

[dtc] Allow multipart property values

At present each property definition in a dts file must give as the
value either a string ("abc..."), a bytestring ([12abcd...]) or a cell
list (<1 2 3 ...>).  This patch allows a property value to be given as
several of these, comma-separated.  The final property value is just
the components appended together.  So a property could have a list of
cells followed by a string, or a bytestring followed by some cells.
Cells are always aligned, so if cells are given following a string or
bytestring which is not a multiple of 4 bytes long, zero bytes are
inserted to align the following cells.

The primary motivation for this feature, however, is to allow defining
a property as a list of several strings.  This is what's needed for
defining OF 'compatible' properties, and is less ugly and fiddly than
using embedded \0s in the strings.

Signed-off-by: David Gibson <dwg@au1.ibm.com>

Index: dtc/data.c
===================================================================
--- dtc.orig/data.c	2007-02-07 14:17:00.000000000 +1100
+++ dtc/data.c	2007-02-07 14:24:34.000000000 +1100
@@ -197,6 +197,33 @@ struct data data_append_data(struct data
 	return d;
 }
 
+struct data data_merge(struct data d1, struct data d2)
+{
+	struct data d;
+	struct fixup **ff;
+	struct fixup *f, *f2;
+
+	d = data_append_data(d1, d2.val, d2.len);
+
+	/* Extract d2's fixups */
+	f2 = d2.refs;
+	d2.refs = NULL;
+
+	/* Tack them onto d's list of fixups */
+	ff = &d.refs;
+	while (*ff)
+		ff = &((*ff)->next);
+	*ff = f2;
+
+	/* And correct them for their new position */
+	for (f = f2; f; f = f->next)
+		f->offset += d1.len;
+
+	data_free(d2);
+
+	return d;
+}
+
 struct data data_append_cell(struct data d, cell_t word)
 {
 	cell_t beword = cpu_to_be32(word);
Index: dtc/dtc-parser.y
===================================================================
--- dtc.orig/dtc-parser.y	2007-02-07 14:17:00.000000000 +1100
+++ dtc/dtc-parser.y	2007-02-07 14:24:34.000000000 +1100
@@ -58,6 +58,7 @@ extern struct boot_info *the_boot_info;
 %token <str> DT_REF
 
 %type <data> propdata
+%type <data> propdataprefix
 %type <re> memreserve
 %type <re> memreserves
 %type <data> celllist
@@ -121,9 +122,15 @@ propdef:	label DT_PROPNAME '=' propdata 
 		}
 	;
 
-propdata:	DT_STRING { $$ = $1; }
-	|	'<' celllist '>' { $$ = $2; }
-	|	'[' bytestring ']' { $$ = $2; }
+propdata:	propdataprefix DT_STRING { $$ = data_merge($1, $2); }
+	|	propdataprefix '<' celllist '>' {
+			$$ = data_merge(data_append_align($1, sizeof(cell_t)), $3);
+		}
+	|	propdataprefix '[' bytestring ']' { $$ = data_merge($1, $3); }
+	;
+
+propdataprefix:	propdata ',' { $$ = $1; }
+	|	/* empty */ { $$ = empty_data; }
 	;
 
 celllist:	celllist DT_CELL { $$ = data_append_cell($1, $2); }
Index: dtc/dtc.h
===================================================================
--- dtc.orig/dtc.h	2007-02-07 14:17:00.000000000 +1100
+++ dtc/dtc.h	2007-02-07 14:24:34.000000000 +1100
@@ -119,6 +119,7 @@ struct data data_copy_escape_string(char
 struct data data_copy_file(FILE *f, size_t len);
 
 struct data data_append_data(struct data d, void *p, int len);
+struct data data_merge(struct data d1, struct data d2);
 struct data data_append_cell(struct data d, cell_t word);
 struct data data_append_re(struct data d, struct reserve_entry *re);
 struct data data_append_addr(struct data d, u64 addr);
Index: dtc/dtc-lexer.l
===================================================================
--- dtc.orig/dtc-lexer.l	2007-02-07 14:17:00.000000000 +1100
+++ dtc/dtc-lexer.l	2007-02-07 14:24:34.000000000 +1100
@@ -123,6 +123,15 @@ REFCHAR		({PROPCHAR}|{UNITCHAR}|[/@])
 			return ']';
 		}
 
+,		{ /* Technically this is a valid property name,
+		     but we'd rather use it as punctuation, so detect it
+		     here in preference */
+			yylloc.first_line = yylineno;
+			DPRINT("Char (propname like): %c (\\x%02x)\n", yytext[0],
+				(unsigned)yytext[0]);
+			return yytext[0];
+		}
+
 {PROPCHAR}+	{
 			yylloc.first_line = yylineno;
 			DPRINT("PropName: %s\n", yytext);
Index: dtc/test.dts
===================================================================
--- dtc.orig/test.dts	2007-02-07 16:35:54.000000000 +1100
+++ dtc/test.dts	2007-02-07 16:37:21.000000000 +1100
@@ -4,7 +4,7 @@
 
 / {
 	model = "MyBoardName";
-	compatible = "MyBoardFamilyName";
+	compatible = "MyBoardName", "MyBoardFamilyName";
 	#address-cells = <2>;
 	#size-cells = <2>;
 
@@ -39,6 +39,7 @@
 		string = "\xff\0stuffstuff\t\t\t\n\n\n";
 		blob = [0a 0b 0c 0d de ea ad be ef];
 		ref = < &/memory@0 >;
+		mixed = "abc", [1234], <a b c>;
 	};
 
 	memory@0 {


-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [dtc] Allow multipart property values

[Jon Loeliger]

So, like, the other day David Gibson mumbled:
> 
> There's nothing in tests/ though - I always meant to make a testsuite,
> but I never got around to it.  Current plan is to implement a
> testsuite by merging libfdt into the dtc tree.
> 
> Here's a revised patch which adds an example usage to test.dts
> 
> [dtc] Allow multipart property values
> 
> At present each property definition in a dts file must give as the
> value either a string ("abc..."), a bytestring ([12abcd...]) or a cell
> list (<1 2 3 ...>).  This patch allows a property value to be given as
> several of these, comma-separated.  The final property value is just
> the components appended together.  So a property could have a list of
> cells followed by a string, or a bytestring followed by some cells.
> Cells are always aligned, so if cells are given following a string or
> bytestring which is not a multiple of 4 bytes long, zero bytes are
> inserted to align the following cells.
> 
> The primary motivation for this feature, however, is to allow defining
> a property as a list of several strings.  This is what's needed for
> defining OF 'compatible' properties, and is less ugly and fiddly than
> using embedded \0s in the strings.
> 
> Signed-off-by: David Gibson <dwg@au1.ibm.com>

Applied.

Thanks,
jdl

Re: [dtc] Allow multipart property values

[Jon Loeliger]

So, like, the other day David Gibson mumbled:
> At present each property definition in a dts file must give as the
> value either a string ("abc..."), a bytestring ([12abcd...]) or a cell
> list (<1 2 3 ...>).  This patch allows a property value to be given as
> several of these, comma-separated.  The final property value is just
> the components appended together.  So a property could have a list of
> cells followed by a string, or a bytestring followed by some cells.
> Cells are always aligned, so if cells are given following a string or
> bytestring which is not a multiple of 4 bytes long, zero bytes are
> inserted to align the following cells.
> 
> The primary motivation for this feature, however, is to allow defining
> a property as a list of several strings.  This is what's needed for
> defining OF 'compatible' properties, and is less ugly and fiddly than
> using embedded \0s in the strings.

David,

I'd like a bit of clarification on the issue of null-padding.
I'm not sure how it is supposed to work for OF originally,
but I'm not sure the null-padding to mod 4 will be the same
(or usable) semantics.

In OF's original definition would "ab\0de" yield something
different than a similar "ab", [0], "de" with your proposal?
I could see that we might have:

	OF   : a b \0 d e
	Yours: a b \0 \0 \0 d e

Or does the OF spec say that it will also pad to mod 4?
Or do all strings always start on mod 4 address?

Thanks,
jdl

Re: [dtc] Allow multipart property values

[David Gibson]

On Wed, Feb 07, 2007 at 08:35:25AM -0600, Jon Loeliger wrote:
> So, like, the other day David Gibson mumbled:
> > At present each property definition in a dts file must give as the
> > value either a string ("abc..."), a bytestring ([12abcd...]) or a cell
> > list (<1 2 3 ...>).  This patch allows a property value to be given as
> > several of these, comma-separated.  The final property value is just
> > the components appended together.  So a property could have a list of
> > cells followed by a string, or a bytestring followed by some cells.
> > Cells are always aligned, so if cells are given following a string or
> > bytestring which is not a multiple of 4 bytes long, zero bytes are
> > inserted to align the following cells.
> > 
> > The primary motivation for this feature, however, is to allow defining
> > a property as a list of several strings.  This is what's needed for
> > defining OF 'compatible' properties, and is less ugly and fiddly than
> > using embedded \0s in the strings.
> 
> David,
> 
> I'd like a bit of clarification on the issue of null-padding.
> I'm not sure how it is supposed to work for OF originally,

I don't see how OF is relevant - the notation used for property values
in dts is my own invention, not an OF specification.

> but I'm not sure the null-padding to mod 4 will be the same
> (or usable) semantics.
> 
> In OF's original definition would "ab\0de" yield something
> different than a similar "ab", [0], "de" with your proposal?
> I could see that we might have:
> 
> 	OF   : a b \0 d e
> 	Yours: a b \0 \0 \0 d e
> 
> Or does the OF spec say that it will also pad to mod 4?
> Or do all strings always start on mod 4 address?

No, it's only the "cell" format that's padded for alignment.  Actually
the example you give (with [0]) is invalid in any case, since
bytestring notation must have a multiple of 2 digits.

	"ab", [00], "de"
is equivalent to
	"ab\0\0de"
or to
	[61620000646500]
There are two zero bytes in the middle, the first is the terminator
for "ab", the second is the explicit [00].

With cell notation, however, alignement padding gives us:

	"abc", <deadbeef>
is equivalent to
	[61626300deadbeef]

but
	"a", <deadbeef>
is equivalent to
	[61000000deadbeef]

I can't think of any good reason you'd want to define a property with
a cell portion following a non-cell portion, and doing so is probably
a bad idea.  However, it falls naturally out of the most obvious way
to allow list-of-strings notation for compatible properties.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson