[PATCH V2] dtc: Implement -d option to write out a dependency file

[PATCH V2] dtc: Implement -d option to write out a dependency file

[Stephen Warren]

This will allow callers to rebuild .dtb files when any of the /include/d
.dtsi files are modified, not just the top-level .dts file.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
---
Michal, this patch is a dependency for the kbuild changes that use it;
http://www.spinics.net/lists/linux-kbuild/msg05883.html
I think it makes sense to apply both this patch, and that patch, to the
kbuild tree. Do you agree?

v2: Replace fputs/fputc with fprintf

This patch has been blessed for application to upstream dtc. Here, I'm
simply applying the same thing to the code in the Linux kernel tree,
resolving some conflicts. See:
http://lists.ozlabs.org/pipermail/devicetree-discuss/2012-January/011172.html

 scripts/dtc/dtc.c    |   21 ++++++++++++++++++++-
 scripts/dtc/srcpos.c |    4 ++++
 scripts/dtc/srcpos.h |    1 +
 3 files changed, 25 insertions(+), 1 deletions(-)

diff --git a/scripts/dtc/dtc.c b/scripts/dtc/dtc.c
index cbc0193..451c92d 100644
--- a/scripts/dtc/dtc.c
+++ b/scripts/dtc/dtc.c
@@ -71,6 +71,7 @@ static void  __attribute__ ((noreturn)) usage(void)
 	fprintf(stderr, "\t\t\tasm - assembler source\n");
 	fprintf(stderr, "\t-V <output version>\n");
 	fprintf(stderr, "\t\tBlob version to produce, defaults to %d (relevant for dtb\n\t\tand asm output only)\n", DEFAULT_FDT_VERSION);
+	fprintf(stderr, "\t-d <output dependency file>\n");
 	fprintf(stderr, "\t-R <number>\n");
 	fprintf(stderr, "\t\tMake space for <number> reserve map entries (relevant for \n\t\tdtb and asm output only)\n");
 	fprintf(stderr, "\t-S <bytes>\n");
@@ -99,6 +100,7 @@ int main(int argc, char *argv[])
 	const char *inform = "dts";
 	const char *outform = "dts";
 	const char *outname = "-";
+	const char *depname = NULL;
 	int force = 0, check = 0, sort = 0;
 	const char *arg;
 	int opt;
@@ -111,7 +113,8 @@ int main(int argc, char *argv[])
 	minsize    = 0;
 	padsize    = 0;
 
-	while ((opt = getopt(argc, argv, "hI:O:o:V:R:S:p:fcqb:vH:s")) != EOF) {
+	while ((opt = getopt(argc, argv, "hI:O:o:V:d:R:S:p:fcqb:vH:s"))
+			!= EOF) {
 		switch (opt) {
 		case 'I':
 			inform = optarg;
@@ -125,6 +128,9 @@ int main(int argc, char *argv[])
 		case 'V':
 			outversion = strtol(optarg, NULL, 0);
 			break;
+		case 'd':
+			depname = optarg;
+			break;
 		case 'R':
 			reservenum = strtol(optarg, NULL, 0);
 			break;
@@ -188,6 +194,14 @@ int main(int argc, char *argv[])
 	fprintf(stderr, "DTC: %s->%s  on file \"%s\"\n",
 		inform, outform, arg);
 
+	if (depname) {
+		depfile = fopen(depname, "w");
+		if (!depfile)
+			die("Couldn't open dependency file %s: %s\n", depname,
+			    strerror(errno));
+		fprintf(depfile, "%s:", outname);
+	}
+
 	if (streq(inform, "dts"))
 		bi = dt_from_source(arg);
 	else if (streq(inform, "fs"))
@@ -197,6 +211,11 @@ int main(int argc, char *argv[])
 	else
 		die("Unknown input format \"%s\"\n", inform);
 
+	if (depfile) {
+		fputc('\n', depfile);
+		fclose(depfile);
+	}
+
 	if (cmdline_boot_cpuid != -1)
 		bi->boot_cpuid_phys = cmdline_boot_cpuid;
 
diff --git a/scripts/dtc/srcpos.c b/scripts/dtc/srcpos.c
index 2dbc874..36a38e9 100644
--- a/scripts/dtc/srcpos.c
+++ b/scripts/dtc/srcpos.c
@@ -40,6 +40,7 @@ static char *dirname(const char *path)
 	return NULL;
 }
 
+FILE *depfile; /* = NULL */
 struct srcfile_state *current_srcfile; /* = NULL */
 
 /* Detect infinite include recursion. */
@@ -67,6 +68,9 @@ FILE *srcfile_relative_open(const char *fname, char **fullnamep)
 			    strerror(errno));
 	}
 
+	if (depfile)
+		fprintf(depfile, " %s", fullname);
+
 	if (fullnamep)
 		*fullnamep = fullname;
 	else
diff --git a/scripts/dtc/srcpos.h b/scripts/dtc/srcpos.h
index bd7966e..ce980ca 100644
--- a/scripts/dtc/srcpos.h
+++ b/scripts/dtc/srcpos.h
@@ -30,6 +30,7 @@ struct srcfile_state {
 	struct srcfile_state *prev;
 };
 
+extern FILE *depfile; /* = NULL */
 extern struct srcfile_state *current_srcfile; /* = NULL */
 
 FILE *srcfile_relative_open(const char *fname, char **fullnamep);
-- 
1.7.0.4

Re: [PATCH V2] dtc: Implement -d option to write out a dependency file

[Michal Marek]

On 11.1.2012 01:27, Stephen Warren wrote:
> This will allow callers to rebuild .dtb files when any of the /include/d
> .dtsi files are modified, not just the top-level .dts file.
> 
> Signed-off-by: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
> ---
> Michal, this patch is a dependency for the kbuild changes that use it;
> http://www.spinics.net/lists/linux-kbuild/msg05883.html
> I think it makes sense to apply both this patch, and that patch, to the
> kbuild tree. Do you agree?

Sure.


> v2: Replace fputs/fputc with fprintf
> 
> This patch has been blessed for application to upstream dtc. Here, I'm
> simply applying the same thing to the code in the Linux kernel tree,
> resolving some conflicts. See:
> http://lists.ozlabs.org/pipermail/devicetree-discuss/2012-January/011172.html

I see it has been applied in the meantime, great. I'll apply this one
and the kbuild patch to kbuild.git#kbuild.

Michal

[PATCH V2] dtc: Implement -d option to write out a dependency file

[Stephen Warren]

This will allow callers to rebuild .dtb files when any of the /include/d
.dtsi files are modified, not just the top-level .dts file.

Signed-off-by: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
---
v2: Replaced fputs/fputc with fprintf

This is an updated patch for dtc itself. Once it's accepted, I'll repost
the equivalent kernel patch.

 Documentation/manual.txt |    3 +++
 dtc.c                    |   20 +++++++++++++++++++-
 srcpos.c                 |    4 ++++
 srcpos.h                 |    1 +
 tests/dependencies.cmp   |    1 +
 tests/dependencies.dts   |    6 ++++++
 tests/deps_inc1.dtsi     |    1 +
 tests/deps_inc2.dtsi     |    1 +
 tests/run_tests.sh       |    4 ++++
 9 files changed, 40 insertions(+), 1 deletions(-)
 create mode 100644 tests/dependencies.cmp
 create mode 100644 tests/dependencies.dts
 create mode 100644 tests/deps_inc1.dtsi
 create mode 100644 tests/deps_inc2.dtsi

diff --git a/Documentation/manual.txt b/Documentation/manual.txt
index 14508f3..989c589 100644
--- a/Documentation/manual.txt
+++ b/Documentation/manual.txt
@@ -106,6 +106,9 @@ Options:
     -O <output_format>
 	The generated output format, as listed above.
 
+    -d <dependency_filename>
+	Generate a dependency file during compilation.
+
     -q
 	Quiet: -q suppress warnings, -qq errors, -qqq all
 
diff --git a/dtc.c b/dtc.c
index 15d2fc2..7a0c605 100644
--- a/dtc.c
+++ b/dtc.c
@@ -71,6 +71,7 @@ static void  __attribute__ ((noreturn)) usage(void)
 	fprintf(stderr, "\t\t\tasm - assembler source\n");
 	fprintf(stderr, "\t-V <output version>\n");
 	fprintf(stderr, "\t\tBlob version to produce, defaults to %d (relevant for dtb\n\t\tand asm output only)\n", DEFAULT_FDT_VERSION);
+	fprintf(stderr, "\t-d <output dependency file>\n");
 	fprintf(stderr, "\t-R <number>\n");
 	fprintf(stderr, "\t\tMake space for <number> reserve map entries (relevant for \n\t\tdtb and asm output only)\n");
 	fprintf(stderr, "\t-S <bytes>\n");
@@ -99,6 +100,7 @@ int main(int argc, char *argv[])
 	const char *inform = "dts";
 	const char *outform = "dts";
 	const char *outname = "-";
+	const char *depname = NULL;
 	int force = 0, sort = 0;
 	const char *arg;
 	int opt;
@@ -111,7 +113,7 @@ int main(int argc, char *argv[])
 	minsize    = 0;
 	padsize    = 0;
 
-	while ((opt = getopt(argc, argv, "hI:O:o:V:R:S:p:fqb:vH:s")) != EOF) {
+	while ((opt = getopt(argc, argv, "hI:O:o:V:d:R:S:p:fqb:vH:s")) != EOF) {
 		switch (opt) {
 		case 'I':
 			inform = optarg;
@@ -125,6 +127,9 @@ int main(int argc, char *argv[])
 		case 'V':
 			outversion = strtol(optarg, NULL, 0);
 			break;
+		case 'd':
+			depname = optarg;
+			break;
 		case 'R':
 			reservenum = strtol(optarg, NULL, 0);
 			break;
@@ -185,6 +190,14 @@ int main(int argc, char *argv[])
 	fprintf(stderr, "DTC: %s->%s  on file \"%s\"\n",
 		inform, outform, arg);
 
+	if (depname) {
+		depfile = fopen(depname, "w");
+		if (!depfile)
+			die("Couldn't open dependency file %s: %s\n", depname,
+			    strerror(errno));
+		fprintf(depfile, "%s:", outname);
+	}
+
 	if (streq(inform, "dts"))
 		bi = dt_from_source(arg);
 	else if (streq(inform, "fs"))
@@ -194,6 +207,11 @@ int main(int argc, char *argv[])
 	else
 		die("Unknown input format \"%s\"\n", inform);
 
+	if (depfile) {
+		fputc('\n', depfile);
+		fclose(depfile);
+	}
+
 	if (cmdline_boot_cpuid != -1)
 		bi->boot_cpuid_phys = cmdline_boot_cpuid;
 
diff --git a/srcpos.c b/srcpos.c
index 2dbc874..36a38e9 100644
--- a/srcpos.c
+++ b/srcpos.c
@@ -40,6 +40,7 @@ static char *dirname(const char *path)
 	return NULL;
 }
 
+FILE *depfile; /* = NULL */
 struct srcfile_state *current_srcfile; /* = NULL */
 
 /* Detect infinite include recursion. */
@@ -67,6 +68,9 @@ FILE *srcfile_relative_open(const char *fname, char **fullnamep)
 			    strerror(errno));
 	}
 
+	if (depfile)
+		fprintf(depfile, " %s", fullname);
+
 	if (fullnamep)
 		*fullnamep = fullname;
 	else
diff --git a/srcpos.h b/srcpos.h
index bd7966e..ce980ca 100644
--- a/srcpos.h
+++ b/srcpos.h
@@ -30,6 +30,7 @@ struct srcfile_state {
 	struct srcfile_state *prev;
 };
 
+extern FILE *depfile; /* = NULL */
 extern struct srcfile_state *current_srcfile; /* = NULL */
 
 FILE *srcfile_relative_open(const char *fname, char **fullnamep);
diff --git a/tests/dependencies.cmp b/tests/dependencies.cmp
new file mode 100644
index 0000000..bcd9432
--- /dev/null
+++ b/tests/dependencies.cmp
@@ -0,0 +1 @@
+dependencies.test.dtb: dependencies.dts deps_inc1.dtsi deps_inc2.dtsi
diff --git a/tests/dependencies.dts b/tests/dependencies.dts
new file mode 100644
index 0000000..2cfe31b
--- /dev/null
+++ b/tests/dependencies.dts
@@ -0,0 +1,6 @@
+/dts-v1/;
+
+/include/ "deps_inc1.dtsi"
+
+/ {
+};
diff --git a/tests/deps_inc1.dtsi b/tests/deps_inc1.dtsi
new file mode 100644
index 0000000..5c607dc
--- /dev/null
+++ b/tests/deps_inc1.dtsi
@@ -0,0 +1 @@
+/include/ "deps_inc2.dtsi"
diff --git a/tests/deps_inc2.dtsi b/tests/deps_inc2.dtsi
new file mode 100644
index 0000000..710cecc
--- /dev/null
+++ b/tests/deps_inc2.dtsi
@@ -0,0 +1 @@
+/* Empty */
diff --git a/tests/run_tests.sh b/tests/run_tests.sh
index da6f970..b96f6d8 100755
--- a/tests/run_tests.sh
+++ b/tests/run_tests.sh
@@ -353,6 +353,10 @@ dtc_tests () {
     run_sh_test dtc-fatal.sh -I dts -O dtb nosuchfile.dts
     run_sh_test dtc-fatal.sh -I dtb -O dtb nosuchfile.dtb
     run_sh_test dtc-fatal.sh -I fs -O dtb nosuchfile
+
+    # Dependencies
+    run_dtc_test -I dts -O dtb -o dependencies.test.dtb -d dependencies.test.d dependencies.dts
+    run_wrap_test cmp dependencies.test.d dependencies.cmp
 }
 
 cmp_tests () {
-- 
1.7.0.4

Re: [PATCH V2] dtc: Implement -d option to write out a dependency file

[David Gibson]

On Tue, Jan 10, 2012 at 10:12:48AM -0700, Stephen Warren wrote:
> This will allow callers to rebuild .dtb files when any of the /include/d
> .dtsi files are modified, not just the top-level .dts file.
> 
> Signed-off-by: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>

Nice.

Acked-by: David Gibson <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org>

Jon, please apply.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [PATCH V2] dtc: Implement -d option to write out a dependency file

[Jon Loeliger]

> On Tue, Jan 10, 2012 at 10:12:48AM -0700, Stephen Warren wrote:
> > This will allow callers to rebuild .dtb files when any of the /include/d
> > .dtsi files are modified, not just the top-level .dts file.
> > 
> > Signed-off-by: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
> 
> Nice.
> 
> Acked-by: David Gibson <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org>
> 
> Jon, please apply.

Will do with a patch for the DTC repo.

Thanks,
jdl

Re: [PATCH V2] dtc: Implement -d option to write out a dependency file

[David Gibson]

On Wed, Jan 11, 2012 at 08:29:58AM -0600, Jon Loeliger wrote:
> > On Tue, Jan 10, 2012 at 10:12:48AM -0700, Stephen Warren wrote:
> > > This will allow callers to rebuild .dtb files when any of the /include/d
> > > .dtsi files are modified, not just the top-level .dts file.
> > > 
> > > Signed-off-by: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
> > 
> > Nice.
> > 
> > Acked-by: David Gibson <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org>
> > 
> > Jon, please apply.
> 
> Will do with a patch for the DTC repo.

Uh.. this patch is for the dtc repo, he sent it in addition to the one
against the kernel version.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [PATCH V2] dtc: Implement -d option to write out a dependency file

[Jon Loeliger]

> On Wed, Jan 11, 2012 at 08:29:58AM -0600, Jon Loeliger wrote:
> > > On Tue, Jan 10, 2012 at 10:12:48AM -0700, Stephen Warren wrote:
> > > > This will allow callers to rebuild .dtb files when any of the /include/d
> > > > .dtsi files are modified, not just the top-level .dts file.
> > > > 
> > > > Signed-off-by: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
> > > 
> > > Nice.
> > > 
> > > Acked-by: David Gibson <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org>
> > > 
> > > Jon, please apply.
> > 
> > Will do with a patch for the DTC repo.
> 
> Uh.. this patch is for the dtc repo, he sent it in addition to the one
> against the kernel version.

I did not receive this patch for the DTC repo.
I have this patch for the kernel only on the arm list.

I'm not saying he didn't do the worl, nor am I saying he
didn't post such a past.  I'm saying *I* don't have it!

Request 3: Please send this patch to *me* for the DTC repo! :-)

jdl

Re: [PATCH V2] dtc: Implement -d option to write out a dependency file

[Jon Loeliger]

> 
> I'm not saying he didn't do the worl, nor am I saying he
> didn't post such a past.  I'm saying *I* don't have it!

Yeeesh.

s/worl/work/
s/past/patch/

jdl

Re: [PATCH V2] dtc: Implement -d option to write out a dependency file

[David Gibson]

On Thu, Jan 12, 2012 at 07:24:04AM -0600, Jon Loeliger wrote:
> > On Wed, Jan 11, 2012 at 08:29:58AM -0600, Jon Loeliger wrote:
> > > > On Tue, Jan 10, 2012 at 10:12:48AM -0700, Stephen Warren wrote:
> > > > > This will allow callers to rebuild .dtb files when any of the /include/d
> > > > > .dtsi files are modified, not just the top-level .dts file.
> > > > > 
> > > > > Signed-off-by: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
> > > > 
> > > > Nice.
> > > > 
> > > > Acked-by: David Gibson <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org>
> > > > 
> > > > Jon, please apply.
> > > 
> > > Will do with a patch for the DTC repo.
> > 
> > Uh.. this patch is for the dtc repo, he sent it in addition to the one
> > against the kernel version.
> 
> I did not receive this patch for the DTC repo.
> I have this patch for the kernel only on the arm list.
> 
> I'm not saying he didn't do the worl, nor am I saying he
> didn't post such a past.  I'm saying *I* don't have it!

Um.. so, the mail I was replying to you above, was for the dtc repo
and you were on the recipient list.  Given the frequency with which I
need to resend my own patches, I'm beginning to wonder if your mail
system is dropping things, Jon..

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [PATCH V2] dtc: Implement -d option to write out a dependency file

[Jon Loeliger]

> 
> Um.. so, the mail I was replying to you above, was for the dtc repo
> and you were on the recipient list.  Given the frequency with which I
> need to resend my own patches, I'm beginning to wonder if your mail
> system is dropping things, Jon..

It is not.  See Stephen's mail, for example.

jdl

Re: [PATCH V2] dtc: Implement -d option to write out a dependency file

[David Gibson]

On Thu, Jan 12, 2012 at 05:56:48PM -0600, Jon Loeliger wrote:
> > 
> > Um.. so, the mail I was replying to you above, was for the dtc repo
> > and you were on the recipient list.  Given the frequency with which I
> > need to resend my own patches, I'm beginning to wonder if your mail
> > system is dropping things, Jon..
> 
> It is not.  See Stephen's mail, for example.

Ah, ok.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

RE: [PATCH V2] dtc: Implement -d option to write out a dependency file

[Stephen Warren]

David Gibson wrote at Thursday, January 12, 2012 4:12 PM:
> On Thu, Jan 12, 2012 at 07:24:04AM -0600, Jon Loeliger wrote:
> > > On Wed, Jan 11, 2012 at 08:29:58AM -0600, Jon Loeliger wrote:
> > > > > On Tue, Jan 10, 2012 at 10:12:48AM -0700, Stephen Warren wrote:
> > > > > > This will allow callers to rebuild .dtb files when any of the /include/d
> > > > > > .dtsi files are modified, not just the top-level .dts file.
> > > > > >
> > > > > > Signed-off-by: Stephen Warren <swarren-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
> > > > >
> > > > > Nice.
> > > > >
> > > > > Acked-by: David Gibson <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org>
> > > > >
> > > > > Jon, please apply.
> > > >
> > > > Will do with a patch for the DTC repo.
> > >
> > > Uh.. this patch is for the dtc repo, he sent it in addition to the one
> > > against the kernel version.
> >
> > I did not receive this patch for the DTC repo.
> > I have this patch for the kernel only on the arm list.
> >
> > I'm not saying he didn't do the worl, nor am I saying he
> > didn't post such a past.  I'm saying *I* don't have it!
> 
> Um.. so, the mail I was replying to you above, was for the dtc repo
> and you were on the recipient list.  Given the frequency with which I
> need to resend my own patches, I'm beginning to wonder if your mail
> system is dropping things, Jon..

In this case, it was my fault.

Jon's email system validates SPF and doesn't accept messages where that
fails. I was sending my patch through my personal email server but using
NVIDIA's email address, hence this failed the SPF check. I think I wasn't
notified of this because the bounce message my server would have
presumably generated probably experienced some similar issue getting back
to my NVIDIA email account.

I've reconfigured git to send email through NVIDIA's servers, although
now that means I have to be on the VPN to send patches:-(

-- 
nvpublic