[0/2] Embed a copy of dtc in the kernel source

[0/2] Embed a copy of dtc in the kernel source

[David Gibson]

These two patches are a respin of my previous patch to merge a copy of
dtc into the kernel tree, so that kernel builds no longer depend on an
externally installed copy of dtc.

This respin embeds a newer revision of dtc, and incorporates Sam
Ravnborg's preferred approach to Makefile integration.  Also, since so
many people whinged about it last time, I've split the patch into two
parts, the first is the too-large-for-the-list patch just verbatim
adding files and the second has the changes to existing kernel files
to build and use the embedded code.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[1/2] Merge dtc

[David Gibson]

This very large patch incorporates a copy of dtc into the kernel
source, in arch/powerpc/boot/dtc-src.  This patch only imports the dtc
sources verbatim, another patch coming to actually link it into the
kernel Makefiles and use the embedded copy of dtc for kernel builds.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>

Too big for this list.  Full patch at:
	http://ozlabs.org/~dgibson/home/merge-dtc.patch

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[2/2] Use embedded dtc in kernel builds

[David Gibson]

This patch alters the kernel makefiles to build dtc from the sources
embedded in the previous patch.  It also changes the
arch/powerpc/boot/wrapper script to use the embedded dtc, rather than
expecting a copy of dtc already installed on the system.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>

Index: working-2.6/arch/powerpc/boot/Makefile
===================================================================
--- working-2.6.orig/arch/powerpc/boot/Makefile	2007-10-22 13:55:49.000000000 +1000
+++ working-2.6/arch/powerpc/boot/Makefile	2007-10-31 14:02:40.000000000 +1100
@@ -108,17 +108,52 @@ $(patsubst %.S,%.o, $(filter %.S, $(src-
 $(obj)/wrapper.a: $(obj-wlib) FORCE
 	$(call if_changed,bootar)
 
-hostprogs-y	:= addnote addRamDisk hack-coff mktree
+hostprogs-y	:= addnote addRamDisk hack-coff mktree dtc
 
 targets		+= $(patsubst $(obj)/%,%,$(obj-boot) wrapper.a)
 extra-y		:= $(obj)/wrapper.a $(obj-plat) $(obj)/empty.o \
 		   $(obj)/zImage.lds $(obj)/zImage.coff.lds $(obj)/zImage.ps3.lds
 
 wrapper		:=$(srctree)/$(src)/wrapper
-wrapperbits	:= $(extra-y) $(addprefix $(obj)/,addnote hack-coff mktree) \
+wrapperbits	:= $(extra-y) $(addprefix $(obj)/,addnote hack-coff mktree dtc) \
 			$(wrapper) FORCE
 
 #############
+# Bits for building dtc
+# DTC_GENPARSER      := 1    # Uncomment to rebuild flex/bison output
+
+dtc-objs := dtc.o flattree.o fstree.o data.o livetree.o treesource.o srcpos.o
+dtc-objs += dtc-lexer.lex.o dtc-parser.tab.o
+dtc-objs := $(addprefix dtc-src/, $(dtc-objs))
+
+# prerequisites on generated files needs to be explicit
+$(obj)/dtc-src/dtc-parser.tab.o: $(obj)/dtc-src/dtc-parser.tab.c $(obj)/dtc-src/dtc-parser.tab.h
+$(obj)/dtc-src/dtc-lexer.lex.o:  $(obj)/dtc-src/dtc-lexer.lex.c $(obj)/dtc-src/dtc-parser.tab.h
+
+HOSTCFLAGS += -I$(src)/dtc-src/
+
+targets += dtc-src/dtc-parser.tab.c
+targets += dtc-src/dtc-lexer.lex.c
+
+ifdef DTC_GENPARSER
+BISON = bison
+FLEX = flex
+
+quiet_cmd_bison = BISON   $@
+      cmd_bison = $(BISON) -o$@ -d $<; cp $@ $@_shipped
+quiet_cmd_flex = FLEX    $@
+      cmd_flex = $(FLEX) -o$@ $<; cp $@ $@_shipped
+
+$(obj)/dtc-src/dtc-parser.tab.c: $(src)/dtc-src/dtc-parser.y FORCE
+     $(call if_changed,bison)
+
+$(obj)/dtc-src/dtc-parser.tab.h: $(obj)/dtc-src/dtc-parser.tab.c
+
+$(obj)/dtc-src/dtc-lexer.lex.c: $(src)/dtc-src/dtc-lexer.l FORCE
+     $(call if_changed,flex)
+endif
+
+#############
 # Bits for building various flavours of zImage
 
 ifneq ($(CROSS32_COMPILE),)
@@ -236,7 +271,7 @@ install: $(CONFIGURE) $(addprefix $(obj)
 
 # anything not in $(targets)
 clean-files += $(image-) $(initrd-) zImage zImage.initrd cuImage.* treeImage.* \
-	otheros.bld
+	otheros.bld $(dtc-clean-files)
 
 # clean up files cached by wrapper
 clean-kernel := vmlinux.strip vmlinux.bin
Index: working-2.6/arch/powerpc/boot/wrapper
===================================================================
--- working-2.6.orig/arch/powerpc/boot/wrapper	2007-10-22 13:55:50.000000000 +1000
+++ working-2.6/arch/powerpc/boot/wrapper	2007-10-31 14:02:40.000000000 +1100
@@ -111,7 +111,7 @@ if [ -n "$dts" ]; then
     if [ -z "$dtb" ]; then
 	dtb="$platform.dtb"
     fi
-    dtc -O dtb -o "$dtb" -b 0 -V 16 "$dts" || exit 1
+    $object/dtc -O dtb -o "$dtb" -b 0 -V 16 "$dts" || exit 1
 fi
 
 if [ -z "$kernel" ]; then


-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [0/2] Embed a copy of dtc in the kernel source

[Kumar Gala]

On Oct 30, 2007, at 10:24 PM, David Gibson wrote:

> These two patches are a respin of my previous patch to merge a copy of
> dtc into the kernel tree, so that kernel builds no longer depend on an
> externally installed copy of dtc.
>
> This respin embeds a newer revision of dtc, and incorporates Sam
> Ravnborg's preferred approach to Makefile integration.  Also, since so
> many people whinged about it last time, I've split the patch into two
> parts, the first is the too-large-for-the-list patch just verbatim
> adding files and the second has the changes to existing kernel files
> to build and use the embedded code.

What about doing part of this via git-submodule?

- k

Re: [0/2] Embed a copy of dtc in the kernel source

[David Gibson]

On Tue, Oct 30, 2007 at 11:37:15PM -0500, Kumar Gala wrote:
> 
> On Oct 30, 2007, at 10:24 PM, David Gibson wrote:
> 
> > These two patches are a respin of my previous patch to merge a copy of
> > dtc into the kernel tree, so that kernel builds no longer depend on an
> > externally installed copy of dtc.
> >
> > This respin embeds a newer revision of dtc, and incorporates Sam
> > Ravnborg's preferred approach to Makefile integration.  Also, since so
> > many people whinged about it last time, I've split the patch into two
> > parts, the first is the too-large-for-the-list patch just verbatim
> > adding files and the second has the changes to existing kernel files
> > to build and use the embedded code.
> 
> What about doing part of this via git-submodule?

Uh.. where do I find out what that does?  My version of git (Ubuntu
gutsy) doesn't seem to know anything about it...

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [0/2] Embed a copy of dtc in the kernel source

[Kumar Gala]

On Oct 30, 2007, at 11:49 PM, David Gibson wrote:

> On Tue, Oct 30, 2007 at 11:37:15PM -0500, Kumar Gala wrote:
>>
>> On Oct 30, 2007, at 10:24 PM, David Gibson wrote:
>>
>>> These two patches are a respin of my previous patch to merge a  
>>> copy of
>>> dtc into the kernel tree, so that kernel builds no longer depend  
>>> on an
>>> externally installed copy of dtc.
>>>
>>> This respin embeds a newer revision of dtc, and incorporates Sam
>>> Ravnborg's preferred approach to Makefile integration.  Also,  
>>> since so
>>> many people whinged about it last time, I've split the patch into  
>>> two
>>> parts, the first is the too-large-for-the-list patch just verbatim
>>> adding files and the second has the changes to existing kernel files
>>> to build and use the embedded code.
>>
>> What about doing part of this via git-submodule?
>
> Uh.. where do I find out what that does?  My version of git (Ubuntu
> gutsy) doesn't seem to know anything about it...

Take a look at:

http://www.kernel.org/pub/software/scm/git/docs/git-submodule.html

- k

[0/2] Merge dtc

[David Gibson]

Here is a revised set of patches to merge dtc into the kernel tree.

The various methods suggested for doing some sort of automagic git
pull of the dtc repository into the kernel git all appear to me to be
more trouble than they're worth at this stage.  So, at this stage this
is just a raw patch importing the source.  I have scripts which make
it an easy job to produce a patch updating the embedded copy of dtc.

If at some future point we want to move dtc to a more general place so
it can be used by microblaze or other architectures as well, we can
revisit how the import is done at that time.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[2/2] Use embedded dtc in kernel builds

[David Gibson]

This patch alters the kernel makefiles to build dtc from the sources
embedded in the previous patch.  It also changes the
arch/powerpc/boot/wrapper script to use the embedded dtc, rather than
expecting a copy of dtc already installed on the system.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>

---
 arch/powerpc/boot/Makefile |   39 +++++++++++++++++++++++++++++++++++++--
 arch/powerpc/boot/wrapper  |    2 +-
 2 files changed, 38 insertions(+), 3 deletions(-)

Index: working-2.6/arch/powerpc/boot/Makefile
===================================================================
--- working-2.6.orig/arch/powerpc/boot/Makefile	2007-12-18 14:30:51.000000000 +1100
+++ working-2.6/arch/powerpc/boot/Makefile	2007-12-18 14:33:22.000000000 +1100
@@ -112,17 +112,52 @@ $(patsubst %.S,%.o, $(filter %.S, $(src-
 $(obj)/wrapper.a: $(obj-wlib) FORCE
 	$(call if_changed,bootar)
 
-hostprogs-y	:= addnote addRamDisk hack-coff mktree
+hostprogs-y	:= addnote addRamDisk hack-coff mktree dtc
 
 targets		+= $(patsubst $(obj)/%,%,$(obj-boot) wrapper.a)
 extra-y		:= $(obj)/wrapper.a $(obj-plat) $(obj)/empty.o \
 		   $(obj)/zImage.lds $(obj)/zImage.coff.lds $(obj)/zImage.ps3.lds
 
 wrapper		:=$(srctree)/$(src)/wrapper
-wrapperbits	:= $(extra-y) $(addprefix $(obj)/,addnote hack-coff mktree) \
+wrapperbits	:= $(extra-y) $(addprefix $(obj)/,addnote hack-coff mktree dtc) \
 			$(wrapper) FORCE
 
 #############
+# Bits for building dtc
+# DTC_GENPARSER      := 1    # Uncomment to rebuild flex/bison output
+
+dtc-objs := dtc.o flattree.o fstree.o data.o livetree.o treesource.o srcpos.o checks.o
+dtc-objs += dtc-lexer.lex.o dtc-parser.tab.o
+dtc-objs := $(addprefix dtc-src/, $(dtc-objs))
+
+# prerequisites on generated files needs to be explicit
+$(obj)/dtc-src/dtc-parser.tab.o: $(obj)/dtc-src/dtc-parser.tab.c $(obj)/dtc-src/dtc-parser.tab.h
+$(obj)/dtc-src/dtc-lexer.lex.o:  $(obj)/dtc-src/dtc-lexer.lex.c $(obj)/dtc-src/dtc-parser.tab.h
+
+HOSTCFLAGS += -I$(src)/dtc-src/ -I$(src)/libfdt/
+
+targets += dtc-src/dtc-parser.tab.c
+targets += dtc-src/dtc-lexer.lex.c
+
+ifdef DTC_GENPARSER
+BISON = bison
+FLEX = flex
+
+quiet_cmd_bison = BISON   $@
+      cmd_bison = $(BISON) -o$@ -d $<; cp $@ $@_shipped
+quiet_cmd_flex = FLEX    $@
+      cmd_flex = $(FLEX) -o$@ $<; cp $@ $@_shipped
+
+$(obj)/dtc-src/dtc-parser.tab.c: $(src)/dtc-src/dtc-parser.y FORCE
+     $(call if_changed,bison)
+
+$(obj)/dtc-src/dtc-parser.tab.h: $(obj)/dtc-src/dtc-parser.tab.c
+
+$(obj)/dtc-src/dtc-lexer.lex.c: $(src)/dtc-src/dtc-lexer.l FORCE
+     $(call if_changed,flex)
+endif
+
+#############
 # Bits for building various flavours of zImage
 
 ifneq ($(CROSS32_COMPILE),)
Index: working-2.6/arch/powerpc/boot/wrapper
===================================================================
--- working-2.6.orig/arch/powerpc/boot/wrapper	2007-12-18 14:30:52.000000000 +1100
+++ working-2.6/arch/powerpc/boot/wrapper	2007-12-18 14:33:22.000000000 +1100
@@ -124,7 +124,7 @@ if [ -n "$dts" ]; then
     if [ -z "$dtb" ]; then
 	dtb="$platform.dtb"
     fi
-    dtc -O dtb -o "$dtb" -b 0 -V 16 "$dts"
+    $object/dtc -O dtb -o "$dtb" -b 0 "$dts"
 fi
 
 if [ -z "$kernel" ]; then


-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson