[path] don't diff generated firmware files

[path] don't diff generated firmware files

[Arjan van de Ven]

With the new firmware infrastructure in 2.6.27, some files are generated and shouldn't be
diffed; add these 2 to the "dontdiff" file

Signed-off-by: Arjan van de Ven <arjan@Linux.intel.com>

diff -purN --exclude-from=linux.trees.git/Documentation/dontdiff /home/arjan/linux.trees.git/Documentation/dontdiff linux.trees.git/Documentation/dontdiff
--- linux.trees.git/Documentation/dontdiff	2008-05-01 13:13:42.000000000 -0700
+++ linux.trees.git/Documentation/dontdiff	2008-09-01 15:09:16.000000000 -0700
@@ -5,6 +5,8 @@
 *.css
 *.dvi
 *.eps
+*.fw.gen.S
+*.fw
 *.gif
 *.grep
 *.grp

Re: [path] don't diff generated firmware files

[Alexey Dobriyan]

On Mon, Sep 01, 2008 at 03:09:51PM -0700, Arjan van de Ven wrote:
> With the new firmware infrastructure in 2.6.27, some files are generated and shouldn't be
> diffed; add these 2 to the "dontdiff" file

> --- linux.trees.git/Documentation/dontdiff
> +++ linux.trees.git/Documentation/dontdiff
> @@ -5,6 +5,8 @@
>  *.css
>  *.dvi
>  *.eps
> +*.fw.gen.S
> +*.fw

Dontdiff is obsoleted because of .gitignore and should probably be auto-generated
from them.

Re: [path] don't diff generated firmware files

[Randy Dunlap]

On Tue, 2 Sep 2008 03:14:19 +0400 Alexey Dobriyan wrote:

> On Mon, Sep 01, 2008 at 03:09:51PM -0700, Arjan van de Ven wrote:
> > With the new firmware infrastructure in 2.6.27, some files are generated and shouldn't be
> > diffed; add these 2 to the "dontdiff" file
> 
> > --- linux.trees.git/Documentation/dontdiff
> > +++ linux.trees.git/Documentation/dontdiff
> > @@ -5,6 +5,8 @@
> >  *.css
> >  *.dvi
> >  *.eps
> > +*.fw.gen.S
> > +*.fw
> 
> Dontdiff is obsoleted because of .gitignore and should probably be auto-generated
> from them.

Let's please not assume that everyone uses git nor force that.

---
~Randy
Linux Plumbers Conference, 17-19 September 2008, Portland, Oregon USA
http://linuxplumbersconf.org/

[PATCH] dontdiff: more updates to be closer to gitignore

[Randy Dunlap]

defkeymap.c_shipped should be diffed if it is changed.
Reported-by: Mike Galbraith <efault@gmx.de>

COPYING, CREDITS, .mailmap should be diffed if they are changed.
keywords.c_shipped & lex.c_shipped should be diffed when changed.
parse.[ch]_shipped should be diffed when changed.
Reported-by: Sam Ravnborg <sam@ravnborg.org>

vsyscall* updates from a .gitignore patch by "Denis V. Lunev" <den@openvz.org>.

*.so.dbg from a .gitignore patch by Thomas Gleixner <tglx@linutronix.de>.

binoffset from a .gitignore patch by Uwe Kleine-K=F6nig <Uwe.Kleine-Koenig@digi.com>.

Module.markers from a .gitignore patch by Matthew Wilcox <willy@linux.intel.com>.

vmlinux*.lds* should be diffed if changed.
Reported-by: Etienne Lorrain <etienne_lorrain@yahoo.fr>

vmlinux.lds from a .gitignore patch by Daniel Guilak <daniel@danielguilak.com>.

*.scr should be diffed if changed.

Lots of updates from http://lkml.org/lkml/2008/5/20/32
Reported-by: Bart Van Assche <bart.vanassche@gmail.com>

Use ncscope.* instead of *cscope* since the latter may catch too many files.

Add *.elf, from a .gitignore patch by Eduard - Gabriel Munteanu <eduard.munteanu@linux360.ro>.

Make firmware entries match .gitignore entries.

Make some entries less greedy by removing trailing '*'.

Remove "make_times_h" (no such file).
Remove "filelist" (no such file).
Remove "dummy_sym.c" (no such file).
Remove "gen-kdb_cmds.c" (no such file).
Remove "gentbl" (no such file).
Remove "kconfig.tk" (no such file).
Remove "tkparse" (no such file).
Remove "sim710_d.h" (no such file).
Remove "53c8xx_d.h" (no such file).
Add "syscalltab.h" (generated file).

---
 Documentation/dontdiff |   59 +++++++++++++++++++++++++++----------------------
 1 file changed, 33 insertions(+), 26 deletions(-)

--- linux-2.6.27-rc6-git2.orig/Documentation/dontdiff
+++ linux-2.6.27-rc6-git2/Documentation/dontdiff
@@ -2,11 +2,13 @@
 *.aux
 *.bin
 *.cpio
-*.css
+*.csp
+*.dsp
 *.dvi
+*.elf
 *.eps
-*.fw.gen.S
 *.fw
+*.gen.S
 *.gif
 *.grep
 *.grp
@@ -30,6 +32,7 @@
 *.s
 *.sgml
 *.so
+*.so.dbg
 *.symtypes
 *.tab.c
 *.tab.h
@@ -38,24 +41,17 @@
 *.xml
 *_MODULES
 *_vga16.c
-*cscope*
 *~
 *.9
 *.9.gz
 .*
-.cscope
-.gitignore
-.mailmap
 .mm
 53c700_d.h
-53c8xx_d.h*
-COPYING
-CREDITS
 CVS
 ChangeSet
 Image
 Kerntypes
-MODS.txt
+Module.markers
 Module.symvers
 PENDING
 SCCS
@@ -73,7 +69,9 @@ autoconf.h*
 bbootsect
 bin2c
 binkernel.spec
+binoffset
 bootsect
+bounds.h
 bsetup
 btfixupprep
 build
@@ -89,39 +87,36 @@ config_data.h*
 config_data.gz*
 conmakehash
 consolemap_deftbl.c*
+cpustr.h
 crc32table.h*
 cscope.*
-defkeymap.c*
+defkeymap.c
 devlist.h*
 docproc
-dummy_sym.c*
 elf2ecoff
 elfconfig.h*
-filelist
 fixdep
 fore200e_mkfirm
 fore200e_pca_fw.c*
 gconf
 gen-devlist
-gen-kdb_cmds.c*
 gen_crc32table
 gen_init_cpio
 genksyms
-gentbl
 *_gray256.c
+ihex2fw
 ikconfig.h*
 initramfs_data.cpio
 initramfs_data.cpio.gz
 initramfs_list
 kallsyms
 kconfig
-kconfig.tk
-keywords.c*
+keywords.c
 ksym.c*
 ksym.h*
 kxgettext
 lkc_defs.h
-lex.c*
+lex.c
 lex.*.c
 logo_*.c
 logo_*_clut224.c
@@ -130,7 +125,6 @@ lxdialog
 mach-types
 mach-types.h
 machtypes.h
-make_times_h
 map
 maui_boot.h
 mconf
@@ -138,6 +132,7 @@ miboot*
 mk_elfconfig
 mkboot
 mkbugboot
+mkcpustr
 mkdep
 mkprep
 mktables
@@ -145,11 +140,12 @@ mktree
 modpost
 modules.order
 modversions.h*
+ncscope.*
 offset.h
 offsets.h
 oui.c*
-parse.c*
-parse.h*
+parse.c
+parse.h
 patches*
 pca200e.bin
 pca200e_ecd.bin2
@@ -157,7 +153,7 @@ piggy.gz
 piggyback
 pnmtologo
 ppc_defs.h*
-promcon_tbl.c*
+promcon_tbl.c
 pss_boot.h
 qconf
 raid6altivec*.c
@@ -168,27 +164,38 @@ series
 setup
 setup.bin
 setup.elf
-sim710_d.h*
 sImage
 sm_tbl*
 split-include
+syscalltab.h
 tags
 tftpboot.img
 timeconst.h
 times.h*
-tkparse
 trix_boot.h
 utsrelease.h*
+vdso-syms.lds
 vdso.lds
+vdso32-int80-syms.lds
+vdso32-syms.lds
+vdso32-syscall-syms.lds
+vdso32-sysenter-syms.lds
+vdso32.lds
+vdso32.so.dbg
+vdso64.lds
+vdso64.so.dbg
 version.h*
 vmlinux
 vmlinux-*
 vmlinux.aout
-vmlinux*.lds*
-vmlinux*.scr
+vmlinux.lds
 vsyscall.lds
+vsyscall_32.lds
 wanxlfw.inc
 uImage
 unifdef
+wakeup.bin
+wakeup.elf
+wakeup.lds
 zImage*
 zconf.hash.c

[RFC/PATCH] dontdiff: generate from gitignore

[Randy Dunlap]

From: Randy Dunlap <randy.dunlap@oracle.com>

Generate the "dontdiff" file from the .gitignore files.
Save it in scripts/dontdiff.

Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
---
 Makefile           |   10 +++++++++-
 scripts/.gitignore |    1 +
 scripts/mkdontdiff |   23 +++++++++++++++++++++++
 3 files changed, 33 insertions(+), 1 deletion(-)

--- linux-2.6.27-rc6-git2.orig/Makefile
+++ linux-2.6.27-rc6-git2/Makefile
@@ -398,7 +398,7 @@ endif
 no-dot-config-targets := clean mrproper distclean \
 			 cscope TAGS tags help %docs check% \
 			 include/linux/version.h headers_% \
-			 kernelrelease kernelversion
+			 kernelrelease kernelversion dontdiff
 
 config-targets := 0
 mixed-targets  := 0
@@ -1251,6 +1251,7 @@ help:
 	@echo  '  dir/            - Build all files in dir and below'
 	@echo  '  dir/file.[ois]  - Build specified target only'
 	@echo  '  dir/file.ko     - Build module including final link'
+	@echo  '  dontdiff        - Create a "dontdiff" file'
 	@echo  '  prepare         - Set up for building external modules'
 	@echo  '  tags/TAGS	  - Generate tags file for editors'
 	@echo  '  cscope	  - Generate cscope index'
@@ -1519,6 +1520,13 @@ endef
 tags: FORCE
 	$(call cmd,tags)
 
+quiet_cmd_dontdiff = MAKE    dontdiff
+      cmd_dontdiff = $(shell $(CONFIG_SHELL) \
+	                  $(srctree)/scripts/mkdontdiff $(srctree))
+
+dontdiff: FORCE
+	$(call cmd,dontdiff)
+
 
 # Scripts to check various things for consistency
 # ---------------------------------------------------------------------------
--- /dev/null
+++ linux-2.6.27-rc6-git2/scripts/mkdontdiff
@@ -0,0 +1,23 @@
+#! /bin/sh
+set -f
+
+TMPFILE=`mktemp ./.tmpdd.XXXXXX`
+srctree=$1
+DDFILE=$srctree/scripts/dontdiff
+
+find . -name .gitignore | xargs cat | grep -v '^[#!]' | grep -v '^[:space:]*$' | sort | uniq > $TMPFILE
+
+rm -f $DDFILE
+touch $DDFILE
+
+while read PATTERN ; do
+	if echo "$PATTERN" | grep -q '/' ; then
+		filename=`basename $PATTERN`
+	else
+		filename="$PATTERN"
+	fi
+
+	echo $filename >> $DDFILE
+done < $TMPFILE
+
+rm $TMPFILE
--- linux-2.6.27-rc6-git2.orig/scripts/.gitignore
+++ linux-2.6.27-rc6-git2/scripts/.gitignore
@@ -2,6 +2,7 @@
 # Generated files
 #
 conmakehash
+dontdiff
 kallsyms
 pnmtologo
 bin2c

Re: [RFC/PATCH] dontdiff: generate from gitignore

[Linus Torvalds]

On Fri, 12 Sep 2008, Randy Dunlap wrote:
> 
> Generate the "dontdiff" file from the .gitignore files.

t's not quite as simple as that, though.

The .gitignore files are fundamentally location hierarchy-aware, so a 
ignore entry in one subdirectory only affects that subdirectory 
(recursively), so if you flatten then, you should take that into account.

As far as I can tell, your script will generate lots of incorrect entries 
due to this. Eg, it will generate

	parse.[ch]

as a dontdiff pattern, because scripts/genksyms/.gitignore has that, but 
that means that now it will ignore parse.c in all the _other_ places, 
where it *isn't* a generated file (ie mm/parse.c).

		Linus

Re: [RFC/PATCH] dontdiff: generate from gitignore

[Randy Dunlap]

Linus Torvalds wrote:
> 
> On Fri, 12 Sep 2008, Randy Dunlap wrote:
>> Generate the "dontdiff" file from the .gitignore files.
> 
> t's not quite as simple as that, though.
> 
> The .gitignore files are fundamentally location hierarchy-aware, so a 
> ignore entry in one subdirectory only affects that subdirectory 
> (recursively), so if you flatten then, you should take that into account.
> 
> As far as I can tell, your script will generate lots of incorrect entries 
> due to this. Eg, it will generate
> 
> 	parse.[ch]
> 
> as a dontdiff pattern, because scripts/genksyms/.gitignore has that, but 
> that means that now it will ignore parse.c in all the _other_ places, 
> where it *isn't* a generated file (ie mm/parse.c).

Yes, that's one of the reasons that it contains "RFC" in $subject.
There's also no handling of !pattern .gitignore lines....

AFAIK, diff with a dontdiff file does not allow/support full pathname
hierarchies like .gitignore does, so this is probably a futile exercise.  ??

-- 
~Randy
Linux Plumbers Conference, 17-19 September 2008, Portland, Oregon USA
http://linuxplumbersconf.org/

Re: [RFC/PATCH] dontdiff: generate from gitignore

[Kai Henningsen]

On Fri, Sep 12, 2008 at 19:41, Randy Dunlap <randy.dunlap@oracle.com> wrote:
> Linus Torvalds wrote:
>>
>> On Fri, 12 Sep 2008, Randy Dunlap wrote:
>>> Generate the "dontdiff" file from the .gitignore files.
>>
>> t's not quite as simple as that, though.
>>
>> The .gitignore files are fundamentally location hierarchy-aware, so a
>> ignore entry in one subdirectory only affects that subdirectory
>> (recursively), so if you flatten then, you should take that into account.
>>
>> As far as I can tell, your script will generate lots of incorrect entries
>> due to this. Eg, it will generate
>>
>>       parse.[ch]
>>
>> as a dontdiff pattern, because scripts/genksyms/.gitignore has that, but
>> that means that now it will ignore parse.c in all the _other_ places,
>> where it *isn't* a generated file (ie mm/parse.c).
>
> Yes, that's one of the reasons that it contains "RFC" in $subject.
> There's also no handling of !pattern .gitignore lines....
>
> AFAIK, diff with a dontdiff file does not allow/support full pathname
> hierarchies like .gitignore does, so this is probably a futile exercise.  ??

I'm presuming you're talking about this diff option?

`-X FILE'
`--exclude-from=FILE'
     When comparing directories, ignore files and subdirectories whose
     basenames match any pattern contained in FILE.  *Note Comparing
     Directories::.

Indeed: it clearly says "basename" here.

If it weren't for that, you could just use git-ls-files --ignore to
find them ...

[RFC PATCH] Script for generating Documentation/dontdiff from .gitignore files.

[Eduard - Gabriel Munteanu]

.gitignore files get more testing than Documentation/dontdiff, since most
developers follow Git workflow when sending patches. The latter tends to
stay outdated and needs to be synchronized every now and then. This patch
provides a script that scans all .gitignore files and generates a
Documentation/dontdiff to stdout.

Signed-off-by: Eduard - Gabriel Munteanu <eduard.munteanu@linux360.ro>
---
 scripts/gen_dontdiff.sh |   21 +++++++++++++++++++++
 1 files changed, 21 insertions(+), 0 deletions(-)
 create mode 100755 scripts/gen_dontdiff.sh

diff --git a/scripts/gen_dontdiff.sh b/scripts/gen_dontdiff.sh
new file mode 100755
index 0000000..cd94747
--- /dev/null
+++ b/scripts/gen_dontdiff.sh
@@ -0,0 +1,21 @@
+#!/bin/bash
+
+# Copyright (C) 2008 Eduard - Gabriel Munteanu
+#
+# This file is released under GPL version 2.
+
+echo "### Generated by gen_dontdiff.sh ###"
+
+grep -Ev "^([[:space:]]*[[#]]*|$)" .gitignore
+
+find . -mindepth 2 -name .gitignore | sed -e "s/^\.\///g" | while read FILE
+do
+	CURR_DIR=`echo "$FILE" | sed -e "s/\.gitignore$//g"`
+	echo "# $FILE"
+	grep -Ev "^([[:space:]]*[[#]]*|$)" $FILE | while read LINE
+	do
+		echo $CURR_DIR$LINE
+	done
+	echo ""
+done
+
-- 
1.5.6.1

Re: [RFC PATCH] Script for generating Documentation/dontdiff from .gitignore files.

[Randy Dunlap]

On Sun, 14 Sep 2008 21:23:57 +0300 Eduard - Gabriel Munteanu wrote:

> .gitignore files get more testing than Documentation/dontdiff, since most
> developers follow Git workflow when sending patches. The latter tends to
> stay outdated and needs to be synchronized every now and then. This patch
> provides a script that scans all .gitignore files and generates a
> Documentation/dontdiff to stdout.
> 
> Signed-off-by: Eduard - Gabriel Munteanu <eduard.munteanu@linux360.ro>

Hi,
You probably missed (& should read) this thread:
  http://lkml.org/lkml/2008/9/12/154


> ---
>  scripts/gen_dontdiff.sh |   21 +++++++++++++++++++++
>  1 files changed, 21 insertions(+), 0 deletions(-)
>  create mode 100755 scripts/gen_dontdiff.sh
> 
> diff --git a/scripts/gen_dontdiff.sh b/scripts/gen_dontdiff.sh
> new file mode 100755
> index 0000000..cd94747
> --- /dev/null
> +++ b/scripts/gen_dontdiff.sh
> @@ -0,0 +1,21 @@
> +#!/bin/bash
> +
> +# Copyright (C) 2008 Eduard - Gabriel Munteanu
> +#
> +# This file is released under GPL version 2.
> +
> +echo "### Generated by gen_dontdiff.sh ###"
> +
> +grep -Ev "^([[:space:]]*[[#]]*|$)" .gitignore
> +
> +find . -mindepth 2 -name .gitignore | sed -e "s/^\.\///g" | while read FILE
> +do
> +	CURR_DIR=`echo "$FILE" | sed -e "s/\.gitignore$//g"`
> +	echo "# $FILE"
> +	grep -Ev "^([[:space:]]*[[#]]*|$)" $FILE | while read LINE
> +	do
> +		echo $CURR_DIR$LINE
> +	done
> +	echo ""
> +done
> +
> -- 

---
~Randy
Linux Plumbers Conference, 17-19 September 2008, Portland, Oregon USA
http://linuxplumbersconf.org/

Re: [RFC PATCH] Script for generating Documentation/dontdiff from .gitignore files. (was Re: [RFC/PATCH] dontdiff: generate from gitignore)

[Eduard - Gabriel Munteanu]

On Mon, Sep 15, 2008 at 08:34:07AM -0700, Randy Dunlap wrote:
> On Sun, 14 Sep 2008 21:23:57 +0300 Eduard - Gabriel Munteanu wrote:
> 
> > .gitignore files get more testing than Documentation/dontdiff, since most
> > developers follow Git workflow when sending patches. The latter tends to
> > stay outdated and needs to be synchronized every now and then. This patch
> > provides a script that scans all .gitignore files and generates a
> > Documentation/dontdiff to stdout.
> > 
> > Signed-off-by: Eduard - Gabriel Munteanu <eduard.munteanu@linux360.ro>
> 
> Hi,
> You probably missed (& should read) this thread:
>   http://lkml.org/lkml/2008/9/12/154
> 
>

Thanks, it makes sense.

It seems like diff can't handle such scenarios at all. I suppose Linux should
supply its own diff tool (which uses regular diff, but implements proper file
matching).

Maybe git-diff alone could be included in the tarballs, I'll look into
it.


	Cheers,
	Eduard

> > ---
> >  scripts/gen_dontdiff.sh |   21 +++++++++++++++++++++
> >  1 files changed, 21 insertions(+), 0 deletions(-)
> >  create mode 100755 scripts/gen_dontdiff.sh
> > 
> > diff --git a/scripts/gen_dontdiff.sh b/scripts/gen_dontdiff.sh
> > new file mode 100755
> > index 0000000..cd94747
> > --- /dev/null
> > +++ b/scripts/gen_dontdiff.sh
> > @@ -0,0 +1,21 @@
> > +#!/bin/bash
> > +
> > +# Copyright (C) 2008 Eduard - Gabriel Munteanu
> > +#
> > +# This file is released under GPL version 2.
> > +
> > +echo "### Generated by gen_dontdiff.sh ###"
> > +
> > +grep -Ev "^([[:space:]]*[[#]]*|$)" .gitignore
> > +
> > +find . -mindepth 2 -name .gitignore | sed -e "s/^\.\///g" | while read FILE
> > +do
> > +	CURR_DIR=`echo "$FILE" | sed -e "s/\.gitignore$//g"`
> > +	echo "# $FILE"
> > +	grep -Ev "^([[:space:]]*[[#]]*|$)" $FILE | while read LINE
> > +	do
> > +		echo $CURR_DIR$LINE
> > +	done
> > +	echo ""
> > +done
> > +
> > -- 
> 
> ---
> ~Randy
> Linux Plumbers Conference, 17-19 September 2008, Portland, Oregon USA
> http://linuxplumbersconf.org/