How to fix (and find) many git-* --check errors?

How to fix (and find) many git-* --check errors?

["Peter Valdemar Mørch (Lists)"]

We have > 37000 white space "errors" in HEAD, mostly trailing 
whitespace, and I'm looking for a

$ git diff --check | git??? --whitespace=fix

command.

Is there such a beast?

I see that git-apply has a --whitespace=<action> option, but I don't 
seem to grock how to be able to use it for fixing my working directory.

Details follow:

I can create a perl script that does this for me (e.g. inspired by 
1.5.6's hooks/pre-commit's perl version of git diff --check) and post it 
here if anybody would like it, but I'd rather use some well-tested 
method if one exists. And it seems git-apply has the functionality somehow.

Of course, I can also:

$ git diff --check > tmpcfile
# (Or some other command to find all of them under ./)
$ vim
:cfile tmpcfile

Thank you for --check having a handy output format, BTW! But I prefer 
automation (and automated auditing of the results) for 37000 lines.

Also the way I found them is like this:

$ git diff --check $(git log --pretty=format:%H | tail -1)..HEAD .

(The diff between "the empty commit" and HEAD - well between the first 
commit and HEAD anyway. Is there a ref for "totally empty" or the 
revision before the first commit? Or a more elegant way to get this list?)

Peter
-- 
Peter Valdemar Mørch
http://www.morch.com

Re: How to fix (and find) many git-* --check errors?

[Jeff King]

On Fri, Aug 08, 2008 at 02:49:00PM +0200, "Peter Valdemar Mørch (Lists)" wrote:

> (The diff between "the empty commit" and HEAD - well between the first
> commit and HEAD anyway. Is there a ref for "totally empty" or the
> revision before the first commit? Or a more elegant way to get this
> list?)

It is not advertised, but we always recognize the empty sha1 of the
empty tree:

  git diff --check 4b825dc642cb6eb9a060e54bf8d69288fbee4904

In fact, just the other day I was using this for the Nth time and got
tired of looking it up in the code, so I wrote the patch below. I don't
know if it is too crazy to be included in mainline git (it was discussed
a long time ago, but I think the general response was "what would it be
good for?").

---
 cache.h     |    4 ++++
 refs.c      |    4 ++++
 sha1_file.c |    3 +--
 3 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/cache.h b/cache.h
index 30f1d62..4aa02ae 100644
--- a/cache.h
+++ b/cache.h
@@ -567,6 +567,10 @@ static inline unsigned int hexval(unsigned char c)
 #define MINIMUM_ABBREV 4
 #define DEFAULT_ABBREV 7
 
+#define EMPTY_TREE_SHA1 \
+	"\x4b\x82\x5d\xc6\x42\xcb\x6e\xb9\xa0\x60" \
+	"\xe5\x4b\xf8\xd6\x92\x88\xfb\xee\x49\x04"
+
 extern int get_sha1(const char *str, unsigned char *sha1);
 extern int get_sha1_with_mode(const char *str, unsigned char *sha1, unsigned *mode);
 extern int get_sha1_hex(const char *hex, unsigned char *sha1);
diff --git a/refs.c b/refs.c
index 39a3b23..0acbcbc 100644
--- a/refs.c
+++ b/refs.c
@@ -427,6 +427,10 @@ const char *resolve_ref(const char *ref, unsigned char *sha1, int reading, int *
 				}
 				list = list->next;
 			}
+			if (!strcmp(ref, "EMPTY")) {
+				hashcpy(sha1, (unsigned char *)EMPTY_TREE_SHA1);
+				return ref;
+			}
 			if (reading || errno != ENOENT)
 				return NULL;
 			hashclr(sha1);
diff --git a/sha1_file.c b/sha1_file.c
index 2aff59b..38aad13 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1985,8 +1985,7 @@ static int cached_object_nr, cached_object_alloc;
 
 static struct cached_object empty_tree = {
 	/* empty tree sha1: 4b825dc642cb6eb9a060e54bf8d69288fbee4904 */
-	"\x4b\x82\x5d\xc6\x42\xcb\x6e\xb9\xa0\x60"
-	"\xe5\x4b\xf8\xd6\x92\x88\xfb\xee\x49\x04",
+	EMPTY_TREE_SHA1,
 	OBJ_TREE,
 	"",
 	0
-- 
1.6.0.rc1.260.g4782

Re: How to fix (and find) many git-* --check errors?

[Björn Steinbrink]

On 2008.08.08 14:49:00 +0200, "Peter Valdemar Mørch (Lists)" wrote:
> We have > 37000 white space "errors" in HEAD, mostly trailing  
> whitespace, and I'm looking for a
>
> $ git diff --check | git??? --whitespace=fix
>
> command.
>
> Is there such a beast?
>
> I see that git-apply has a --whitespace=<action> option, but I don't  
> seem to grock how to be able to use it for fixing my working directory.

I'd probably do something like:

# Create a commit with an empty tree
rm .git/index
git commit -m tmp

rm -r * (include dotfiles if required, ie. remove all tracked files)

git diff -R HEAD^ | git apply --index --whitespace=fix
git commit --amend -m "Whitespace fixed up"

But probably there's some smarter way than that.

Björn

Re: How to fix (and find) many git-* --check errors?

["Peter Valdemar Mørch (Lists)"]

Thanks to hints from both Jeff King and Björn Stenbrink I got it 
working. Here is my script, if anybody is interested.

It handles both fixing whitespace in current workspace diffs:

$ git-fix-whitespace file

(requires existing diffs)

and fixing all whitespace problems in a file:

$ git-fix-whitespace -f file

(asserts no existing diffs)

Now all I need to do is grock how to fix stuff already in the index, but 
I'll leave that for another day...

Peter

-----------------
#!/bin/bash

# Take a -f option to fix all whitespace problems in entire files
# completely. Otherwise only fix current diffs.

# Option handling
while getopts fh? o
do	
     case "$o" in
	f)	fixEntireFile=1;;
	[h?])	echo >&2 "Usage: $0 [-f] file ..."
		echo >&2 "-f: Fix the entire file"
		echo >&2 "    Otherwise only fix current diffs"
		exit 1;;
     esac
done

shift $(($OPTIND-1))

# A constant for a rev that is empty
EMPTY_TREE_SHA1=4b825dc642cb6eb9a060e54bf8d69288fbee4904

# Check to see we're on a clean HEAD just to be sure.
# (Not sure this is necessary, but just in case.)
git --no-pager diff --exit-code HEAD -- "$@"  > /dev/null
if [ $? = 0 ] ; then
	# There is no current diff
	if [ "$fixEntireFile" != "" ] ; then
		# Good, there is no diff, and we've been asked to fix
		# the entire file

		# Remove all files - the -f for rm is just needed if
		# $@ happens to contain only untracked files
		git-ls-tree -z  -r --name-status HEAD "$@"  | \
			xargs --null rm -f

		# Re-create the files - but with whitespace fixed
		git diff $EMPTY_TREE_SHA1 HEAD -- "$@" | \
			git-apply --whitespace=fix
	else
		echo >&2 '*Error*: there no diff with HEAD'
		exit 1
	fi
else
	# There is a current diff
	if [ "$fixEntireFile" = "" ] ; then
		# Good, there is a current diff, that we need to fix
		TEMP_FILE=$(tempfile)
		git diff -- "$@" > $TEMP_FILE
		git checkout HEAD -- "$@"
		git apply --whitespace=fix $TEMP_FILE
		rm $TEMP_FILE
	else
		echo >&2 '*Error*: there is diff with HEAD'
		exit 1
	fi
fi

Re: How to fix (and find) many git-* --check errors?

[Junio C Hamano]

"Peter Valdemar Mørch (Lists)"  <4ux6as402@sneakemail.com> writes:

> We have > 37000 white space "errors" in HEAD, mostly trailing
> whitespace, and I'm looking for a
>
> $ git diff --check | git??? --whitespace=fix
>
> command.

Starting from a clean checkout, you could do something like this:

	$ git reset --hard
        $ rm .git/index
        $ git diff --binary -R HEAD >P.diff
        $ git apply --whitespace=fix --cached <P.diff
	$ git commit -m "Fixed all whitespace gotchas"

P.diff contains essentially everything, and you are recreating everything
from that patch.

Re: How to fix (and find) many git-* --check errors?

["Peter Valdemar Mørch (Lists)"]

Junio C Hamano gitster-at-pobox.com |Lists| wrote:
> Starting from a clean checkout, you could do something like this:
> 
> 	$ git reset --hard
>         $ rm .git/index
>         $ git diff --binary -R HEAD >P.diff
>         $ git apply --whitespace=fix --cached <P.diff
> 	$ git commit -m "Fixed all whitespace gotchas"
> 
> P.diff contains essentially everything, and you are recreating everything
> from that patch.

Thanks for taking the time to answer my many mails and questions. It 
really is helpful to me.

Rambling on:

Scary with the "rm .git/index"! It bascially creates an empty index it 
seems... That was news to me. As a newbie I try not to meddle too much 
in .git/ ...

I tried the above on git.git, and it gave:
warning: 485 lines applied after fixing whitespace errors.

I would suggest finishing the above with:
$ git checkout HEAD
otherwise, the working dir is stuck at the old unfixed state.

But like I posted earlier, I have another solution that works on 
selected files. With git, there is always more than one way to do 
things, it seems...

Peter
-- 
Peter Valdemar Mørch
http://www.morch.com