bug with .gitattributes diff and embedded NUL

bug with .gitattributes diff and embedded NUL

[Eric Blake]

The man page for gitattributes suggests that one use of the diff attribute
is to force text-based diffs, even when the file contains embedded NULs.
Yet actually trying this falls flat, since git generates invalid diffs
when encountering embedded NUL.

Here is a pared-down example showing the bug.  I based it off of what I
was trying to do in the git repository for GNU m4, where I was working on
adding testcases to make m4 seamlessly handle embedded NUL.

$ git --version
git version 1.5.4.rc0.67.gf9c5
$ mkdir sample
$ cd sample
$ git init
Initialized empty Git repository in .git/
$ printf 'context with\0embedded nul\nbefore\nmore context\n' > file
$ git add file
$ git commit -m initial
Created initial commit 25108f6: initial
 1 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 file
$ git tag t1
$ sed -i 's/before/after/' file
$ git add file
$ git commit -m later
Created commit a5ee10c: later
 1 files changed, 0 insertions(+), 0 deletions(-)
$ git tag t2
$ git reset --hard t1
HEAD is now at 25108f6... initial
$ sed -i 's/more/some &/' file
$ echo 'file diff' > .gitattributes
$ git add .gitattributes file
$ git commit -m 'switch file to text mode'
Created commit 0f2a9b2: switch file to text mode
 2 files changed, 2 insertions(+), 1 deletions(-)
 create mode 100644 .gitattributes
$ git tag t3
$ git reset --hard t2
HEAD is now at a5ee10c... later
$ git rebase --onto t3 t1
First, rewinding head to replay your work on top of it...
HEAD is now at 0f2a9b2... switch file to text mode
Applying later
error: patch failed: file:1
error: file: patch does not apply
Using index info to reconstruct a base tree...
error: patch failed: file:1
error: file: patch does not apply
Did you hand edit your patch?
It does not apply to blobs recorded in its index.
Cannot fall back to three-way merge.
Patch failed at 0001.

When you have resolved this problem run "git rebase --continue".
If you would prefer to skip this patch, instead run "git rebase --skip".
To restore the original branch and stop rebasing run "git rebase --abort".

Looking closer at .dotest/0001, the diff is indeed invalid, containing the
single line

 context with-before

instead of the desired two lines

 context with^@embedded nul
-before

I'm also wondering if using GIT_EXTERNAL_DIFF would provide a way to work
around this bug in git diff, since GNU diffutils doesn't seem to have any
problems when doing text-mode diffs with embedded NUL:

$ diff -u -a <(git show t1:file) <(git show t2:file) | cat -v
--- /dev/fd/63	2006-11-30 17:00:00.000000000 -0700
+++ /dev/fd/62	2006-11-30 17:00:00.000000000 -0700
@@ -1,3 +1,3 @@
 context with^@embedded nul
-before
+after
 more context

-- 
Don't work too hard, make some time for fun as well!

Eric Blake             ebb9@byu.net

Re: bug with .gitattributes diff and embedded NUL

[Junio C Hamano]

Eric Blake <ebb9@byu.net> writes:

> Looking closer at .dotest/0001, the diff is indeed invalid, containing the
> single line

That is not a bug in diff but actually in bug in non-merge form of
rebase, which reuses "format patch as if we are sending it out in
e-mail, and apply that on top of a different commit as if we are
accepting that e-mail" machinery.  That dataflow loses NUL.  Fixing
rebase not to reuse the e-mailed patch dataflow is on my to-do list, but
has been slipping.

So, no, I do not think GIT_EXTERNAL_DIFF would help, as the problem is
not diff generation.

I think you can use "rebase -m" to work this issue around.

Re: bug with .gitattributes diff and embedded NUL

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

According to Junio C Hamano on 12/18/2007 11:40 PM:
> Eric Blake <ebb9@byu.net> writes:
> 
>> Looking closer at .dotest/0001, the diff is indeed invalid, containing the
>> single line
> 
> That dataflow loses NUL.  Fixing
> rebase not to reuse the e-mailed patch dataflow is on my to-do list, but
> has been slipping.
> 
> I think you can use "rebase -m" to work this issue around.

Indeed, after learning about git-rebase -m and git-mergetool, adding the
merge option alongside the diff option for the file in question, and
teaching emacs that it should use emerge-diff-options "-a" so that it can
handle embedded NUL, I was finally able to use git-mergetool to do a sane
merge of the differences in the mostly-text file.  Thanks!

But it raised some additional issues:

Is there a way to specify custom merge tools, rather than the current
hard-coded list valid for merge.tool?  For example, what if I prefer
emacs' ediff driver over its emerge driver?

Using merge.tool of emerge leaves junk around when the merge is aborted.
Before I edited emacs' emerge-diff-options, the subsidiary diff3 was
failing because it treated the file to merge as binary even though git was
trying to treat it as text.  As a result, emerge left behind all of its
command-line argument files (file.{BASE,LOCAL,REMOTE}.pid), and git did
not clean any of them up.

- --
Don't work too hard, make some time for fun as well!

Eric Blake             ebb9@byu.net
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHaShk84KuGfSFAYARAkDtAJ0XhhhbeJZJS6td3/aeOlxd7iuoawCgiVY6
3s+aMxRO1vjKf/EqncKW2jE=
=GrcF
-----END PGP SIGNATURE-----