How can a custom merge tool get access to file shell variables?

How can a custom merge tool get access to file shell variables?

[Keith Amidon]

I'm in a situation where I have to fairly regularly merge two different
trees where multiple files are being simultaneously edited in both trees
and whitespace in some of the files is different in both trees as well.
I would like to be able to use git mergetool to walk me through
resolving conflicts in merges.  My preferred graphical merge tool is
xxdiff but in 3-way mode it doesn't allow you to ignore whitespace while
doing the diff, only while displaying.  As a result, all these files
appear to have one giant diff and separating out what has really changed
is a giant pain.  Since my use-case is simple, I thought I would create
a custom mergetool as a shell script that invokes xxdiff in 2-way mode
and moves around files to get what I need.  I've pasted that script at
the end of the message.

I've added this script as custom mergetool as indicated in comments at
the top of the script, but it does not work.  It appears to me that the
problem is that the shell variables containing the required filenames
(BASE, LOCAL, REMOTE, MERGED) are not set.  Looking
at /usr/bin/git-mergetool, it appears that custom merge tools are called
without arguments in a subshell and those variables are not exported
anywhere so they are not in the environment of the subshell.  Given
this, how can a custom mergetool ever get access to this information to
do its work?

If there is another better way to try to accomplish this (other than
forcing whitespace to be the same in both trees for now!) I'd appreciate
it if someone would clue me in!

             --- Keith


#!/bin/bash
#
# Perform a merge for GIT in which whitespace is ignored.  Because the
# underlying diff tools don't seem to be able to ignore whitespace on
# a three-way merge, a 2-way merge is done where the first file is a
# copy of the remote file to be merged and the second file is copy of
# the current file from the branch into which files are being merged.
# The success of the merge is determined by whether a merged file is
# written out from xxdiff.  If you don't save the merged file, the
# merge will be aborted.
#
# This script should be configured as a merge tool in git as follows:
#
# git-config --global --add mergetool.xxdiff-2way-ignorews.cmd
xxdiff-2way-ignorews-git-mergetool
# git-config --global --add mergetool.xxdiff-2way-ignorews.trustExitCode
true
# 

rtmpfile=""
ltmpfile=""

function remove_tmpfiles () {
    if [ -n "$ltmpfile" ]; then
        rm -f "$ltmpfile"
    fi
    if [ -n "$rtmpfile" ]; then
        rm -f "$rtmpfile"
        rm -f "$rtmpfile.merge"
    fi
}

function die () {
    printf "%b" "$1"
    remove_tmpfiles
    exit 1
}

[ -f "$LOCAL" ] || die "Local file does not exists: $LOCAL\n"
[ -f "$REMOTE" ] || die "Remote file does not exists: $REMOTE\n"

# Be really paranoid and copy git's temp files to new ones so we aren't
# playing around with git's files...
ltmpfile=$(mktemp --tmpdir xxdiff-2way-tmp.remote.XXXXXXXXXX)
cp "$LOCAL" "$ltmpfile" || die "Could not copy local file to temporary
merge file.\nLocal file is: $LOCAL\nTemporary merge file is: $ltmpfile
\n"

rtmpfile=$(mktemp --tmpdir xxdiff-2way-tmp.local.XXXXXXXXXX)
cp "$REMOTE" "$rtmpfile" || die "Could not copy remote file to temporary
merge result file.\nLocal file is: $LOCAL\nTemporary merge file is:
$tmpfile\n"

xxdiff -w "$rtmpfile" "$ltmpfile"

if [ -f "$rtmpfile.merge" ]; then
    if [ -f "$MERGED" ]; then
        printf "Merged file already exists, removing so merge result can
be renamed.\n"
        rm -f "$MERGED"
    fi
    mv "$rtmpfile.merge" "$MERGED" || die "Could not rename merge result
to merged file: $MERGED\n"
    remove_tmpfiles
else
    die "Did not find merge output file from xxdiff, aborting merge.\n"
fi

Re: How can a custom merge tool get access to file shell variables?

[Keith Amidon]

In case anyone is interested, the following does exactly what I want.
Hopefully this example will help someone else that wants to do something
similar avoid spending time on writing an unnecessary shell script
wrapper tool.

[mergetool "xxdiff-2way-ignorews"]
    cmd = xxdiff -w $REMOTE $LOCAL --merged-filename $MERGED
    trustExitCode = false

I think I was thrown off by the description of mergetool.<name>.cmd in
the git-config man page.  While on close reading it is definitely
correct, for me at least it seemed natural to assume that the invoked
command was supposed to get its information from the environment, not
that the command line itself could substitute from the environment.
Would an example such as the above in the man page might help direct
people toward the best way to do this?

           --- Keith

On Tue, 2008-08-19 at 11:20 -0700, Keith Amidon wrote:
> Please ignore this.  I was not effective enough in looking for an
> answer even though I did search the archives a bit before writing
> this.  The answer is here:
> 
> http://article.gmane.org/gmane.comp.version-control.git/82371/match=mergetool+export
> 
> I'm off to try a custom way of invoking xxdiff that does what I want. 
> 
> --- Keith
> 
> 
> On Tue, 2008-08-19 at 11:13 -0700, Keith Amidon wrote: 
> > I'm in a situation where I have to fairly regularly merge two different
> > trees where multiple files are being simultaneously edited in both trees
> > and whitespace in some of the files is different in both trees as well.
> > I would like to be able to use git mergetool to walk me through
> > resolving conflicts in merges.  My preferred graphical merge tool is
> > xxdiff but in 3-way mode it doesn't allow you to ignore whitespace while
> > doing the diff, only while displaying.  As a result, all these files
> > appear to have one giant diff and separating out what has really changed
> > is a giant pain.  Since my use-case is simple, I thought I would create
> > a custom mergetool as a shell script that invokes xxdiff in 2-way mode
> > and moves around files to get what I need.  I've pasted that script at
> > the end of the message.
> > 
> > I've added this script as custom mergetool as indicated in comments at
> > the top of the script, but it does not work.  It appears to me that the
> > problem is that the shell variables containing the required filenames
> > (BASE, LOCAL, REMOTE, MERGED) are not set.  Looking
> > at /usr/bin/git-mergetool, it appears that custom merge tools are called
> > without arguments in a subshell and those variables are not exported
> > anywhere so they are not in the environment of the subshell.  Given
> > this, how can a custom mergetool ever get access to this information to
> > do its work?
> > 
> > If there is another better way to try to accomplish this (other than
> > forcing whitespace to be the same in both trees for now!) I'd appreciate
> > it if someone would clue me in!
> > 
> >              --- Keith
> > 
> > 
> > #!/bin/bash
> > #
> > # Perform a merge for GIT in which whitespace is ignored.  Because the
> > # underlying diff tools don't seem to be able to ignore whitespace on
> > # a three-way merge, a 2-way merge is done where the first file is a
> > # copy of the remote file to be merged and the second file is copy of
> > # the current file from the branch into which files are being merged.
> > # The success of the merge is determined by whether a merged file is
> > # written out from xxdiff.  If you don't save the merged file, the
> > # merge will be aborted.
> > #
> > # This script should be configured as a merge tool in git as follows:
> > #
> > # git-config --global --add mergetool.xxdiff-2way-ignorews.cmd
> > xxdiff-2way-ignorews-git-mergetool
> > # git-config --global --add mergetool.xxdiff-2way-ignorews.trustExitCode
> > true
> > # 
> > 
> > rtmpfile=""
> > ltmpfile=""
> > 
> > function remove_tmpfiles () {
> >     if [ -n "$ltmpfile" ]; then
> >         rm -f "$ltmpfile"
> >     fi
> >     if [ -n "$rtmpfile" ]; then
> >         rm -f "$rtmpfile"
> >         rm -f "$rtmpfile.merge"
> >     fi
> > }
> > 
> > function die () {
> >     printf "%b" "$1"
> >     remove_tmpfiles
> >     exit 1
> > }
> > 
> > [ -f "$LOCAL" ] || die "Local file does not exists: $LOCAL\n"
> > [ -f "$REMOTE" ] || die "Remote file does not exists: $REMOTE\n"
> > 
> > # Be really paranoid and copy git's temp files to new ones so we aren't
> > # playing around with git's files...
> > ltmpfile=$(mktemp --tmpdir xxdiff-2way-tmp.remote.XXXXXXXXXX)
> > cp "$LOCAL" "$ltmpfile" || die "Could not copy local file to temporary
> > merge file.\nLocal file is: $LOCAL\nTemporary merge file is: $ltmpfile
> > \n"
> > 
> > rtmpfile=$(mktemp --tmpdir xxdiff-2way-tmp.local.XXXXXXXXXX)
> > cp "$REMOTE" "$rtmpfile" || die "Could not copy remote file to temporary
> > merge result file.\nLocal file is: $LOCAL\nTemporary merge file is:
> > $tmpfile\n"
> > 
> > xxdiff -w "$rtmpfile" "$ltmpfile"
> > 
> > if [ -f "$rtmpfile.merge" ]; then
> >     if [ -f "$MERGED" ]; then
> >         printf "Merged file already exists, removing so merge result can
> > be renamed.\n"
> >         rm -f "$MERGED"
> >     fi
> >     mv "$rtmpfile.merge" "$MERGED" || die "Could not rename merge result
> > to merged file: $MERGED\n"
> >     remove_tmpfiles
> > else
> >     die "Did not find merge output file from xxdiff, aborting merge.\n"
> > fi
> > 

Re: How can a custom merge tool get access to file shell variables?

[Charles Bailey]

On Tue, Aug 19, 2008 at 11:32:31AM -0700, Keith Amidon wrote:
> In case anyone is interested, the following does exactly what I want.
> Hopefully this example will help someone else that wants to do something
> similar avoid spending time on writing an unnecessary shell script
> wrapper tool.
> 
> [mergetool "xxdiff-2way-ignorews"]
>     cmd = xxdiff -w $REMOTE $LOCAL --merged-filename $MERGED
>     trustExitCode = false
> 
> I think I was thrown off by the description of mergetool.<name>.cmd in
> the git-config man page.  While on close reading it is definitely
> correct, for me at least it seemed natural to assume that the invoked
> command was supposed to get its information from the environment, not
> that the command line itself could substitute from the environment.
> Would an example such as the above in the man page might help direct
> people toward the best way to do this?
> 
>            --- Keith

Did you also try the git mergetool man page (not that it's much
better!)?

The point of the custom mergetool patch was to make using a new,
previously unknown merge tool a 'simple' configuration exercise rather
than a patch or scripting exercise. At the time, an 'eval' approach
was the compromise between ease of implementation and preventing
environmental pollution.

It's not the most beautiful of solutions, especially since escaping
quotes and spaces in either .gitconfig or in a git config command line
of something that is later going to be expanded by the shell is
something of a mind bender.

An example in the documentation would be a really good idea - you are
not the first person to have asked about how to use the custom merge
tool feature. I'm feeling a little guilty about not adding my name to
the man page when I submitted the patch. It's not Ted's fault that the
custom merge tool section is badly explained; it's mine.

-- 
Charles Bailey
http://ccgi.hashpling.plus.com/blog/

Re: How can a custom merge tool get access to file shell variables?

[Keith Amidon]

Hi Charles,

Thanks for your reply to my issue.  To answer your question, I did read
the git mergetool man page too.  For some reason I felt that the
information in the git config page was more complete/specific for this
case.  Anyway, now I understand how it works (and quoting issues I
haven't had to worry about because I don't have weird filenames aside),
the setup actually works very nicely.  If you would be interested and
could provide some guidance on roughly what you would like to see I'd be
happy to take a crack at an initial modification to the git mergetool
man page to include an example similar to mine below with proper quoting
so paths with spaces in them work, etc.

Thanks again for your response and for the excellent work you and
everyone involved in git development have been doing.

             --- Keith

P.S. -- Sorry if you get two copies Charles.  My previous version of
this message was rejected by the mailing list because I forgot to send
w/o HTML.

On Thu, 2008-08-21 at 23:15 +0100, Charles Bailey wrote:
> Did you also try the git mergetool man page (not that it's much
> better!)?
> 
> The point of the custom mergetool patch was to make using a new,
> previously unknown merge tool a 'simple' configuration exercise rather
> than a patch or scripting exercise. At the time, an 'eval' approach
> was the compromise between ease of implementation and preventing
> environmental pollution.
> 
> It's not the most beautiful of solutions, especially since escaping
> quotes and spaces in either .gitconfig or in a git config command line
> of something that is later going to be expanded by the shell is
> something of a mind bender.
> 
> An example in the documentation would be a really good idea - you are
> not the first person to have asked about how to use the custom merge
> tool feature. I'm feeling a little guilty about not adding my name to
> the man page when I submitted the patch. It's not Ted's fault that the
> custom merge tool section is badly explained; it's mine.
>