merging .gitignore

merging .gitignore

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 1220 bytes --]

Dear gits (oh dear…),

we just ran into a problem in a git-managed project and I'd be
interested to learn how you approach this.

Our main line ("upstream"), which tracks a remote repository, does
not have a .gitignore file. For new features, we use feature
branches, and we merge those into an integration branch ("master")
and track them separately of upstream.

Feature branch A has a .gitignore file, and it's been merged into
master for a while. Today, feature branch B failed to merge into
master because it also provides a .gitignore file. We can obviously
resolve the conflict, but I wonder whether there is a better way to
deal with this since we deal with quite a large number of new
feature branches and it's only a matter of time until the next one
will conflict because of .gitignore.

(and yes, this is basically a reincarnation of my case for
.gitignore.d [http://lists.zerezo.com/git/msg627581.html]).

Thoughts,

-- 
martin;              (greetings from the heart of the sun.)
  \____ echo mailto: !#^."<*>"|tr "<*> mailto:" net@madduck
 
kermit: why are there so many songs about rainbows?
fozzy: that's part of what rainbows do.
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: merging .gitignore

[Andy Parkins]

On Monday 2007 October 01, martin f krafft wrote:

> Feature branch A has a .gitignore file, and it's been merged into
> master for a while. Today, feature branch B failed to merge into
> master because it also provides a .gitignore file. We can obviously
> resolve the conflict, but I wonder whether there is a better way to
> deal with this since we deal with quite a large number of new
> feature branches and it's only a matter of time until the next one
> will conflict because of .gitignore.

But it _is_ a conflict.  Conflicts have to be resolved.  I'm having difficulty 
understanding what you think git should be doing in these cases?

> (and yes, this is basically a reincarnation of my case for
> .gitignore.d [http://lists.zerezo.com/git/msg627581.html]).

I don't see that that would help.  All you are doing with a gitignore.d is 
swapping lines for files, the conflicts would still exist.  Presumably you 
are hoping that the separate branches will make different files in 
gitignore.d and hence can't conflict; but then you've just pushed the 
conflict to a place where it won't be seen (and also made a terrible mess of 
the merged branch gitignore.d).

 branchA:.gitignore.d/branchAignores
 branchB:.gitignore.d/branchBignores

Over time you would get:

 master:.gitignore.d/branchAignores
 master:.gitignore.d/branchBignores
 master:.gitignore.d/branchCignores
 master:.gitignore.d/branchDignores
 master:.gitignore.d/branchEignores
 master:.gitignore.d/branchFignores

Then, assuming the conflicts you get now occur for a reason, you will get 
conflicts within the .gitignore.d/ directory.  Let's say branchCignores adds 
*.o and branchFignores removes *.o from the ignores.  Who is right?  Who 
knows, and worse than that you didn't see the conflict when it happened so it 
wasn't resolved and the master branch was left with conflicts in it.

Of course the conflicts in that case aren't in the form of "<<<<<" markers, 
but they are no less conflicts just because they're invisible.

Eventually someone is going to want to combine this ever-increasing set of 
ignore files into one.  i.e. they'll merge them.  In which case why couldn't 
you do the merge at the correct time - when the branch was merged?



Andy
-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com

Re: merging .gitignore

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 1680 bytes --]

also sprach Johannes Schindelin <Johannes.Schindelin@gmx.de> [2007.10.01.1457 +0100]:
> You might be interested in writing a merge driver.  See 
> Documentation/gitattributes.txt.

This is an excellent idea; thanks.



also sprach Andy Parkins <andyparkins@gmail.com> [2007.10.01.1448 +0100]:
> Then, assuming the conflicts you get now occur for a reason, you
> will get conflicts within the .gitignore.d/ directory.  Let's say
> branchCignores adds *.o and branchFignores removes *.o from the
> ignores.  Who is right?  Who knows, and worse than that you didn't
> see the conflict when it happened so it wasn't resolved and the
> master branch was left with conflicts in it.

Well, with gitignore I am ready to say that merges should be
resolved in an additive way. Remember that I am talking about an
intergration branch, and if feature branches A and B used to ignore
.o files, and now B suddenly does not ignore them anymore, the only
real reason I can think of is that it was rewritten in a languages
other than C*. So then you *still* want to ignore .o files in the
integration branch.

Basically I am saying that it should be

  cat $gitignore_files | sort -u

and obviously, this is something for a specific merge driver, as
Johannes suggested.

Thanks for the feedback,

-- 
martin;              (greetings from the heart of the sun.)
  \____ echo mailto: !#^."<*>"|tr "<*> mailto:" net@madduck
 
minchinhampton (n.): the expression on a man's face when he has just
zipped up his trousers without due care and attention.
                               -- douglas adams, the meaning of liff
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: merging .gitignore

[Junio C Hamano]

martin f krafft <madduck@madduck.net> writes:

> Basically I am saying that it should be
>
>   cat $gitignore_files | sort -u
>
> and obviously, this is something for a specific merge driver, as
> Johannes suggested.

Perhaps you can use the existing union merge there.

Re: merging .gitignore

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 541 bytes --]

also sprach Junio C Hamano <gitster@pobox.com> [2007.10.02.2055 +0100]:
> Perhaps you can use the existing union merge there.

Many thanks for the pointer, I did not even have time to look at
gitattributes.txt yet...

-- 
martin;              (greetings from the heart of the sun.)
  \____ echo mailto: !#^."<*>"|tr "<*> mailto:" net@madduck
 
"self-denial is the shining sore
 on the leprous body of christianity."
                                                        -- oscar wilde
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: merging .gitignore

[Pierre Habouzit]

[-- Attachment #1: Type: text/plain, Size: 2135 bytes --]

On Tue, Oct 02, 2007 at 07:51:48PM +0000, martin f krafft wrote:
> Well, with gitignore I am ready to say that merges should be
> resolved in an additive way. Remember that I am talking about an
> intergration branch, and if feature branches A and B used to ignore
> .o files, and now B suddenly does not ignore them anymore, the only
> real reason I can think of is that it was rewritten in a languages
> other than C*. So then you *still* want to ignore .o files in the
> integration branch.
> 
> Basically I am saying that it should be
> 
>   cat $gitignore_files | sort -u

  Except that this would not work, just take that example (for the sake
of conciseness I put lines as members of a set):


  Common ancestor content: (bar, foo, quux)
  Left child: (bar, baz, foo, quux)
  Right child: (bar, quux)

  This one is a conflict, and if you apply your method, the merge always
"works" (as in has no cases where it fails) and would yield a result
like:

  (bar, baz, foo, quux) whereas it's probably (bar, baz, quux) that
would be the proper one (aka left branch added a new ignore `baz` and
the right one removed it).

  The proper way for gitignore is probably to work on the sets
operations, like diff does with lines, but without taking ordering into
account. What gets harder is when your lists are:

  Ancestor: (aa*, aaa, bbb)
  Left child: (aa*, bbb)   <-- remove aaa because aa* covers it
  Right child: (aaa, aabcd, bbb, cc*) <-- remove aa* and be explicit

  The proper result is probably: (aaa, aabcd, bbb, cc*) but is in fact a
case of conflict, because the "left" child could have used the fact that
aa* was present and hide say a aaXXX that the right child did not had,
and the merge would be wrong.

  Of course, .gitignore aren't _that_ important and if you ignore one
less file, or one too many, git will continue to behave properly, but
well, merge implementations aren't _that_ trivial.

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: merging .gitignore

[Pierre Habouzit]

[-- Attachment #1: Type: text/plain, Size: 1315 bytes --]

On mar, oct 02, 2007 at 08:13:18 +0000, Pierre Habouzit wrote:
>   Ancestor: (aa*, aaa, bbb)
>   Left child: (aa*, bbb)   <-- remove aaa because aa* covers it
>   Right child: (aaa, aabcd, bbb, cc*) <-- remove aa* and be explicit
> 
>   The proper result is probably: (aaa, aabcd, bbb, cc*) but is in fact a
> case of conflict, because the "left" child could have used the fact that
> aa* was present and hide say a aaXXX that the right child did not had,
> and the merge would be wrong.

  Okay this example blows, I believe this one is better:

       (a*)
      /    \
  (ab*)    (ac*)
      \    /
       ????

  gitignore are subsets of the set of words.  if S is the ancestor set,
S1 and S2 the left and right sets. let Δ1 and Δ2 be S1 \ S and S2 \ S
respectively.  I think there is a conflict if
  Δ1 n Δ2 != 0 and (Δ1 is not a subset of Δ2) and (Δ2 is not a subset of Δ1)

  If the condition holds, then I believe that the "merged" .gitignore
would be: (S1 u S2) \ (Δ1 u Δ2)

  Though, don't take my word for it, I've only sketched this on a small
piece of paper, and have no rigorous proof.
-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: merging .gitignore

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 644 bytes --]

also sprach Pierre Habouzit <madcoder@debian.org> [2007.10.02.2147 +0100]:
>        (a*)
>       /    \
>   (ab*)    (ac*)
>       \    /
>        ????

(a*, ab*, ac*)

-- 
 .''`.   martin f. krafft <madduck@debian.org>
: :'  :  proud Debian developer, author, administrator, and user
`. `'`   http://people.debian.org/~madduck - http://debiansystem.info
  `-  Debian - when you have better things to do than fixing systems
 
"once ... in the wilds of afghanistan, i lost my corkscrew, and we
 were forced to live on nothing but food and water for days."
                                -- w. c. fields, "my little chickadee"

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: merging .gitignore

[Pierre Habouzit]

[-- Attachment #1: Type: text/plain, Size: 858 bytes --]

On Tue, Oct 02, 2007 at 08:56:18PM +0000, martin f krafft wrote:
> also sprach Pierre Habouzit <madcoder@debian.org> [2007.10.02.2147 +0100]:
> >        (a*)
> >       /    \
> >   (ab*)    (ac*)
> >       \    /
> >        ????
> 
> (a*, ab*, ac*)

  Definitely not. a* -> ab* is making a?* unignored for any value of ?
except b. So adding a* is definitely invalid.

  So your final merge is definitely invalid. I'd say that it's likely
that the final merge is (ab*, ac*) but on the left branch the ab* could
have been chosen because the programmer added a thing named ac.c, so the
merge would still have (probably little but still some) chances to be invalid.

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: merging .gitignore

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 904 bytes --]

also sprach Pierre Habouzit <madcoder@debian.org> [2007.10.02.2207 +0100]:
> > >        (a*)
> > >       /    \
> > >   (ab*)    (ac*)
> > >       \    /
> > >        ????
> > 
> > (a*, ab*, ac*)
> 
>   Definitely not. a* -> ab* is making a?* unignored for any value of ?
> except b. So adding a* is definitely invalid.

In left, ab* is still ignored, in right ac* is still ignored, and in
the integration branch, they're all ignored. We don't merge up in
this model...

-- 
 .''`.   martin f. krafft <madduck@debian.org>
: :'  :  proud Debian developer, author, administrator, and user
`. `'`   http://people.debian.org/~madduck - http://debiansystem.info
  `-  Debian - when you have better things to do than fixing systems
 
"if there's anything more important than my ego,
 i want it caught and shot now."
                                                -- zaphod beeblebrox

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: merging .gitignore

[Pierre Habouzit]

[-- Attachment #1: Type: text/plain, Size: 966 bytes --]

On Tue, Oct 02, 2007 at 09:49:19PM +0000, martin f krafft wrote:
> also sprach Pierre Habouzit <madcoder@debian.org> [2007.10.02.2207 +0100]:
> > > >        (a*)
> > > >       /    \
> > > >   (ab*)    (ac*)
> > > >       \    /
> > > >        ????
> > > 
> > > (a*, ab*, ac*)
> > 
> >   Definitely not. a* -> ab* is making a?* unignored for any value of ?
> > except b. So adding a* is definitely invalid.
> 
> In left, ab* is still ignored, in right ac* is still ignored, and in
> the integration branch, they're all ignored. We don't merge up in
> this model...

  err maybe you didn't get my little picture

     (a*)
    /    \
   v      v
(ab*)    (ac*)
    \    /
     v  v
     ????

  This is a perfectly sensible history. Or I miss sth on your end.

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: merging .gitignore

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 670 bytes --]

also sprach Pierre Habouzit <madcoder@debian.org> [2007.10.02.2307 +0100]:
>      (a*)
>     /    \
>    v      v
> (ab*)    (ac*)
>     \    /
>      v  v
>      ????
> 
>   This is a perfectly sensible history. Or I miss sth on your end.

So these are revs, not branches?

-- 
 .''`.   martin f. krafft <madduck@debian.org>
: :'  :  proud Debian developer, author, administrator, and user
`. `'`   http://people.debian.org/~madduck - http://debiansystem.info
  `-  Debian - when you have better things to do than fixing systems
 
seen on an advertising for an elaborate swiss men's watch:
  "almost as complicated as a woman. except it's on time"

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: merging .gitignore

[Pierre Habouzit]

[-- Attachment #1: Type: text/plain, Size: 878 bytes --]

On Wed, Oct 03, 2007 at 08:42:59AM +0000, martin f krafft wrote:
> also sprach Pierre Habouzit <madcoder@debian.org> [2007.10.02.2307 +0100]:
> >      (a*)
> >     /    \
> >    v      v
> > (ab*)    (ac*)
> >     \    /
> >      v  v
> >      ????
> > 
> >   This is a perfectly sensible history. Or I miss sth on your end.
> 
> So these are revs, not branches?

  Yes, those are the contents of the .gitignores on refs, on top is the
common ancestor, and you have two branches that you want to merge into
one. I pretend that if in one branch the content of the .gitignore was
a* and becomes ab* and than on the other it was a* and became ac*, then
you cannot know how to merge.

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: merging .gitignore

[Pierre Habouzit]

[-- Attachment #1: Type: text/plain, Size: 2290 bytes --]

On Tue, Oct 02, 2007 at 08:47:48PM +0000, Pierre Habouzit wrote:
> On mar, oct 02, 2007 at 08:13:18 +0000, Pierre Habouzit wrote:
> >   Ancestor: (aa*, aaa, bbb)
> >   Left child: (aa*, bbb)   <-- remove aaa because aa* covers it
> >   Right child: (aaa, aabcd, bbb, cc*) <-- remove aa* and be explicit
> > 
> >   The proper result is probably: (aaa, aabcd, bbb, cc*) but is in fact a
> > case of conflict, because the "left" child could have used the fact that
> > aa* was present and hide say a aaXXX that the right child did not had,
> > and the merge would be wrong.
> 
>   Okay this example blows, I believe this one is better:
> 
>        (a*)
>       /    \
>   (ab*)    (ac*)
>       \    /
>        ????
> 
>   gitignore are subsets of the set of words.  if S is the ancestor set,
> S1 and S2 the left and right sets. let Δ1 and Δ2 be S1 \ S and S2 \ S
   I meant S \ S1 and S \ S2 in fact here ...
> respectively.  I think there is a conflict if
>   Δ1 n Δ2 != 0 and (Δ1 is not a subset of Δ2) and (Δ2 is not a subset of Δ1)
> 
>   If the condition holds, then I believe that the "merged" .gitignore
> would be: (S1 u S2) \ (Δ1 u Δ2)

  after some more thoughts, as basically merging the complementary of
the sets I talk about here should yield the same "conflicts" (as it's
the dual problem), I suppose that the same restrictions should be
checked wrt the "added" deltas between S -> S1 (aka S1 \ S for real this
time) and S -> S2 (aka S2 \ S).

  so if Δ(0,n) is Sn \ S and Δ(n, 0) is S \ Sn, it would mean that if:

   { Δ(0,1) n Δ(0,2) == 0 || ∃ i ∋ (1,2), Δ(0,i) ⊆ Δ(0, 3 - i) }
&& { Δ(1,0) n Δ(2,0) == 0 || ∃ i ∋ (1,2), Δ(i,0) ⊆ Δ(3 - i, 0) }

  Then the correct merge (without conflicts) would be:

  (S u Δ(1,0) u Δ(2,0)) \ (Δ(0,1) u Δ(0,2)) aka S + what was added -
what was removed.

  in fact, I think that wrt the sets usual operations, there is a
conflict if the expression I just wrote does not commutes wrt the \ or
sth very similar.

  Anyway, I'm going pretty off topic here, so I'll shut up now :)

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: merging .gitignore

[Andy Parkins]

On Tuesday 2007 October 02, martin f krafft wrote:

> Well, with gitignore I am ready to say that merges should be
> resolved in an additive way. Remember that I am talking about an
> intergration branch, and if feature branches A and B used to ignore
> .o files, and now B suddenly does not ignore them anymore, the only

Okay; *.o was obviously not a good example.  A more detailed one: how about a 
change like this to a makefile (excuse bastardised diff format)

diff Makefile
-include depends.make
+include depends.mak

diff .gitignore
-depends.make
+depends.mak

>   cat $gitignore_files | sort -u

Now, say there is another branch that makes exactly this change but 
chooses "depends.inc" as the filename.  Your "additive only" merge 
of .gitignore will not flag the conflict and will leave a .gitignore with

depends.mak
depends.inc

The makefile conflict will have been resolved one way or the other but the 
gitignore conflict will not.  While it's not a serious fault it is wrong, and 
no one was signalled that it was wrong.

I am still having difficult seeing why you want to hide conflicts 
in .gitignore.  It's just as possible to get and resolve conflicts in 
gitignore as in any other file.


Andy
-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com

Re: merging .gitignore

[Johan Herland]

On Wednesday 03 October 2007, Andy Parkins wrote:
> I am still having difficult seeing why you want to hide conflicts 
> in .gitignore.  It's just as possible to get and resolve conflicts in 
> gitignore as in any other file.

Agreed. What about the following:

- No special merge rules for .gitignore

- Teach the .gitignore parser to ignore conflict markers (i.e. regard them 
as comments)


This way, the user will have to merge .gitignore like any other file, but if 
for some reason, the user is not able to do so (say, git needs to 
consult .gitignore before the user has had a chance to resolve the 
conflict), the fallback result will be similar to a union merge, which 
shouldn't be extremely harmful in .gitignore's case.

I do not think we really want to create an auto-merge algorithm 
for .gitignore, and then depend on this auto-merge algorithm to _always_ 
succeed with the _correct_ result and _no_ conflicts. These algorithms tend 
to be extremely tricky to get right, especially for the "always correct" 
requirement.

<rant>
Mercurial had a similar problem some months ago. They have their tag 
definitions stored in a versioned file in the working tree (.hgtags IIRC). 
But the repo tag state (i.e. Mercurial's opinion at any time as to _which_ 
tags are defined and _where_ they point) is not deduced from the copy in 
your current working tree at all. (That would of course limit you to only 
ba able to refer to tags defined earlier on the particular branch you're 
currently on.) Instead the repo tag state was found by consulting 
the "head-most" copy of the .hgtags file (for some definition 
of "head-most" including non-obvious things like which branch has the most 
recent commit, etc). The end result was that you could get some _really_ 
interesting behaviour depending on the order in which you commited totally 
unrelated changes to two different branches which happened to have 
different .hgtags files. (E.g.: Given two branches A, B, and  - in .hgtags 
on branch A - tag Foo is defined to point at rev X, and - in .hgtags on 
branch B - Foo points at rev Y. Now, whether you got rev X or rev Y when 
you checked out Foo, depended on which of branch A or branch B had the most 
recent (totally unrelated, i.e. not even touching .hgtags) commit.)

I (and others) pointed out this bug on their ML, and instead of fixing the 
braindeadness of allowing branched tag definitions in the repo in the first 
place, they set about trying to create an auto-merge algorithm for deducing 
the repo tag state from the various versions/branches of .hgtags found in 
the repo. I didn't stick around for long enough to see how well this 
auto-merge algorithm works (the misdesign of tags in Mercurial was one of 
the reasons I started looking at git), but I would be surprised if 
Mercurial today has a simple and straightforward way of deducing the repo 
tag state that _always_ gives _correct_ (i.e. unsurprising) results, even 
in the corner cases.
</rant>


Have fun! :)


...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: merging .gitignore

[Johannes Schindelin]

Hi,

On Wed, 3 Oct 2007, Johan Herland wrote:

> - Teach the .gitignore parser to ignore conflict markers (i.e. regard them 
> as comments)

You might be delighted to know that in practice, it works already (because 
you usually do not have a file named "<<<<<< blablub" or "======" or 
">>>>>> blablub"...

Ciao,
Dscho

Re: merging .gitignore

[Johan Herland]

On Wednesday 03 October 2007, Johannes Schindelin wrote:
> On Wed, 3 Oct 2007, Johan Herland wrote:
> > - Teach the .gitignore parser to ignore conflict markers (i.e. regard them 
> > as comments)
> 
> You might be delighted to know that in practice, it works already (because 
> you usually do not have a file named "<<<<<< blablub" or "======" or 
> ">>>>>> blablub"...

I suspected so... ;)


...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: merging .gitignore

[Junio C Hamano]

Johan Herland <johan@herland.net> writes:

> On Wednesday 03 October 2007, Johannes Schindelin wrote:
>> On Wed, 3 Oct 2007, Johan Herland wrote:
>> > - Teach the .gitignore parser to ignore conflict markers (i.e. regard them 
>> > as comments)
>> 
>> You might be delighted to know that in practice, it works already (because 
>> you usually do not have a file named "<<<<<< blablub" or "======" or 
>> ">>>>>> blablub"...
>
> I suspected so... ;)

Yeah, and that is one of the reasons why we made gitignore and
gitattributes one-entry-per-line format.

Re: merging .gitignore

[Johannes Schindelin]

Hi,

On Mon, 1 Oct 2007, martin f krafft wrote:

> Feature branch A has a .gitignore file, and it's been merged into
> master for a while. Today, feature branch B failed to merge into
> master because it also provides a .gitignore file.

You might be interested in writing a merge driver.  See 
Documentation/gitattributes.txt.

Hth,
Dscho