libreoffice merge(tool?) issue #3 ...

libreoffice merge(tool?) issue #3 ...

[Michael Meeks]

Hi there,

        So - yet again, I'm still a completely clueless git user :-)
basically the same setup and reproduction issue as last time, still
using a stable git: version 1.7.3.4

Setup:
        git clone git://anongit.freedesktop.org/libreoffice/libs-core
        git checkout integration/dev300_m98
        git remote add stage git://anongit.freedesktop.org/libreoffice/staging/libs-core
        git fetch stage

Reproduce:
	rm -Rf *
	git reset --hard # ie. totally clean tree.
        git merge stage/dev300

	I get this output from the merge:

error: refusing to lose untracked file at 'ucb/source/ucp/ext/makefile.mk'

	Then when I run:

$ git mergetool ucb/source/ucp/ext/makefile.mk
..

Deleted merge conflict for 'ucb/source/ucp/ext/makefile.mk':
  {local}: created
  {remote}: deleted
Use (c)reated or (d)eleted file, or (a)bort?

	It seems to suggest that the file is deleted somewhere (in the branch I
am trying to merge in) it seems.

	Interestingly, though - the file is present in both
integration/dev300_m98:

	git log -n1 ucb/source/ucp/ext/makefile.mk | tee
commit 80b61d9c6762b4f195edd1246b903b11ad3f2252
Author: Thomas Arnhold <thomas@arnhold.org>
Date:   Fri Jan 21 14:11:55 2011 +0100

    Remove old RCS lines.

	And also present in the branch I'm merging: stage/dev300:

commit 0c87cb97cf3790fa98bcbb0eef9d174140a4e847
Author: sb <sb@openoffice.org>
Date:   Fri Sep 10 13:10:07 2010 +0200

    sb129: #i113189# change UNO components to use passive registration


	Which makes me wonder - why the deleted / untracked file message ?
probably something obvious, but I found it rather confusing, and again
I've seen a number of examples of this.

	Thanks,

		Michael.

PS. of course, perhaps this is 'just me' - for space / time /
simplicty / certainty reasons, I do a lot of "cp -lR foo/.git baa/" to
duplicate trees - but AFAIK all git operations are atomic and use
renames rather than in-place re-writing: right ?
-- 
 michael.meeks@novell.com  <><, Pseudo Engineer, itinerant idiot

Re: libreoffice merge(tool?) issue #3 ...

[Brian Gernhardt]

On Feb 22, 2011, at 10:34 AM, Michael Meeks wrote:

> PS. of course, perhaps this is 'just me' - for space / time /
> simplicty / certainty reasons, I do a lot of "cp -lR foo/.git baa/" to
> duplicate trees - but AFAIK all git operations are atomic and use
> renames rather than in-place re-writing: right ?

FYI: `git clone foo bar` will use hard-links to copy the object files and is both very fast and space efficient.  (See the description of `--local` in git-clone(1), which is used by default for local repositories since git 1.5.3.)  It's also guaranteed to work while the correctness of `cp -lR` depends on implementation details of git.

~~ Brian

Re: libreoffice merge(tool?) issue #3 ... (bogus)

[Michael Meeks]

Hi Brian,

	First - it seems that the issue here was entirely bogus, not least
because we had a bug with re-writing these makefiles as we checked them
in; so hopefully only 2 issues pending ;-)

	Anyhow - I tried your kind advice:

On Tue, 2011-02-22 at 10:55 -0500, Brian Gernhardt wrote:
> FYI: `git clone foo bar` will use hard-links to copy the object
> files and is both very fast and space efficient.  (See the
> description of `--local` in git-clone(1), which is used by
> default for local repositories since git 1.5.3.)  It's also
> guaranteed to work while the correctness of `cp -lR` depends
> on implementation details of git.

	Sounds like just what I need. Unfortunately, it didn't clone some of
the pieces I needed; eg. other configured remotes, I ended up with just
'origin' - which was unexpected (and less wonderful than cp -lR ;-).

	Is that a feature ?

	Thanks,

		Michael.

-- 
 michael.meeks@novell.com  <><, Pseudo Engineer, itinerant idiot

Re: libreoffice merge(tool?) issue #3 ... (bogus)

[Michael J Gruber]

Michael Meeks venit, vidit, dixit 24.02.2011 17:39:
> Hi Brian,
> 
> 	First - it seems that the issue here was entirely bogus, not least
> because we had a bug with re-writing these makefiles as we checked them
> in; so hopefully only 2 issues pending ;-)
> 
> 	Anyhow - I tried your kind advice:
> 
> On Tue, 2011-02-22 at 10:55 -0500, Brian Gernhardt wrote:
>> FYI: `git clone foo bar` will use hard-links to copy the object
>> files and is both very fast and space efficient.  (See the
>> description of `--local` in git-clone(1), which is used by
>> default for local repositories since git 1.5.3.)  It's also
>> guaranteed to work while the correctness of `cp -lR` depends
>> on implementation details of git.
> 
> 	Sounds like just what I need. Unfortunately, it didn't clone some of
> the pieces I needed; eg. other configured remotes, I ended up with just
> 'origin' - which was unexpected (and less wonderful than cp -lR ;-).
> 
> 	Is that a feature ?

Yes, because by cloning someone else's config they could make you do
what they want (alias...).

I think in your case you can just copy over the .git/config and maybe
set up "alternates" so that you don't have to refetch the remote objects
which are not referenced by local refs. (Alternatively, clone --mirror,
then copy over config and turn into non bare.)

Maybe we do need "clone --copy" or something as a safe version of "cp -al"?

Michael

Re: libreoffice merge(tool?) issue #3 ... (bogus)

[Andres Freund]

Hi,

On Thursday 24 February 2011 17:39:20 Michael Meeks wrote:
> 	Anyhow - I tried your kind advice:
> 
> On Tue, 2011-02-22 at 10:55 -0500, Brian Gernhardt wrote:
> > FYI: `git clone foo bar` will use hard-links to copy the object
> > files and is both very fast and space efficient.  (See the
> > description of `--local` in git-clone(1), which is used by
> > default for local repositories since git 1.5.3.)  It's also
> > guaranteed to work while the correctness of `cp -lR` depends
> > on implementation details of git.
> 
> 	Sounds like just what I need. Unfortunately, it didn't clone some of
> the pieces I needed; eg. other configured remotes, I ended up with just
> 'origin' - which was unexpected (and less wonderful than cp -lR ;-).
See the --mirror option for clone

Per default only local refs get copied over.

Andres

copying git repositories ...

[Michael Meeks]

Hi Andres & Brian,

	Thanks for your help;

On Fri, 2011-02-25 at 21:55 +0100, Andres Freund wrote:
> > On Tue, 2011-02-22 at 10:55 -0500, Brian Gernhardt wrote:
> > > FYI: `git clone foo bar` will use hard-links to copy the object
> > > files and is both very fast and space efficient.  (See the
> > > description of `--local` in git-clone(1), which is used by
> > > default for local repositories since git 1.5.3.)  It's also
> > > guaranteed to work while the correctness of `cp -lR` depends
> > > on implementation details of git.
..
> > 	Sounds like just what I need. Unfortunately, it didn't clone some of
> > the pieces I needed; eg. other configured remotes, I ended up with just
> > 'origin' - which was unexpected (and less wonderful than cp -lR ;-).
..
> See the --mirror option for clone

	I tried all of this; none of it did what I had hoped :-)

	git clone src dest

	yields a different repository, with a totally different config /
remotes setup - missing all but a new synthetic origin pointing at the
local files and which can't be git pulled from in the normal way; ie. it
behaves extremely differently to the cp -lR result.

	git clone --mirror src dest

	yields a similar problem-repository, which is for extra measure a bare
checkout, and thus also not what I need.

	I'm really looking for an equivalent of 'cp -lR foo baa' that:

	* uses hard links to save space
	* produces precisely-a-duplicate repository

	For #2 - I would use the verb 'clone' except of course the 'clone' I'm
talking about would be one that is identical, the same, with no
differences (not eg. missing a few limbs ;-) 

	Clearly cp -lR is bad & evil and all that; but it yields exactly what I
need to effectively manage my local trees, multiple checkouts, and
different builds without burning the entire disk.

	Is there a blessed 'cp -lR' wrapper for git that is functionally
identical ? [ and I'm happy of course for some slow divergence, and loss
of efficiency as I pull more changes from time to time into each tree ].

	Sorry for the noise,

		Michael.

-- 
 michael.meeks@novell.com  <><, Pseudo Engineer, itinerant idiot

Re: copying git repositories ...

[Andres Freund]

Hi Michael,

On Monday, February 28, 2011 03:25:02 PM Michael Meeks wrote:
> Hi Andres & Brian,
> 	For #2 - I would use the verb 'clone' except of course the 'clone' I'm
> talking about would be one that is identical, the same, with no
> differences (not eg. missing a few limbs ;-)
> 
> 	Clearly cp -lR is bad & evil and all that; but it yields exactly what I
> need to effectively manage my local trees, multiple checkouts, and
> different builds without burning the entire disk.
> 
> 	Is there a blessed 'cp -lR' wrapper for git that is functionally
> identical ? [ and I'm happy of course for some slow divergence, and loss
> of efficiency as I pull more changes from time to time into each tree ].
What about git clone --reference=oldrepo ssh://upstream/ ?

Andres

Re: copying git repositories ...

[Brian Gernhardt]

On Feb 28, 2011, at 9:25 AM, Michael Meeks wrote:

> 	I'm really looking for an equivalent of 'cp -lR foo baa' that:
> 
> 	* uses hard links to save space
> 	* produces precisely-a-duplicate repository

I mostly handle this sort of thing by branch switching in the same checkout, which I assume you're trying not to do because of recompilation time.  Your way, eventually the repositories in question would start to diverge and you'd have to keep them in sync manually, which sounds far less than ideal.  Eventual repacks would break the hard links and you'd even lose the space savings.

What you might be interested in is the git-new-workdir script in git.git/contrib/workdir.  It uses symbolic links to create a new working directory backed by the exact same object store and config as the original.


This is not a situation git is well designed for, honestly.  The more "git-like" work flow would be to maintain a single central, probably bare, repository on your machine that pulls from everything that all your local repositories need.  Then set up the other repositories to get all the references from the central repo.  Then there's only one that needs to be updated from the remotes.  If you use the --reference option to git-clone, you're not even duplicating the object store and each clone only has the objects it needs.

~~ Brian

Re: libreoffice merge(tool?) issue #3 ...

[Michael J Gruber]

Michael Meeks venit, vidit, dixit 22.02.2011 16:34:
> Hi there,
> 
>         So - yet again, I'm still a completely clueless git user :-)
> basically the same setup and reproduction issue as last time, still
> using a stable git: version 1.7.3.4
<PG>
I'm sorry you're having reproduction issues. At least your git is stable.
</PG>
> 
> Setup:
>         git clone git://anongit.freedesktop.org/libreoffice/libs-core
cd libs-core # I assume
>         git checkout integration/dev300_m98
>         git remote add stage git://anongit.freedesktop.org/libreoffice/staging/libs-core
>         git fetch stage
> 
> Reproduce:
> 	rm -Rf *
> 	git reset --hard # ie. totally clean tree.
# Those two should be no-ops after following the above.
>         git merge stage/dev300
Is that stage/ooo/dev300?
> 
> 	I get this output from the merge:

I get thousands of conflicts. Have the branches moved since your post?
It may be better to give us sha1 or stable tags.

Are you doing any builds before merging?

> PS. of course, perhaps this is 'just me' - for space / time /
> simplicty / certainty reasons, I do a lot of "cp -lR foo/.git baa/" to
> duplicate trees - but AFAIK all git operations are atomic and use
> renames rather than in-place re-writing: right ?

Ouch. See Brian's post :)

Michael

Re: libreoffice merge(tool?) issue #3 ...

[Michael Meeks]

Hi Michael,

On Tue, 2011-02-22 at 18:30 +0100, Michael J Gruber wrote:
> I get thousands of conflicts. Have the branches moved since your post?
> It may be better to give us sha1 or stable tags.

	Nope; there are thousands of conflicts; a subset of these are (I would
like to think ;-) erroneous; but I'm picking out individual files with
problems that I can repeat easily and that have (I hope) simple history
to try to isolate the problems for you.

> Are you doing any builds before merging?

	Builds of what ? git - no; LibreOffice - sure, it builds before the
merge - and then there is a huge slew of work to do to make it build
afterwards ;-) but then that is not so suprising.

	Anyhow - thanks for looking at it; can you replicate the suprising
result in that file: ucb/source/ucp/ext/makefile.mk ? what does
'refusing to loose untracked file' mean in that context ?

	Thanks,

		Michael.

-- 
 michael.meeks@novell.com  <><, Pseudo Engineer, itinerant idiot

Re: libreoffice merge(tool?) issue #3 ...

[Michael J Gruber]

Michael Meeks venit, vidit, dixit 22.02.2011 19:14:
> Hi Michael,
> 
> On Tue, 2011-02-22 at 18:30 +0100, Michael J Gruber wrote:
>> I get thousands of conflicts. Have the branches moved since your post?
>> It may be better to give us sha1 or stable tags.
> 
> 	Nope; there are thousands of conflicts; a subset of these are (I would
> like to think ;-) erroneous; but I'm picking out individual files with
> problems that I can repeat easily and that have (I hope) simple history
> to try to isolate the problems for you.
> 
>> Are you doing any builds before merging?
> 
> 	Builds of what ? git - no; LibreOffice - sure, it builds before the
> merge - and then there is a huge slew of work to do to make it build
> afterwards ;-) but then that is not so suprising.

Builds of LO, naturally. The point is that possibly one branch tracks
some build products that the other doesn't track - e.g., autogenerated
make or autoconf stuff etc. (This happens easily when you change your
build chain.) That would lead to a message like that:

> 	Anyhow - thanks for looking at it; can you replicate the suprising
> result in that file: ucb/source/ucp/ext/makefile.mk ? what does
> 'refusing to loose untracked file' mean in that context ?

Say, you build on branch A, that generates an untracked file foo, but
branch B (which you want to merge) tracks that. Merges refuses to
overwrite foo (even though A does not have foo) so that you don't loose
the contents of the untracked file.

But I'm really wondering whether we're merging the same revs. As I
mentioned, I don't see that branch in "stage" that you're merging, only
"stage/ooo/dev300", see below. Also, I'm getting

AA ucb/source/ucp/ext/makefile.mk

which has a somewhat surprising markup (the left side introduces no
change, the right side does - should have a trivial resolution), without
building LO before.

Note that you're merging branches which are way off,

git rev-list --count --left-right
origin/integration/dev300_m98...stage/ooo/dev300
3566    3126

and that the merge base is quite old:

af61642 (#i105937# Fixed a few remaining gradient glitches, 2010-01-16)

The latter explains many of the problems (and the "surprising" above):
compared to the merge base, both branches add
ucb/source/ucp/ext/makefile.mk as a *new* file with different contents,
so that the conflict can't be resolved automatically, and that's how it
is marked up.

Is that merge really what you're after?

Michael

Branches with your recipe:

* integration/dev300_m98
  master
  remotes/origin/HEAD -> origin/master
  remotes/origin/feature/bootstrap-build
  remotes/origin/feature/currency-64bit
  remotes/origin/feature/gnumake2.1
  remotes/origin/feature/helppack
  remotes/origin/feature/layout
  remotes/origin/feature/pptx-export-ooxml11
  remotes/origin/feature/rodatastrings
  remotes/origin/feature/sqlite
  remotes/origin/feature/winshrink
  remotes/origin/integration/dev300_m98
  remotes/origin/libreoffice-3-3
  remotes/origin/libreoffice-3-3-0
  remotes/origin/libreoffice-3-3-1
  remotes/origin/master
  remotes/stage/ooo/dev300
  remotes/stage/ooo/dev300_m100
  remotes/stage/premerge/dev300_m98


Michael

Re: libreoffice merge(tool?) issue #3 ...

[Michael J Gruber]

Michael J Gruber venit, vidit, dixit 23.02.2011 12:18:
> Note that you're merging branches which are way off,
> 
> git rev-list --count --left-right
> origin/integration/dev300_m98...stage/ooo/dev300
> 3566    3126
> 
> and that the merge base is quite old:
> 
> af61642 (#i105937# Fixed a few remaining gradient glitches, 2010-01-16)

Following up on this:

git rev-list --count --left-right
origin/integration/dev300_m98...stage/ooo/dev300
3566    3126

git rev-list --count --left-right --no-merges
origin/integration/dev300_m98...stage/ooo/dev300
2794    2180

git rev-list --count --left-right --no-merges --cherry-pick
origin/integration/dev300_m98...stage/ooo/dev300
1136    528

That is, 2500 of these different commits are patch-equivalent
(cherry-picks). I don't think "merge" is the best tool to combine "fake"
branches like those. You may be better off with rebase... Although I'm
wondering what the branch policy was that lead to this.

Michael