* Re: several quick questions
From: Junio C Hamano @ 2006-02-15 9:21 UTC (permalink / raw)
To: Andreas Ericsson; +Cc: git
In-Reply-To: <43F2EF09.5060603@op5.se>
Andreas Ericsson <ae@op5.se> writes:
> The good thing about being past 1.0 in a project is that it's
> feature-complete, or close to. The bad thing is that bloat usually
> starts to happen around 1.1.
Thanks -- be kind and stop me whenever somebody else tempts me
to add excess things to the core, please. Always good to have
adult supervision ;-), eh, voice of sanity.
^ permalink raw reply
* [PATCH] Detect misspelled pathspec to git-add
From: Junio C Hamano @ 2006-02-15 9:14 UTC (permalink / raw)
To: git
This is in the same spirit as an earlier patch for git-commit.
It does an extra ls-files to avoid complaining when a fully
tracked directory name is given on the command line (otherwise
--others restriction would say the pathspec does not match).
Signed-off-by: Junio C Hamano <junkio@cox.net>
---
* I noticed this when I accidentally said "git add /a/b/c"
when I meant "git add a/b/c", and I did not notice it until
I said "git commit" which said "Nothing to commit."
It does one extra ls-files only if any pathspec is given; I
am hoping this slight performance hit is tolerated. I think
people who does "git add ." all the time to mean "add
everything I added anywhere since I did git add last time"
deserve it ;-).
Comments?
git-add.sh | 11 +++++++++++
1 files changed, 11 insertions(+), 0 deletions(-)
ca28a7e03232b9a3b66df8519d9ee827882c83c1
diff --git a/git-add.sh b/git-add.sh
index f719b4b..13fad82 100755
--- a/git-add.sh
+++ b/git-add.sh
@@ -24,6 +24,17 @@ while : ; do
shift
done
+# Check misspelled pathspec
+case "$#" in
+0) ;;
+*)
+ git-ls-files --error-unmatch --others --cached -- "$@" >/dev/null || {
+ echo >&2 "Maybe you misspelled it?"
+ exit 1
+ }
+ ;;
+esac
+
if test -f "$GIT_DIR/info/exclude"
then
git-ls-files -z \
--
1.2.0.gcfba7
^ permalink raw reply related
* Re: several quick questions
From: Andreas Ericsson @ 2006-02-15 9:06 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vwtfxm917.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano wrote:
> Andreas Ericsson <ae@op5.se> writes:
>
>
>>Junio C Hamano wrote:
>>
>>
>>>Now, read-only ref does not interest me, but "do not commit on
>>>top of this yourself, only fast-forward from somewhere else is
>>>allowed" may be useful, for the reason why you mentioned
>>>"origin".
>>
>>Do my suggestion and you wouldn't have to worry about read-only
>>branches, and although merging any changes from it might be more
>>trouble than its worth, it might be possible to cherry-pick the commit
>>rather than reverting and re-applying it.
>
>
> Sorry, is this "do my suggestion" a solution to my "do not
> commit on top of this yourself, only fast-forward from somewhere
> else is allowed -- e.g. to protect 'origin'" issue, or is it
> something completely different?
>
The "git checkout -b foo HEAD~15", which was already supported, although
I missed that. All programmers have names to use just for throwaway
variables (never heard "frotz" and "nitfol" before though), so adding
the burden of selecting a name for the throw-away search branch
shouldn't be too hard on them. Then it would be possible to commit to
it, and merge or cherry-pick from it later. It's usually preferrable to
amend to a broken patch than to revert it completely.
In essence, I claim that git-seek is superfluous and inferior to "git
checkout -b foo <commit-ish>" and shouldn't be implemented. If anyone
wants to distribute the source to non-scm people as per a given point in
the history I think "git tar-tree" works marvelously as is.
The good thing about being past 1.0 in a project is that it's
feature-complete, or close to. The bad thing is that bloat usually
starts to happen around 1.1.
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
^ permalink raw reply
* Re: StGIT refreshes all added files - limitation of git-write-tree?
From: Pavel Roskin @ 2006-02-15 9:06 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vacctnnx4.fsf@assigned-by-dhcp.cox.net>
On Tue, 2006-02-14 at 22:20 -0800, Junio C Hamano wrote:
> Pavel Roskin <proski@gnu.org> writes:
>
> > Or maybe git-write-tree and other utilities could be changed to work on
> > a copy of the index file? I would prefer not to move the
> > actual .git/index away, but to make a copy for the current "stg refresh"
> > operation.
>
> There is no need to change the core side.
>
> GIT_INDEX_FILE=temporary-index git-write-tree
>
> would do the job. See the current round of git-commit and how
> it handles "git commit --only these files" case.
Thank you! It's comforting to know that the issue is not unique to
StGIT.
--
Regards,
Pavel Roskin
^ permalink raw reply
* [FYI] pack idx format
From: Junio C Hamano @ 2006-02-15 8:39 UTC (permalink / raw)
To: git
This is still WIP but if anybody is interested... Once done, it
should become Documentation/technical/pack-format.txt.
The reason I started doing this is to prototype this one:
<7v4q3453qu.fsf@assigned-by-dhcp.cox.net>
-- >8 --
Idx file:
The idx file is to map object name SHA1 to offset into the
corresponding pack file. There is the 'first-level fan-out'
table at the beginning, and then the main part of the index
follows. This is a table whose entries are sorted by their
object name SHA1. The file ends with some trailer information.
The main part is a table of 24-byte entries, and each entry is:
offset : 4-byte network byte order integer.
SHA1 : 20-byte object name SHA1.
The data for the named object begins at byte offset "offset" in
the corresponding pack file.
Before this main table, at the beginning of the idx file, there
is a table of 256 4-byte network byte order integers. This is
called "first-level fan-out". N-th entry of this table records
the offset into the main index for the first object whose object
name SHA1 starts with N+1. fanout[255] points at the end of
main index. The offset is expressed in 24-bytes unit.
Example:
idx
+--------------------------------+
| fanout[0] = 2 |-.
+--------------------------------+ |
| fanout[1] | |
+--------------------------------+ |
| fanout[2] | |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| fanout[255] | |
+--------------------------------+ |
main | offset | |
index | object name 00XXXXXXXXXXXXXXXX | |
table +--------------------------------+ |
| offset | |
| object name 00XXXXXXXXXXXXXXXX | |
+--------------------------------+ |
.-| offset |<+
| | object name 01XXXXXXXXXXXXXXXX |
| +--------------------------------+
| | offset |
| | object name 01XXXXXXXXXXXXXXXX |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| | offset |
| | object name FFXXXXXXXXXXXXXXXX |
| +--------------------------------+
trailer | | packfile checksum |
| +--------------------------------+
| | idxfile checksum |
| +--------------------------------+
.-------.
|
Pack file entry: <+
packed object header:
1-byte type (upper 4-bit)
size0 (lower 4-bit)
n-byte sizeN (as long as MSB is set, each 7-bit)
size0..sizeN form 4+7+7+..+7 bit integer, size0
is the most significant part.
packed object data:
If it is not DELTA, then deflated bytes (the size above
is the size before compression).
If it is DELTA, then
20-byte base object name SHA1 (the size above is the
size of the delta data that follows).
delta data, deflated.
^ permalink raw reply
* Re: several quick questions
From: Carl Worth @ 2006-02-15 8:21 UTC (permalink / raw)
To: Keith Packard
Cc: Martin Langhoff, Linus Torvalds, Nicolas Vilz 'niv', git
In-Reply-To: <1139981145.4341.137.camel@evo.keithp.com>
[-- Attachment #1.1: Type: text/plain, Size: 2040 bytes --]
On Tue, 14 Feb 2006 21:25:45 -0800, Keith Packard wrote:
> On Wed, 2006-02-15 at 17:11 +1300, Martin Langhoff wrote:
>
> > Did that lead to finding any problems with the import? Can I get my
> > hands on that script you've written to run the comparison?
>
> The only issues we had were with manual changes to the repository;
For anyone interested, here are the problems our script found after
importing cairo with git-cvsimport:
1) Some "future" files existed at old tags since we had copied ,v
files to preserve per-file history. This one was no surprise.
2) We had a couple tags in CVS that didn't tag the current head,
(instead a file had been manually reverted before the tag). In the
git checkout of the same tag name, we got the results as if HEAD
had been tagged.
Those two weren't too surprising. We remembered quite clearly what had
happened as soon as we saw the results. I've fixed these up
post-import by making git commits to fix the problems and then moving
the tags.
3) There are some sub-tree tags in cairo's CVS tree as
well. Obviously, those aren't very interesting for direct
comparison.
Also not surprising. For these, I just modified the script to
whitelist the tags I actually cared about checking.
4) There was a branch that diverged from the main line two commits
"late" in the git history.
I'm not sure what caused this, but it's obviously happening in the
cvsps output. I fixed the problem by capturing the cvsps output into a
file, reordering the branching patchset up two positions in the list,
then feeding the result into git-cvsimport with its -P option.
I've attached the script I used to do the git and cvs comparisons. I
was getting perfect results with a local rsync of cairo's CVS
repository. The version here does a pserver checkout instead. When I
run this I'm apparently getting different substitution of some of
those annoying RCS $Id:...$ strings. Looks like the formatting of the
date is different. So your mileage may vary.
But here it is if its of any interest.
[-- Attachment #1.2: git-cvs-compare --]
[-- Type: application/octet-stream, Size: 1976 bytes --]
#!/bin/sh
set -e
CVSROOT=:pserver:anoncvs@cairographics.org:/cvs/cairo
GITMASTER=git://git.cairographics.org/cairo
if [ ! -e cairo-cvs ]; then
echo -n "Performing CVS checkout to cairo-cvs..."
cvs -d $CVSROOT co -d cairo-cvs cairo > /dev/null
echo "done."
fi
if [ ! -e cairo-git ]; then
echo -n "Cloning git repository to cairo-git..."
git clone $GITMASTER cairo-git
echo "done."
fi
# This is what you probably want to check all tags
#for tag in $(ls cairo-git/.git/refs/tags); do
# Instead, for cairo we whitelist the tags to check since there are
# some bogus partial-tree tags that just aren't interesting.
for tag in SNAPSHOT_0_1_16 SNAPSHOT_0_1_20 SNAPSHOT_0_1_21 SNAPSHOT_0_1_22 SNAPSHOT_0_1_23 LGPL_CHANGE_BEFORE LGPL_CHANGE_AFTER SNAPSHOT_0_2_0 SNAPSHOT_0_3_0 SNAPSHOT_0_4_0 SNAPSHOT_0_5_0 SNAPSHOT_0_5_1 SNAPSHOT_0_5_2 SNAPSHOT_0_6_0 RELEASE_0_9_0 RELEASE_0_9_2 RELEASE_1_0_0 RELEASE_1_0_2; do
echo -n "Performing cvs update to $tag..."
(cd cairo-cvs; cvs -Q update -r $tag > /dev/null)
echo "done."
echo -n "Performing git checkout of $tag..."
# Linus says this is the most efficient way to do this, but it
# seems to leave some empty directories around that shouldn't be
# there.
# (cd cairo-git; git checkout -b cvs-compare >& /dev/null || true; git checkout cvs-compare; git reset --hard $tag)
# This might be slower, but it's plenty fast still and it does the right thing
(cd cairo-git; git checkout master; git branch -D cvs-compare >& /dev/null || true; git checkout -b cvs-compare $tag)
echo "done."
echo -n "Comparing cvs and git trees for $tag..."
if diff -r -x CVS -x .git cairo-cvs cairo-git >& cairo.diff; then
echo "perfect."
rm cairo.diff
else
echo "different. :("
echo " Saving trees to cairo-git-$tag & cairo-cvs-$tag and diff to cairo-$tag.diff"
cp -a cairo-git cairo-git-$tag
cp -a cairo-cvs cairo-cvs-$tag
mv cairo.diff cairo-$tag.diff
fi
done
[-- Attachment #1.3: Type: text/plain, Size: 1 bytes --]
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: "git reset --hard" leaves empty directories that shouldn't exist
From: Junio C Hamano @ 2006-02-15 8:09 UTC (permalink / raw)
To: git
In-Reply-To: <87y80dhxfd.wl%cworth@cworth.org>
Carl Worth <cworth@cworth.org> writes:
> I've been exploring the potential for git-sync, and I found some odd
> behavior with "git reset --hard". It appears that if the current tree
> has some directory structure (at least two levels deep) that does not
> exist in the tree being reset to, that empty directories are left
> around after the reset:
>
> $ git --version
> git version 1.2.0.gf6e8
> $ git init-db
> defaulting to local storage area
> $ touch file; git add file; git commit -m "Add file"
> Committing initial tree df2b8fc99e1c1d4dbc0a854d9f72157f1d6ea078
> $ git tag OLD
> $ mkdir -p a/b/c; touch a/b/c/foo; git add a/b/c/foo; git commit -m
> "Add foo"
> $ git checkout -b bogus
> $ git reset --hard OLD
> $ find a
> a
> a/b
> a/b/c
> $
>
> Is this operator error?
Git does not track directories but bends backwards to make empty
directories go away. I do not know if it is an operator error
or not, but it appears sometimes it does not bend hard enough.
Interestingly enough, this seems to do things more carefully.
$ ...
$ git commit -m 'Add file'
$ git branch OLD
$ ...
$ git commit -m 'Add a/b/c/foo'
$ git checkout -b bogo
$ git checkout OLD
^ permalink raw reply
* Re: "git reset --hard" leaves empty directories that shouldn't exist
From: Shawn Pearce @ 2006-02-15 8:06 UTC (permalink / raw)
To: Carl Worth; +Cc: git
In-Reply-To: <87y80dhxfd.wl%cworth@cworth.org>
Carl Worth <cworth@cworth.org> wrote:
> I've been exploring the potential for git-sync, and I found some odd
> behavior with "git reset --hard". It appears that if the current tree
> has some directory structure (at least two levels deep) that does not
> exist in the tree being reset to, that empty directories are left
> around after the reset:
>
> $ git --version
> git version 1.2.0.gf6e8
> $ git init-db
> defaulting to local storage area
> $ touch file; git add file; git commit -m "Add file"
> Committing initial tree df2b8fc99e1c1d4dbc0a854d9f72157f1d6ea078
> $ git tag OLD
> $ mkdir -p a/b/c; touch a/b/c/foo; git add a/b/c/foo; git commit -m
> "Add foo"
> $ git checkout -b bogus
> $ git reset --hard OLD
> $ find a
> a
> a/b
> a/b/c
> $
>
> Is this operator error? I don't see any extra options I might be
> missing in the documentation of git-reset.
Its not operator error.
I just dug though git-reset.sh in 1.2.0 and it won't cull
directories, only files. Culling the directories is a little bit
on the ugly side obviously as you must cull bottom-up. The perl
code which git-reset.sh is using to cull files definately won't
cull the directories.
No patch attached. Maybe someone not on the east coast can write
one; I need to go catch some sleep. :-)
--
Shawn.
^ permalink raw reply
* Re: bisect ends with a commit marked good and bad
From: Junio C Hamano @ 2006-02-15 8:01 UTC (permalink / raw)
To: Sanjoy Mahajan; +Cc: git
In-Reply-To: <E1F9Gwm-0001Dy-3b@approximate.corpus.cam.ac.uk>
Sanjoy Mahajan <sanjoy@mrao.cam.ac.uk> writes:
> How would I use gitk to show the those three good and three bad commits,
> and their spanning tree? Neither the man entry nor the examples of its
> use in git-bisect taught me enough to use it properly.
The way bisect narrows down bug hunting is to find a single
regression. IOW, the only question you can ask bisect is "this
older one was known to be good, but somehow this new one is bad.
If it was caused by one bug introduced somewhere during the
course of changes between the two, which one is it?"
So you always start by giving one good and one bad (or one bad
and one good, the order does not matter). That leaves * as
suspects.
o---o---O---*---*---*---*---X---o
\ /
*---*---*---*
O (good) X (bad)
Since it is finding a single regression, o commits are
irrelevant. commits after bad one X may have fixed the bug but
you are not interested in it. Commits before good one O might
have been broken in some other way, but we are looking for a
single regression, so they do not really matter either.
Then bisect suggests you one of (*) commits. Not at random, but
somewhere midway.
o---o---O---*---*---*---*---X---o
\ /
*---*---*---?
O (good) X (bad)
If ? is good, then three * suspects are good so you then have to
worry about remaining four * between the first O and X. That's
how you narrow things down.
You say a2d823bf is bad and 0f442aa2 is good. However, the
a2d823bf is a proper ancestor of 0f442aa2. That is, earlier it
was good but now it is good. As if you are trying to locate the
commit that fixed the bug. This is an impossible situation for
the bisect.
So your initial input may be truly good and bad commits, but
there might be more than one regression involved.
To look at the graph, you could do something like this:
for r in $good; do git tag good-$r $(git-rev-parse --verify $r); done
for r in $bad; do git tag bad-$r $(git-rev-parse --verify $r); done
gitk $(git-rev-parse $good) $(git-rev-parse --not $bad)
You will notice there is only one bad-* way below all the good-*
ones. Another possibility is maybe you have your good and bad
swapped?
^ permalink raw reply
* "git reset --hard" leaves empty directories that shouldn't exist
From: Carl Worth @ 2006-02-15 7:51 UTC (permalink / raw)
To: git
[-- Attachment #1: Type: text/plain, Size: 883 bytes --]
I've been exploring the potential for git-sync, and I found some odd
behavior with "git reset --hard". It appears that if the current tree
has some directory structure (at least two levels deep) that does not
exist in the tree being reset to, that empty directories are left
around after the reset:
$ git --version
git version 1.2.0.gf6e8
$ git init-db
defaulting to local storage area
$ touch file; git add file; git commit -m "Add file"
Committing initial tree df2b8fc99e1c1d4dbc0a854d9f72157f1d6ea078
$ git tag OLD
$ mkdir -p a/b/c; touch a/b/c/foo; git add a/b/c/foo; git commit -m
"Add foo"
$ git checkout -b bogus
$ git reset --hard OLD
$ find a
a
a/b
a/b/c
$
Is this operator error? I don't see any extra options I might be
missing in the documentation of git-reset.
I haven't looked into the implementation at all yet to see what might
be going on.
-Carl
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* bisect ends with a commit marked good and bad
From: Sanjoy Mahajan @ 2006-02-15 7:19 UTC (permalink / raw)
To: Git Mailing List
In doing a huge bisect (from 1875 or so revisions but fortunately
log(anything) is not too big) to chase a S3 sleep bug (bugme #5989), I
also noticed another problem. In several 'good' kernels, where S3
sleep-wake worked twice, the fan would not turn on despite the
temperature getting high, after the system went through two sleep-wake
cycles. [In the 'bad' kernels the system hangs on the second sleep, so
I couldn't collect any data about the fan.]
The result is that, for several kernels, I got free bisect data about
this second bug. I tried feeding this data to a fresh bisect, but
bisect says that one particular commit is good and bad. Here's the
script to run bisect:
#!/bin/bash -x
bad="a2d823bf ee408c79 d1138cf0"
good="0f442aa2 49799291 4a90c7e8"
git bisect start
for r in $bad ; do git-bisect bad $r ; done
for r in $good ; do git-bisect good $r ; done
The output:
+ bad='a2d823bf ee408c79 d1138cf0'
+ good='0f442aa2 49799291 4a90c7e8'
+ git bisect start
+ for r in '$bad'
+ git-bisect bad a2d823bf
+ for r in '$bad'
+ git-bisect bad ee408c79
+ for r in '$bad'
+ git-bisect bad d1138cf0
+ for r in '$good'
+ git-bisect good 0f442aa2
d1138cf035ad5a8dc0796b213bd078a2fb92eb7c was both good and bad
+ for r in '$good'
+ git-bisect good 49799291
d1138cf035ad5a8dc0796b213bd078a2fb92eb7c was both good and bad
+ for r in '$good'
+ git-bisect good 4a90c7e8
d1138cf035ad5a8dc0796b213bd078a2fb92eb7c was both good and bad
At first I thought it was impossible for a bug to behave this way, and
concluded that my data must be corrupt, since d1138cf0 is marked bad.
But on further thought, I'm not sure. Maybe there are quadratic
interactions between patches. For example, if a particular bug happens
only when two patches occur together? No, that would result in the
merge of their branches being the problematic commit. So I'm stuck.
Before I conclude that the data is corrupt and spend several hours
retesting several kernels, I thought I'd ask for advice from the
entomologists and bisectors.
How would I use gitk to show the those three good and three bad commits,
and their spanning tree? Neither the man entry nor the examples of its
use in git-bisect taught me enough to use it properly.
-Sanjoy
^ permalink raw reply
* Re: [ANNOUNCE] pg - A patch porcelain for GIT
From: Shawn Pearce @ 2006-02-15 6:54 UTC (permalink / raw)
To: J. Bruce Fields
Cc: Sam Vilain, Petr Baudis, Chuck Lever, Karl Hasselström,
Catalin Marinas, git
In-Reply-To: <20060215041142.GA21048@fieldses.org>
"J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Tue, Feb 14, 2006 at 07:35:10PM -0500, Shawn Pearce wrote:
> > Publishing a repository with a stg (or pg) patch series isn't
> > a problem; the problem is that no clients currently know how to
> > follow along with the remote repository's patch series. And I can't
> > think of a sensible behavior for doing so that isn't what git-core is
> > already doing today for non patch series type clients (as in don't go
> > backwards by popping but instead by pushing a negative delta). :-)
>
> If you represent each patch as a branch, with each modification to the
> patch a commit on the corresponding branch, and each "push" operation a
> merge from the branch corresponding to the previous patch to a branch
> corresponding to the new patch (isn't that what pg's trying to do?),
> then it should be possible just to track the branch corresponding to the
> top patch.
Yes that's pg in a nutshell.
But what happens when I pop back two patches (of three) and then push
down a different (fourth) patch? The tree just rewound backwards
and then forwards again in a different direction. (I apologize for
not being able to draw a nice ASCII art diagram of this, that's a
skill I'll have to learn to keep up with you guys.) This is the
issue with Junio's pu branch in git.git and is why some people
apparently don't follow it.
StGIT and pg aren't the only ones who suffer from this wonderful
little feature of GIT.
> In theory I guess it should also be possible to merge patch series that
> have followed two lines of development, by merging each corresponding
> branch.
Of course. If I delete all of the refs used by pg to mark the patch
boundaries its just another GIT branch. Ditto for StGIT. So clearly
you can merge them together just like any other GIT branch.
The open question is could you preserve the patch boundaries
while doing the merge. Probably not. It would become way to
complicated as you would want to merge the entire branch and not each
individual patch as the individual patch merges may not work but the
larger branch merge might go through without human intervention.
Of course you can try to keep the patch boundaries by exporting
all of the patches from the one branch and push them on top of
the current branch. But isn't that what a 3 way merge is anyway?
And again that might not work as well as taking the larger patch
and pushing that down. :-)
> The history would be really complicated. You'd need to figure out how
> to track the patch comments too, and you'd need scripts to convert to
> just a simple series of commits for submitting upstream. Probably not
> worth the trouble, but I don't know.
I think I'm almost there with pg. One of my next tasks is the
patch log ripping code. This is really only complicated because GIT
won't let me store the base of a 3 way merge as part of a commit;
all I can store is the set of parents. If I had the base in the
commit (and specifically marked as such so I can tell it from the
end points) then I could easily walk through the log to extract all
commits relevant to a patch and seek forward and backward over it.
Perhaps I could cheat and record 3 parents: (HEAD, base, last).
I wonder what gitk would make of that mess. I doubt it would display
any better than the current (HEAD, last) format I'm using now.
> If you really want revision control on patches the simplest thing might
> be just to run quilt or Andrew Morton's scripts on top of a git
> repository--the documentation with Andrew's scripts recommends doing
> that with CVS.
True but you also then run into problems about needing to know which
base each patch revision was applied against so you can reproduce
a source tree plus patch at a specific point in time.
When I started pg I first thought about recording a patch in a
secondary index/working directory where each patch was its own file
and use git-diff-tree to extract the patch, commit it as a blob in
the secondary index/directory, and push it onto the series by reading
the blob in and running git-apply on the patch stored within it.
I ruled this strategy out as it just felt like it would be too
slow and rather unnatural to work on with existing GIT tools.
I didn't want to write a full porcelain; I was really hoping to just
extend Cogito to get a patch stack that worked the way *I* wanted a
patch stack to work, which was close to StGIT but not quite StGIT.
(Of course it didn't work out this way when I picked a prefix of
'pg' instead of 'cg' for my new commands.) :-|
--
Shawn.
^ permalink raw reply
* Re: StGIT refreshes all added files - limitation of git-write-tree?
From: Shawn Pearce @ 2006-02-15 6:28 UTC (permalink / raw)
To: Pavel Roskin; +Cc: Catalin Marinas, git
In-Reply-To: <1139978528.28292.41.camel@dv>
Pavel Roskin <proski@gnu.org> wrote:
> Or maybe git-write-tree and other utilities could be changed to work on
> a copy of the index file? I would prefer not to move the
> actual .git/index away, but to make a copy for the current "stg refresh"
> operation.
git-write-tree pays attention to the GIT_INDEX_FILE environment
variable; if this, GIT_DIR and GIT_OBJECT_DIRECTORY are set you can
redirect git-write-tree to look at a different index. One way to
create a tree from a mixture of your current index and the current
HEAD's tree is to set GIT_INDEX_FILE to a temporary file name
(which doesn't exist), use git-read-tree to load the current tree
unmodified into that index, copy the modified records of interest
from the current index to the temporary index, then git-write-tree
from the temporary index. But now you need to update your current
index with the real SHA1s of the written files.
Somewhat convoluted. I think that git-commit.sh and cg-commit.sh
play such games to do partial commits based on what the user has
passed on the command line or modified in their editor (which is
actually a rather cool feature of Cogito). I'm using a similar
trick in pg to load binary objects from a patch directory (for a
specific use case that I have). git-am.sh uses a similar trick
to perform a 3 way merge if git-apply fails.
--
Shawn.
^ permalink raw reply
* Re: several quick questions
From: Junio C Hamano @ 2006-02-15 6:27 UTC (permalink / raw)
To: Andreas Ericsson; +Cc: git
In-Reply-To: <43F26129.4040804@op5.se>
Andreas Ericsson <ae@op5.se> writes:
> Junio C Hamano wrote:
>
>> Now, read-only ref does not interest me, but "do not commit on
>> top of this yourself, only fast-forward from somewhere else is
>> allowed" may be useful, for the reason why you mentioned
>> "origin".
>
> Do my suggestion and you wouldn't have to worry about read-only
> branches, and although merging any changes from it might be more
> trouble than its worth, it might be possible to cherry-pick the commit
> rather than reverting and re-applying it.
Sorry, is this "do my suggestion" a solution to my "do not
commit on top of this yourself, only fast-forward from somewhere
else is allowed -- e.g. to protect 'origin'" issue, or is it
something completely different?
^ permalink raw reply
* Re: StGIT refreshes all added files - limitation of git-write-tree?
From: Junio C Hamano @ 2006-02-15 6:20 UTC (permalink / raw)
To: Pavel Roskin; +Cc: git
In-Reply-To: <1139978528.28292.41.camel@dv>
Pavel Roskin <proski@gnu.org> writes:
> Or maybe git-write-tree and other utilities could be changed to work on
> a copy of the index file? I would prefer not to move the
> actual .git/index away, but to make a copy for the current "stg refresh"
> operation.
There is no need to change the core side.
GIT_INDEX_FILE=temporary-index git-write-tree
would do the job. See the current round of git-commit and how
it handles "git commit --only these files" case.
^ permalink raw reply
* Re: several quick questions
From: Keith Packard @ 2006-02-15 5:25 UTC (permalink / raw)
To: Martin Langhoff
Cc: keithp, Linus Torvalds, Carl Worth, Nicolas Vilz 'niv',
git
In-Reply-To: <46a038f90602142011o36b975b7s1833953db3b6d376@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 842 bytes --]
On Wed, 2006-02-15 at 17:11 +1300, Martin Langhoff wrote:
> Did that lead to finding any problems with the import? Can I get my
> hands on that script you've written to run the comparison?
The only issues we had were with manual changes to the repository; other
than that, we now has a usable git repository for cairo (visible at
git://git.cairographics.org/cairo). The comparison tool that I wrote was
a cheesy shell script; I think Carl has updated it to do something less
severe than rm -rf *; git-reset --hard; if he can share that, I think
you'll like it a lot better than mine.
Our CVS import script has some magic ChangeLog-style mangling which
we've posted to the list before; that clearly needs to be encapsulated
in an optional log-reformatting bit for it to be generally useful.
--
keith.packard@intel.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* StGIT refreshes all added files - limitation of git-write-tree?
From: Pavel Roskin @ 2006-02-15 4:42 UTC (permalink / raw)
To: Catalin Marinas, git
Hello!
Current StGIT would refresh all added files on "stg refresh", even if
they are not specified on the command line. I believe the same is true
for removed files.
That's when it can hurt. I apply a large patch that modifies and adds
several files. I want to split the patch into several StGIT patches by
selecting which files belong to which patch. All files that need to be
modified are already modifies. It's natural that I add new files using
"stg add" at this stage. To me, adding files looks very similar to
modifying them. However, then I create a patch by "stg new" and commit
_some_ files to it using "stg refresh" with those files on the command
line, I discover that _all_ added files have been refreshed.
The only workaround seems to be to add only those files that will be
included in the current patch. The same should be true for renamed
files.
Debugging StGIT shows that it builds correct lists of the files to be
included in the current patch (update_cache in git.py), but they are
never used. Instead, StGIT runs "git-write-tree" without arguments
(commit in git.py). While StGIT is careful to only add user-specified
modified files to the directory cache, it does nothing to the added
files, which are in the cache already.
Purely StGIT way of handling this problem would be to remove added files
from the directory cache if they are not being committed. The problem
is, "stg add" uses the cache, so this would undo "stg add" on files
unused in the current operation. Either StGIT should restore files in
the cache after the refresh, or there should be a separate StGIT cache
that "stg add" would work on. The former is potentially unreliable
(what if refresh is interrupted), the later creates an extra layer,
another directory cache no less, this reducing usability of the pure git
commands.
Another approach would be to get rid of "stg add" and "stg rm" and use
"stg refresh --add" and "stg refresh --rm" to add files to the current
patch or to remove them. It does feel like removal of a useful feature,
since "stg diff" without arguments will no longer show added or removed
files. Adding and removing files would be too immediate compared to
modifying them.
What I really hope to hear is that there is or there will be a git based
solution. Either git-write-tree could be changed to only process files
specified on the command line, or there should be some other command
doing that.
Or maybe git-write-tree and other utilities could be changed to work on
a copy of the index file? I would prefer not to move the
actual .git/index away, but to make a copy for the current "stg refresh"
operation.
--
Regards,
Pavel Roskin
^ permalink raw reply
* Re: [ANNOUNCE] pg - A patch porcelain for GIT
From: J. Bruce Fields @ 2006-02-15 4:11 UTC (permalink / raw)
To: Sam Vilain, Petr Baudis, Chuck Lever, Karl Hasselström,
Catalin Marinas, git
In-Reply-To: <20060215003510.GA25715@spearce.org>
On Tue, Feb 14, 2006 at 07:35:10PM -0500, Shawn Pearce wrote:
> Publishing a repository with a stg (or pg) patch series isn't
> a problem; the problem is that no clients currently know how to
> follow along with the remote repository's patch series. And I can't
> think of a sensible behavior for doing so that isn't what git-core is
> already doing today for non patch series type clients (as in don't go
> backwards by popping but instead by pushing a negative delta). :-)
If you represent each patch as a branch, with each modification to the
patch a commit on the corresponding branch, and each "push" operation a
merge from the branch corresponding to the previous patch to a branch
corresponding to the new patch (isn't that what pg's trying to do?),
then it should be possible just to track the branch corresponding to the
top patch.
In theory I guess it should also be possible to merge patch series that
have followed two lines of development, by merging each corresponding
branch.
The history would be really complicated. You'd need to figure out how
to track the patch comments too, and you'd need scripts to convert to
just a simple series of commits for submitting upstream. Probably not
worth the trouble, but I don't know.
If you really want revision control on patches the simplest thing might
be just to run quilt or Andrew Morton's scripts on top of a git
repository--the documentation with Andrew's scripts recommends doing
that with CVS.
--b,
^ permalink raw reply
* Re: several quick questions
From: Martin Langhoff @ 2006-02-15 4:11 UTC (permalink / raw)
To: Keith Packard; +Cc: Linus Torvalds, Carl Worth, Nicolas Vilz 'niv', git
In-Reply-To: <1139945967.4341.71.camel@evo.keithp.com>
On 2/15/06, Keith Packard <keithp@keithp.com> wrote:
> I was validating the cvs import by comparing every tagged version. Trust
> me, the git tree-rewriting stage was somewhat faster than the CVS
> checkout of the same content. And, as an egg, one often prefers BFI to
> finesse.
Keith,
Did that lead to finding any problems with the import? Can I get my
hands on that script you've written to run the comparison?
cheers,
martin
^ permalink raw reply
* Re: Handling large files with GIT
From: Sam Vilain @ 2006-02-15 4:03 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vslqlo0wo.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano wrote:
> So I think the order of questions you should be asking is:
>
> 1 - what operations are you trying to help?
Primarily, tracing history when dealing with history/changeset based
revision systems like SVN or darcs, and doing this in a manner that we
can make guarantees about behaving in the same way as those systems
would.
> 2 - what information you would need to achieve those operations
> better?
Minimally, this tuple:
( merge|copy, source_path, source_tree|source_commit,
destination_path, destination_commit )
It makes sense to record this with commits, as conceptually it is a part
of the intended commit history along with the change comment.
> 3 - among the second one, what will be necessary to be set in
> stone (IOW, cannot be computed later), and what are
> computable but expensive to recompute every time?
The only operation you cannot automatically and with certainty detect a
rename and change content without inserting a dummy commit between the
name change and the content change. But in a sense this is the same as
my suggestion - using the commit object history to record information
that normally doesn't matter when you are doing content-keyed
operations.
> I am getting an impression that you are doing only the first
> half of (2) without other parts, which somewhat bothers me.
Well, thank you for spending so much time to reply to me given that was
your assessment. I think the best direction from here would be to start
molding some porcelain, then I can cross this bridge when I come to it
rather than simply speculating and hand-waving.
Besides, I can always prototype it for discussion using the commit
description as a surrogate container for the information.
Sam.
ps I also responded to the rest of your e-mail, but decided that the
answers to the above questions were more important.
>> 2. forensic - extra stuff at the end of the commit object?
> (except "extra at the end of commit", which does not make it out
> of the tree).
It is a part of the repository, but more a property of the commit itself
- like the commit description. Like somebody writing "I renamed this
file to that file and changed its contents", but in a parsable form
that can _optionally_ be used to prevent the relevant git-core tools
from having to do content comparison, or perhaps something subtler like
increasing the score of the recorded history branch when scoring
alternatives looking for history.
>> eg
>> Copied: /new/path from /old/path:commit:c0bb171d..
>> (for SVN case where history matters)
>> Copied: /new/path from blob:b10b1d..
>> (for general pre-caching case)
>> Merged: /new/path from /old/path:commit:C0bb171d..
>> (for an SVK clone, so we know that subsequent merges on
>> /new/path need only merge from /old/path starting at commit
>> C0bb171d..)
> I am not sure if recording the bare SVN ``copied'' is very
> useful. You would need to infer things from what SVN did to
> tell if the copy is a tree copy inside a project (e.g. cp -r
> i386 x86_64), tagging (e.g. svn-cp rHEAD trunk tags/v1.2), or
> branching, wouldn't you? SVK merge ticket is a bit more useful
> in that sense.
In the SVN model there really is no difference between these cases. Of
course the actual representation of these in the object does not matter;
the above is the what, not the how. But in general, SVN only records
copying; it has no repository concept of merge, branch, tag, rename.
SVK adds merging to the picture.
Representing an SVN tree copy as a new sub-tree in a git repository
should still be a "cheap copy", it's just that all the tools will not
(and probably should not) see it as a branch but a copy.
> So far, git philosophy is to record things you _know_ about and
> defer such guesswork to the future, so limiting what you record
> to what you can actually see from the foreign SCM would be more
> in line with it.
Yes, and if I am mirroring an SVN repository, then I only know that in
that repository, the history /was recorded/ as such. Not the history
/is/ as such, that's a different question, and is the guesswork worth
being defered to the future.
> For the same reason, if you are talking about
> maildir managed under git, you should not have record anything
> other than what git already records: "we used to have these
> files, now we have these instead".
Ok. As Martin pointed out, the Maildir situation is actually a simple
case. In a sense, I hijacked a vaguely related thread to resolve my
Warnock dilemma :)
> But I thought you were talking about caching what earlier
> inference declared what happened, so that you do not have to do
> the same inference every time. If that is the case, SVN level
> "Copied:" is probably not what you would want to record, I
> suspect. You would do some inference with the given information
> ("SVN says it copied this tree to that tree, what was it that it
> really wanted to do? Was it a copy, or was it to create a
> branch which was implemented as a copy?"), and record that,
> hoping that information would help your other operations this
> time and later.
Well, this is already guesswork defered to the future that the
Subversion authors inflict on the users of Subversion repositories. If
you read the Subversion manual you will find recommendations to
studiously record this information and to use a standard repository
layout so that other people will understand what your copies were
intended to be.
^ permalink raw reply
* Re: Handling large files with GIT
From: Linus Torvalds @ 2006-02-15 3:58 UTC (permalink / raw)
To: Junio C Hamano, Fredrik Kuivinen; +Cc: Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0602141829080.3691@g5.osdl.org>
On Tue, 14 Feb 2006, Linus Torvalds wrote:
>
> So in case people want to try, here's a third patch. Oh, and it's against
> my _original_ path, not incremental to the middle one (ie both patches two
> and three are against patch #1, it's not a nice series).
>
> Now I'm really done, and won't be sending out any more patches today.
Still true. I've just been thinking about the last state.
As far as I can tell, the output from git-merge-tree with that fix to only
simplify subdirectories that match exactly in all of base/branch1/branch2
is precisely the output that git-merge-recursive actually wants.
Rather than doing a three-way merge with "git-read-tree", and then doing
"git-ls-files --unmerged", I think this gives the same result much more
efficiently.
That said, I can't follow the python code, so maybe I'm missing something.
Fredrik cc'd, in case he can put me right.
Linus
^ permalink raw reply
* Re: Handling large files with GIT
From: Linus Torvalds @ 2006-02-15 2:33 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0602141811050.3691@g5.osdl.org>
On Tue, 14 Feb 2006, Linus Torvalds wrote:
>
> Here, btw, is the trivial diff to turn my previous "tree-resolve" into a
> "resolve tree relative to the current branch".
Gaah. It was trivial, and it happened to work fine for my test-case, but
when I started looking at not doing that extremely aggressive subdirectory
merging, that showed a few other issues...
So in case people want to try, here's a third patch. Oh, and it's against
my _original_ path, not incremental to the middle one (ie both patches two
and three are against patch #1, it's not a nice series).
Now I'm really done, and won't be sending out any more patches today.
Sorry for the noise.
Linus
----
diff --git a/merge-tree.c b/merge-tree.c
index 0d6d434..6381118 100644
--- a/merge-tree.c
+++ b/merge-tree.c
@@ -55,9 +55,26 @@ static int same_entry(struct name_entry
a->mode == b->mode;
}
-static void resolve(const char *base, struct name_entry *result)
+static const char *sha1_to_hex_zero(const unsigned char *sha1)
{
- printf("0 %06o %s %s%s\n", result->mode, sha1_to_hex(result->sha1), base, result->path);
+ if (sha1)
+ return sha1_to_hex(sha1);
+ return "0000000000000000000000000000000000000000";
+}
+
+static void resolve(const char *base, struct name_entry *branch1, struct name_entry *result)
+{
+ char branch1_sha1[50];
+
+ /* If it's already branch1, don't bother showing it */
+ if (!branch1)
+ return;
+ memcpy(branch1_sha1, sha1_to_hex_zero(branch1->sha1), 41);
+
+ printf("0 %06o->%06o %s->%s %s%s\n",
+ branch1->mode, result->mode,
+ branch1_sha1, sha1_to_hex_zero(result->sha1),
+ base, result->path);
}
static int unresolved_directory(const char *base, struct name_entry n[3])
@@ -100,9 +117,12 @@ static void unresolved(const char *base,
{
if (unresolved_directory(base, n))
return;
- printf("1 %06o %s %s%s\n", n[0].mode, sha1_to_hex(n[0].sha1), base, n[0].path);
- printf("2 %06o %s %s%s\n", n[1].mode, sha1_to_hex(n[1].sha1), base, n[1].path);
- printf("3 %06o %s %s%s\n", n[2].mode, sha1_to_hex(n[2].sha1), base, n[2].path);
+ if (n[0].sha1)
+ printf("1 %06o %s %s%s\n", n[0].mode, sha1_to_hex(n[0].sha1), base, n[0].path);
+ if (n[1].sha1)
+ printf("2 %06o %s %s%s\n", n[1].mode, sha1_to_hex(n[1].sha1), base, n[1].path);
+ if (n[2].sha1)
+ printf("3 %06o %s %s%s\n", n[2].mode, sha1_to_hex(n[2].sha1), base, n[2].path);
}
/*
@@ -183,21 +203,21 @@ static void merge_trees(struct tree_desc
/* Same in both? */
if (same_entry(entry+1, entry+2)) {
if (entry[0].sha1) {
- resolve(base, entry+1);
+ resolve(base, NULL, entry+1);
continue;
}
}
if (same_entry(entry+0, entry+1)) {
- if (entry[2].sha1) {
- resolve(base, entry+2);
+ if (entry[2].sha1 && !S_ISDIR(entry[2].mode)) {
+ resolve(base, entry+1, entry+2);
continue;
}
}
if (same_entry(entry+0, entry+2)) {
- if (entry[1].sha1) {
- resolve(base, entry+1);
+ if (entry[1].sha1 && !S_ISDIR(entry[1].mode)) {
+ resolve(base, NULL, entry+1);
continue;
}
}
^ permalink raw reply related
* Re: Handling large files with GIT
From: Linus Torvalds @ 2006-02-15 2:18 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0602141741210.3691@g5.osdl.org>
On Tue, 14 Feb 2006, Linus Torvalds wrote:
>
> Finally (a more serious caveat):
> - doing things through stdout may end up being so expensive that we'd
> need to do something else. In particular, it's likely that I should
> not actually output the "merge results", but instead output a "merge
> results as they _differ_ from branch1"
>
> In many ways, the really _interesting_ part of a merge is not the result,
> but how it _changes_ the branch we're merging into. That's particularly
> important as it should hopefully also mean that the output size for any
> reasonable case is minimal (and tracks what we actually need to do to the
> current state to create the final result).
Here, btw, is the trivial diff to turn my previous "tree-resolve" into a
"resolve tree relative to the current branch".
In particular, it makes the example merge perhaps even more interesting,
and makes the "merging directories and merging files should use different
heuristics more obvious". It's quite instructive, I think.
So if you want to test this, the merge I have been testing with is the
last infiniband merge in the kernel:
git-merge-tree 3c3b809 4cbf876 7d2babc
and you'll need to spend a few moments on thinking about what the
"directory merge" thing there means: in particular, we should probably
make the
if (entry[2].sha1) {
test be
if (entry[2].sha && !S_ISDIR(entry[2].mode)) {
(and same for "resolve to entry[1]" case for that matter) so that we never
create a "resolve()" that picks a whole subdirectory from one of the
branches.
The current logic is "logical", just probably not what we want.
Linus
----
diff --git a/merge-tree.c b/merge-tree.c
index 0d6d434..0bf871c 100644
--- a/merge-tree.c
+++ b/merge-tree.c
@@ -55,9 +55,19 @@ static int same_entry(struct name_entry
a->mode == b->mode;
}
-static void resolve(const char *base, struct name_entry *result)
+static void resolve(const char *base, struct name_entry *branch1, struct name_entry *result)
{
- printf("0 %06o %s %s%s\n", result->mode, sha1_to_hex(result->sha1), base, result->path);
+ char branch1_sha1[50];
+
+ /* If it's already branch1, don't bother showing it */
+ if (!branch1)
+ return;
+ memcpy(branch1_sha1, sha1_to_hex(branch1->sha1), 41);
+
+ printf("0 %06o->%06o %s->%s %s%s\n",
+ branch1->mode, result->mode,
+ branch1_sha1, sha1_to_hex(result->sha1),
+ base, result->path);
}
static int unresolved_directory(const char *base, struct name_entry n[3])
@@ -183,21 +193,21 @@ static void merge_trees(struct tree_desc
/* Same in both? */
if (same_entry(entry+1, entry+2)) {
if (entry[0].sha1) {
- resolve(base, entry+1);
+ resolve(base, NULL, entry+1);
continue;
}
}
if (same_entry(entry+0, entry+1)) {
if (entry[2].sha1) {
- resolve(base, entry+2);
+ resolve(base, entry+1, entry+2);
continue;
}
}
if (same_entry(entry+0, entry+2)) {
if (entry[1].sha1) {
- resolve(base, entry+1);
+ resolve(base, NULL, entry+1);
continue;
}
}
^ permalink raw reply related
* Re: Handling large files with GIT
From: Martin Langhoff @ 2006-02-15 2:07 UTC (permalink / raw)
To: Sam Vilain; +Cc: Junio C Hamano, git, Linus Torvalds
In-Reply-To: <43F27878.50701@vilain.net>
On 2/15/06, Sam Vilain <sam@vilain.net> wrote:
> Excellent. Any speculations on where they might fit? Clearly, it needs
> to be out of the "tree".
I think Junio & Linus are talking about alternative mergers, something
that can be called instead of git-read-tree -m (which is the way
merges seem to kick off). Or perhaps an additional flag to
git-read-tree to be used in conjunction with -m, something like
--optimize-for-identity that lets git-read-tree know to do a first
pass keying things on file identity rather than file path.
So we are _not_ touching the object database, at all. Only optimising
merges for very large trees there mv is a popular operation. All the
cases you discuss can be tackled very efficiently without making *any*
change to the object database.
> Martin, is that enough for your CVS case?
Oh, I don't need it at all. It's just that there's been some lazy talk
of tracking mboxes and maildirs with git, and look where it's led.
Blame Roland Stigge who got me started down this track.
I'm sure it's because the other optimisations are a lot harder to
tackle ;-) though Linus mentions that it'd be trivial for
git-read-tree -m to detect unchanged directories and perhaps do things
a bit faster. Not as revolutionary as an --optimize-for-identity but
not as risky either.
In any case, don't count in me for any of this git-checkout hacking. I
know better than start learning C posting patches to *this* list.
cheers,
martin
^ permalink raw reply
* Re: Handling large files with GIT
From: Linus Torvalds @ 2006-02-15 2:05 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vy80dpo9g.fsf@assigned-by-dhcp.cox.net>
On Tue, 14 Feb 2006, Junio C Hamano wrote:
> Linus Torvalds <torvalds@osdl.org> writes:
>
> > If somebody is interested in making the "lots of filename changes" case go
> > fast, I'd be more than happy to walk them through what they'd need to
> > change. I'm just not horribly motivated to do it myself. Hint, hint.
>
> In case anybody is wondering, I share the same feeling. I
> cannot say I'd be "more than happy to" clean up potential
> breakages during the development of such changes, but if the
> change eventually would help certain use cases, I can be
> persuaded to help debugging such a mess ;-).
Actually, I got interested in seeing how hard this is, and wrote a simple
first cut at doing a tree-optimized merger.
Let me shout a bit first:
THIS IS WORKING CODE, BUT BE CAREFUL: IT'S A TECHNOLOGY DEMONSTRATION
RATHER THAN THE FINAL PRODUCT!
With that out of the way, let me descibe what this does (and then describe
the missing parts).
This is basically a three-way merge that works entirely on the "tree"
level, rather than on the index. A lot of the _concepts_ are the same,
though, and if you're familiar with the results of an index merge, some of
the output will make more sense.
You give it three trees: the base tree (tree 0), and the two branches to
be merged (tree 1 and tree 2 respectively). It will then walk these three
trees, and resolve them as it goes along.
The interesting part is:
- it can resolve whole sub-directories in one go, without actually even
looking recursively at them. A whole subdirectory will resolve the same
way as any individual files will (although that may need some
modification, see later).
- if it has a "content conflict", for subdirectories that means "try to
do a recursive tree merge", while for non-subdirectories it's just a
content conflict and we'll output the stage 1/2/3 information.
- a successful merge will output a single stage 0 ("merged") entry,
potentially for a whole subdirectory.
- it outputs all the resolve information on stdout, so something like the
recursive resolver can pretty easily parse it all.
Now, the caveats:
- we probably need to be more careful about subdirectory resolves. The
trivial case (both branches have the exact same subdirectory) is a
trivial resolve, but the other cases ("branch1 matches base, branch2 is
different" probably can't be silently just resolved to the "branch2"
subdirectory state, since it might involve renames into - or out of -
that subdirectory)
- we do not track the current index file at all, so this does not do the
"check that index matches branch1" logic that the three-way merge in
git-read-tree does. The theory is that we'd do a full three-way merge
(ignoring the index and working directory), and then to update the
working tree, we'd do a two-way "git-read-tree branch1->result"
- I didn't actually make it do all the trivial resolve cases that
git-read-tree does. It's a technology demonstration.
Finally (a more serious caveat):
- doing things through stdout may end up being so expensive that we'd
need to do something else. In particular, it's likely that I should
not actually output the "merge results", but instead output a "merge
results as they _differ_ from branch1"
However, I think this patch is already interesting enough that people who
are interested in merging trees might want to look at it. Please keep in
mind that tech _demo_ part, and in particular, keep in mind the final
"serious caveat" part.
In many ways, the really _interesting_ part of a merge is not the result,
but how it _changes_ the branch we're merging into. That's particularly
important as it should hopefully also mean that the output size for any
reasonable case is minimal (and tracks what we actually need to do to the
current state to create the final result).
The code very much is organized so that doing the result as a "diff
against branch1" should be quite easy/possible. I was actually going to do
it, but I decided that it probably makes the output harder to read. I
dunno.
Anyway, let's think about this kind of approach.. Note how the code itself
is actually quite small and short, although it's prbably pretty "dense".
As an interesting test-case, I'd suggest this merge in the kernel:
git-merge-tree $(git-merge-base 4cbf876 7d2babc) 4cbf876 7d2babc
which resolves beautifully (there are no actual file-level conflicts), and
you can look at the output of that command to start thinking about what
it does.
The interesting part (perhaps) is that timing that command for me shows
that it takes all of 0.004 seconds.. (the git-merge-base thing takes
considerably more ;)
The point is, we _can_ do the actual merge part really really quickly.
Linus
PS. Final note: when I say that it is "WORKING CODE", that is obviously by
my standards. IOW, I tested it once and it gave reasonable results - so it
must be perfect.
Whether it works for anybody else, or indeed for any other test-case, is
not my problem ;)
---
diff-tree f0e6b454ff873429237322c846603d2e1fffc867 (from 6a9b87972f27edfe53da4ce016adf4c0cd42f5e6)
Author: Linus Torvalds <torvalds@osdl.org>
Date: Tue Feb 14 17:39:15 2006 -0800
Add "git-merge-tree" functionality
This is basically a tree-optimized merge. Or rather, it is the first
stages _towards_ such a merge.
Given a base tree and two branches to merge, it will do a trivial merge,
optimizing away the case of identical subdirectories, and resolving
trivial merges. It outputs the list of file/directory resolves.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
diff --git a/Makefile b/Makefile
index d40aa6a..4d04f49 100644
--- a/Makefile
+++ b/Makefile
@@ -151,7 +151,7 @@ PROGRAMS = \
git-upload-pack$X git-verify-pack$X git-write-tree$X \
git-update-ref$X git-symbolic-ref$X git-check-ref-format$X \
git-name-rev$X git-pack-redundant$X git-repo-config$X git-var$X \
- git-describe$X
+ git-describe$X git-merge-tree$X
# what 'all' will build and 'install' will install, in gitexecdir
ALL_PROGRAMS = $(PROGRAMS) $(SIMPLE_PROGRAMS) $(SCRIPTS)
diff --git a/merge-tree.c b/merge-tree.c
new file mode 100644
index 0000000..0d6d434
--- /dev/null
+++ b/merge-tree.c
@@ -0,0 +1,238 @@
+#include "cache.h"
+#include "diff.h"
+
+static const char merge_tree_usage[] = "git-merge-tree <base-tree> <branch1> <branch2>";
+static int resolve_directories = 1;
+
+static void merge_trees(struct tree_desc t[3], const char *base);
+
+static void *fill_tree_descriptor(struct tree_desc *desc, const unsigned char *sha1)
+{
+ unsigned long size = 0;
+ void *buf = NULL;
+
+ if (sha1) {
+ buf = read_object_with_reference(sha1, "tree", &size, NULL);
+ if (!buf)
+ die("unable to read tree %s", sha1_to_hex(sha1));
+ }
+ desc->size = size;
+ desc->buf = buf;
+ return buf;
+}
+
+struct name_entry {
+ const unsigned char *sha1;
+ const char *path;
+ unsigned int mode;
+ int pathlen;
+};
+
+static void entry_clear(struct name_entry *a)
+{
+ memset(a, 0, sizeof(*a));
+}
+
+static int entry_compare(struct name_entry *a, struct name_entry *b)
+{
+ return base_name_compare(
+ a->path, a->pathlen, a->mode,
+ b->path, b->pathlen, b->mode);
+}
+
+static void entry_extract(struct tree_desc *t, struct name_entry *a)
+{
+ a->sha1 = tree_entry_extract(t, &a->path, &a->mode);
+ a->pathlen = strlen(a->path);
+}
+
+/* An empty entry never compares same, not even to another empty entry */
+static int same_entry(struct name_entry *a, struct name_entry *b)
+{
+ return a->sha1 &&
+ b->sha1 &&
+ !memcmp(a->sha1, b->sha1, 20) &&
+ a->mode == b->mode;
+}
+
+static void resolve(const char *base, struct name_entry *result)
+{
+ printf("0 %06o %s %s%s\n", result->mode, sha1_to_hex(result->sha1), base, result->path);
+}
+
+static int unresolved_directory(const char *base, struct name_entry n[3])
+{
+ int baselen;
+ char *newbase;
+ struct name_entry *p;
+ struct tree_desc t[3];
+ void *buf0, *buf1, *buf2;
+
+ if (!resolve_directories)
+ return 0;
+ p = n;
+ if (!p->mode) {
+ p++;
+ if (!p->mode)
+ p++;
+ }
+ if (!S_ISDIR(p->mode))
+ return 0;
+ baselen = strlen(base);
+ newbase = xmalloc(baselen + p->pathlen + 2);
+ memcpy(newbase, base, baselen);
+ memcpy(newbase + baselen, p->path, p->pathlen);
+ memcpy(newbase + baselen + p->pathlen, "/", 2);
+
+ buf0 = fill_tree_descriptor(t+0, n[0].sha1);
+ buf1 = fill_tree_descriptor(t+1, n[1].sha1);
+ buf2 = fill_tree_descriptor(t+2, n[2].sha1);
+ merge_trees(t, newbase);
+
+ free(buf0);
+ free(buf1);
+ free(buf2);
+ free(newbase);
+ return 1;
+}
+
+static void unresolved(const char *base, struct name_entry n[3])
+{
+ if (unresolved_directory(base, n))
+ return;
+ printf("1 %06o %s %s%s\n", n[0].mode, sha1_to_hex(n[0].sha1), base, n[0].path);
+ printf("2 %06o %s %s%s\n", n[1].mode, sha1_to_hex(n[1].sha1), base, n[1].path);
+ printf("3 %06o %s %s%s\n", n[2].mode, sha1_to_hex(n[2].sha1), base, n[2].path);
+}
+
+/*
+ * Merge two trees together (t[1] and t[2]), using a common base (t[0])
+ * as the origin.
+ *
+ * This walks the (sorted) trees in lock-step, checking every possible
+ * name. Note that directories automatically sort differently from other
+ * files (see "base_name_compare"), so you'll never see file/directory
+ * conflicts, because they won't ever compare the same.
+ *
+ * IOW, if a directory changes to a filename, it will automatically be
+ * seen as the directory going away, and the filename being created.
+ *
+ * Think of this as a three-way diff.
+ *
+ * The output will be either:
+ * - successful merge
+ * "0 mode sha1 filename"
+ * NOTE NOTE NOTE! FIXME! We really really need to walk the index
+ * in parallel with this too!
+ *
+ * - conflict:
+ * "1 mode sha1 filename"
+ * "2 mode sha1 filename"
+ * "3 mode sha1 filename"
+ * where not all of the 1/2/3 lines may exist, of course.
+ *
+ * The successful merge rules are the same as for the three-way merge
+ * in git-read-tree.
+ */
+static void merge_trees(struct tree_desc t[3], const char *base)
+{
+ for (;;) {
+ struct name_entry entry[3];
+ unsigned int mask = 0;
+ int i, last;
+
+ last = -1;
+ for (i = 0; i < 3; i++) {
+ if (!t[i].size)
+ continue;
+ entry_extract(t+i, entry+i);
+ if (last >= 0) {
+ int cmp = entry_compare(entry+i, entry+last);
+
+ /*
+ * Is the new name bigger than the old one?
+ * Ignore it
+ */
+ if (cmp > 0)
+ continue;
+ /*
+ * Is the new name smaller than the old one?
+ * Ignore all old ones
+ */
+ if (cmp < 0)
+ mask = 0;
+ }
+ mask |= 1u << i;
+ last = i;
+ }
+ if (!mask)
+ break;
+
+ /*
+ * Update the tree entries we've walked, and clear
+ * all the unused name-entries.
+ */
+ for (i = 0; i < 3; i++) {
+ if (mask & (1u << i)) {
+ update_tree_entry(t+i);
+ continue;
+ }
+ entry_clear(entry + i);
+ }
+
+ /* Same in both? */
+ if (same_entry(entry+1, entry+2)) {
+ if (entry[0].sha1) {
+ resolve(base, entry+1);
+ continue;
+ }
+ }
+
+ if (same_entry(entry+0, entry+1)) {
+ if (entry[2].sha1) {
+ resolve(base, entry+2);
+ continue;
+ }
+ }
+
+ if (same_entry(entry+0, entry+2)) {
+ if (entry[1].sha1) {
+ resolve(base, entry+1);
+ continue;
+ }
+ }
+
+ unresolved(base, entry);
+ }
+}
+
+static void *get_tree_descriptor(struct tree_desc *desc, const char *rev)
+{
+ unsigned char sha1[20];
+ void *buf;
+
+ if (get_sha1(rev, sha1) < 0)
+ die("unknown rev %s", rev);
+ buf = fill_tree_descriptor(desc, sha1);
+ if (!buf)
+ die("%s is not a tree", rev);
+ return buf;
+}
+
+int main(int argc, char **argv)
+{
+ struct tree_desc t[3];
+ void *buf1, *buf2, *buf3;
+
+ if (argc < 4)
+ usage(merge_tree_usage);
+
+ buf1 = get_tree_descriptor(t+0, argv[1]);
+ buf2 = get_tree_descriptor(t+1, argv[2]);
+ buf3 = get_tree_descriptor(t+2, argv[3]);
+ merge_trees(t, "");
+ free(buf1);
+ free(buf2);
+ free(buf3);
+ return 0;
+}
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox