new file leaked onto release branch

new file leaked onto release branch

[Len Brown]

I'm suspecting that this issue is technically a pilot error of me issuing
git-updated-index at the wrong time, but perhaps the list
should be aware of this type of issue.  And perhaps somebody
can suggest a better work flow that is immune to this?

Somehow a new file leaked from my "acpica" branch onto my "release" branch
without me pulling "acpica" into "release".

I use the latest git and I follow Tony's Documentation/howto/using-topic-branches.txt.

The new file, rsinfo.c,  was added in one of the patches in the acpi branch,
but then was sucked into the release branch in this commit:
9115a6c787596e687df03010d97fccc5e0762506
which is on the release.broken branch of this tree:
git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux-acpi-2.6.git

Even though the file didn't exist on either of the parents of the merge.

I think I probably did a git-update-index while working on the acpica
branch, and then git remembered that while I was on the release branch
doing a routine pull from linus to make sure I was up-to-date before
doing a push to kernel.org.

why did I do this?

I use quilt to manage a stack of patches in my repo.
I like to build them on several build machines (i386, x86_64, ia64)
before doing a git commit.

I package up a tar-file for the remote machines with
git-tar-tree $BRANCH $REPO | gzip -1 > $TARFILE
But I still need to generate a patch containing all the local
changes that I haven't checked into git yet.

git diff > my.patch
does this for me.  But when it failed to pick up a new file,
I manually did a git-update-index --add IIR, and that seems
to be how rsinfo.c got sucked into the subsequent git pull from linus.

perhaps somebody has a better idiom for packaging up a  current
working tree for a remote build machine to crunch on it?

thanks,
-Len

Re: new file leaked onto release branch

[Junio C Hamano]

Len Brown <len.brown@intel.com> writes:

> Somehow a new file leaked from my "acpica" branch onto my "release" branch
> without me pulling "acpica" into "release".

I'll take a look at your "broken" branch later today (but will
go to bed first ;-), so please leave it around.

> I package up a tar-file for the remote machines with
> git-tar-tree $BRANCH $REPO | gzip -1 > $TARFILE
> But I still need to generate a patch containing all the local
> changes that I haven't checked into git yet.

Let me understand what you have correctly:

    You have the latest commit, changes needed to be tested in
    working tree, and index is somewhere in-between, because of
    "git-update-index --add" you did for patch generation; but
    ideally you would want to keep HEAD and index in sync.

And you need a way to do the above tar-tree + patch equivanent
to move what you have in your working tree to other machines for
pre-commit testing.  Some random thoughts on ways to do it.

(1) NFS with "make O=dir/to/store/output/files/"?

(2) Do you have git on other build boxes?  Perhaps

    mainbox$ edit on "acpica" branch
    mainbox$ git checkout -b build-test ;# branch from it
    mainbox$ git add some files
    mainbox$ git commit -a -m "Build test $(date)" ;# take all

    mainbox$ ssh i386box
    i386box$ git pull mainbox:/home/lnx/.git build-test
    Fast forward.
    i386box$ make ;# happy

    mainbox$ git checkout acpica ; git pull . build-test ;# fast forward

(3) Tarball approach, but taking the local modifications along
    as well:

    mainbox$ git update-index all-changed-paths-not-just-add
    mainbox$ git-tar-tree $(git-write-tree) $REPO | gzip >$TARFILE
    mainbox$ git reset ;# unregisters all mods since HEAD from index
    mainbox$ scp $TARFILE i386box:/home/lnx/

I suspect the last one is the least disruptive to your existing
workflow.

Re: new file leaked onto release branch

[Junio C Hamano]

Len Brown <len.brown@intel.com> writes:

> I'm suspecting that this issue is technically a pilot error of me issuing
> git-updated-index at the wrong time, but perhaps the list
> should be aware of this type of issue.

Merging is so fundamental to git workflow, and it is not
supposed to be prone to pilot errors, so I am puzzled and
very worried, so I am still awake.

Even if you added some file to the index prior to merge, this
sequence:

        $ git checkout one-branch
        $ git add some-file
        $ git pull . another-branch

should *never* succeed.  It should always fail, saying:

        fatal: Entry 'some-file' would be overwritten by merge. Cannot merge.

and I just tried it with both recursive (default) and resolve
merge strategies.  They both do the right thing.

One exception is if you did something like this:

	$ git checkout one-branch
        $ git pull . another-branch ;# oops, merge conflicts!
        $ edit conflicted files and fix
        $ git update-index --add some-file
        $ git commit

You can introduce unrelated changes to commits like that.  Lemme
see what your release.broken looks like.

Gahhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.

Forget about what I said.  Although all of what I said above are
correct, I do not think it applies to your case at all --- these
commits do not appear to be created by git-pull nor git-merge,
and honestly, they look quite bogus.  There is something else
going on.

        $ git cat-file commit 9115a6
        tree da65904b8c0c0bc6c778b39bc25d06e07ca7c82c
        parent 927fe18397b3b1194a5b26b1d388d97e391e5fd2
        parent e4f5c82a92c2a546a16af1614114eec19120e40a
        author Len Brown <len.brown@intel.com> 1133904460 -0500
        committer Len Brown <len.brown@intel.com> 1133904460 -0500

        Auto-update from upstream

        $ git-merge-base 9115a6^1 9115a6^2; git-rev-parse 9115a6^2
        e4f5c82a92c2a546a16af1614114eec19120e40a
        e4f5c82a92c2a546a16af1614114eec19120e40a

What is this "Auto-update from upstream" merge?  It is not a
merge at all, not even a fast forward.  Your branch tip was
927fe1 and the other branch tip e4f5c8 was behind.  In fact, I
just tried it and both of my attempts correctly said "Already
up-to-date.", with or without local changes to the index.

        $ git show-branch --sha1-name --topo-order lenb/merge*
        * [lenb/mergetest] Pull 5165 into release branch
         ! [lenb/mergeother] V4L/DVB (3087) fix analog NTSC for pcHDTV 3000
        --
        +  [927fe18] Pull 5165 into release branch
        +  [1e48396] [ACPI] correct earlier SMP deep C-states on HT patch
        ++ [e4f5c82] V4L/DVB (3087) fix analog NTSC for pcHDTV 3000

        $ git pull . lenb/mergeother
        Already up-to-date.

What do you use to create these "Auto-update" commits?

I think 9115a6 commit should not *even* *exist*.  The merge
between 927fe1 and e4f5c8 should have resulted in "Oh, we are
already up-to-date and they are behind, so there is nothing for
us to do".

Or if your tool somehow records your tree as the second parent
(goes against the usual convention but is a valid thing to do),
it should have resulted in "Updating from e4f5c8... to 927fe1.
Fast forward".

In either case, 9115a6 commit should never have been created in
the first place.

What do you use to create these "Auto-update" commits, and how
can I help you to fix it?

RE: new file leaked onto release branch

[Brown, Len]

>> I package up a tar-file for the remote machines with
>> git-tar-tree $BRANCH $REPO | gzip -1 > $TARFILE
>> But I still need to generate a patch containing all the local
>> changes that I haven't checked into git yet.
>
>Let me understand what you have correctly:
>
>    You have the latest commit, changes needed to be tested in
>    working tree, and index is somewhere in-between, because of
>    "git-update-index --add" you did for patch generation; but
>    ideally you would want to keep HEAD and index in sync.

Right, I updated the index so that git diff would work.

I guess I don't really have my head around the index concept.
Yes, it seems perhaps that HEAD and index disagreeing about what is
in the repo is what caused the problem at hand.

>And you need a way to do the above tar-tree + patch equivanent
>to move what you have in your working tree to other machines for
>pre-commit testing.  Some random thoughts on ways to do it.
>
>(1) NFS with "make O=dir/to/store/output/files/"?

I think I prefer the build boxes to be more independent.

They build about a dozen kernels from scratch each,
so I can't really keep my working repo frozen for
the duration of the builds.

Also, sometimes I have them building different branches
and different versions, depending on what is going on.

Also, sometimes I apply a local test patch to the
build box that isn't (yet) shared with the other boxes.
 
>(2) Do you have git on other build boxes?  Perhaps
>
>    mainbox$ edit on "acpica" branch
>    mainbox$ git checkout -b build-test ;# branch from it
>    mainbox$ git add some files
>    mainbox$ git commit -a -m "Build test $(date)" ;# take all
>
>    mainbox$ ssh i386box
>    i386box$ git pull mainbox:/home/lnx/.git build-test
>    Fast forward.
>    i386box$ make ;# happy
>
>    mainbox$ git checkout acpica ; git pull . build-test ;# 
>fast forward

sure, I have git everwhere.  But check-in is not a lightweight
process for me -- I like to get the contributor & comments right
on the check-in as if I'm creating permanent history.  I hate
checking in a patch just to have to check in a build fix for
it 20 minutes later.

this is why I use quilt.  I just pop the patches, fix the
broken one, and push them back.  after build and boot tests,
that is when I do the git commit.

>(3) Tarball approach, but taking the local modifications along
>    as well:
>
>    mainbox$ git update-index all-changed-paths-not-just-add
>    mainbox$ git-tar-tree $(git-write-tree) $REPO | gzip >$TARFILE
>    mainbox$ git reset ;# unregisters all mods since HEAD from index
>    mainbox$ scp $TARFILE i386box:/home/lnx/
>
>I suspect the last one is the least disruptive to your existing
>workflow.

yes, this is along the lines of what I do now.
do these steps capture the changed uncommited source files in
my repo?

actually, since the mis-hap with the update-index, what I do now
is a make clean mrproper, move the .git directory out of the source tree
tar what is left, and then restore the .git directory...
this has the nice side effect of taking along the quilt patches directory,
and sometimes I've found it useful to do individual build/test
between patches in the quilt stack, though I'd probably lose track
of what I'm doing if I did that all the time.

thanks,
-Len

RE: new file leaked onto release branch

[Brown, Len]

[-- Attachment #1: Type: text/plain, Size: 1279 bytes --]

>What do you use to create these "Auto-update" commits, and how
>can I help you to fix it?

I use the attached script, which I pasted from
git/Documentation/howto/using-topic-branches.txt

git.update linus

to get the latest stuff from Linus onto the linus branch

git.update release

to pull linus branch into my release branch

git.update release

to pull linus branch into my test branch

I also use the git.merge script to pull topic branches
into either release or test, or to pull release into test.
I used git.merge to pull release into acpica because
acpica had fallen behind the base at one point, and this
may be where the issue happened.

in the case of the acpica topic branch, it was not pulled
back into either release or into test after the file in question
was added.  Indeed, you can pull the acpica branch from the tree
on kernel.org.

I'm trying to remember if there were git error message along the way.
I do remember an issue with a commit where I had re-named a file
and git didn't recognize it as such.  so I put this in my git.commit
wrapper:

git-udpate-index --add --remove `quilt files`
git commit

dunno if that may be related to the issue at hand.
I don't really understand this index stuff...

thanks,
-Len

[-- Attachment #2: git.update --]
[-- Type: application/octet-stream, Size: 624 bytes --]

#!/bin/bash
# Update a branch in my GIT tree.  If the branch to be updated
# is "linus", then pull from kernel.org.  Otherwise merge local
# linus branch into test|release branch

case "$1" in
test|release)
        git checkout $1 && git merge "Auto-update from upstream" $1 linus
        ;;
linus)
        before=$(cat .git/refs/heads/linus)
        git fetch linus
        after=$(cat .git/refs/heads/linus)
        if [ $before != $after ]
        then
                git-whatchanged $after ^$before | git-shortlog
        fi
        ;;
*)
        echo "Usage: $0 linus|test|release" 1>&2
        exit 1
        ;;
esac

[-- Attachment #3: git.merge --]
[-- Type: application/octet-stream, Size: 549 bytes --]

#!/bin/bash
# Merge a branch into either the test or release branch

pname=$0

usage()
{
        echo "Usage: $pname branch test|release" 1>&2
        exit 1
}

if [ ! -f .git/refs/heads/"$1" ]
then
        echo "Can't see branch <$1>" 1>&2
        usage
fi

case "$2" in
test|release)
        if [ $(git-rev-list $1 ^$2 | wc -c) -eq 0 ]
        then
                echo $1 already merged into $2 1>&2
                exit 1
        fi
        git checkout $2 && git merge "Pull $1 into $2 branch" $2 $1
        ;;
*)
        usage
        ;;
esac

Re: new file leaked onto release branch

[Linus Torvalds]

On Wed, 14 Dec 2005, Junio C Hamano wrote:
> 
> Gahhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.
> 
> Forget about what I said.  Although all of what I said above are
> correct, I do not think it applies to your case at all --- these
> commits do not appear to be created by git-pull nor git-merge,
> and honestly, they look quite bogus.  There is something else
> going on.
> 
>         $ git cat-file commit 9115a6
>         tree da65904b8c0c0bc6c778b39bc25d06e07ca7c82c
>         parent 927fe18397b3b1194a5b26b1d388d97e391e5fd2
>         parent e4f5c82a92c2a546a16af1614114eec19120e40a
>         author Len Brown <len.brown@intel.com> 1133904460 -0500
>         committer Len Brown <len.brown@intel.com> 1133904460 -0500
> 
>         Auto-update from upstream
> 
>         $ git-merge-base 9115a6^1 9115a6^2; git-rev-parse 9115a6^2
>         e4f5c82a92c2a546a16af1614114eec19120e40a
>         e4f5c82a92c2a546a16af1614114eec19120e40a

Argh. 

Len, how old a git are you using? "git-merge" really should _never_ have 
done this, but old (and I mean really old) git merges sometimes got 
confused with the base parent and would create a merge even when it was 
unnecessary.

		Linus

Re: new file leaked onto release branch

[Linus Torvalds]

On Wed, 14 Dec 2005, Linus Torvalds wrote:
> 
> Argh. 
> 
> Len, how old a git are you using? "git-merge" really should _never_ have 
> done this, but old (and I mean really old) git merges sometimes got 
> confused with the base parent and would create a merge even when it was 
> unnecessary.

Going through the real kernel tree, this has happened before:

	commit 8e9887cc3b8d9f1c88c6f3842346a9478e52718f
	Merge: 0a47c906342e2447003e207d23917dfa5c912071 d2149b542382bfc206cb28485108f6470c979566
	Author: Len Brown <len.brown@intel.com>
	Date:   Wed Nov 30 22:22:52 2005 -0500
	
	    Auto-update from upstream

and doing a

	git-merge-base -a 0a47c906342 d2149b542

results in the second parent: d2149b542382bfc206cb28485108f6470c979566.

We also have two old ones:

	commit b129a8ccd53f74c43e4c83c8e0031a4990040830
	commit 603fff54420a0ccc4c3b48bfef43896fb4e33161

but both of those are from June, and of known causes (Russell King using 
an old git that didn't clean up after a failed merge, and he ended up 
committing a perfectly normal change as a bogus merge because git would 
remember that broken MERGE_HEAD).

So Len, since you seem to use "git merge" in your scripts, I suspect you 
have an old version of git lying around. Can you try doing just

	git merge-base -a 0a47c906342e2447003e207d23917dfa5c912071 d2149b542382bfc206cb28485108f6470c979566

to see what the result is for you?

Also, maybe the _reason_ you have an old git lying around is that you have 
two installations - if you have at some point installed both an RPM _and_ 
recompiled your own git, you may have different git versions in ~/bin and 
in /usr/bin, and then depending on how you log in to the machine, you may 
end up with different versions being used..

			Linus

RE: new file leaked onto release branch

[Brown, Len]

 
>Len, how old a git are you using? "git-merge" really should 
>_never_ have done this, but old (and I mean really old)
>git merges sometimes got confused with the base parent
>and would create a merge even when it was unnecessary.

should be the top of tree on the day the merge was made.

I generally pull from git://git.kernel.org/pub/scm/git/git.git
at the beginning of the day, build it and install it.

Right now git --version answers 0.99.9.GIT

-Len

RE: new file leaked onto release branch

[Brown, Len]

[-- Attachment #1: Type: text/plain, Size: 1813 bytes --]

 
>So Len, since you seem to use "git merge" in your scripts, I 
>suspect you have an old version of git lying around. Can you try doing just

Should I be using something different than git merge?
is Documentation/howto/using-topic-branches out of date?

>	git merge-base -a 
>0a47c906342e2447003e207d23917dfa5c912071 
>d2149b542382bfc206cb28485108f6470c979566
>
>to see what the result is for you?

$ git merge-base -a 0a47c906342e2447003e207d23917dfa5c912071 d2149b542382bfc206cb28485108f6470c979566
d2149b542382bfc206cb28485108f6470c979566

>Also, maybe the _reason_ you have an old git lying around is 
>that you have two installations

Doesn't appear to be the case, as I don't have a /usr/bin/git
IIR, months ago I tried to install the rpm and
it failed due to some incompatibility like not groking
a SuSE destination.  I got Dave's git tarball according
to Jeff's howto: http://linux.yyz.us/git-howto.html
and have been building and installing from a git repo since.
(I found git-current tarball dated 7/21/05, so maybe it was then)
I did, however a few months ago copy my i386 home directory over to the
x86_64 box I use now, re-build and re-install.  Dunno
if there may have been a hickup in that process...
I found a backup copy of my i386 bin directory from 2005-08-25 --
binaries still in i386 format.  But I don't think I ran that b/c
it isn't on any PATH.  Git lives in ~/bin which is 1st in my PATH.

I think the lesson I'm taking away from this is that
as I continue to stumble forward using git I should
immediately report anything that doesn't look quite right
while I can still guarantee that all the clues are still
at the scene of the crime.  I expect that I've re-built
and re-installed git several times since the merge
in question was made.

-Len

[-- Attachment #2: to-linus --]
[-- Type: application/octet-stream, Size: 141 bytes --]

git merge-base -a 0a47c906342e2447003e207d23917dfa5c912071 d2149b542382bfc206cb28485108f6470c979566
d2149b542382bfc206cb28485108f6470c979566

RE: new file leaked onto release branch

[Linus Torvalds]

On Wed, 14 Dec 2005, Brown, Len wrote:
>
> >So Len, since you seem to use "git merge" in your scripts, I 
> >suspect you have an old version of git lying around. Can you try doing just
> 
> Should I be using something different than git merge?

No, "git merge" should be fine. It's what "git pull" ends up doing 
internally, which is why I suspected an old git version: "git merge" 
should be well-tested, since it's very much what I end up using every day 
when I pull stuff.

> >	git merge-base -a 0a47c906342e2447003e207d23917dfa5c912071 d2149b542382bfc206cb28485108f6470c979566
> >
> >to see what the result is for you?
> 
> $ git merge-base -a 0a47c906342e2447003e207d23917dfa5c912071 d2149b542382bfc206cb28485108f6470c979566
> d2149b542382bfc206cb28485108f6470c979566

Ok, that's correct.

git-merge does:

	common=$(git-merge-base --all $head "$@")

and then it _should_ have triggered this case:

	case "$#,$common,$no_commit" in
	..
	1,"$1",*)
		# If head can reach all the merge then we are up to date.
		# but first the most common case of merging one remote
		echo "Already up-to-date."
		dropsave
		exit 0
		;;
	..

and thus never have created any merge messages.

That's what I get when I try this:

	git checkout -b test-merge 0a47c906342e2447003e207d23917dfa5c912071
	git merge "Testing merging" HEAD d2149b542382bfc206cb28485108f6470c979566

results in a very immediate

	"Already up-to-date."

message. Does it do that for you too?

I tested not only with current git, but also the gits that were valid on 
Nov 29 and Nov 30. All of them did this.

> Doesn't appear to be the case, as I don't have a /usr/bin/git
> IIR, months ago I tried to install the rpm and
> it failed due to some incompatibility like not groking
> a SuSE destination.  I got Dave's git tarball according
> to Jeff's howto: http://linux.yyz.us/git-howto.html
> and have been building and installing from a git repo since.
> (I found git-current tarball dated 7/21/05, so maybe it was then)
> I did, however a few months ago copy my i386 home directory over to the
> x86_64 box I use now, re-build and re-install.  Dunno
> if there may have been a hickup in that process...
> I found a backup copy of my i386 bin directory from 2005-08-25 --
> binaries still in i386 format.  But I don't think I ran that b/c
> it isn't on any PATH.  Git lives in ~/bin which is 1st in my PATH.

Hmm. It really looks like it should have been impossible to generate that 
commit with current git, which is why I'm still a bit suspicious. 

> I think the lesson I'm taking away from this is that
> as I continue to stumble forward using git I should
> immediately report anything that doesn't look quite right
> while I can still guarantee that all the clues are still
> at the scene of the crime.

I think this list has been pretty responsive to reports of strange 
behaviour, so yes. 

			Linus

Re: new file leaked onto release branch

[Junio C Hamano]

"Brown, Len" <len.brown@intel.com> writes:

> Should I be using something different than git merge?
> is Documentation/howto/using-topic-branches out of date?

I reviewed it once again right now.  The document claims to be
last updated for 0.99.9f, but I do not see anything outdated in
there for the latest.  Tony's procedure looks valid [*1*], so do
the scripts you sent in this thread.

Sorry, but I do not seem to be able to spot anything obviously
wrong with your troubled commits nor scripts.  I'll do some more
digging, including rewinding to an older git and trying them,
but I am pessimistic.

I pointed out one anomaly which is the commit should never have
been created because it was not even a fast forward but already
up-to-date case, and it was followed up with exchange of a few
messages between Linus and you.  But even if we got that mixed
up, the resulting merge should not have contained the file
neither parents had.  That part worries me the most.

One question.  You mentioned these in your message, you have
a "git.commit wrapper" that contains these lines:

    git-update-index --add --remove `quilt files`
    git commit

I am not familiar with 'quilt', but is "quilt files" the command
to show the list of files with patches applied to the working
tree?

If so, the above do tell git about the modified (including added
or removed) files that the applied quilt patches touch, which
sounds like the correct thing to do.

But the resulting commit from that procedure would not be a
merge commit, and the commit in question that had the rsinfo
file magically appeared from nowhere is a merge, so this does
not seem to have much to do with the current problem...

Still puzzlled, sorry.


[Footnote]

*1* Except that the rsync transport is probably suboptimal for
people who stay reasonably up-to-date with Linus and I would
apply the following change if I were Tony, but that shouldn't
have anything to do with the trouble we are discussing here.

---
diff --git a/Documentation/howto/using-topic-branches.txt b/Documentation/howto/using-topic-branches.txt
index 4698abe..4944297 100644
--- a/Documentation/howto/using-topic-branches.txt
+++ b/Documentation/howto/using-topic-branches.txt
@@ -31,7 +31,7 @@ test tree and then pull to the release t
 patches blocked in the test tree waiting for complex changes to accumulate
 enough test time to graduate.
 
-Back in the BitKeeper days I achieved this my creating small forests of
+Back in the BitKeeper days I achieved this by creating small forests of
 temporary trees, one tree for each logical grouping of patches, and then
 pulling changes from these trees first to the test tree, and then to the
 release tree.  At first I replicated this in GIT, but then I realised
@@ -42,7 +42,8 @@ So here is the step-by-step guide how th
 
 First create your work tree by cloning Linus's public tree:
 
- $ git clone rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git work
+ $ git clone \
+   master.kernel.org:/pub/scm/linux/kernel/git/torvalds/linux-2.6.git work
 
 Change directory into the cloned tree you just created
 
@@ -52,7 +53,7 @@ Set up a remotes file so that you can fe
 branch into a local branch named "linus":
 
  $ cat > .git/remotes/linus
- URL: rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
+ URL: master.kernel.org:/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
  Pull: master:linus
  ^D
 

Re: new file leaked onto release branch

[Tom Prince]

On Wed, Dec 14, 2005 at 12:45:51PM -0800, Junio C Hamano wrote:
> 
> I pointed out one anomaly which is the commit should never have
> been created because it was not even a fast forward but already
> up-to-date case, and it was followed up with exchange of a few
> messages between Linus and you.
> 

I don't remember any of the details now, but I remember that an old
version of git or cogito would create bogus fast-forward merges, if they
were used without GNU coreutils. The machine it happend on was running
FreeBSD 4.10, but current versions work fine.

  Tom

RE: new file leaked onto release branch

[Brown, Len]

>> I pointed out one anomaly which is the commit should never have
>> been created because it was not even a fast forward but already
>> up-to-date case, and it was followed up with exchange of a few
>> messages between Linus and you.
>> 
>
>I don't remember any of the details now, but I remember that an old
>version of git or cogito would create bogus fast-forward 
>merges, if they
>were used without GNU coreutils. The machine it happend on was running
>FreeBSD 4.10, but current versions work fine.

I'm running SuSE Linux 10.0 x86_64 on this box
# rpm -q coreutils
coreutils-5.3.0-20

RE: new file leaked onto release branch

[Brown, Len]

 
>That's what I get when I try this:
>
>	git checkout -b test-merge 0a47c906342e2447003e207d23917dfa5c912071
>	git merge "Testing merging" HEAD d2149b542382bfc206cb28485108f6470c979566
>
>results in a very immediate
>
>	"Already up-to-date."
>
>message. Does it do that for you too?

Yes.

>I think this list has been pretty responsive to reports of strange 
>behaviour, so yes. 

Agreed.
If I see anything strange like this in the future I'll report it immediately.

thanks,
-Len

RE: new file leaked onto release branch

[Brown, Len]

 
>I pointed out one anomaly which is the commit should never have
>been created because it was not even a fast forward but already
>up-to-date case, and it was followed up with exchange of a few
>messages between Linus and you.  But even if we got that mixed
>up, the resulting merge should not have contained the file
>neither parents had.  That part worries me the most.
>
>One question.  You mentioned these in your message, you have
>a "git.commit wrapper" that contains these lines:
>
>    git-update-index --add --remove `quilt files`
>    git commit
>
>I am not familiar with 'quilt', but is "quilt files" the command
>to show the list of files with patches applied to the working
>tree?

yes, but only by the last applied "on the top of the stack."
When I do a commit, there is only 1 patch applied so this works okay.
(then I push the next patch after the commit and repeat...)

>If so, the above do tell git about the modified (including added
>or removed) files that the applied quilt patches touch, which
>sounds like the correct thing to do.
>
>But the resulting commit from that procedure would not be a
>merge commit, and the commit in question that had the rsinfo
>file magically appeared from nowhere is a merge, so this does
>not seem to have much to do with the current problem...
>
>Still puzzlled, sorry.

I'm not stuck by this problem, since I created a new release
branch based before the funny merge and it seems to be
working properly.  I'm sorry I can't re-create the problem
again.  Next time something like this happens and I notice it
I'll be sure to keep the ingredients for the failure on hand.

thanks,
-Len

Re: new file leaked onto release branch

[Junio C Hamano]

"Brown, Len" <len.brown@intel.com> writes:

> I'm not stuck by this problem, since I created a new release
> branch based before the funny merge and it seems to be
> working properly.  I'm sorry I can't re-create the problem
> again.  Next time something like this happens and I notice it
> I'll be sure to keep the ingredients for the failure on hand.

Thanks.

BTW, are 5165, 3410, 5452, 5571... topic branch names?

RE: new file leaked onto release branch

[Brown, Len]

>BTW, are 5165, 3410, 5452, 5571... topic branch names?

yes, the are bugzilla ids

RE: new file leaked onto release branch

[Johannes Schindelin]

Hi,

On Wed, 14 Dec 2005, Brown, Len wrote:

> >BTW, are 5165, 3410, 5452, 5571... topic branch names?
> 
> yes, the are bugzilla ids

So, it could have been

	git pull . 5165

which mistakes 5165 for a short SHA1?

Hth,
Dscho

Re: new file leaked onto release branch

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> So, it could have been
>
> 	git pull . 5165
>
> which mistakes 5165 for a short SHA1?

I do not think git-pull would work on arbitrary SHA1
expressions, so you should be safe.

Interestingly...

        $ git rev-parse 5165
        error: short SHA1 5165 is ambiguous.
        5165

that short SHA1 is ambiguous.  But a branch name immediately
under .git/refs/heads takes precedence:

        $ git branch 5165 master
        $ git rev-parse 5165 master
        acd9b7b4e08a3f0f48afa922d8e371414cf2d3b2
        acd9b7b4e08a3f0f48afa922d8e371414cf2d3b2

And this makes it safer and unambiguous:

        $ git branch -d 5165
        Deleted branch 5165.
        $ git branch bug/5165 master
        $ git rev-parse bug/5165
        acd9b7b4e08a3f0f48afa922d8e371414cf2d3b2

We might want to detect collisions between SHA1 prefix and
branch names, but I am not sure if it is worth it in practice.

Re: new file leaked onto release branch

[Johannes Schindelin]

Hi,

On Wed, 14 Dec 2005, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > So, it could have been
> >
> > 	git pull . 5165
> >
> > which mistakes 5165 for a short SHA1?
> 
> I do not think git-pull would work on arbitrary SHA1
> expressions, so you should be safe.

D'oh. I wanted to write it short, but I guess that made it only more 
confusing. In reality, Len used "git merge", and that works quite well 
with short SHA1s.

However, I just verified (as you did also), that they do not take 
precedence over branch names (the relevant piece of code is in get_sha1_1: 
get_short_sha1() is called only if get_sha1_basic() fails).

> Interestingly...
> 
>         $ git rev-parse 5165
>         error: short SHA1 5165 is ambiguous.
>         5165
> 
> that short SHA1 is ambiguous.

I don't want to be a PITA, but it could be ambiguous only since short time 
ago.

> But a branch name immediately under .git/refs/heads takes precedence:
>
>         $ git branch 5165 master
>         $ git rev-parse 5165 master
>         acd9b7b4e08a3f0f48afa922d8e371414cf2d3b2
>         acd9b7b4e08a3f0f48afa922d8e371414cf2d3b2

There is an interesting side effect: If 5165 as a short SHA1 would be 
unique, and there is a tag *and* a branch named 5165, git-rev-parse would 
expand the short SHA1...

However, it still does not solve the original riddle.

Ciao,
Dscho

Re: new file leaked onto release branch

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> git-merge does:
>
> 	common=$(git-merge-base --all $head "$@")
>
> and then it _should_ have triggered this case:
>
> 	case "$#,$common,$no_commit" in
> 	..
> 	1,"$1",*)
> 		# If head can reach all the merge then we are up to date.
> 		# but first the most common case of merging one remote
> 		echo "Already up-to-date."
> 		dropsave
> 		exit 0
> 		;;
> 	..
>
> and thus never have created any merge messages.
>...
> Hmm. It really looks like it should have been impossible to generate that 
> commit with current git, which is why I'm still a bit suspicious. 

Two good news (one puzzle fully explained, one bug fixed) and
one not so good news (one puzzle still remains).

First good news.  I solved this puzzle.  This has been fixed as
a part of a seemingly independent fix:

    commit 9954f5b876abb6118f9bdf1d113239d86acca7bd
    Author: Junio C Hamano <junkio@cox.net>
    Date:   Tue Dec 13 17:01:23 2005 -0800

        [PATCH] allow merging any committish

        Although "git-merge" is advertised as the end-user level command
        (instead of being a "git-pull" backend), it was not prepared to
        take tag objects that point at commits and barfed when fed one.
        Sanitize the input while we validate them, for which we already
        have a loop.

        Signed-off-by: Junio C Hamano <junkio@cox.net>

There was a bug in git-merge which used the user input without
converting them to object names.  When the part you quoted above
was executed, $1..${$#} were remote ref parameters from the
command line, so in the case of Len's commit, which did:

      git merge "Auto-update from upstream" release linus

"$1" at that point was string "linus", not the object name
returned from "git-rev-parse --verify linus".  The case pattern
match did not match because $common was object name and $1 was
not.  This was fixed by the above commit; the user supplied refs
are already converted into object names at that point with the
current code.

I have never seen this problem myself because git-pull feeds
object names after converting refnames to git-merge, but people
who used the git-merge command themselves could have been
affected by the bug.

So I think I am done with the "this is "already-up-to-date"; why
does that commit exists in the first place?" commit we have
discussed in this thread.

Second good news.  I have been working on a theory on the "where
did this file come from?" problem.  I found a real bug that can
cause a bad mismerge that can introduce completely unrelated
changes to the tree, but after digging a bit deeper, I do not
think it matches Len's problematic commit.  It still is a bug.

If you run the sequence attached at the end in an empty
repository, you will have a repository suitable for this
demonstration.  After the script runs, the commit structure
would look like this:

! [heads/7589] add xyzzy
 * [master] Merge 7589 branch
  ! [nitfol] add nitfol
---
+   [8eec60c] add xyzzy			<tag 7589>
 +  [db5bc99] edit frotz
  + [758916c] add nitfol
+++ [70c4319] initial

There are three branches: master, nitfol, and "7589".
They all start from the initial commit which has one file
"frotz" and each branch adds one commit.  Also the tip of 7589
branch is tagged as "7589".  Now, we will run this:

	$ git merge "Merge 7589 branch" HEAD 7589

With this setup, the current tip of the "master" branch
mismerges and adds "nitfol" file which did not exist in either
branch heads (and it is not fixed with the 9954f5 commit above).

A change I introduced mid November causes get_sha1_basic() to
misinterpret "7589" to be neither the tag 7589 nor branch 7589
tip, but by mistake it does not outright fail, but returns the
758916c commit!  This merge ends up pulling nitfol branch head
into master branch, not 7589 branch as the user intended.  The
resulting merge commit has db5bc99 and 758916c as its parents.

The "revert misguided disambiguation" patch I posted earlier
fixes this problem.  I'll push it out tonight.

This theory however does not seem to match what really happened.
Len did mention that he has "5165" branch (there is a commit
marked "Pull 5165 into release branch" near a problematic
merge), but he did not say he also has a 5165 tag; the bug does
not trigger if you do not have the tag of the same name.  Also
if this theory holds true, the problematic commit should have a
commit whose object name begins with 5165 as the second parent
but that is not the case.  And the problem happened with a
commit that is not a merge between release/test and topic
branch anyway; it is with an "Auto-update from upstream" commit.

So I am still puzzled by the "where did this file come from"
problem.  The most plausible explanation was the driver error
mentioned already in the thread: "update-index --add" in the
middle of merge with manual committing.

----------------------------------------------------------------
#!/bin/sh

GIT_AUTHOR_DATE='1995-01-29T15:00:00 -0800'
GIT_AUTHOR_EMAIL='author@example.com'
GIT_AUTHOR_NAME='A U Thor'
GIT_COMMITTER_DATE='1995-01-29T15:00:00 -0800'
GIT_COMMITTER_EMAIL='committer@example.com'
GIT_COMMITTER_NAME='C O Mmitter'

export GIT_AUTHOR_DATE
export GIT_AUTHOR_EMAIL
export GIT_AUTHOR_NAME
export GIT_COMMITTER_DATE
export GIT_COMMITTER_EMAIL
export GIT_COMMITTER_NAME

git init-db

echo frotz >frotz
git add frotz
git commit -m 'initial'

git checkout -b nitfol
echo nitfol >nitfol
git add nitfol
git commit -m 'add nitfol'

git checkout -b 7589 master
echo xyzzy >xyzzy
git add xyzzy
git commit -m 'add xyzzy'
git tag 7589

git checkout master
echo FROTZ >frotz
git update-index frotz
git commit -m 'edit frotz'

RE: new file leaked onto release branch

[Brown, Len]

 
>This theory however does not seem to match what really happened.
>Len did mention that he has "5165" branch (there is a commit
>marked "Pull 5165 into release branch" near a problematic
>merge), but he did not say he also has a 5165 tag; the bug does
>not trigger if you do not have the tag of the same name.  Also
>if this theory holds true, the problematic commit should have a
>commit whose object name begins with 5165 as the second parent
>but that is not the case.  And the problem happened with a
>commit that is not a merge between release/test and topic
>branch anyway; it is with an "Auto-update from upstream" commit.

Per Tony's topic-branches howto, I do plenty of branching,
where the topic branch name , #### below , is a 4-digit number:

git checkout -b #### linus

and I use the wrapper script to do this:

git checkout test && git merge "Pull #### into test branch" test ####
git checkout release && git merge "Pull #### into release branch" release ####


But
1. I don't create tags with #### names, indeed, I never use tags at all
2. I agree that while #### names may be an area of a potential
   problem, it may not be related to the observed failure.  This is
   because the file in question was added to to a topic branch called "acpica"
   which IIR never interacted in any way with a branched named ####,
   but only with the branches called "linus" and "test".

>So I am still puzzled by the "where did this file come from"
>problem.  The most plausible explanation was the driver error
>mentioned already in the thread: "update-index --add" in the
>middle of merge with manual committing.

I too suspect that a failed automatic merge is related.
Likely the sequence of events is not something that either
a git expert or a git beginner would ever issue, but takes
somebody like me who knows just enough to be dangerous:-)

-Len

Re: new file leaked onto release branch

[Junio C Hamano]

"Brown, Len" <len.brown@intel.com> writes:

> 2. I agree that while #### names may be an area of a potential
>    problem, it may not be related to the observed failure.

I made it a potential problem by mistake and left it so for a
couple of weeks, but there was no reason for #### names to be a
problem.  Even when #### is a unique prefix of a valid object
name, if you have .git/refs/heads/#### branch, that should be
the one that takes precedence over a random object name, and
that is what the fixed code does.  Sorry about the earlier
breakage.

Just please keep in mind that the search order is "tags then
heads" (see sha1_name.c::get_sha1_basic()::prefix[]), so if you
happen to have #### branch *and* tag then you will be merging
the tag with this:

> git checkout test && git merge "Pull #### into test branch" test ####

If the git.merge wrapper you posted earlier is always used to
pull a topic branch, it might be safer to update it to do:

	git checkout "$2" &&
        git merge "Pull $1 into $2 branch" HEAD "refs/heads/$1"

to disambiguate.