git-bisect not working correctly in the kvm.git repository?

git-bisect not working correctly in the kvm.git repository?

[walt]

I'm trying to find a bug introduced somewhere between kvm-69
and kvm-70, so of course I want to git-bisect down to it.

The weirdness is that when I give git-bisect the good and
bad tags, almost everything in my working directory just
disappears, leaving only the qemu subdirectory behind.

Can anyone else reproduce this with the kvm.git repo?
(I don't see this problem with the linux.git repo or the
git.git repo, so it seems to be something with the kvm.git
repo itself.)

Thanks!

Re: git-bisect not working correctly in the kvm.git repository?

[Linus Torvalds]

On Mon, 17 Nov 2008, walt wrote:
>
> I'm trying to find a bug introduced somewhere between kvm-69
> and kvm-70, so of course I want to git-bisect down to it.
> 
> The weirdness is that when I give git-bisect the good and
> bad tags, almost everything in my working directory just
> disappears, leaving only the qemu subdirectory behind.

Have you done subdirectory merges or other odd merges?

If you merge stuff from another project that isn't a full repository, then 
you are merging commits that simply don't _have_ the full tree of your 
repo. If you then bisect into those commits, the checkout will then (of 
course) end up having only the files that are in that partial tree.

This will happen in the git repository too, if "git bisect" ends up 
jumping into one of the commits that were done in the 'gitk' repository. 

For an example of this, do something like

	git clean -dqfx
	git checkout -b oddbranch eb33a67f218b612f6fb1456e19b40a1f97ff02c0

in the git repository, and see all files disappear, and gitk is no longer 
in the 'gitk-git' subdirectory, but got moved to the top level.

Because in the history that is that commit, that is literally how things 
were. The data from that commit just got merged into a subdirectory of its 
own, and together with all the rest of git, by using a subtree merge.

			Linus

Re: git-bisect not working correctly in the kvm.git repository?

[Christian Couder]

Le lundi 17 novembre 2008, walt a écrit :
> I'm trying to find a bug introduced somewhere between kvm-69
> and kvm-70, so of course I want to git-bisect down to it.
>
> The weirdness is that when I give git-bisect the good and
> bad tags, almost everything in my working directory just
> disappears, leaving only the qemu subdirectory behind.
>
> Can anyone else reproduce this with the kvm.git repo?
> (I don't see this problem with the linux.git repo or the
> git.git repo, so it seems to be something with the kvm.git
> repo itself.)

I cannot reproduce this:

$ git bisect start kvm-70 kvm-69
Bisecting: 846 revisions left to test after this
[80bfc25f42db6d4715c7688ae2352c5a8038fe7e] ntfs: le*_add_cpu conversion
$ find . -type f |wc -l
24286
$ git bisect reset
Previous HEAD position was 80bfc25... ntfs: le*_add_cpu conversion
Switched to branch "master"
$ find . -type f |wc -l
24384

Could you give more information, like your git version and the result of the 
above commands?

Thanks,
Christian.

Re: git-bisect not working correctly in the kvm.git repository?

[walt]

Linus Torvalds wrote:
>
> On Mon, 17 Nov 2008, walt wrote:
>> I'm trying to find a bug introduced somewhere between kvm-69
>> and kvm-70, so of course I want to git-bisect down to it.
>>
>> The weirdness is that when I give git-bisect the good and
>> bad tags, almost everything in my working directory just
>> disappears, leaving only the qemu subdirectory behind.

> ...
> If you merge stuff from another project that isn't a full repository, then
> you are merging commits that simply don't _have_ the full tree of your
> repo...

Okay, that's clearly what's happening to me.  The kvm.git repo
contains hundred of commits just like that.  How would you deal
with the same situation?  Is git-bisect skip the easiest way?

Re: git-bisect not working correctly in the kvm.git repository?

[Linus Torvalds]

On Mon, 17 Nov 2008, walt wrote:
> 
> Okay, that's clearly what's happening to me.  The kvm.git repo
> contains hundred of commits just like that.  How would you deal
> with the same situation?  Is git-bisect skip the easiest way?

Yeah. A much fancier alternative is to do a (temporary) branch that 
rewrites the history using the same sub-tree merge strategy, so that you 
can actually build the whole thing in that branch. 

This, btw, is why I asked Chris Mason to not do a subtree merge for the 
btrfs code. I wrote him a helper script to do exactly that with a nice 
index script for 'git filter-branch'. That way his whole tree is nicely 
bisectable when merged into the rest of the kernel.

But that's really a rather bigger thing to do, and I'd suggest that only 
if you really can't find things any other way.

		Linus

Re: git-bisect not working correctly in the kvm.git repository?

[walt]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1933 bytes --]



On Mon, 17 Nov 2008, Christian Couder wrote:

> Le lundi 17 novembre 2008, walt a écrit :
> > I'm trying to find a bug introduced somewhere between kvm-69
> > and kvm-70, so of course I want to git-bisect down to it.
> >
> > The weirdness is that when I give git-bisect the good and
> > bad tags, almost everything in my working directory just
> > disappears, leaving only the qemu subdirectory behind.
> >
> > Can anyone else reproduce this with the kvm.git repo?
> > (I don't see this problem with the linux.git repo or the
> > git.git repo, so it seems to be something with the kvm.git
> > repo itself.)
>
> I cannot reproduce this:
>
> $ git bisect start kvm-70 kvm-69
> Bisecting: 846 revisions left to test after this
> [80bfc25f42db6d4715c7688ae2352c5a8038fe7e] ntfs: le*_add_cpu conversion
> $ find . -type f |wc -l
> 24286
> $ git bisect reset
> Previous HEAD position was 80bfc25... ntfs: le*_add_cpu conversion
> Switched to branch "master"
> $ find . -type f |wc -l
> 24384

Your numbers are so different from mine I'm wondering if we are using
different repositories.

$~/src/kvm-userspace find . -type f | wc -l
1715

$~/src/kvm-userspace git bisect start kvm-70 kvm-69
Bisecting: 176 revisions left to test after this
[06508be8108f785caead849a717c9e6d67da62d7] Un-break MIPS conditional moves, by
Richard Sandiford.
wa1ter@k9:~/src/kvm-userspace find . -type f | wc -l
1413

$~/src/kvm-userspace cat .git/config
[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[remote "origin"]
        url = git://git.kernel.org/pub/scm/virt/kvm/kvm-userspace.git
        fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
        remote = origin
        merge = refs/heads/master

I track Junio's git.git and it's current as of this morning, but I
also tried git v1.5.0 (chosen at random) and got the same results.
What repo are you using?

Thanks.

Re: git-bisect not working correctly in the kvm.git repository?

[Christian Couder]

Le lundi 17 novembre 2008, walt a écrit :
> On Mon, 17 Nov 2008, Christian Couder wrote:
> > Le lundi 17 novembre 2008, walt a écrit :
> > > I'm trying to find a bug introduced somewhere between kvm-69
> > > and kvm-70, so of course I want to git-bisect down to it.
> > >
> > > The weirdness is that when I give git-bisect the good and
> > > bad tags, almost everything in my working directory just
> > > disappears, leaving only the qemu subdirectory behind.
> > >
> > > Can anyone else reproduce this with the kvm.git repo?
> > > (I don't see this problem with the linux.git repo or the
> > > git.git repo, so it seems to be something with the kvm.git
> > > repo itself.)
> >
> > I cannot reproduce this:
> >
> > $ git bisect start kvm-70 kvm-69
> > Bisecting: 846 revisions left to test after this
> > [80bfc25f42db6d4715c7688ae2352c5a8038fe7e] ntfs: le*_add_cpu conversion
> > $ find . -type f |wc -l
> > 24286
> > $ git bisect reset
> > Previous HEAD position was 80bfc25... ntfs: le*_add_cpu conversion
> > Switched to branch "master"
> > $ find . -type f |wc -l
> > 24384
>
> Your numbers are so different from mine I'm wondering if we are using
> different repositories.
>
> $~/src/kvm-userspace find . -type f | wc -l
> 1715
>
> $~/src/kvm-userspace git bisect start kvm-70 kvm-69
> Bisecting: 176 revisions left to test after this
> [06508be8108f785caead849a717c9e6d67da62d7] Un-break MIPS conditional
> moves, by Richard Sandiford.
> wa1ter@k9:~/src/kvm-userspace find . -type f | wc -l
> 1413
>
> $~/src/kvm-userspace cat .git/config
> [core]
>         repositoryformatversion = 0
>         filemode = true
>         bare = false
>         logallrefupdates = true
> [remote "origin"]
>         url = git://git.kernel.org/pub/scm/virt/kvm/kvm-userspace.git

I cloned this one:

[remote "origin"]
        url = git://git.kernel.org/pub/scm/linux/kernel/git/amit/kvm.git

Sorry I didn't know which should be used, as I am not a kernel developer.

Regards,
Christian.

>         fetch = +refs/heads/*:refs/remotes/origin/*
> [branch "master"]
>         remote = origin
>         merge = refs/heads/master
>
> I track Junio's git.git and it's current as of this morning, but I
> also tried git v1.5.0 (chosen at random) and got the same results.
> What repo are you using?
>
> Thanks.

Re: git-bisect not working correctly in the kvm.git repository?

[Johannes Sixt]

walt schrieb:
> Linus Torvalds wrote:
>>
>> On Mon, 17 Nov 2008, walt wrote:
>>> I'm trying to find a bug introduced somewhere between kvm-69
>>> and kvm-70, so of course I want to git-bisect down to it.
>>>
>>> The weirdness is that when I give git-bisect the good and
>>> bad tags, almost everything in my working directory just
>>> disappears, leaving only the qemu subdirectory behind.
> 
>> ...
>> If you merge stuff from another project that isn't a full repository,
>> then
>> you are merging commits that simply don't _have_ the full tree of your
>> repo...
> 
> Okay, that's clearly what's happening to me.  The kvm.git repo
> contains hundred of commits just like that.  How would you deal
> with the same situation?  Is git-bisect skip the easiest way?

If you are pretty sure that the bug you are looking for was *not*
introduced in the branch that was subtree-merged, you can mark it as
"good" like so:

  $ git bisect start kvm-69 kvm-70 $BRANCH

where $BRANCH is the most recent commit on the subtree-merged branch.
Watch out that you do *not* accidentally pick the *merge* commit that
merged the branch in, but the (second) parent[*], the one that was merged
in.  Then your bisect sequence won't ever end up in that branch.

[*] If you happen to have the branch lying around as a remote ref, you can
use that instead, even if it is ahead of what was merged in.

-- Hannes

Re: git-bisect not working correctly in the kvm.git repository?

[Avi Kivity]

Johannes Sixt wrote:
> If you are pretty sure that the bug you are looking for was *not*
> introduced in the branch that was subtree-merged, you can mark it as
> "good" like so:
>
>   $ git bisect start kvm-69 kvm-70 $BRANCH
>
> where $BRANCH is the most recent commit on the subtree-merged branch.
> Watch out that you do *not* accidentally pick the *merge* commit that
> merged the branch in, but the (second) parent[*], the one that was merged
> in.  Then your bisect sequence won't ever end up in that branch.
>   

Unfortunately, it is fairly likely that the bug was indeed introduced by 
the merge, since most of the commits in that repository come via that merge.

It is possible that the bug exists in that branch (which you need to 
build as pure qemu, not kvm), but again, it is more likely that the bug 
is an interaction between a qemu feature introduced during the merge, 
and the existing kvm glue code.

One way around this is to redo the merge from the point git bisect 
suggests, but that is difficult, time consuming, and error prone.  It 
also needs a lot of familiarity with the code.

Fortunately, long term the problem is going away with qemu upstream 
gaining kvm support.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.