git bisect with history manipulation

git bisect with history manipulation

[Kalle Pokki]

Hi,

I'm still pretty new with git, and cannot quite figure out how to use
"git bisect" effectively in this special case: I'm running an embedded
powerpc board, to which I need about a dozen platform patches in the
kernel. Originally I made the patches with quilt on top of 2.6.15.4. I
recently started using git, and just applied the patches on top of
v2.6.18. However, the system seems to oops at every boot now. So I did
"git branch downgrade && git reset --hard v2.6.15" and applied my
patches on top of it to create a starting state similar to what I had
previously. There everything is ok.

Wanting to try to bisect the kernel versions, I then merged the master
branch into the downgrade branch. Then I marked my last platform
commit as good, and v2.6.18 as bad. However the bisect algorithm seems
to group the platform patches near v2.6.18 instead of v2.6.15, since I
don't have the platform files in the bisect checkout. And since I
don't have the platform files, I can't compile a kernel that would run
on my board.

So is there any way to insert a few patches to an arbitrary point
backwards in time and start bisecting from that to the present time?
Or am I thinking this somehow all wrong?

Re: git bisect with history manipulation

[Linus Torvalds]

On Mon, 23 Oct 2006, Kalle Pokki wrote:
> 
> So is there any way to insert a few patches to an arbitrary point
> backwards in time and start bisecting from that to the present time?
> Or am I thinking this somehow all wrong?

It's possible, but you have to do it the patch-application manually at 
each stage, so it's not a lot of fun. I've had to do it for the opposite 
reason: hunting down a bug with "git bisect" when there was _another_ 
unrelated bug that I already had fixed, but that made it impossible to 
test for the first one. Again, in order to see the first one, I did git 
bisect, but then at each stage applied the fix for the second one if 
needed.

So what you would do is to simply do the bisect _as_if_ those patches 
weren't there, and then apply the patches on top of the kernel that "git 
bisect" suggests. Then (and this is important), when you mark the result 
good or bad, don't use just "git bisect good/bad" - but name explicitly 
the commit that you applied your patch-series on.

(This gets a bit easier if you instead of actually cherry-picking the 
patches you want to apply, just apply it as a single patch _without_ 
committing it - then all your changes will be effectively "invisible" to 
git bisect anyway, and you don't need to do much of anything special, 
except do a "git checkout -f" to remove the patch before you say "that was 
bad" or "that was good").

HOWEVER. It's quite possibly easier to just do it the other way: if your 
series of patches causes problems when rebased on top of a newer kernel, 
just bisect your rebased series itself (at which point it's all a totally 
normal "git bisect", and you don't need to do anything special). It may be 
that once you see _which_ patch in the series caused problems, you'll 
immediately say "Oh, duh, that got mis-merged" or (perhaps more likely) by 
pointing to a particular commit it will point to a certain area that 
changed and you suddenly realize _why_ the series caused a problem.

So depending on the problem, you can try two different approaches.

It might make sense to extend "git bisect" with a "apply this patch at 
each point" capability, but that doesn't exist right now. If you do end up 
going that way, and automate it, please do send the patches out for 
discussion.

		Linus

Re: git bisect with history manipulation

[Jakub Narebski]

Linus Torvalds wrote:

> So depending on the problem, you can try two different approaches.

[The approaches being: 1) applying patchseries before testing, and marking
the commit before applying as good or bad for bisect; 2) rebasing
(applying) the patch-series on top of current kernel, and bisecting the
series]

You can try yet another approach, namely rebase v2.6.15..v2.6.18 on top of
your patch-series applied to v2.6.15, and bisect that.

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: git bisect with history manipulation

[Linus Torvalds]

On Mon, 23 Oct 2006, Jakub Narebski wrote:
> 
> You can try yet another approach, namely rebase v2.6.15..v2.6.18 on top of
> your patch-series applied to v2.6.15, and bisect that.

Oooh. 

Yeah, it's a great idea, but likely not practical (or even possible).

git-rebase really wants a linear set of patches to rebase, which you 
definitely don't have for that big history. In fact, even if rebase did 
the full history rebasing, if later history did a merge of an earlier 
thing (ie if the patch-series that you're trying to rebase onto was based 
on something that wasn't an "epoch tip", but that was in the middle of 
intertwined history), you'd really be screwed.

Also, let's face it, rebasing isn't _that_ fast. So trying to rebase huge 
swaths of code would be painful as hell, even if it was a nice linear 
series.

But yes, for simpler situations, you could try to switch the problem 
around like you suggest.

		Linus

Re: git bisect with history manipulation

[Sean]

On Mon, 23 Oct 2006 17:22:41 +0300
"Kalle Pokki" <kalle.pokki@iki.fi> wrote:

> I'm still pretty new with git, and cannot quite figure out how to use
> "git bisect" effectively in this special case: I'm running an embedded
> powerpc board, to which I need about a dozen platform patches in the
> kernel. Originally I made the patches with quilt on top of 2.6.15.4. I
> recently started using git, and just applied the patches on top of
> v2.6.18. However, the system seems to oops at every boot now. So I did
> "git branch downgrade && git reset --hard v2.6.15" and applied my
> patches on top of it to create a starting state similar to what I had
> previously. There everything is ok.
> 
> Wanting to try to bisect the kernel versions, I then merged the master
> branch into the downgrade branch. Then I marked my last platform
> commit as good, and v2.6.18 as bad. However the bisect algorithm seems
> to group the platform patches near v2.6.18 instead of v2.6.15, since I
> don't have the platform files in the bisect checkout. And since I
> don't have the platform files, I can't compile a kernel that would run
> on my board.
> 
> So is there any way to insert a few patches to an arbitrary point
> backwards in time and start bisecting from that to the present time?
> Or am I thinking this somehow all wrong?
> 

Well, there is no way to insert the patches into the history at an old
point so that they're always available at each bisection point in your
search.  The history is immutable.

What you will have to do, is apply your patches manually at each
bisection point before you compile/test.  After testing you'll have
to remove your manual patches and continue with your next git
bisect command.  However, there are a few ways that you can make this
easier to do.

You could use the Stacked Git utility (which is a Quilt like clone
built on top of Git) to help you along in this process.  Actually,
you may be able to do the same thing with Quilt itself, but i don't
know i've never used it.

That said, i'll try to outline one way to do this that should work
using just native Git commands.  The first thing you need to do is
prepare a git branch based on the first known-good commit that 
contains all your extra patches.  So:

$ git checkout -b mypatches v2.6.15
$ patch < patch1
$ git commit -a
$ patch < patch2
$ git commit -a
etc...

Then you can go back to the master branch and start the bisection:

$ git checkout master
$ git bisect start
$ git bisect bad
$ git bisect good v2.6.15

At which point git will create and move you to a temporary "bisect"
branch to do your compile/testing.  Now before you compile, you want
to append your patch series from the branch you prepared earlier, so:

$ git pull . mypatches

Now compile and test.  After you've tested, you will  have to remove
your patches from this branch before continuing with the git bisection
process so:

$ git reset --hard ORIG_HEAD

And now continue with your bisection, let's assume the first bisection
point still didn't work:

$ git bisect bad

Which will move you to a new bisection point where you can go back to
the "git pull . mypatches" step to apply your patch series again and
do another compile/test.  Continue with this process as needed.

Hope that is clear enough,
Sean

Re: git bisect with history manipulation

[Sean]

On Mon, 23 Oct 2006 17:22:41 +0300
"Kalle Pokki" <kalle.pokki@iki.fi> wrote:

> I'm still pretty new with git, and cannot quite figure out how to use
> "git bisect" effectively in this special case: I'm running an embedded
> powerpc board, to which I need about a dozen platform patches in the
> kernel. Originally I made the patches with quilt on top of 2.6.15.4. I
> recently started using git, and just applied the patches on top of
> v2.6.18. However, the system seems to oops at every boot now. So I did
> "git branch downgrade && git reset --hard v2.6.15" and applied my
> patches on top of it to create a starting state similar to what I had
> previously. There everything is ok.
> 
> Wanting to try to bisect the kernel versions, I then merged the master
> branch into the downgrade branch. Then I marked my last platform
> commit as good, and v2.6.18 as bad. However the bisect algorithm seems
> to group the platform patches near v2.6.18 instead of v2.6.15, since I
> don't have the platform files in the bisect checkout. And since I
> don't have the platform files, I can't compile a kernel that would run
> on my board.
> 
> So is there any way to insert a few patches to an arbitrary point
> backwards in time and start bisecting from that to the present time?
> Or am I thinking this somehow all wrong?
> 

Well, there is no way to insert the patches into the history at an old
point so that they're always available at each bisection point in your
search.  The history is immutable.

What you will have to do, is apply your patches manually at each
bisection point before you compile/test.  After testing you'll have
to remove your manual patches and continue with your next git
bisect command.  However, there are a few ways that you can make this
easier to do.

You could use the Stacked Git utility (which is a Quilt like clone
built on top of Git) to help you along in this process.  Actually,
you may be able to do the same thing with Quilt itself, but i don't
know i've never used it.

That said, i'll try to outline one way to do this that should work
using just native Git commands.  The first thing you need to do is
prepare a git branch based on the first known-good commit that 
contains all your extra patches.  So:

$ git checkout -b mypatches v2.6.15
$ patch < patch1
$ git commit -a
$ patch < patch2
$ git commit -a
etc...

Then you can go back to the master branch and start the bisection:

$ git checkout master
$ git bisect start
$ git bisect bad
$ git bisect good v2.6.15

At which point git will create and move you to a temporary "bisect"
branch to do your compile/testing.  Now before you compile, you want
to append your patch series from the branch you prepared earlier, so:

$ git pull . mypatches

Now compile and test.  After you've tested, you will  have to remove
your patches from this branch before continuing with the git bisection
process so:

$ git reset --hard ORIG_HEAD

And now continue with your bisection, let's assume the first bisection
point still didn't work:

$ git bisect bad

Which will move you to a new bisection point where you can go back to
the "git pull . mypatches" step to apply your patch series again and
do another compile/test.  Continue with this process as needed.

Hope that is clear enough,
Sean

Re: git bisect with history manipulation

[Jakub Narebski]

Sean wrote:

> $ patch < patch1

We have git-apply for that (sometimes git-am would work on whole
series of patches).
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: git bisect with history manipulation

[Sean]

On Mon, 23 Oct 2006 18:19:56 +0200
Jakub Narebski <jnareb@gmail.com> wrote:

> Sean wrote:
> 
> > $ patch < patch1
> 
> We have git-apply for that (sometimes git-am would work on whole
> series of patches).

Point taken.  git-rebase and git-cherrypick may also be appropriate
to snag the commits from another branch rather than reapplying them
from patches.  In fact you may already have a branch that works
which contains just your patches on top of it.  I was trying to
make the example as clear as possible rather than exhaustive.

Sean

Re: git bisect with history manipulation

[Sean]

On Mon, 23 Oct 2006 18:19:56 +0200
Jakub Narebski <jnareb@gmail.com> wrote:

> Sean wrote:
> 
> > $ patch < patch1
> 
> We have git-apply for that (sometimes git-am would work on whole
> series of patches).

Point taken.  git-rebase and git-cherrypick may also be appropriate
to snag the commits from another branch rather than reapplying them
from patches.  In fact you may already have a branch that works
which contains just your patches on top of it.  I was trying to
make the example as clear as possible rather than exhaustive.

Sean

Re: git bisect with history manipulation

[Catalin Marinas]

Sean <seanlkml@sympatico.ca> wrote:
> On Mon, 23 Oct 2006 17:22:41 +0300
> "Kalle Pokki" <kalle.pokki@iki.fi> wrote:
>
>> So is there any way to insert a few patches to an arbitrary point
>> backwards in time and start bisecting from that to the present time?
>> Or am I thinking this somehow all wrong?
[...]
> You could use the Stacked Git utility (which is a Quilt like clone
> built on top of Git) to help you along in this process.  Actually,
> you may be able to do the same thing with Quilt itself, but i don't
> know i've never used it.

It's on my todo list to actually add bisect support to StGIT (someone
suggested it on the mailing list) but I can't give any estimates about
when this would be done. The idea is that it will only bisect the base
and push the patches on top of the new tree (at a first though, it
doesn't look difficult at all).

Otherwise, use StGIT to manage the patches and the following sequence
for bisecting:

$ git bisect start
$ stg init
$ stg pick <patch@branch>
$ stg pick <patch@branch>
$ ...
$ stg pop -a
$ git bisect good v2.6.x
$ git bisect bad
$ stg push -a
$ ... test ...
$ stg pop -a
$ git bisect good|bad
$ stg push -a
$ ...

-- 
Catalin