Git bisect skip

Git bisect skip

[Michael Gofron]

Hello,

My question is can using `git bisect skip` cause a bisect to be indeterminate and/or fail if the commits that are skipped couldn't have caused the issue? 

Consider if my commits are like this:

1P - 2P - 3B - 4P - 5P - 6B - 7B - 8F - 9F. 
P for "Pass", B for "Broken", and F for "Fail". 
Broken commits are commits that we can't create a build for but wouldn't cause the issue.
Failing commits are failing because of a bug. 

In this case, 8F caused the bug.
If you tell git bisect that 1P is good and 9F is bad, bisect picks a commit between the known newest Good commit (1P) and the known oldest Bad commit (9F).

1P -- 2P - 3B - 4P - 5P - 6B - 7B - 8F - 9F
G                                        B
      <------------------------------>
Perhaps 4P. That builds and passes, so it marks that as Good.

If it then goes to 6B which is a Broken commit and we do `git bisect skip` what happens next? It seems from the code it uses a psuedo random number generator with bias to determine the next commit. Would it ever get in a state where it can't determine the commit that caused the issue even if these broken commits would never cause an issue? 

-Thank you
Michael Gofron

Re: Git bisect skip

[Christian Couder]

Hi,

On Fri, Jun 21, 2024 at 6:01 AM Michael Gofron <gofronm@yahoo.com> wrote:
>
> Hello,
>
> My question is can using `git bisect skip` cause a bisect to be indeterminate and/or fail if the commits that are skipped couldn't have caused the issue?

If some skipped commits haven't caused the issue but they are direct
ancestors of the commit that caused the issue, then git bisect might
not be able to tell which one among those commits and the commit that
caused the issue is the actual commit that caused the issue.

For example if there is the following history:

G1-S1-G2-G3-S2-S3-B1-S4-S5-B2-B3-S6

(where G* are "good" commits, S* are skipped commits and B* are "bad"
commits) then git bisect will not be able to determine which one is
the first bad commit between S2, S3 and B1.

> Consider if my commits are like this:
>
> 1P - 2P - 3B - 4P - 5P - 6B - 7B - 8F - 9F.
> P for "Pass", B for "Broken", and F for "Fail".
> Broken commits are commits that we can't create a build for but wouldn't cause the issue.
> Failing commits are failing because of a bug.

Similarly as the case above, I would say that git bisect will not be
able to determine which one is the first bad between 6B, 7B and 8F.

> In this case, 8F caused the bug.
> If you tell git bisect that 1P is good and 9F is bad, bisect picks a commit between the known newest Good commit (1P) and the known oldest Bad commit (9F).

Yeah, it should pick one in the middle, so likely 4P, 5P or 6B.

> 1P -- 2P - 3B - 4P - 5P - 6B - 7B - 8F - 9F
> G                                        B
>       <------------------------------>
> Perhaps 4P. That builds and passes, so it marks that as Good.
>
> If it then goes to 6B which is a Broken commit and we do `git bisect skip` what happens next? It seems from the code it uses a psuedo random number generator with bias to determine the next commit.

Yeah, because it tries to avoid testing broken commits that are likely
to be around the broken commit it already picked, while at the same
time it tries to be efficient which means to pick a commit near the
middle of the range of commits left.

> Would it ever get in a state where it can't determine the commit that caused the issue even if these broken commits would never cause an issue?

It can't know that the broken commits that are direct ancestors of a
"bad" (or "failed" commit as you call them) would never cause the
issue, so it should eventually say that it can't determine which
commit is the first bad commit among 6B, 7B and 8F.

If you are sure that some broken commits don't have the issue you can
mark them as "good" to help git bisect which should then be able to
find the first "bad" commit.

Best,
Christian.

Re: Git bisect skip

[Michael Gofron]

Thank you. This was very detailed. 

Another question I have is does the exclude filepath include commits which have changes in the excluded filepaths and in the included filepaths? 

I noticed for instance I could specify to include and exclude changes to certain files with a format like the following: 

`git bisect start bad_commit good_commit -- 'mobile/' ':!mobile/tests.yaml' ':!mobile/GeneratedSources``

The includes filepath is any changes in `mobile` and the exclude is changes in `mobile/tests.yaml` and `mobile/GeneratedSources`. 

In the prior example, I'd like to ignore commits where the only changes are in `mobile/tests.yaml` and/or `mobile/GeneratedSources`. If there were a change in `mobile/tests.yaml` and a change in say `mobile/highly_important_fragile_code.swift` would it include that in the bisect? 

-Thank you
Michael Gofron






On Friday, June 21, 2024 at 04:38:05 PM GMT+9, Christian Couder <christian.couder@gmail.com> wrote: 





Hi,

On Fri, Jun 21, 2024 at 6:01 AM Michael Gofron <gofronm@yahoo.com> wrote:
>
> Hello,
>
> My question is can using `git bisect skip` cause a bisect to be indeterminate and/or fail if the commits that are skipped couldn't have caused the issue?

If some skipped commits haven't caused the issue but they are direct
ancestors of the commit that caused the issue, then git bisect might
not be able to tell which one among those commits and the commit that
caused the issue is the actual commit that caused the issue.

For example if there is the following history:

G1-S1-G2-G3-S2-S3-B1-S4-S5-B2-B3-S6

(where G* are "good" commits, S* are skipped commits and B* are "bad"
commits) then git bisect will not be able to determine which one is
the first bad commit between S2, S3 and B1.

> Consider if my commits are like this:
>
> 1P - 2P - 3B - 4P - 5P - 6B - 7B - 8F - 9F.
> P for "Pass", B for "Broken", and F for "Fail".
> Broken commits are commits that we can't create a build for but wouldn't cause the issue.
> Failing commits are failing because of a bug.

Similarly as the case above, I would say that git bisect will not be
able to determine which one is the first bad between 6B, 7B and 8F.

> In this case, 8F caused the bug.
> If you tell git bisect that 1P is good and 9F is bad, bisect picks a commit between the known newest Good commit (1P) and the known oldest Bad commit (9F).

Yeah, it should pick one in the middle, so likely 4P, 5P or 6B.

> 1P -- 2P - 3B - 4P - 5P - 6B - 7B - 8F - 9F
> G                                        B
>      <------------------------------>
> Perhaps 4P. That builds and passes, so it marks that as Good.
>
> If it then goes to 6B which is a Broken commit and we do `git bisect skip` what happens next? It seems from the code it uses a psuedo random number generator with bias to determine the next commit.

Yeah, because it tries to avoid testing broken commits that are likely
to be around the broken commit it already picked, while at the same
time it tries to be efficient which means to pick a commit near the
middle of the range of commits left.


> Would it ever get in a state where it can't determine the commit that caused the issue even if these broken commits would never cause an issue?


It can't know that the broken commits that are direct ancestors of a
"bad" (or "failed" commit as you call them) would never cause the
issue, so it should eventually say that it can't determine which
commit is the first bad commit among 6B, 7B and 8F.

If you are sure that some broken commits don't have the issue you can
mark them as "good" to help git bisect which should then be able to
find the first "bad" commit.

Best,
Christian.

Re: Git bisect skip

[Michael Gofron]

I ended up testing out the filepath exclusion with a test repo I'd set up with mock data. 

I have two more questions regarding bisect:

1. Is there a way to specify regex with regard to the filepath spec? So I could specify something like `git bisect start bad_commit good_commit ':!*.md'` to avoid changes that only happen in `.md` documentation files? I had tested locally in my git test repo I'd set up and couldn't seem to find a way filter changes with a regex or file extensions rather than specific paths. 
2. Is there a way to check the contents of files and exclude based off a script if the file was changed in a particular way? In my case, I had been thinking if a file only added comments to avoid testing that specific commit.

-Thank you
Michael Gofron






On Saturday, June 22, 2024 at 06:53:50 AM GMT+9, Michael Gofron <gofronm@yahoo.com> wrote: 





Thank you. This was very detailed. 

Another question I have is does the exclude filepath include commits which have changes in the excluded filepaths and in the included filepaths? 

I noticed for instance I could specify to include and exclude changes to certain files with a format like the following: 

`git bisect start bad_commit good_commit -- 'mobile/' ':!mobile/tests.yaml' ':!mobile/GeneratedSources``

The includes filepath is any changes in `mobile` and the exclude is changes in `mobile/tests.yaml` and `mobile/GeneratedSources`. 

In the prior example, I'd like to ignore commits where the only changes are in `mobile/tests.yaml` and/or `mobile/GeneratedSources`. If there were a change in `mobile/tests.yaml` and a change in say `mobile/highly_important_fragile_code.swift` would it include that in the bisect? 

-Thank you
Michael Gofron






On Friday, June 21, 2024 at 04:38:05 PM GMT+9, Christian Couder <christian.couder@gmail.com> wrote: 





Hi,

On Fri, Jun 21, 2024 at 6:01 AM Michael Gofron <gofronm@yahoo.com> wrote:
>
> Hello,
>
> My question is can using `git bisect skip` cause a bisect to be indeterminate and/or fail if the commits that are skipped couldn't have caused the issue?

If some skipped commits haven't caused the issue but they are direct
ancestors of the commit that caused the issue, then git bisect might
not be able to tell which one among those commits and the commit that
caused the issue is the actual commit that caused the issue.

For example if there is the following history:

G1-S1-G2-G3-S2-S3-B1-S4-S5-B2-B3-S6

(where G* are "good" commits, S* are skipped commits and B* are "bad"
commits) then git bisect will not be able to determine which one is
the first bad commit between S2, S3 and B1.

> Consider if my commits are like this:
>
> 1P - 2P - 3B - 4P - 5P - 6B - 7B - 8F - 9F.
> P for "Pass", B for "Broken", and F for "Fail".
> Broken commits are commits that we can't create a build for but wouldn't cause the issue.
> Failing commits are failing because of a bug.

Similarly as the case above, I would say that git bisect will not be
able to determine which one is the first bad between 6B, 7B and 8F.

> In this case, 8F caused the bug.
> If you tell git bisect that 1P is good and 9F is bad, bisect picks a commit between the known newest Good commit (1P) and the known oldest Bad commit (9F).

Yeah, it should pick one in the middle, so likely 4P, 5P or 6B.

> 1P -- 2P - 3B - 4P - 5P - 6B - 7B - 8F - 9F
> G                                        B
>      <------------------------------>
> Perhaps 4P. That builds and passes, so it marks that as Good.
>
> If it then goes to 6B which is a Broken commit and we do `git bisect skip` what happens next? It seems from the code it uses a psuedo random number generator with bias to determine the next commit.

Yeah, because it tries to avoid testing broken commits that are likely
to be around the broken commit it already picked, while at the same
time it tries to be efficient which means to pick a commit near the
middle of the range of commits left.


> Would it ever get in a state where it can't determine the commit that caused the issue even if these broken commits would never cause an issue?


It can't know that the broken commits that are direct ancestors of a
"bad" (or "failed" commit as you call them) would never cause the
issue, so it should eventually say that it can't determine which
commit is the first bad commit among 6B, 7B and 8F.

If you are sure that some broken commits don't have the issue you can
mark them as "good" to help git bisect which should then be able to
find the first "bad" commit.

Best,
Christian.

Re: Git bisect skip

[Michael Gofron]

To add to that I had another question: 

1. Is there a way to preview the next commit that bisect would pick if it were a good commit, a bad commit, or a skipped commit? 

-Thank you






On Tuesday, June 25, 2024 at 01:14:30 PM GMT+9, Michael Gofron <gofronm@yahoo.com> wrote: 





I ended up testing out the filepath exclusion with a test repo I'd set up with mock data. 

I have two more questions regarding bisect:

1. Is there a way to specify regex with regard to the filepath spec? So I could specify something like `git bisect start bad_commit good_commit ':!*.md'` to avoid changes that only happen in `.md` documentation files? I had tested locally in my git test repo I'd set up and couldn't seem to find a way filter changes with a regex or file extensions rather than specific paths. 
2. Is there a way to check the contents of files and exclude based off a script if the file was changed in a particular way? In my case, I had been thinking if a file only added comments to avoid testing that specific commit.

-Thank you
Michael Gofron






On Saturday, June 22, 2024 at 06:53:50 AM GMT+9, Michael Gofron <gofronm@yahoo.com> wrote: 





Thank you. This was very detailed. 

Another question I have is does the exclude filepath include commits which have changes in the excluded filepaths and in the included filepaths? 

I noticed for instance I could specify to include and exclude changes to certain files with a format like the following: 

`git bisect start bad_commit good_commit -- 'mobile/' ':!mobile/tests.yaml' ':!mobile/GeneratedSources``

The includes filepath is any changes in `mobile` and the exclude is changes in `mobile/tests.yaml` and `mobile/GeneratedSources`. 

In the prior example, I'd like to ignore commits where the only changes are in `mobile/tests.yaml` and/or `mobile/GeneratedSources`. If there were a change in `mobile/tests.yaml` and a change in say `mobile/highly_important_fragile_code.swift` would it include that in the bisect? 

-Thank you
Michael Gofron






On Friday, June 21, 2024 at 04:38:05 PM GMT+9, Christian Couder <christian.couder@gmail.com> wrote: 





Hi,

On Fri, Jun 21, 2024 at 6:01 AM Michael Gofron <gofronm@yahoo.com> wrote:
>
> Hello,
>
> My question is can using `git bisect skip` cause a bisect to be indeterminate and/or fail if the commits that are skipped couldn't have caused the issue?

If some skipped commits haven't caused the issue but they are direct
ancestors of the commit that caused the issue, then git bisect might
not be able to tell which one among those commits and the commit that
caused the issue is the actual commit that caused the issue.

For example if there is the following history:

G1-S1-G2-G3-S2-S3-B1-S4-S5-B2-B3-S6

(where G* are "good" commits, S* are skipped commits and B* are "bad"
commits) then git bisect will not be able to determine which one is
the first bad commit between S2, S3 and B1.

> Consider if my commits are like this:
>
> 1P - 2P - 3B - 4P - 5P - 6B - 7B - 8F - 9F.
> P for "Pass", B for "Broken", and F for "Fail".
> Broken commits are commits that we can't create a build for but wouldn't cause the issue.
> Failing commits are failing because of a bug.

Similarly as the case above, I would say that git bisect will not be
able to determine which one is the first bad between 6B, 7B and 8F.

> In this case, 8F caused the bug.
> If you tell git bisect that 1P is good and 9F is bad, bisect picks a commit between the known newest Good commit (1P) and the known oldest Bad commit (9F).

Yeah, it should pick one in the middle, so likely 4P, 5P or 6B.

> 1P -- 2P - 3B - 4P - 5P - 6B - 7B - 8F - 9F
> G                                        B
>      <------------------------------>
> Perhaps 4P. That builds and passes, so it marks that as Good.
>
> If it then goes to 6B which is a Broken commit and we do `git bisect skip` what happens next? It seems from the code it uses a psuedo random number generator with bias to determine the next commit.

Yeah, because it tries to avoid testing broken commits that are likely
to be around the broken commit it already picked, while at the same
time it tries to be efficient which means to pick a commit near the
middle of the range of commits left.


> Would it ever get in a state where it can't determine the commit that caused the issue even if these broken commits would never cause an issue?


It can't know that the broken commits that are direct ancestors of a
"bad" (or "failed" commit as you call them) would never cause the
issue, so it should eventually say that it can't determine which
commit is the first bad commit among 6B, 7B and 8F.

If you are sure that some broken commits don't have the issue you can
mark them as "good" to help git bisect which should then be able to
find the first "bad" commit.

Best,
Christian.