Teach git how to propagate a fix across a file split

Teach git how to propagate a fix across a file split

[Jon Smirl]

Are there any git merge experts looking for something to do? We just
hit this situation again merging some ALSA branches for 2.6.31.

A basic problem description:
Pending change for 2.6.31 splits file A into A and B
Fix is added to 2.6.30 in file A
The fix in A is in a section that is going to be in file B for 2.6.31

Merge the two branches and note that git totally messes up on the
merge. It can't figure out that the fix needs to go into file B.

On Mon, May 4, 2009 at 10:54 AM, Michael J Gruber
<git@drmicha.warpmail.net> wrote:
> Jon Smirl venit, vidit, dixit 04.05.2009 15:52:
>> On Mon, May 4, 2009 at 9:49 AM, Michael J Gruber
>> <git@drmicha.warpmail.net> wrote:
>>> Jon Smirl venit, vidit, dixit 04.05.2009 15:42:
>>>> On Mon, May 4, 2009 at 9:27 AM, Michael J Gruber
>>>> <git@drmicha.warpmail.net> wrote:
>>>>> Jon Smirl venit, vidit, dixit 04.05.2009 14:53:
>>>>>> I keep running into this problem, is there anything I can do to make
>>>>>> it better? I'm using stgit but this is a problem in git itself.
>>>>>>
>>>>>> I have a patch that splits file A into two files, A and B.
>>>>>> Now I merge with another tree and bring in a one line fix to A.
>>>>>> The fix touches the pre-split file A in a section that is going to end up in B.
>>>>>> Next I re-apply the patch that splits A into A and B.
>>>>>>
>>>>>> This results in a large conflict in the post split file A.
>>>>>> And no patch being applied to file B which is where the fix belongs.
>>>>>>
>>>>>> Repeat this process with a multi-line fix and the whole automated
>>>>>> merge process breaks down and I have to carefully figure everything
>>>>>> out by hand.
>>>>>>
>>>>>> The merge process seems to be unaware of the newly created file B. No
>>>>>> patches or conflict ever end up in it.
>>>>>>
>>>>>
>>>>> Can you provide a test case or at least a list of commands which you are
>>>>> issuing? You complain about "merge", but you say you are "applying a
>>>>> patch". Are you merging that patch from another branch, or are you
>>>>> really applying it as a patch (git-apply/cherry-pick/rebase/what-not)?
>>>>
>>>> What git command does stgit use internally on push/pop?
>>>>
>>>> It's the stg push of a patch creating a split on top of a change to
>>>> the section that is going to end up in file B that causes the problem.
>>>
>>> I see. So it's really rebasing/applying here rather then merging. I
>>> don't think they have the necessary info in order to do content-based
>>> patching across file boundaries.
>>
>> Are there git commands that can do this properly? stgit is just a
>> bunch of Python executing git commands, they can change which commands
>> are getting called.
>
> OK, I checked the source, and in fact stgit uses git's merge strategies.
>
> I also checked a simple example (splitting `seq 1 20`) in two. It seems
> that git's rename detection does not detect that to be a rename/copy
> (split). As a consequence, the merge strategy performs only a file based
> merge between the versions of A. One would need to teach git about "a
> better split detection" in order to cope with such a situation.
>
> On the other hand, if you know the split, can't you redo that manually?
> I mean, checkout the new version (with the multi-line fixes), split it,
> git-add and git-commit to resolve.
>
> Michael
>



-- 
Jon Smirl
jonsmirl@gmail.com

Re: Teach git how to propagate a fix across a file split

[Jon Smirl]

On Thu, May 14, 2009 at 5:00 PM, Jon Smirl<jonsmirl@gmail.com> wrote:
> Are there any git merge experts looking for something to do? We just
> hit this situation again merging some ALSA branches for 2.6.31.
>
> A basic problem description:
> Pending change for 2.6.31 splits file A into A and B
> Fix is added to 2.6.30 in file A
> The fix in A is in a section that is going to be in file B for 2.6.31
>
> Merge the two branches and note that git totally messes up on the
> merge. It can't figure out that the fix needs to go into file B.

I just hit this problem again.

It can happen in other forms. In this case I'm trying to rebase three
year old patches forward (good old embedded vendor port and forget
behavior). Part of these patches made a four line change to one file.
About a year latter a 1,000 line section from this file was moved to
another pre-existing file. When I rebase the four line change forward
it generates a merge conflict over the entire 1,000 line section that
was moved. This conflict is in the file the section was moved out of
which is not what you want.

To fix this I have to search for where the 1,000 lines section was
moved to. After I find it I can redo the four line patch.

Doesn't rebase need to be smarter about processing the intervening
deltas in order to track the block of code being changed? Diff isn't
recognizing that the block of code it wants to diff against to is no
longer in the original file.

Maybe there is another way to do this? I'll try commuting the patch to
the old version in a temp tree and then merge in the newer code. Then
generate a new diff at the end.


>
> On Mon, May 4, 2009 at 10:54 AM, Michael J Gruber
> <git@drmicha.warpmail.net> wrote:
>> Jon Smirl venit, vidit, dixit 04.05.2009 15:52:
>>> On Mon, May 4, 2009 at 9:49 AM, Michael J Gruber
>>> <git@drmicha.warpmail.net> wrote:
>>>> Jon Smirl venit, vidit, dixit 04.05.2009 15:42:
>>>>> On Mon, May 4, 2009 at 9:27 AM, Michael J Gruber
>>>>> <git@drmicha.warpmail.net> wrote:
>>>>>> Jon Smirl venit, vidit, dixit 04.05.2009 14:53:
>>>>>>> I keep running into this problem, is there anything I can do to make
>>>>>>> it better? I'm using stgit but this is a problem in git itself.
>>>>>>>
>>>>>>> I have a patch that splits file A into two files, A and B.
>>>>>>> Now I merge with another tree and bring in a one line fix to A.
>>>>>>> The fix touches the pre-split file A in a section that is going to end up in B.
>>>>>>> Next I re-apply the patch that splits A into A and B.
>>>>>>>
>>>>>>> This results in a large conflict in the post split file A.
>>>>>>> And no patch being applied to file B which is where the fix belongs.
>>>>>>>
>>>>>>> Repeat this process with a multi-line fix and the whole automated
>>>>>>> merge process breaks down and I have to carefully figure everything
>>>>>>> out by hand.
>>>>>>>
>>>>>>> The merge process seems to be unaware of the newly created file B. No
>>>>>>> patches or conflict ever end up in it.
>>>>>>>
>>>>>>
>>>>>> Can you provide a test case or at least a list of commands which you are
>>>>>> issuing? You complain about "merge", but you say you are "applying a
>>>>>> patch". Are you merging that patch from another branch, or are you
>>>>>> really applying it as a patch (git-apply/cherry-pick/rebase/what-not)?
>>>>>
>>>>> What git command does stgit use internally on push/pop?
>>>>>
>>>>> It's the stg push of a patch creating a split on top of a change to
>>>>> the section that is going to end up in file B that causes the problem.
>>>>
>>>> I see. So it's really rebasing/applying here rather then merging. I
>>>> don't think they have the necessary info in order to do content-based
>>>> patching across file boundaries.
>>>
>>> Are there git commands that can do this properly? stgit is just a
>>> bunch of Python executing git commands, they can change which commands
>>> are getting called.
>>
>> OK, I checked the source, and in fact stgit uses git's merge strategies.
>>
>> I also checked a simple example (splitting `seq 1 20`) in two. It seems
>> that git's rename detection does not detect that to be a rename/copy
>> (split). As a consequence, the merge strategy performs only a file based
>> merge between the versions of A. One would need to teach git about "a
>> better split detection" in order to cope with such a situation.
>>
>> On the other hand, if you know the split, can't you redo that manually?
>> I mean, checkout the new version (with the multi-line fixes), split it,
>> git-add and git-commit to resolve.
>>
>> Michael
>>
>
>
>
> --
> Jon Smirl
> jonsmirl@gmail.com
>



-- 
Jon Smirl
jonsmirl@gmail.com

Re: Teach git how to propagate a fix across a file split

[Jon Smirl]

On Fri, Jul 31, 2009 at 11:19 AM, Jon Smirl<jonsmirl@gmail.com> wrote:
> On Thu, May 14, 2009 at 5:00 PM, Jon Smirl<jonsmirl@gmail.com> wrote:
>> Are there any git merge experts looking for something to do? We just
>> hit this situation again merging some ALSA branches for 2.6.31.
>>
>> A basic problem description:
>> Pending change for 2.6.31 splits file A into A and B
>> Fix is added to 2.6.30 in file A
>> The fix in A is in a section that is going to be in file B for 2.6.31
>>
>> Merge the two branches and note that git totally messes up on the
>> merge. It can't figure out that the fix needs to go into file B.
>
> I just hit this problem again.
>
> It can happen in other forms. In this case I'm trying to rebase three
> year old patches forward (good old embedded vendor port and forget
> behavior). Part of these patches made a four line change to one file.
> About a year latter a 1,000 line section from this file was moved to
> another pre-existing file. When I rebase the four line change forward
> it generates a merge conflict over the entire 1,000 line section that
> was moved. This conflict is in the file the section was moved out of
> which is not what you want.
>
> To fix this I have to search for where the 1,000 lines section was
> moved to. After I find it I can redo the four line patch.
>
> Doesn't rebase need to be smarter about processing the intervening
> deltas in order to track the block of code being changed? Diff isn't
> recognizing that the block of code it wants to diff against to is no
> longer in the original file.
>
> Maybe there is another way to do this? I'll try commuting the patch to
> the old version in a temp tree and then merge in the newer code. Then
> generate a new diff at the end.

That hits the same problem.

The diff mechanism seems to be file based. Would it work to take the
context of the initial diff fragment and then track it forward through
the log to figure out the final location of the context taking into
account movement across files? Once you know the final location of the
context, the diffs could be rewritten.

In all of the cases I am hitting the diff context was moved intact to
another file. If the context was updated to the new location the patch
applies with no conflict.

>
>
>>
>> On Mon, May 4, 2009 at 10:54 AM, Michael J Gruber
>> <git@drmicha.warpmail.net> wrote:
>>> Jon Smirl venit, vidit, dixit 04.05.2009 15:52:
>>>> On Mon, May 4, 2009 at 9:49 AM, Michael J Gruber
>>>> <git@drmicha.warpmail.net> wrote:
>>>>> Jon Smirl venit, vidit, dixit 04.05.2009 15:42:
>>>>>> On Mon, May 4, 2009 at 9:27 AM, Michael J Gruber
>>>>>> <git@drmicha.warpmail.net> wrote:
>>>>>>> Jon Smirl venit, vidit, dixit 04.05.2009 14:53:
>>>>>>>> I keep running into this problem, is there anything I can do to make
>>>>>>>> it better? I'm using stgit but this is a problem in git itself.
>>>>>>>>
>>>>>>>> I have a patch that splits file A into two files, A and B.
>>>>>>>> Now I merge with another tree and bring in a one line fix to A.
>>>>>>>> The fix touches the pre-split file A in a section that is going to end up in B.
>>>>>>>> Next I re-apply the patch that splits A into A and B.
>>>>>>>>
>>>>>>>> This results in a large conflict in the post split file A.
>>>>>>>> And no patch being applied to file B which is where the fix belongs.
>>>>>>>>
>>>>>>>> Repeat this process with a multi-line fix and the whole automated
>>>>>>>> merge process breaks down and I have to carefully figure everything
>>>>>>>> out by hand.
>>>>>>>>
>>>>>>>> The merge process seems to be unaware of the newly created file B. No
>>>>>>>> patches or conflict ever end up in it.
>>>>>>>>
>>>>>>>
>>>>>>> Can you provide a test case or at least a list of commands which you are
>>>>>>> issuing? You complain about "merge", but you say you are "applying a
>>>>>>> patch". Are you merging that patch from another branch, or are you
>>>>>>> really applying it as a patch (git-apply/cherry-pick/rebase/what-not)?
>>>>>>
>>>>>> What git command does stgit use internally on push/pop?
>>>>>>
>>>>>> It's the stg push of a patch creating a split on top of a change to
>>>>>> the section that is going to end up in file B that causes the problem.
>>>>>
>>>>> I see. So it's really rebasing/applying here rather then merging. I
>>>>> don't think they have the necessary info in order to do content-based
>>>>> patching across file boundaries.
>>>>
>>>> Are there git commands that can do this properly? stgit is just a
>>>> bunch of Python executing git commands, they can change which commands
>>>> are getting called.
>>>
>>> OK, I checked the source, and in fact stgit uses git's merge strategies.
>>>
>>> I also checked a simple example (splitting `seq 1 20`) in two. It seems
>>> that git's rename detection does not detect that to be a rename/copy
>>> (split). As a consequence, the merge strategy performs only a file based
>>> merge between the versions of A. One would need to teach git about "a
>>> better split detection" in order to cope with such a situation.
>>>
>>> On the other hand, if you know the split, can't you redo that manually?
>>> I mean, checkout the new version (with the multi-line fixes), split it,
>>> git-add and git-commit to resolve.
>>>
>>> Michael
>>>
>>
>>
>>
>> --
>> Jon Smirl
>> jonsmirl@gmail.com
>>
>
>
>
> --
> Jon Smirl
> jonsmirl@gmail.com
>



-- 
Jon Smirl
jonsmirl@gmail.com

Re: Teach git how to propagate a fix across a file split

[Björn Steinbrink]

On 2009.07.31 11:19:28 -0400, Jon Smirl wrote:
> On Thu, May 14, 2009 at 5:00 PM, Jon Smirl<jonsmirl@gmail.com> wrote:
> > Are there any git merge experts looking for something to do? We just
> > hit this situation again merging some ALSA branches for 2.6.31.
> >
> > A basic problem description:
> > Pending change for 2.6.31 splits file A into A and B
> > Fix is added to 2.6.30 in file A
> > The fix in A is in a section that is going to be in file B for 2.6.31
> >
> > Merge the two branches and note that git totally messes up on the
> > merge. It can't figure out that the fix needs to go into file B.
> 
> I just hit this problem again.
> 
> It can happen in other forms. In this case I'm trying to rebase three
> year old patches forward (good old embedded vendor port and forget
> behavior). Part of these patches made a four line change to one file.
> About a year latter a 1,000 line section from this file was moved to
> another pre-existing file. When I rebase the four line change forward
> it generates a merge conflict over the entire 1,000 line section that
> was moved. This conflict is in the file the section was moved out of
> which is not what you want.
> 
> To fix this I have to search for where the 1,000 lines section was
> moved to. After I find it I can redo the four line patch.

The search can be done via blame:
git blame -C -C --reverse <old_rev>.. -- <file>

That shows the contents of <file> as of <old_rev> and where those
contents are now (as of HEAD). Of course, if the contents were changed a
lot since they were moved, the results won't be very useful, as git
simply can't find the old code in the new files anymore.

HTH
Björn