Grafting mis-aligned trees.

Grafting mis-aligned trees.

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 1895 bytes --]

I haven't gotten a response from my subscription email, so please CC me on any replies.

So, I've been managaing the source I had from a client project in git and 
have a non-linear history.  Currently, two tips (production and testing) 
but there are many feature branches that were git-merge'd in, not rebased.

Now, I've gotten the full tree.  Turns out all the source code I was 
working on was in a subdirectory "project/web".  I'd like to "graft" the 
*changes* I made onto the full tree.

I figured this might be a job for git-filter-branch.  Certainly, that did 
the job of moving all my changes into the subdirectory.  But, now I want to 
do something that's a combination or git-rebase and git-filter-branch.  I 
want to replay the *patches/deltas* (like rebase) on top of the full tree I 
have, but *maintain the non-liear history* (like filter-branch).

Can anyone think of a recipe for me?

Trees look something like this right now.

<some history> -> FT

TI -> <non-linear history> -> A -> <non-linear history> -> C
   \                            \                           \
    -> PI ------------------------> B ------------------------> D

I'd like to have it look something like:

<some history> -> FT -> <non-linear history> -> A' -> <non-linear history> -> C'
                    \                            \                           \
                     -> PI' ----------------------> B' -----------------------> D'

A', B', C', and D' are different commits, but the diff (and history) between 
FT and A' is the same as the diff (and history) between TI and A.

Again, please CC me on any replies.
-- 
Boyd Stephen Smith Jr.                     ,= ,-_-. =. 
bss03@volumehost.net                      ((_/)o o(\_))
ICQ: 514984 YM/AIM: DaTwinkDaddy           `-'(. .)`-' 
http://iguanasuicide.org/                      \_/     

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Grafting mis-aligned trees.

[Michael J Gruber]

Boyd Stephen Smith Jr. venit, vidit, dixit 17.11.2008 23:45:
> I haven't gotten a response from my subscription email, so please CC me on any replies.
> 
> So, I've been managaing the source I had from a client project in git and 
> have a non-linear history.  Currently, two tips (production and testing) 
> but there are many feature branches that were git-merge'd in, not rebased.
> 
> Now, I've gotten the full tree.  Turns out all the source code I was 
> working on was in a subdirectory "project/web".  I'd like to "graft" the 
> *changes* I made onto the full tree.
> 
> I figured this might be a job for git-filter-branch.  Certainly, that did 
> the job of moving all my changes into the subdirectory.  But, now I want to 
> do something that's a combination or git-rebase and git-filter-branch.  I 
> want to replay the *patches/deltas* (like rebase) on top of the full tree I 
> have, but *maintain the non-liear history* (like filter-branch).
> 
> Can anyone think of a recipe for me?
> 
> Trees look something like this right now.
> 
> <some history> -> FT
> 
> TI -> <non-linear history> -> A -> <non-linear history> -> C
>    \                            \                           \
>     -> PI ------------------------> B ------------------------> D
> 
> I'd like to have it look something like:
> 
> <some history> -> FT -> <non-linear history> -> A' -> <non-linear history> -> C'
>                     \                            \                           \
>                      -> PI' ----------------------> B' -----------------------> D'
> 
> A', B', C', and D' are different commits, but the diff (and history) between 
> FT and A' is the same as the diff (and history) between TI and A.
> 
> Again, please CC me on any replies.

[CCing is customary here anyways.]

So, your base directory for TI and FT is different, right? I.e.: In the
TI repo, your project sits at the root, whereas in the FT repo it sits
in project/web? Has FT advanced since you took the initial subdir
snapshot for TI?

Michael

Re: Grafting mis-aligned trees.

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 2827 bytes --]

On Tuesday 2008 November 18 03:24, Michael J Gruber wrote:
> Boyd Stephen Smith Jr. venit, vidit, dixit 17.11.2008 23:45:
> > I haven't gotten a response from my subscription email, so please CC me
> > on any replies.
> >
> > So, I've been managaing the source I had from a client project in git and
> > have a non-linear history.  Currently, two tips (production and testing)
> > but there are many feature branches that were git-merge'd in, not
> > rebased.
> >
> > Now, I've gotten the full tree.  Turns out all the source code I was
> > working on was in a subdirectory "project/web".  I'd like to "graft" the
> > *changes* I made onto the full tree.
> >
> > I figured this might be a job for git-filter-branch.  Certainly, that did
> > the job of moving all my changes into the subdirectory.  But, now I want
> > to do something that's a combination or git-rebase and git-filter-branch.
> >  I want to replay the *patches/deltas* (like rebase) on top of the full
> > tree I have, but *maintain the non-liear history* (like filter-branch).
> >
> > Can anyone think of a recipe for me?
> >
> > Trees look something like this right now.
> >
> > <some history> -> FT
> >
> > TI -> <non-linear history> -> A -> <non-linear history> -> C
> >    \                            \                           \
> >     -> PI ------------------------> B ------------------------> D
> >
> > I'd like to have it look something like:
> >
> > <some history> -> FT -> <non-linear history> -> A' -> <non-linear
> > history> -> C' \                            \                           \
> > -> PI' ----------------------> B' -----------------------> D'
> >
> > A', B', C', and D' are different commits, but the diff (and history)
> > between FT and A' is the same as the diff (and history) between TI and A.
> >
> > Again, please CC me on any replies.
>
> [CCing is customary here anyways.]
>
> So, your base directory for TI and FT is different, right? I.e.: In the
> TI repo, your project sits at the root, whereas in the FT repo it sits
> in project/web?

Yes.

> Has FT advanced since you took the initial subdir 
> snapshot for TI?

No.  Well, maybe.  I think the subdir diff is fairly trivial if not empty.  TI 
is an import from the code actually present on the testing server.  FT was 
the a subversion repository obtained later after some hullabaloo with the 
ex-development house.

Right now this tree is effectively all mine, so I can always graft in commits 
to synchronize the common subtree of FT and TI, if that makes things easier.
-- 
Boyd Stephen Smith Jr.                     ,= ,-_-. =. 
bss03@volumehost.net                      ((_/)o o(\_))
ICQ: 514984 YM/AIM: DaTwinkDaddy           `-'(. .)`-' 
http://iguanasuicide.org/                      \_/     

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Grafting mis-aligned trees.

[Michael J Gruber]

Boyd Stephen Smith Jr. venit, vidit, dixit 29.11.2008 00:01:
> On Tuesday 2008 November 18 03:24, Michael J Gruber wrote:
>> Boyd Stephen Smith Jr. venit, vidit, dixit 17.11.2008 23:45:
>>> I haven't gotten a response from my subscription email, so please CC me
>>> on any replies.
>>>
>>> So, I've been managaing the source I had from a client project in git and
>>> have a non-linear history.  Currently, two tips (production and testing)
>>> but there are many feature branches that were git-merge'd in, not
>>> rebased.
>>>
>>> Now, I've gotten the full tree.  Turns out all the source code I was
>>> working on was in a subdirectory "project/web".  I'd like to "graft" the
>>> *changes* I made onto the full tree.
>>>
>>> I figured this might be a job for git-filter-branch.  Certainly, that did
>>> the job of moving all my changes into the subdirectory.  But, now I want
>>> to do something that's a combination or git-rebase and git-filter-branch.
>>>  I want to replay the *patches/deltas* (like rebase) on top of the full
>>> tree I have, but *maintain the non-liear history* (like filter-branch).
>>>
>>> Can anyone think of a recipe for me?
>>>
>>> Trees look something like this right now.
>>>
>>> <some history> -> FT
>>>
>>> TI -> <non-linear history> -> A -> <non-linear history> -> C
>>>    \                            \                           \
>>>     -> PI ------------------------> B ------------------------> D
>>>
>>> I'd like to have it look something like:
>>>
>>> <some history> -> FT -> <non-linear history> -> A' -> <non-linear
>>> history> -> C' \                            \                           \
>>> -> PI' ----------------------> B' -----------------------> D'
>>>
>>> A', B', C', and D' are different commits, but the diff (and history)
>>> between FT and A' is the same as the diff (and history) between TI and A.
>>>
>>> Again, please CC me on any replies.
>> [CCing is customary here anyways.]
>>
>> So, your base directory for TI and FT is different, right? I.e.: In the
>> TI repo, your project sits at the root, whereas in the FT repo it sits
>> in project/web?
> 
> Yes.
> 
>> Has FT advanced since you took the initial subdir 
>> snapshot for TI?
> 
> No.  Well, maybe.  I think the subdir diff is fairly trivial if not empty.  TI 
> is an import from the code actually present on the testing server.  FT was 
> the a subversion repository obtained later after some hullabaloo with the 
> ex-development house.
> 
> Right now this tree is effectively all mine, so I can always graft in commits 
> to synchronize the common subtree of FT and TI, if that makes things easier.

OK, here's a possibly primitive solution, but it works with my little
toy model of your layout:

- filter-branch your TI branches so that they are in the proper subdir
(you did that already)

- take a snapshot (say ftstuff.tar) of everything in FT's head (assuming
this is where TI branched off, or else take that point) *but exclude
project/web*

- using filter-branch again, rewrite your TI branches to contain those
missing FT files:
git filter-branch --tree-filter 'tar -xf /tmp/ftstuff.tar && git add .'
-f -- ti/master ti/whatever

Now your TI branches produce the same diffs as before, but are based on
the full tree. You can happily graft FT's head onto TI's root as a parent.
In fact those two should produce no diff in between them, so you might
as well get rid of one of them.

[cleaning out refs/original and repack -adf might be in order afterwards]

The tree-filter part feels hacky but does the job (probably with -f). I
don't think a subtree merge can do what you want.

Cheers,
Michael

Re: Grafting mis-aligned trees.

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 3605 bytes --]

On Tuesday 02 December 2008, Michael J Gruber <git@drmicha.warpmail.net> 
wrote about 'Re: Grafting mis-aligned trees.':
>Boyd Stephen Smith Jr. venit, vidit, dixit 29.11.2008 00:01:
>> On Tuesday 2008 November 18 03:24, Michael J Gruber wrote:
>>> Boyd Stephen Smith Jr. venit, vidit, dixit 17.11.2008 23:45:
>>>> Trees look something like this right now.
>>>>
>>>> <some history> -> FT
>>>>
>>>> TI -> <non-linear history> -> A -> <non-linear history> -> C
>>>>    \                            \                           \
>>>>     -> PI ------------------------> B ------------------------> D
>>>>
>>>> I'd like to have it look something like:
>>>>
>>>> <some history> -> FT -> <non-linear history> -> A' -> <non-linear
>>>> history> -> C' \                            \                        
>>>>   \ -> PI' ----------------------> B' -----------------------> D'
>>>>
>>>> A', B', C', and D' are different commits, but the diff (and history)
>>>> between FT and A' is the same as the diff (and history) between TI
>>>> and A.
>>>
>>> So, your base directory for TI and FT is different, right? I.e.: In
>>> the TI repo, your project sits at the root, whereas in the FT repo it
>>> sits in project/web?
>>
>> Yes.
>>
>>> Has FT advanced since you took the initial subdir
>>> snapshot for TI?
>>
>> No.
>
>OK, here's a possibly primitive solution, but it works with my little
>toy model of your layout:

That sounds like it will work fine.  Thank you very much.

It does feel a bit "hacky", I was hoping git would have better support 
this, through the subtree merge or something else.  It seems like 
something that might happen to others, perhaps as a side-effect of a 
failed attempt at using submodules.

I can't help thinking that rebase -ip might have helped.  I wasn't aware 
of -p when I was initially working on this problem.  (It doesn't help that 
I generally use Debian stable, and git 1.4 did not have -p.)

>- filter-branch your TI branches so that they are in the proper subdir
>(you did that already)

If I need to "undo" this, it's really easy.

>- take a snapshot (say ftstuff.tar) of everything in FT's head (assuming
>this is where TI branched off, or else take that point) *but exclude
>project/web*
>
>- using filter-branch again, rewrite your TI branches to contain those
>missing FT files:
>git filter-branch --tree-filter 'tar -xf /tmp/ftstuff.tar && git add .'
>-f -- ti/master ti/whatever

I probably don't need the -f.  If there are files that should be ignored 
(and thus shouldn't be in the repo), I'll filter-branch to cut them out of 
the history at some point.

Now is as good a time as any.

>Now your TI branches produce the same diffs as before, but are based on
>the full tree. You can happily graft FT's head onto TI's root as a
> parent. In fact those two should produce no diff in between them, so you
> might as well get rid of one of them.

Makes sense.

>[cleaning out refs/original and repack -adf might be in order afterwards]

I generally do these after a successful filter-branch.

>The tree-filter part feels hacky but does the job (probably with -f). I
>don't think a subtree merge can do what you want.

What *exactly* is the subtree merge.  The documentation I've read sounds 
like this case, sort of, but it's rather unclear to me.
-- 
Boyd Stephen Smith Jr.                     ,= ,-_-. =. 
bss03@volumehost.net                      ((_/)o o(\_))
ICQ: 514984 YM/AIM: DaTwinkDaddy           `-'(. .)`-' 
http://iguanasuicide.org/                      \_/     

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Grafting mis-aligned trees.

[Michael J Gruber]

Boyd Stephen Smith Jr. venit, vidit, dixit 02.12.2008 18:19:
...
> It does feel a bit "hacky", I was hoping git would have better support 
> this, through the subtree merge or something else.  It seems like 
> something that might happen to others, perhaps as a side-effect of a 
> failed attempt at using submodules.

Well, except for 'tar' my solution uses only git commands ;)

> I can't help thinking that rebase -ip might have helped.  I wasn't aware 
> of -p when I was initially working on this problem.  (It doesn't help that 
> I generally use Debian stable, and git 1.4 did not have -p.)

rebase rebases one branch at a time, but you need to rebase/rewrite
several, and the merge info between depends on rewritten sha1s.

...
> I probably don't need the -f.  If there are files that should be ignored 
> (and thus shouldn't be in the repo), I'll filter-branch to cut them out of 
> the history at some point.

'-f' is about not having to clean out refs/original from a previous
filter-branch run.

> What *exactly* is the subtree merge.  The documentation I've read sounds 
> like this case, sort of, but it's rather unclear to me.

I think 'subtree' does what you want, but 'merge' doesn't! 'subtree'
saves you the rewriting (putting TI into project/web), but you want a
one-time conversion anyway. 'subtree' allows you to repeatedly merge
branches with a different root. What it does is it looks for subdir,
'rewrites' the incoming tree automatically and merges the result.

But you don't want a merge, do you? Or else your whole TI history would
be tacked onto FT's head "to the left": a new (subtree) merge commit
would have FT's and TI's head as parents. This is one way of storing TI
history in the full repo, but not the one you said you wanted.

Michael

Re: Grafting mis-aligned trees.

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 2327 bytes --]

On Tuesday 02 December 2008, Michael J Gruber <git@drmicha.warpmail.net> 
wrote about 'Re: Grafting mis-aligned trees.':
>Boyd Stephen Smith Jr. venit, vidit, dixit 02.12.2008 18:19:
>> I can't help thinking that rebase -ip might have helped.  I wasn't
>> aware of -p when I was initially working on this problem.  (It doesn't
>> help that I generally use Debian stable, and git 1.4 did not have -p.)
>
>rebase rebases one branch at a time, but you need to rebase/rewrite
>several, and the merge info between depends on rewritten sha1s.

Yes.  I guess that's why I employed filter-branch to begin with.

>> I probably don't need the -f.  If there are files that should be
>> ignored (and thus shouldn't be in the repo), I'll filter-branch to cut
>> them out of the history at some point.
>
>'-f' is about not having to clean out refs/original from a previous
>filter-branch run.

Okay, I misread your command line.  For some reason I thought "-f" was an 
option to add.

>> What *exactly* is the subtree merge.  The documentation I've read
>> sounds like this case, sort of, but it's rather unclear to me.
>
>I think 'subtree' does what you want, but 'merge' doesn't!

*giggle*  I'm not quite sure what makes this funny to me, but it made me 
laugh.

>'subtree' 
>saves you the rewriting (putting TI into project/web), but you want a
>one-time conversion anyway. 'subtree' allows you to repeatedly merge
>branches with a different root. What it does is it looks for subdir,
>'rewrites' the incoming tree automatically and merges the result.
> 
>But you don't want a merge, do you? Or else your whole TI history would
>be tacked onto FT's head "to the left": a new (subtree) merge commit
>would have FT's and TI's head as parents. This is one way of storing TI
>history in the full repo, but not the one you said you wanted.

You are right; that's not what I want.  But, it is a good second-place 
result that I'll keep in mind.  Sometimes keeping the history at all is 
more important that keeping the history orderly.

Again, thanks for the help.
-- 
Boyd Stephen Smith Jr.                     ,= ,-_-. =. 
bss03@volumehost.net                      ((_/)o o(\_))
ICQ: 514984 YM/AIM: DaTwinkDaddy           `-'(. .)`-' 
http://iguanasuicide.org/                      \_/     

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]