Opera release Git-splitter, a sub-modularizing tool for Git

Opera release Git-splitter, a sub-modularizing tool for Git

[Yngve N. Pettersen (Developer Opera Software ASA)]

Hello all,

Today Opera Software released the "Git-splitter", a small tool for  
sub-modularizing code in a git repo, with complete commit history, under  
the Apache 2.0 license.

It's functionality is similar to "git-subtree", but also include a command  
for reversing the process.

The code is hosted on GitHub:  
<https://github.com/operasoftware/git-splitter>

We have announced the release as part of another announcement of released  
code at the Opera Security Group home page:  
<http://my.opera.com/securitygroup/blog/2012/12/18/tls-prober-source-released-under-apache-2-0-license>

-- 
Sincerely,
Yngve N. Pettersen
********************************************************************
Senior Developer		     Email: yngve@opera.com
Opera Software ASA                   http://www.opera.com/
Phone:  +47 96 90 41 51              Fax:    +47 23 69 24 01
********************************************************************

Re: Opera release Git-splitter, a sub-modularizing tool for Git

[Michael J Gruber]

Yngve N. Pettersen (Developer Opera Software ASA) venit, vidit, dixit
18.12.2012 15:51:
> Hello all,
> 
> Today Opera Software released the "Git-splitter", a small tool for  
> sub-modularizing code in a git repo, with complete commit history, under  
> the Apache 2.0 license.
> 
> It's functionality is similar to "git-subtree", but also include a command  
> for reversing the process.

Is there something keeping you technically from adding a join command to
git-subtree?

> The code is hosted on GitHub:  
> <https://github.com/operasoftware/git-splitter>
> 
> We have announced the release as part of another announcement of released  
> code at the Opera Security Group home page:  
> <http://my.opera.com/securitygroup/blog/2012/12/18/tls-prober-source-released-under-apache-2-0-license>
> 

Re: Opera release Git-splitter, a sub-modularizing tool for Git

[Yngve Nysaeter Pettersen]

Hi,

On Fri, 21 Dec 2012 13:23:46 +0100, Michael J Gruber
<git@drmicha.warpmail.net> wrote:

> Yngve N. Pettersen (Developer Opera Software ASA) venit, vidit, dixit
> 18.12.2012 15:51:
>> Hello all,
>>
>> Today Opera Software released the "Git-splitter", a small tool for
>> sub-modularizing code in a git repo, with complete commit history, under
>> the Apache 2.0 license.
>>
>> It's functionality is similar to "git-subtree", but also include a  
>> command
>> for reversing the process.
>
> Is there something keeping you technically from adding a join command to
> git-subtree?

Probably not, but within the process I was working I did not want to merge
the branch with the recreated history for that path into the existing
codebase (I don't like duplicate histories) so I used rebasing to move the
new commits over, instead, and therefore did not need a join command.

Feel free to add a join command, if you want one.

>> The code is hosted on GitHub:
>> <https://github.com/operasoftware/git-splitter>
>>
>> We have announced the release as part of another announcement of  
>> released
>> code at the Opera Security Group home page:
>> <http://my.opera.com/securitygroup/blog/2012/12/18/tls-prober-source-released-under-apache-2-0-license>
>>


-- 
Sincerely,
Yngve N. Pettersen
********************************************************************
Senior Developer		     Email: yngve@opera.com
Opera Software ASA                   http://www.opera.com/
Phone:  +47 96 90 41 51              Fax:    +47 23 69 24 01
********************************************************************

Re: Opera release Git-splitter, a sub-modularizing tool for Git

[Michael J Gruber]

Yngve Nysaeter Pettersen venit, vidit, dixit 21.12.2012 13:43:
> Hi,
> 
> On Fri, 21 Dec 2012 13:23:46 +0100, Michael J Gruber
> <git@drmicha.warpmail.net> wrote:
> 
>> Yngve N. Pettersen (Developer Opera Software ASA) venit, vidit, dixit
>> 18.12.2012 15:51:
>>> Hello all,
>>>
>>> Today Opera Software released the "Git-splitter", a small tool for
>>> sub-modularizing code in a git repo, with complete commit history, under
>>> the Apache 2.0 license.
>>>
>>> It's functionality is similar to "git-subtree", but also include a  
>>> command
>>> for reversing the process.
>>
>> Is there something keeping you technically from adding a join command to
>> git-subtree?
> 
> Probably not, but within the process I was working I did not want to merge
> the branch with the recreated history for that path into the existing
> codebase (I don't like duplicate histories) so I used rebasing to move the
> new commits over, instead, and therefore did not need a join command.
> 
> Feel free to add a join command, if you want one.

Im sorry, but that was a total misunderstanding. You said git-splitter
is like git-subtree but adds a command for reversing the process. My
question was: What kept you from adding that to git-subtree (rather than
redoing stiff that git-subtree does)?

I just assumed that reversing the process of splitting must be joining.

It may very well be that git-splitter does things differently, i.e. that
there are more differences than just added functionality (compared to
git-subtree), but that is not clear from the announcement.

>>> The code is hosted on GitHub:
>>> <https://github.com/operasoftware/git-splitter>
>>>
>>> We have announced the release as part of another announcement of  
>>> released
>>> code at the Opera Security Group home page:
>>> <http://my.opera.com/securitygroup/blog/2012/12/18/tls-prober-source-released-under-apache-2-0-license>
>>>
> 
> 

Re: Opera release Git-splitter, a sub-modularizing tool for Git

[Yngve Nysaeter Pettersen]

On Fri, 21 Dec 2012 14:49:26 +0100, Michael J Gruber  
<git@drmicha.warpmail.net> wrote:

> Yngve Nysaeter Pettersen venit, vidit, dixit 21.12.2012 13:43:
>> Hi,
>>
>> On Fri, 21 Dec 2012 13:23:46 +0100, Michael J Gruber
>> <git@drmicha.warpmail.net> wrote:
>>
>>> Yngve N. Pettersen (Developer Opera Software ASA) venit, vidit, dixit
>>> 18.12.2012 15:51:
>>>> Hello all,
>>>>
>>>> Today Opera Software released the "Git-splitter", a small tool for
>>>> sub-modularizing code in a git repo, with complete commit history,  
>>>> under
>>>> the Apache 2.0 license.
>>>>
>>>> It's functionality is similar to "git-subtree", but also include a
>>>> command
>>>> for reversing the process.
>>>
>>> Is there something keeping you technically from adding a join command  
>>> to
>>> git-subtree?
>>
>> Probably not, but within the process I was working I did not want to  
>> merge
>> the branch with the recreated history for that path into the existing
>> codebase (I don't like duplicate histories) so I used rebasing to move  
>> the
>> new commits over, instead, and therefore did not need a join command.
>>
>> Feel free to add a join command, if you want one.
>
> Im sorry, but that was a total misunderstanding. You said git-splitter
> is like git-subtree but adds a command for reversing the process. My
> question was: What kept you from adding that to git-subtree (rather than
> redoing stiff that git-subtree does)?
>
> I just assumed that reversing the process of splitting must be joining.
>
> It may very well be that git-splitter does things differently, i.e. that
> there are more differences than just added functionality (compared to
> git-subtree), but that is not clear from the announcement.


The split command will create a new repository for all files foo in a  
folder (path/foo) and their commit history.

The replant command reverses that process, re-adding the path prefix for  
each file. It may be possible to extend that process into one that  
automatically reintegrates the new commits in the original history, but I  
never had time to complete that work.

I did originally add the "replant" functionality into my version of the  
git-subtree script, but given the number of commits in the original  
repository, git-subtree turned out to be inefficient, due to the use of  
temporary files (tens of thousands of files IIRC).

Those problems led to my development of git-splitter in Python (bypassing  
the problem of temporary files), but just including the functionality I  
needed, join was not one of those functions.

>>>> The code is hosted on GitHub:
>>>> <https://github.com/operasoftware/git-splitter>
>>>>
>>>> We have announced the release as part of another announcement of
>>>> released
>>>> code at the Opera Security Group home page:
>>>> <http://my.opera.com/securitygroup/blog/2012/12/18/tls-prober-source-released-under-apache-2-0-license>
>>>>
>>
>>


-- 
Sincerely,
Yngve N. Pettersen
********************************************************************
Senior Developer		     Email: yngve@opera.com
Opera Software ASA                   http://www.opera.com/
Phone:  +47 96 90 41 51              Fax:    +47 23 69 24 01
********************************************************************

Re: Opera release Git-splitter, a sub-modularizing tool for Git

[Matthieu Moy]

"Yngve Nysaeter Pettersen" <yngve@opera.com> writes:

> The split command will create a new repository for all files foo in a
> folder (path/foo) and their commit history.
>
> The replant command reverses that process, re-adding the path prefix
> for each file. It may be possible to extend that process into one that
> automatically reintegrates the new commits in the original history,
> but I never had time to complete that work.
>
> I did originally add the "replant" functionality into my version of
> the git-subtree script, but given the number of commits in the
> original repository, git-subtree turned out to be inefficient, due to
> the use of temporary files (tens of thousands of files IIRC).
>
> Those problems led to my development of git-splitter in Python
> (bypassing the problem of temporary files), but just including the
> functionality I needed, join was not one of those functions.

That still doesn't answer the question: why did you need to write a new
tool instead of extending git-subtree?

If one doesn't use "replant", is your tool different from git-subtree?

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: Opera release Git-splitter, a sub-modularizing tool for Git

[Yngve Nysaeter Pettersen]

On Fri, 21 Dec 2012 16:49:21 +0100, Matthieu Moy  
<Matthieu.Moy@grenoble-inp.fr> wrote:

> "Yngve Nysaeter Pettersen" <yngve@opera.com> writes:
>
>> The split command will create a new repository for all files foo in a
>> folder (path/foo) and their commit history.
>>
>> The replant command reverses that process, re-adding the path prefix
>> for each file. It may be possible to extend that process into one that
>> automatically reintegrates the new commits in the original history,
>> but I never had time to complete that work.
>>
>> I did originally add the "replant" functionality into my version of
>> the git-subtree script, but given the number of commits in the
>> original repository, git-subtree turned out to be inefficient, due to
>> the use of temporary files (tens of thousands of files IIRC).
>>
>> Those problems led to my development of git-splitter in Python
>> (bypassing the problem of temporary files), but just including the
>> functionality I needed, join was not one of those functions.
>
> That still doesn't answer the question: why did you need to write a new
> tool instead of extending git-subtree?

The primary problem with git-subtree was that I ended up with a temporary  
file directory containing 100+K files, which I tracked back to being used  
to manage the commit-to-tree mapping. On Windows, at least, that literally  
slowed down the process to a crawl.

> If one doesn't use "replant", is your tool different from git-subtree?

No, it is not different. However, my tool will use RAM, not diskspace to  
manage information.


-- 
Sincerely,
Yngve N. Pettersen
********************************************************************
Senior Developer		     Email: yngve@opera.com
Opera Software ASA                   http://www.opera.com/
Phone:  +47 96 90 41 51              Fax:    +47 23 69 24 01
********************************************************************

Re: Opera release Git-splitter, a sub-modularizing tool for Git

[Michael J Gruber]

Yngve Nysaeter Pettersen venit, vidit, dixit 21.12.2012 21:13:
> On Fri, 21 Dec 2012 16:49:21 +0100, Matthieu Moy  
> <Matthieu.Moy@grenoble-inp.fr> wrote:
> 
>> "Yngve Nysaeter Pettersen" <yngve@opera.com> writes:
>>
>>> The split command will create a new repository for all files foo in a
>>> folder (path/foo) and their commit history.
>>>
>>> The replant command reverses that process, re-adding the path prefix
>>> for each file. It may be possible to extend that process into one that
>>> automatically reintegrates the new commits in the original history,
>>> but I never had time to complete that work.
>>>
>>> I did originally add the "replant" functionality into my version of
>>> the git-subtree script, but given the number of commits in the
>>> original repository, git-subtree turned out to be inefficient, due to
>>> the use of temporary files (tens of thousands of files IIRC).
>>>
>>> Those problems led to my development of git-splitter in Python
>>> (bypassing the problem of temporary files), but just including the
>>> functionality I needed, join was not one of those functions.
>>
>> That still doesn't answer the question: why did you need to write a new
>> tool instead of extending git-subtree?
> 
> The primary problem with git-subtree was that I ended up with a temporary  
> file directory containing 100+K files, which I tracked back to being used  
> to manage the commit-to-tree mapping. On Windows, at least, that literally  
> slowed down the process to a crawl.
> 
>> If one doesn't use "replant", is your tool different from git-subtree?
> 
> No, it is not different. However, my tool will use RAM, not diskspace to  
> manage information.

That is some valuable input. It can help improve git-subtree for Windows
users, or replace it by something else.

Michael