[ANN] mtn2git v0.1

[ANN] mtn2git v0.1

[Felipe Contreras]

Hi,

This is the result of various experiments I've been doing while trying
to import mtn repositories into git. I've looked into other mtn2git
scripts but none fitted my needs.

After some rfcs on git and monotone mailing lists it seems now the
script is going in the right direction.

There are two modes:

== checkout ==

In this mode each revision is checked out and imported directly into
git. This means it's 100% sure that the result would be an exact
clone.

The disadvantage is that it's extremely slow (1 day for 25,000 commits).

== fast-import ==

This mode requires a few patches for git fast-import, it's very fast
(40 min for 25,000 commits), but not 100% reliable yet.

There are also some missing features like branch creation and updates.


My plan is to keep these two modes in the code until fast-import
method is reliable enough.

I've tried this with Pidgin's repository. The result is here:
http://github.com/felipec/pidgin-clone

It would be interesting to do something similar with OE's repo. Any takers?

Comments and patches are welcome :)

-- 
Felipe Contreras

Re: [ANN] mtn2git v0.1

[Jakub Narebski]

"Felipe Contreras" <felipe.contreras@gmail.com> writes:

> This is the result of various experiments I've been doing while trying
> to import mtn repositories into git. I've looked into other mtn2git
> scripts but none fitted my needs.

mtn or mnt?

> After some RFCs on git and monotone mailing lists it seems now that
> the script is going in the right direction.

When you feel this script to be ready, could you add it to the
"Interaction with other Revision Control Systems" section on
  http://git.or.cz/gitwiki/InterfacesFrontendsAndTools
As far as I can see there ain't any Monotone to Git converter on this
list.

TIA
-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: [ANN] mtn2git v0.1

[Felipe Contreras]

On Sat, Sep 13, 2008 at 12:45 PM, Jakub Narebski <jnareb@gmail.com> wrote:
> "Felipe Contreras" <felipe.contreras@gmail.com> writes:
>
>> This is the result of various experiments I've been doing while trying
>> to import mtn repositories into git. I've looked into other mtn2git
>> scripts but none fitted my needs.
>
> mtn or mnt?

monotone = mtn

>> After some RFCs on git and monotone mailing lists it seems now that
>> the script is going in the right direction.
>
> When you feel this script to be ready, could you add it to the
> "Interaction with other Revision Control Systems" section on
>  http://git.or.cz/gitwiki/InterfacesFrontendsAndTools
> As far as I can see there ain't any Monotone to Git converter on this
> list.

Ok, done. I think it's ready if you can bare the slowness of the
'checkout' method. The only missing feature is tags, but should be
easy to implement.

-- 
Felipe Contreras

Re: [ANN] mtn2git v0.1

[Felipe Contreras]

On Sat, Sep 13, 2008 at 12:45 PM, Jakub Narebski <jnareb@gmail.com> wrote:
> "Felipe Contreras" <felipe.contreras@gmail.com> writes:
>
>> This is the result of various experiments I've been doing while trying
>> to import mtn repositories into git. I've looked into other mtn2git
>> scripts but none fitted my needs.
>
> mtn or mnt?

monotone = mtn

>> After some RFCs on git and monotone mailing lists it seems now that
>> the script is going in the right direction.
>
> When you feel this script to be ready, could you add it to the
> "Interaction with other Revision Control Systems" section on
>  http://git.or.cz/gitwiki/InterfacesFrontendsAndTools
> As far as I can see there ain't any Monotone to Git converter on this
> list.

Ok, done. I think it's ready if you can bare the slowness of the
'checkout' method. The only missing feature is tags, but should be
easy to implement.

-- 
Felipe Contreras

Re: [ANN] mtn2git v0.1

[Jakub Narebski]

On Sat, 13 Sep 2008, Felipe Contreras wrote:
> On Sat, Sep 13, 2008 at 12:45 PM, Jakub Narebski <jnareb@gmail.com> wrote:
>> "Felipe Contreras" <felipe.contreras@gmail.com> writes:
>>
>>> This is the result of various experiments I've been doing while trying
>>> to import mtn repositories into git. I've looked into other mtn2git
>>> scripts but none fitted my needs.
>>
>> mtn or mnt?
> 
> monotone = mtn

Thanks.

My confusion resulted from the fact that 'monotone' has 'n' both
before and after 't'.
 
>>> After some RFCs on git and monotone mailing lists it seems now that
>>> the script is going in the right direction.
>>
>> When you feel this script to be ready, could you add it to the
>> "Interaction with other Revision Control Systems" section on
>>  http://git.or.cz/gitwiki/InterfacesFrontendsAndTools
>> As far as I can see there ain't any Monotone to Git converter on this
>> list.
> 
> Ok, done. I think it's ready if you can bare the slowness of the
> 'checkout' method. The only missing feature is tags, but should be
> easy to implement.

Thank you.

BTW. did you have any problems with (from what I understand) slightly
different concept of branches between Monotone and Git?

-- 
Jakub Narebski
Poland

Re: [ANN] mtn2git v0.1

[Felipe Contreras]

On Sat, Sep 13, 2008 at 3:02 PM, Jakub Narebski <jnareb@gmail.com> wrote:
> On Sat, 13 Sep 2008, Felipe Contreras wrote:
>> On Sat, Sep 13, 2008 at 12:45 PM, Jakub Narebski <jnareb@gmail.com> wrote:
>>> "Felipe Contreras" <felipe.contreras@gmail.com> writes:

<snip/>

>>>> After some RFCs on git and monotone mailing lists it seems now that
>>>> the script is going in the right direction.
>>>
>>> When you feel this script to be ready, could you add it to the
>>> "Interaction with other Revision Control Systems" section on
>>>  http://git.or.cz/gitwiki/InterfacesFrontendsAndTools
>>> As far as I can see there ain't any Monotone to Git converter on this
>>> list.
>>
>> Ok, done. I think it's ready if you can bare the slowness of the
>> 'checkout' method. The only missing feature is tags, but should be
>> easy to implement.
>
> Thank you.
>
> BTW. did you have any problems with (from what I understand) slightly
> different concept of branches between Monotone and Git?

Monotone can have multiple heads in one single branch, but from what I
understand that mostly happens locally (not on the published repo).
Anyway, If that happens the commits are still there, just dangling
temporarily in no branch.

There isn't much we can do for that situation, except maybe create
branch_n or something. I don't think it's a big problem.

-- 
Felipe Contreras

Re: [Monotone-devel] Re: [ANN] mtn2git v0.1

[Markus Wanner]

Hi,

Felipe Contreras wrote:
> Monotone can have multiple heads in one single branch, but from what I
> understand that mostly happens locally (not on the published repo).

Uh.. no, I'd say quite the opposite: if multiple developers work on the 
same branch and do their commits, the first time both heads appear in 
the same branch on the same machine will be the public repository.

> Anyway, If that happens the commits are still there, just dangling
> temporarily in no branch.

..for the mtn repo imported into git that is, right?

Regards

Markus Wanner (ne Schiltknecht)

Re: [ANN] mtn2git v0.1

[cyril Romain]

Hi Felipe,

Felipe Contreras wrote:
> This is the result of various experiments I've been doing while trying
> to import mtn repositories into git. I've looked into other mtn2git
> scripts but none fitted my needs
Did you take a look at the mtn2git python script available in 
org.openembedded.dev/contrib/mtn2git/mtn2git.py ?
Since it was intended and used to convert the OE monotone repository to 
git, I think it should fit your needs :-)

Best regards,

  Cyril

Re: [ANN] mtn2git v0.1

[Felipe Contreras]

On Sat, Sep 13, 2008 at 7:03 PM, cyril Romain <c.romain@laposte.net> wrote:
> Hi Felipe,
>
> Felipe Contreras wrote:
>>
>> This is the result of various experiments I've been doing while trying
>> to import mtn repositories into git. I've looked into other mtn2git
>> scripts but none fitted my needs
>
> Did you take a look at the mtn2git python script available in
> org.openembedded.dev/contrib/mtn2git/mtn2git.py ?
> Since it was intended and used to convert the OE monotone repository to git,
> I think it should fit your needs :-)

I did look at it, but there are some issues. For starters it seems to
be using extensively the manifest, which would probably make it very
slow. Also, it seems to be keeping the trees of all the revisions.

Superficially it seemed to have issues so I didn't try it, but I also,
I can't stand Python code =/ Do you know how long does it takes to
import the OE repo?

Anyway, the patches I did for git fast-import should also help your
script if there is a plan to stop using the manifest.

-- 
Felipe Contreras

Re: [ANN] mtn2git v0.1

[Michael Krelin]

> 
> I did look at it, but there are some issues. For starters it seems to
> be using extensively the manifest, which would probably make it very
> slow. Also, it seems to be keeping the trees of all the revisions.

Back when mtn2git script was on the early stages of development and I've 
had some time I've started doing my own mtn2git gateway in c++ (never 
finished it, though). For what I remember using manifest was the only 
way, because the changes information mtn automate returns was incomplete 
or simply wrong, I don't remember the details... Do check your results 
carefully if you're trying to go this way. Luckily, being slow due to 
use of manifest should only matter on initial import.

Love,
H

Re: [ANN] mtn2git v0.1

[Felipe Contreras]

On Sat, Sep 13, 2008 at 8:43 PM, Michael Krelin <hacker@klever.net> wrote:
>>
>> I did look at it, but there are some issues. For starters it seems to
>> be using extensively the manifest, which would probably make it very
>> slow. Also, it seems to be keeping the trees of all the revisions.
>
> Back when mtn2git script was on the early stages of development and I've had
> some time I've started doing my own mtn2git gateway in c++ (never finished
> it, though). For what I remember using manifest was the only way, because
> the changes information mtn automate returns was incomplete or simply wrong,
> I don't remember the details... Do check your results carefully if you're
> trying to go this way. Luckily, being slow due to use of manifest should
> only matter on initial import.

It's complicated, but hopefully doable.

That's why there are two methods; one that does a full checkout of the
revision and imports it into git, and the other one that uses mtn
automate and fast-import.

I'm able to import about 60% of the Pidgin code correctly, but after
that there are issues. That's the benefit of having two modes... I can
compare the fast one with the exact one and improve the fast one.

-- 
Felipe Contreras

Re: [ANN] mtn2git v0.1

[Michael Krelin]

> On Sat, Sep 13, 2008 at 8:43 PM, Michael Krelin <hacker@klever.net> wrote:
>>> I did look at it, but there are some issues. For starters it seems to
>>> be using extensively the manifest, which would probably make it very
>>> slow. Also, it seems to be keeping the trees of all the revisions.
>> Back when mtn2git script was on the early stages of development and I've had
>> some time I've started doing my own mtn2git gateway in c++ (never finished
>> it, though). For what I remember using manifest was the only way, because
>> the changes information mtn automate returns was incomplete or simply wrong,
>> I don't remember the details... Do check your results carefully if you're
>> trying to go this way. Luckily, being slow due to use of manifest should
>> only matter on initial import.
> 
> It's complicated, but hopefully doable.
> 
> That's why there are two methods; one that does a full checkout of the
> revision and imports it into git, and the other one that uses mtn
> automate and fast-import.
> 
> I'm able to import about 60% of the Pidgin code correctly, but after
> that there are issues. That's the benefit of having two modes... I can
> compare the fast one with the exact one and improve the fast one.

I ended up using some hybrid mode myself, but the point is - there's no 
way to avoid pulling manifests all the time and that doesn't imply 
complete checkout.

Love,
H

Re: [ANN] mtn2git v0.1

[Felipe Contreras]

On Sat, Sep 13, 2008 at 9:18 PM, Michael Krelin <hacker@klever.net> wrote:
>> On Sat, Sep 13, 2008 at 8:43 PM, Michael Krelin <hacker@klever.net> wrote:
>>>>
>>>> I did look at it, but there are some issues. For starters it seems to
>>>> be using extensively the manifest, which would probably make it very
>>>> slow. Also, it seems to be keeping the trees of all the revisions.
>>>
>>> Back when mtn2git script was on the early stages of development and I've
>>> had
>>> some time I've started doing my own mtn2git gateway in c++ (never
>>> finished
>>> it, though). For what I remember using manifest was the only way, because
>>> the changes information mtn automate returns was incomplete or simply
>>> wrong,
>>> I don't remember the details... Do check your results carefully if you're
>>> trying to go this way. Luckily, being slow due to use of manifest should
>>> only matter on initial import.
>>
>> It's complicated, but hopefully doable.
>>
>> That's why there are two methods; one that does a full checkout of the
>> revision and imports it into git, and the other one that uses mtn
>> automate and fast-import.
>>
>> I'm able to import about 60% of the Pidgin code correctly, but after
>> that there are issues. That's the benefit of having two modes... I can
>> compare the fast one with the exact one and improve the fast one.
>
> I ended up using some hybrid mode myself, but the point is - there's no way
> to avoid pulling manifests all the time and that doesn't imply complete
> checkout.

Do you remember the problem?

From what I can see it's doable.

-- 
Felipe Contreras

Re: [ANN] mtn2git v0.1

[Michael Krelin]

>> I ended up using some hybrid mode myself, but the point is - there's no way
>> to avoid pulling manifests all the time and that doesn't imply complete
>> checkout.
> 
> Do you remember the problem?
> 
> From what I can see it's doable.

I don't remember the details - it was about a year ago. For what I 
remember, it *looked* doable and would be doable if mtn provided 
accurate information. In short, it looked like a bug in mtn, but I 
didn't investigate its insides.

Love,
H