importing cvs logical modules

importing cvs logical modules

[Geoff Russell]

Hi,

Firstly, the code to automagically
repack a git repository on-the-fly during a big load has solved one of
my problems - thanks, it is great. Unfortunately  it has bought me to
showstopper number 2.

- cvs modules.

cvs allows you to define modules which rearrange the physical repository into
a different logical structure.  This sounds great and we use it, but it gives
us other headaches because "cvs update" doesn't always do the right
thing with these modules.

Furthermore cvsps doesn't appear to handle this module feature at all and
is tricked into thinking that rearranged directories come from somewhere
else and issues its "file xxx doesn't match strip_path" message.

I have tried to hack cvsps to go around the problem, but without success.

Another alternative that I thought might be easier would be to unload the cvs
repository in clean pieces - each being a git repository. Then to join the
git repositories together in the desired way. I think this would be
generally useful and not just solve my problem.

e.g. Suppose I have 3 git repositories: Progs, xxxx, yyyy

           Progs/.git
           xxxx/.git
           yyyy/.git

I'd like to make 1 git repository Progs with xxxx and yyyy as child trees.

           Progs/.git
           Progs/xxxx
           Progs/yyyy

Does this sound useful to anyone else, or is it already possible?


Cheers,
Geoff Russell.

Re: importing cvs logical modules

[Jakub Narebski]

Geoff Russell wrote:

> Then to join the git repositories together in the desired way. I think
> this would be generally useful and not just solve my problem.
> 
> e.g. Suppose I have 3 git repositories: Progs, xxxx, yyyy
> 
>            Progs/.git
>            xxxx/.git
>            yyyy/.git
> 
> I'd like to make 1 git repository Progs with xxxx and yyyy as child trees.
> 
>            Progs/.git
>            Progs/xxxx
>            Progs/yyyy
> 
> Does this sound useful to anyone else, or is it already possible?

It is done. TODO branch in git.git repository on kernel.org is pushed to
from *separate* git-todo.git repository. BTW. working are of repositories
can be joined (although it needs entry in .gitignore):

   Progs/.git
   Progs/TODO/.git

-- 
Jakub Narebski
Warsaw, Poland

Re: importing cvs logical modules

[Junio C Hamano]

"Geoff Russell" <geoffrey.russell@gmail.com> writes:

> I'd like to make 1 git repository Progs with xxxx and yyyy as child trees.
>
>           Progs/.git
>           Progs/xxxx
>           Progs/yyyy
>
> Does this sound useful to anyone else, or is it already possible?

I would do it like this:

           Progs/.git
           Progs/xxxx/.git
           Progs/yyyy/.git

I do not know what you have in Progs/ hierarchy -- if it is just
scaffolding to house subdirectories and nothing else you may not
even need Progs/.git repository.

This is a very useful and handy structure, and you do not need
any tool support once you have these as separate repositories.
If you want a single distribution point, you can push from these
separate repositories into separate branches of a single
distribution point repository [*1*].

One potential disadvantage is that you would not get revision
linkage between these "modules", but not having tight linkage is
the point of modular structure, so depending on your workflow it
probably may not matter.


[Footnote]

*1* In fact, my git working area has the "master", "next" or
"pu" branch checked out in /path/to/work/area/git (and the
repository is in /path/to/work/area/git/.git directory), and I
check out the "todo" branch in /path/to/work/area/git/Meta
directory, with /path/to/work/area/git/Meta/.git directory as
its repository.

Re: importing cvs logical modules

[Geoff Russell]

On 5/25/06, Junio C Hamano <junkio@cox.net> wrote:
> "Geoff Russell" <geoffrey.russell@gmail.com> writes:
>
> > I'd like to make 1 git repository Progs with xxxx and yyyy as child trees.
> >
> >           Progs/.git
> >           Progs/xxxx
> >           Progs/yyyy
> >
> > Does this sound useful to anyone else, or is it already possible?
>
> I would do it like this:
>
>            Progs/.git
>            Progs/xxxx/.git
>            Progs/yyyy/.git
>
> I do not know what you have in Progs/ hierarchy -- if it is just
> scaffolding to house subdirectories and nothing else you may not
> even need Progs/.git repository.
>
> This is a very useful and handy structure, and you do not need
> any tool support once you have these as separate repositories.
> If you want a single distribution point, you can push from these
> separate repositories into separate branches of a single
> distribution point repository [*1*].
>
> One potential disadvantage is that you would not get revision
> linkage between these "modules", but not having tight linkage is
> the point of modular structure, so depending on your workflow it
> probably may not matter.

The tight linkage is absolutely essential.

When we tag the system, we
want to tag everything (not individually tag all 300 programs)
so that later we can to branch at that tag. Very few of our
programs are self contained, they generally make heavy use
of one or more libraries --- which must also be part of the
tree. The full structure should end up looking like:

        Top/.git
        Libraries/libx
        Libraries/liby
        Libraries/boost
        Libraries/...
        Programs/xxx
        Programs/yyy
        Programs/...
        ...etc.

When we build a full system, we want to tag that point so
we can know exactly what went into that build.

We used to do it the "other" way.

         cvsroot/xxx
         cvsroot/yyy
         cvsroot/liby
         cvsroot/libx
         etc

But then when we did a system build, someone always forgot
to tag something. So we got cvs to build a build logical tree
and then cvs was quite happy to tag the lot with one go and
nothing got forgotten. So now we are trying to pull the
lot into git as one big tree.

Cheers,
Geoff.


>
>
> [Footnote]
>
> *1* In fact, my git working area has the "master", "next" or
> "pu" branch checked out in /path/to/work/area/git (and the
> repository is in /path/to/work/area/git/.git directory), and I
> check out the "todo" branch in /path/to/work/area/git/Meta
> directory, with /path/to/work/area/git/Meta/.git directory as
> its repository.
>
>

Re: importing cvs logical modules

[Martin Langhoff]

On 5/25/06, Geoff Russell <geoffrey.russell@gmail.com> wrote:
> The tight linkage is absolutely essential.
>
> When we tag the system, we
> want to tag everything (not individually tag all 300 programs)
> so that later we can to branch at that tag. Very few of our

Then you want a single git repo/tree/project. The thing is how to work
through your mangled CVS history.

Two options there...

 - Don't. Import from after the last directory reorg or from your last
interesting release. Keep the cvs tree for people who really want to
dig into the past. this has several advantages, as initial checkouts
will be faster, import times shorter, less pain overall.

 - Mangle import to match your reorganizations:
   - Generate the cvsps output file
   - Duplicate/copy files in the cvs repo so that files are in both
places if you've been mving them around.
   - Mangle the cvsps output file from each reorg onwards. This is
nasty, but it will define the history that cvsimport sees.

cheers,


martin

Re: importing cvs logical modules

[Geoff Russell]

On 5/25/06, Martin Langhoff <martin.langhoff@gmail.com> wrote:
> On 5/25/06, Geoff Russell <geoffrey.russell@gmail.com> wrote:
> > The tight linkage is absolutely essential.
> >
> > When we tag the system, we
> > want to tag everything (not individually tag all 300 programs)
> > so that later we can to branch at that tag. Very few of our
>
> Then you want a single git repo/tree/project. The thing is how to work
> through your mangled CVS history.
>
> Two options there...
>
>  - Don't. Import from after the last directory reorg or from your last
> interesting release. Keep the cvs tree for people who really want to
> dig into the past. this has several advantages, as initial checkouts
> will be faster, import times shorter, less pain overall.

Yes, this is definitely on the shortlist of options.
If we can't keep all the history, we may as well make
a clean start. Thanks for the advice.

Cheers,
Geoff.

Re: importing cvs logical modules

[Jakub Narebski]

Geoff Russell wrote:

> On 5/25/06, Martin Langhoff <martin.langhoff@gmail.com> wrote:
>> On 5/25/06, Geoff Russell <geoffrey.russell@gmail.com> wrote:
>>> The tight linkage is absolutely essential.
>>>
>>> When we tag the system, we
>>> want to tag everything (not individually tag all 300 programs)
>>> so that later we can to branch at that tag. Very few of our
>>
>> Then you want a single git repo/tree/project. The thing is how to work
>> through your mangled CVS history.
>>
>> Two options there...
>>
>>  - Don't. Import from after the last directory reorg or from your last
>> interesting release. Keep the cvs tree for people who really want to
>> dig into the past. this has several advantages, as initial checkouts
>> will be faster, import times shorter, less pain overall.
> 
> Yes, this is definitely on the shortlist of options.
> If we can't keep all the history, we may as well make
> a clean start. Thanks for the advice.

Well, you can always copy history and graft old CVS history later. Perhaps
when "bind" (or Cogito "bind lite") mature enough...

-- 
Jakub Narebski
Warsaw, Poland