Mozilla version control requirements and git

Mozilla version control requirements and git

[Jon Smirl]

The Mozilla people have a web page describing what they are looking
for in a new version control system. How does git stack up?

http://wiki.mozilla.org/Version_Control_System_Requirements

They have also recently put up a Subversion server for limited use.

http://weblogs.mozillazine.org/preed/2006/08/subversive_subversion_conversi.html

-- 
Jon Smirl
jonsmirl@gmail.com

-- 
VGER BF report: U 0.506426

Re: Mozilla version control requirements and git

[Martin Langhoff]

On 9/3/06, Jon Smirl <jonsmirl@gmail.com> wrote:
> The Mozilla people have a web page describing what they are looking
> for in a new version control system. How does git stack up?

Hi Jon,

you've been playing with GIT quite a bit by now, so I guess you know
the answer ;-) Is there anything in particular you are wondering
about?

The one item that sticks out for me as not there is ACLs, but access
controls can be implemented in hooks for direct pushes. Maintain an
.htaccess-like file and have a perl script to check it on pushes to
the repo.

Oh, and "partial tree pulls for localisers". Perhaps git-cvsserver can
help there? Localisers can just use TortoiseCVS and get a checkout of
the language pack subdir.

cheers,



martin

-- 
VGER BF report: U 0.544429

Re: Mozilla version control requirements and git

[Jon Smirl]

On 9/2/06, Martin Langhoff <martin.langhoff@gmail.com> wrote:
> On 9/3/06, Jon Smirl <jonsmirl@gmail.com> wrote:
> > The Mozilla people have a web page describing what they are looking
> > for in a new version control system. How does git stack up?
>
> Hi Jon,
>
> you've been playing with GIT quite a bit by now, so I guess you know
> the answer ;-) Is there anything in particular you are wondering
> about?

Shallow clones are a big problem. We have Mozilla down to 450MB is
git, but that is still gigantic for anyone doing an initial check out,
especially if they don't have good broadband. It is over 10x the data
that CVS brings down. Even without doing shallow clones git still
needs to be modified to restart an interrupted pack transfer.

>
> The one item that sticks out for me as not there is ACLs, but access
> controls can be implemented in hooks for direct pushes. Maintain an
> .htaccess-like file and have a perl script to check it on pushes to
> the repo.

I don't see ACLs as that big an issue. If you convert to model where
everyone has their own repo and you just push changesets at each
other, ACLs are much less important. You need ACLs when 2,000 people
have commit privs.

>
> Oh, and "partial tree pulls for localisers". Perhaps git-cvsserver can
> help there? Localisers can just use TortoiseCVS and get a checkout of
> the language pack subdir.

Partial repo pulls and an issue to. The mozilla repo has much more
than a browser in it, it also has a large mail/news program. A partial
repo pull may not be what is needed for git, instead git needs a
partial repo checkout.

-- 
Jon Smirl
jonsmirl@gmail.com

-- 
VGER BF report: U 0.827771

Re: Mozilla version control requirements and git

[Jakub Narebski]

Jon Smirl wrote:

>> Oh, and "partial tree pulls for localisers". Perhaps git-cvsserver can
>> help there? Localisers can just use TortoiseCVS and get a checkout of
>> the language pack subdir.
> 
> Partial repo pulls and an issue to. The mozilla repo has much more
> than a browser in it, it also has a large mail/news program. A partial
> repo pull may not be what is needed for git, instead git needs a
> partial repo checkout.

Or better support for subprojects.

I think you can fake subproject support using separate repositories
for subprojects, in the directory matching the one in the whole project,
and the project repository, where we pull/push subprojects into.
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git



-- 
VGER BF report: U 0.877495

Re: Mozilla version control requirements and git

[Jon Smirl]

On 9/3/06, Jakub Narebski <jnareb@gmail.com> wrote:
> Jon Smirl wrote:
>
> >> Oh, and "partial tree pulls for localisers". Perhaps git-cvsserver can
> >> help there? Localisers can just use TortoiseCVS and get a checkout of
> >> the language pack subdir.
> >
> > Partial repo pulls and an issue to. The mozilla repo has much more
> > than a browser in it, it also has a large mail/news program. A partial
> > repo pull may not be what is needed for git, instead git needs a
> > partial repo checkout.
>
> Or better support for subprojects.
>
> I think you can fake subproject support using separate repositories
> for subprojects, in the directory matching the one in the whole project,
> and the project repository, where we pull/push subprojects into.

You can't really fake it. Many of the projects in Mozilla are
dependent on each other. If you break them up into separate
repositories you lose the ability to do a cross project commit. This
is common when you are changing an interface between the subprojects.

Would it make sense for git to treat a tree of projects like you
describe as a single entity? So a commit would transparently commit to
all of the subprojects. Maybe the commit entry's sha1 could be set up
so that it would be identical in all subprojects. That would let you
string them back together.

> --
> Jakub Narebski
> Warsaw, Poland
> ShadeHawk on #git
>
>
>
> --
> VGER BF report: U 0.877495
> -
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>


-- 
Jon Smirl
jonsmirl@gmail.com

-- 
VGER BF report: S 0.995201

Re: Mozilla version control requirements and git

[Jakub Narebski]

On 03-09-2006 15:46, Jon Smirl wrote:
> On 9/3/06, Jakub Narebski <jnareb@gmail.com> wrote:
> > Jon Smirl wrote:
> >
> > >> Oh, and "partial tree pulls for localisers". Perhaps git-cvsserver can
> > >> help there? Localisers can just use TortoiseCVS and get a checkout of
> > >> the language pack subdir.
> > >
> > > Partial repo pulls and an issue to. The mozilla repo has much more
> > > than a browser in it, it also has a large mail/news program. A partial
> > > repo pull may not be what is needed for git, instead git needs a
> > > partial repo checkout.
> >
> > Or better support for subprojects.
> >
> > I think you can fake subproject support using separate repositories
> > for subprojects, in the directory matching the one in the whole project,
> > and the project repository, where we pull/push subprojects into.
> 
> You can't really fake it. Many of the projects in Mozilla are
> dependent on each other. If you break them up into separate
> repositories you lose the ability to do a cross project commit. This
> is common when you are changing an interface between the subprojects.
> 
> Would it make sense for git to treat a tree of projects like you
> describe as a single entity? So a commit would transparently commit to
> all of the subprojects. Maybe the commit entry's sha1 could be set up
> so that it would be identical in all subprojects. That would let you
> string them back together.

Scratch that. You can fake subproject support using something similar
to the way Junio works with TODO branch. Namely, separate repositories
for subprojects, each as if it would start from it's own directory
(although you can manage without that, it only would be slightly harder
to set it up), and main repository working directory composed from
subproject repositories checked out to correct place. Add to this
ingnoring object database (i.e. .git repositories), and you have whole
repository... where you can do whole repository commits to.

E.g. the following directory structure

	main-project/
	main-project/.git
	main-project/subproject/
	main-project/subproject/.git
	main-project/subproject/...subproject files...

where second .git is in .gitignore or info/excludes, but the subrpoject/
directory itself isn't.


If this is not enough, check out ideas in Subpro.txt in the 'todo' branch
in git repository. It adds additional 'bind' header... the implementation
(search for 'gitlink') were if I remember correctly never fully implemented
and is not in git, although parts of it like git-read-tree and 
git-write-tree --prefix option made it.
  http://www.kernel.org/git/?p=git/git.git;a=blob;f=Subpro.txt;hb=todo


As commit id depends on the contents, including _parents_ and _tree_,
it is I think impossible to set commit entry sha1 to be identical.

P.S. I just hope that it would pass VGER filter. I don't have much luck
with it lately...
-- 
Jakub Narebski
Poland

-- 
VGER BF report: S 0.994192

Re: Mozilla version control requirements and git

[Martin Langhoff]

On 9/4/06, Jon Smirl <jonsmirl@gmail.com> wrote:
> You can't really fake it. Many of the projects in Mozilla are
> dependent on each other. If you break them up into separate
> repositories you lose the ability to do a cross project commit. This
> is common when you are changing an interface between the subprojects.

In many projects this is handled by having tags and generally
versioning modules and interfaces. Being independent gives the
subprojects/modules a lot more freedom to work/branch on wild
tangents, and the versioned interfaces mean that -- with the
contraints of the versioned interfaces -- you can mix and match
branches/releases of the different subprojects.

In short, I'd definitely break those up :-)

cheers,




martin

-- 
VGER BF report: U 0.513231

Re: Mozilla version control requirements and git

[Junio C Hamano]

"Jon Smirl" <jonsmirl@gmail.com> writes:

> Partial repo pulls and an issue to. The mozilla repo has much more
> than a browser in it, it also has a large mail/news program. A partial
> repo pull may not be what is needed for git, instead git needs a
> partial repo checkout.

That does not contradict with the underlying data model of git,
and it would certainly be possible with a specialized Porcelain.

Since git tracks the entire tree, the tree object contained in
the commit you create _must_ describe the entire project tree,
but that does not necessarily mean that your working tree needs
to have the entire tree checked out.

The git corelevel is designed to be operable with sparsely
populated working tree, and the problem you are seeing is
primarily coming from the fact that some wrappers you are seeing
as "git UI" are specifically done to help workflows that _wants_
to have fully populated working tree (e.g. "git commit -a" and
"git checkout").  There are other wrappers that are designed to
work well in sparsely populated working tree, including "git-am"
and "git-merge".  They should be usable pretty much as they are.

To support "narrow checkout" workflow, some wrappers either need
to be told about the "interesting area", or a set of different
wrappers specifically designed to support "narrow checkout"
workflow can be written.  I am not sure which is more desirable.
I have a gut feeling that "git commit" can be used as is (you cd
into the area you are interested in, and say "git commit ."
instead of saying "git commit -a"), while it would be less
confusing to keep "git checkout" the whole-tree operation and
have a specialized porcelain for a narrow checkout, but this is
me saying things without thinking things through.

I imagine such a "narrow checkout" workflow Porcelain would work
like this:

 - the user tells it which parts of the whole project she is
   interested in.

 - when checking things out, the Porcelain populates the index
   in full [*1*] and then checks out only specified subtrees to
   the working tree.

 - when checking things in, the changes outside the specified
   area are ignored (well, outside there should be no files in
   working tree so ignoring them should be trivial).  write-tree
   writes from the whole index and creates the tree object and
   then commit object just as usual.  The net effect is that the
   parts outside the areas of interest stay the same as the
   previous HEAD commit, which is exactly what the user wanted
   anyway.

You could optimize the above further by populating the index
sparsely as well (i.e. *1* above is not a requirement), but you
would need to merge in the changes made to that sparse index
into the full tree read from the HEAD commit before writing the
next tree object out anyway, so I doubt it is worth it.


-- 
VGER BF report: U 0.842228

Re: Mozilla version control requirements and git

[Jon Smirl]

On 9/3/06, Martin Langhoff <martin.langhoff@gmail.com> wrote:
> On 9/4/06, Jon Smirl <jonsmirl@gmail.com> wrote:
> > You can't really fake it. Many of the projects in Mozilla are
> > dependent on each other. If you break them up into separate
> > repositories you lose the ability to do a cross project commit. This
> > is common when you are changing an interface between the subprojects.
>
> In many projects this is handled by having tags and generally
> versioning modules and interfaces. Being independent gives the
> subprojects/modules a lot more freedom to work/branch on wild
> tangents, and the versioned interfaces mean that -- with the
> contraints of the versioned interfaces -- you can mix and match
> branches/releases of the different subprojects.
>
> In short, I'd definitely break those up :-)

Mozilla always ships as a complete system so there is no need to deal
with old versions of the pieces. Only the external interfaces are
frozen and versioned. This is similar to the Linux kernel. The user
space API is frozen, but when an internal device driver interface
changes all of the drivers in the kernel tree get updated in lock
step. Doing it this way removes a lot of overhead in maintaining
binary compatibility.

-- 
Jon Smirl
jonsmirl@gmail.com

-- 
VGER BF report: U 0.502564