Subproject status

Subproject status

[Daniel Barkalow]

I remember that last time I checked, there were a number of designs for 
subprojects and at least a couple of implementations, but none that was 
complete to the point of being mergeable. Are there any subproject 
implementations available that haven't run into unsolveable problems? I'm 
presently only looking at stuff that totals to a reasonable single project 
size and rate of growth, so I'm not worried about the large-scale storage 
requirement issue.

	-Daniel
*This .sig left intentionally blank*

Re: Subproject status

[Jakub Narebski]

[Cc: Daniel Barkalow <barkalow@iabervon.org>, git@vger.kernel.org]

Daniel Barkalow wrote:

> I remember that last time I checked, there were a number of designs for 
> subprojects and at least a couple of implementations, but none that was 
> complete to the point of being mergeable. Are there any subproject 
> implementations available that haven't run into unsolveable problems? I'm 
> presently only looking at stuff that totals to a reasonable single project 
> size and rate of growth, so I'm not worried about the large-scale storage 
> requirement issue.

Check out http://git.or.cz/gitwiki/SubprojectSupport


Could _please_ anyone who wrote some kind of submodule support, add info
about it (links to archive, to repository, perhaps to working prototype)
to this page? Thanks in advance!
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: Subproject status

[Han-Wen Nienhuys]

Jakub Narebski wrote:
> [Cc: Daniel Barkalow <barkalow@iabervon.org>, git@vger.kernel.org]
> 
> Daniel Barkalow wrote:
> 
>> I remember that last time I checked, there were a number of designs for 
>> subprojects and at least a couple of implementations, but none that was 
>> complete to the point of being mergeable. Are there any subproject 
>> implementations available that haven't run into unsolveable problems? I'm 
>> presently only looking at stuff that totals to a reasonable single project 
>> size and rate of growth, so I'm not worried about the large-scale storage 
>> requirement issue.
> 
> Check out http://git.or.cz/gitwiki/SubprojectSupport

this points to

  http://www.kernel.org/git/?p=git/git.git;a=blob;hb=todo;f=Subpro.txt

which is a broken. It results in



1Reading blob failed.


Han-Wen

Re: Subproject status

[Han-Wen Nienhuys]

Jakub Narebski wrote:
> [Cc: Daniel Barkalow <barkalow@iabervon.org>, git@vger.kernel.org]
> 
> Daniel Barkalow wrote:
> 
>> I remember that last time I checked, there were a number of designs for 
>> subprojects and at least a couple of implementations, but none that was 
>> complete to the point of being mergeable. Are there any subproject 
>> implementations available that haven't run into unsolveable problems? I'm 
>> presently only looking at stuff that totals to a reasonable single project 
>> size and rate of growth, so I'm not worried about the large-scale storage 
>> requirement issue.
> 
> Check out http://git.or.cz/gitwiki/SubprojectSupport

this points to

  http://www.kernel.org/git/?p=git/git.git;a=blob;hb=todo;f=Subpro.txt

which is a broken. It results in



1Reading blob failed.


Han-Wen

Re: Subproject status

[Jakub Narebski]

Han-Wen Nienhuys wrote:

> Jakub Narebski wrote:

>> Check out http://git.or.cz/gitwiki/SubprojectSupport
> 
> this points to
> 
>   http://www.kernel.org/git/?p=git/git.git;a=blob;hb=todo;f=Subpro.txt
> 
> which is a broken. It results in
> 
> 
> 
> 1Reading blob failed.

Thanks. Subpro.txt was removed from todo, as it described old, abandoned
idea about subproject support (the "mountpoint" in commit object one).

Fixed (the link, not gitweb: I don't know where this '1' comes from).
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: Subproject status

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 2031 bytes --]

hoi :)

On Sun, Mar 25, 2007 at 01:37:25PM -0400, Daniel Barkalow wrote:
> I remember that last time I checked, there were a number of designs for 
> subprojects and at least a couple of implementations, but none that was 
> complete to the point of being mergeable. Are there any subproject 
> implementations available that haven't run into unsolveable problems? I'm 
> presently only looking at stuff that totals to a reasonable single project 
> size and rate of growth, so I'm not worried about the large-scale storage 
> requirement issue.

You can try to play with my prototype:
git.admingilde.org/tali/git.git, branch module2
(to get an example of how to use it, look at the test script in
t/t7500-submodule.sh).

The core operations should work, but not all the user interface is
adapted to support submodules.  E.g. git-status will not show if a
submodule has dirty changes, it will only show submodule changes which
are already commited to the submodule but not yet to the supermodule.
But at least simple merges do work with submodules.

I abondoned this branch (no further work, only occasionally pulling in
updates from upstream git.git) as it has scalability problems with
large projects.
At the moment I'm working on the module3 branch, which tries to address
the scalability problem by separating the object store.  This is still
in the phase of early prototyping, just to see if it can work.
But I am already at the point that I am confident to be able to finish
it this way.

The objects created by the module2 and module3 branches are the same,
module3 only moves those belonging to the submodule to another location.
So If you start using module2 branch now you should be able to upgrade
later.  But to be extra sure, we should have some discussion about the
object format here.  (There is nothing new here, really: Just one more
tree entry file mode which says that this is not a file or directory
entry, but a submodule, represented by one commit.)

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Subproject status

[Daniel Barkalow]

On Mon, 26 Mar 2007, Martin Waitz wrote:

> You can try to play with my prototype:
> git.admingilde.org/tali/git.git, branch module2
> (to get an example of how to use it, look at the test script in
> t/t7500-submodule.sh).

I tried that yesterday, but I fail reading comprehension; I lost the 
"module2" bit by the time I actually fetched. I'll look into this.

> The core operations should work, but not all the user interface is
> adapted to support submodules.  E.g. git-status will not show if a
> submodule has dirty changes, it will only show submodule changes which
> are already commited to the submodule but not yet to the supermodule.
> But at least simple merges do work with submodules.
> 
> I abondoned this branch (no further work, only occasionally pulling in
> updates from upstream git.git) as it has scalability problems with
> large projects.

That's what I remember. But it's only an issue with super-scale projects 
(i.e., where your subprojects are full-sized projects in their own right, 
and you've got a hundred of them), right? I'm working on the other end 
(factoring out the common bit from a bunch of similar small projects), so 
I should be fine with respect to scalability of the implementation.

Would you guess that a patch series to complete the module2 user interface 
adaptation would also apply to module3 and therefore be useful in the 
future?

> The objects created by the module2 and module3 branches are the same,
> module3 only moves those belonging to the submodule to another location.
> So If you start using module2 branch now you should be able to upgrade
> later.  But to be extra sure, we should have some discussion about the
> object format here.  (There is nothing new here, really: Just one more
> tree entry file mode which says that this is not a file or directory
> entry, but a submodule, represented by one commit.)

So, I had the nutty idea that it would be convenient if I could make 
different files in a single directory come from different projects. But I 
can't think of a sane user interface, so I think that this isn't 
practical from that direction, so it's probably not worth worrying about 
from the data structure end, either. (Answer for the usecase: 
"ln -s make/Makefile Makefile; git add Makefile", and mock systems that 
don't handle symlinks).

But, just to be clear, the semantics of having a commit abcd at a path foo 
are, with respect to the tree this represents, that abcd:* appears at 
foo/*. Right?

Are there any standards we should discuss with respect to refs related to 
subprojects?

I've updated the wiki page with this information.

	-Daniel
*This .sig left intentionally blank*

Re: Subproject status

[David Lang]

On Mon, 26 Mar 2007, Daniel Barkalow wrote:

>
> So, I had the nutty idea that it would be convenient if I could make
> different files in a single directory come from different projects. But I
> can't think of a sane user interface, so I think that this isn't
> practical from that direction, so it's probably not worth worrying about
> from the data structure end, either. (Answer for the usecase:
> "ln -s make/Makefile Makefile; git add Makefile", and mock systems that
> don't handle symlinks).
>

this would definantly be useful (managing files in /etc or /home are easy 
examples

for this sort of thing doing 'git add file -index project' would be a fair 
first-cut at a UI. later on how about creating a dummy .git directory that would 
record 'file X is part of project Y' so that you could get away with only 
specifying it once, and then either have a default, or have git add complain if 
there are multiple places to put things and you don't specify which one (and if 
you add fules to multiple projects at the same time then you could either prompt 
for additional checkin comments as needed, or use the same comment for all 
projects)

David Lang