Re: RFC: Subprojects

[A Large Angry SCM <gitzilla@gmail.com>]

Daniel Barkalow wrote:
> On Mon, 16 Jan 2006, A Large Angry SCM wrote:
> 
>>Daniel Barkalow wrote:
>>[...]
>>>So the problem with handling subprojects with the build system is that it is
>>>too tempting to use the revision control system directly on the subproject,
>>>at which point the thing you're developing and testing isn't at all what
>>>other people will get if they check out your commit. You want "git status"
>>>to report it as an uncommitted change if you have a different revision of
>>>the subproject than your previous commit had, and it can't tell if this
>>>information is buried in the build system.
>>Using "git-status" is the wrong tool to use there. What you should be using is
>>"make project_status". Claiming "that it is too tempting to use the revision
>>control system on the subproject" is wrong; you should use the SCM (of the
>>subproject) to manage the subproject. You use the build system to manage the
>>_entire_ project.
> 
> I'm talking about using "git status" on the main project, in case you're 
> misunderstanding me. If you can manage the entire project with the build 
> system, then you don't need git or any version control at all, aside from 
> your build system. But you'd also lose the ability to use webgit, bisect, 
> gitk, git log, and so forth on the project as a whole.

When you say "main project", do you mean the top level project and all 
of its subprojects? Or just the top level project without any of its 
subprojects?

A build system and a SCM working together can be a powerful very tool, 
even if one or both of the components is not. Since managing a project 
with a build system and no SCM means that you don't have any history and 
since you seem to want access to the project's history, I'll ignore you 
statement. (Backups and directory snapshots are primitive SCMs.)

Consider the following:

1) For a project to be a sub-project, it must also be a project.

2) So the standard SCM tools will work on the project.

3) The super-project is also a project and the standard SCM tools will 
work on it.

4) Projects managed by an SCM are only considered consistent and usable 
at specific points in that project's history; it may be every recorded 
point in it's history or it may be just a few specific recorded points.

5) A super-project only cares about specific states of its sub-projects, 
corresponding to points in the sub-project's history.

6) For each sub-project, the super-project needs to record the 
sub-project's state identifier for each recorded point in the 
super-project's history.

7) The super-project can record each sub-project's state identifier 
somewhere in the build system.

8) If the super-project's SCM is Git then webgit, bisect, gitk, git log, 
and so forth all work and will identify when and where the recorded 
state identifier of each sub-project is changed.

9) The information is available to the build system to permit using 
git-bisect in the super-project and notice that the breakage occurred 
when the recorded state identifier of a sub-project changed. If the 
sub-project used Git, the build system can automatically start 
git-bisect'ing in the sub-project.

10) The information is available to the build system to permit doing 
similar things for git-log and so forth.

11) Webgit and gitk are a little more work but by creating a git 
repository that contains all of the history and refs of each project in 
the entire project, you can navigate and view the history of any project 
in the entire project.

12) If the SCM of some of the sub-projects is not Git, the build system 
can still do git-bisect, git-log, etc. equivalents for the entire 
project (subject to the limitations of the SCMs involved).


>>>The tricky question is whether we should permit the "subproject" objects to
>>>specify a revision that isn't a hash, for use in identifying revisions of
>>>subprojects in other systems.
>>Why would you want to limit how required versions of subprojects are
>>specified? Your project policies and procedures may require that subprojects
>>be specified by a subproject SCM specific immutable revision but the policies
>>and procedures of other projects may not be so restrictive and could accept a
>>tag identifying the latest "stable" (or something) revision.
> 
> If you accept a tag identifying the latest stable revision, then you might 
> as well not bother. The point of revision controlling a project is to be 
> able to reconstruct previous states. If you allow any event, especially 
> outside, unrelated, events to change the reconstructed state for a 
> revision, then this is not the case. Your normal debugging situation will 
> be "It's broken, and I didn't change anything." because someone somewhere 
> else changed something, and you have no record of what last worked. And 
> you can obviously forget any hope of "git bisect" working.

CVS does not have the concept of "hash" for use in identifying the state 
of a particular set of files but it does have tags and CVS tags work 
like Git tags. The form of identifier that is used to identify the 
particular state of interest of a sub-project is dependent on the 
policies and procedures of the super-project, and the SCM of the 
sub-project.

The need to reproduce a particular state at sometime in the future is 
well understood, even in organizations that use tools that only support 
tags. Implying that it's not possible without using Git's immutable 
hashes as the state identifier is just wrong.

--