Can you do this with GIT?

Can you do this with GIT?

[CPD]

Hi All,

I hope this is the right forum, it's all I could find. Sincere apologies in
advance if I in the wrong place.

I set up a source control system for the company around CVS, but GIT has
some very attractive features and I'd like to migrate if it can do some
other things that we need.

We produce variations based on a (mostly) common codebase. In CVS I set up
"environment" modules for each platform, then when you are working on that
platform, you simply check out the correct environment and build. Only the
needed code and tools are exposed in that environment (this is important as
clients must NOT see each other's code and most customers have some
customization). I do this by defining and renaming modules in the CVSROOT
modules file.

Does GIT support anything like this? Or another way to acheive the same end?

Thanks muchly, in advance, Colin
-- 
View this message in context: http://www.nabble.com/Can-you-do-this-with-GIT--tf4159350.html#a11834063
Sent from the git mailing list archive at Nabble.com.

Re: Can you do this with GIT?

[Johannes Schindelin]

Hi,

On Fri, 27 Jul 2007, CPD wrote:

> I hope this is the right forum, it's all I could find. Sincere apologies 
> in advance if I in the wrong place.

You might be interested in http://git.or.cz: there is a lot of useful 
information to find; amongst others what is the right forum...

So yes, this is the right forum.

> I set up a source control system for the company around CVS, but GIT has
> some very attractive features and I'd like to migrate if it can do some
> other things that we need.
> 
> We produce variations based on a (mostly) common codebase. In CVS I set 
> up "environment" modules for each platform, then when you are working on 
> that platform, you simply check out the correct environment and build. 
> Only the needed code and tools are exposed in that environment (this is 
> important as clients must NOT see each other's code and most customers 
> have some customization). I do this by defining and renaming modules in 
> the CVSROOT modules file.

I would use branches for that.  A base branch with the common code, and 
the customisations in all the branches, which merge from the base branch.

If you have an interesting change in a custom branch, you can percolate 
that back into the base branch, by checking out that base branch and 
cherry-picking the commit you want to have.

Of course, you can automate this merging (or even rebasing, if you are 
interested in keeping your customisations nice and tidy) with a script.  
With a git alias even.

Hth,
Dscho

Re: Can you do this with GIT?

[Ray Lehtiniemi]

On Friday 27 July 2007 13:05, Johannes Schindelin wrote:
> On Fri, 27 Jul 2007, CPD wrote:

> > We produce variations based on a (mostly) common codebase. In CVS I set
> > up "environment" modules for each platform, then when you are working on
> > that platform, you simply check out the correct environment and build.
> > Only the needed code and tools are exposed in that environment (this is
> > important as clients must NOT see each other's code and most customers
> > have some customization). I do this by defining and renaming modules in
> > the CVSROOT modules file.
>
> I would use branches for that.  A base branch with the common code, and
> the customisations in all the branches, which merge from the base branch.

this would break down if there were client-specific modules in the base 
branch, though... how could those be hidden from the other clients?


ray

Re: Can you do this with GIT?

[Linus Torvalds]

On Fri, 27 Jul 2007, CPD wrote:
> 
> We produce variations based on a (mostly) common codebase. In CVS I set up
> "environment" modules for each platform, then when you are working on that
> platform, you simply check out the correct environment and build. Only the
> needed code and tools are exposed in that environment (this is important as
> clients must NOT see each other's code and most customers have some
> customization). I do this by defining and renaming modules in the CVSROOT
> modules file.
> 
> Does GIT support anything like this? Or another way to acheive the same end?

Depending on how big the individual pieces are, you can do it either with 
branches or subprojects.

If the "mostly common" codebase means that 90% of it really ends up being 
common, and what you have is a few config things and perhaps a library or 
two that either shows up or not, git branches would work really well. 

And branches are totally independent of each other, so it's quite possible 
to have even "secret" things in one branch, and it would never get exposed 
to anybody else. The biggest worry you'd need to make sure of is that you 
don't want to merge such private branches directly with each other - since 
then the merged branch would see everything that was in the history of the 
other branch, even if the merge itself ended up not picking the "secret 
stuff".

So with branches that have private information, you'd have to have one 
"common" branch that doesn't have any of the private stuff at all, and do 
the general development on that, and then you can merge that common stuff 
into all the individual private branches (and if you want to go the other 
way, moving something from a private branch to the "common area", you'd 
have to cherry-pick just that particular detail into the common branch).

If you have bigger building blocks, and some of the whole building blocks 
need to be private, you'd probably want to use the subproject support, 
which would make for more of a CVS "modules" kind of thing and in that 
sense perhaps closer to what you do now, but I suspect that just using 
branches would actually end up being much simpler for you. It would depend 
mostly on how big those building blocks that you need to keep separate 
are.

		Linus

Re: Can you do this with GIT?

[Johannes Schindelin]

Hi,

On Fri, 27 Jul 2007, Ray Lehtiniemi wrote:

> On Friday 27 July 2007 13:05, Johannes Schindelin wrote:
> > On Fri, 27 Jul 2007, CPD wrote:
> 
> > > We produce variations based on a (mostly) common codebase. In CVS I set
> > > up "environment" modules for each platform, then when you are working on
> > > that platform, you simply check out the correct environment and build.
> > > Only the needed code and tools are exposed in that environment (this is
> > > important as clients must NOT see each other's code and most customers
> > > have some customization). I do this by defining and renaming modules in
> > > the CVSROOT modules file.
> >
> > I would use branches for that.  A base branch with the common code, and
> > the customisations in all the branches, which merge from the base branch.
> 
> this would break down if there were client-specific modules in the base 
> branch, though... how could those be hidden from the other clients?

Umm.  Don't put the client-specific modules in the base branch, then?  The 
base branch is the common code, the code that every client may look at.  
Nothing else.

Maybe I did not get the whole picture...  do you want your _clients_ to 
access your main repo with Git?

Ciao,
Dscho

Re: Can you do this with GIT?

[Ray Lehtiniemi]

On Friday 27 July 2007 12:02, CPD wrote:

> We produce variations based on a (mostly) common codebase. In CVS I set up 
> "environment" modules for each platform, then when you are working on that
> platform, you simply check out the correct environment and build. Only the
> needed code and tools are exposed in that environment (this is important as
> clients must NOT see each other's code and most customers have some
> customization). I do this by defining and renaming modules in the CVSROOT
> modules file.

this sounds very close to what i'm trying to achieve at the moment.


> Does GIT support anything like this? Or another way to acheive the same
> end?

i couldn;t find any ready way to do it.... but i've hacked up a few 
preliminary thoughts on how this might work in git.  comments and feedback 
are welcome :-)

the basic idea is to import each snapshot of each vendor release as a tagged 
root commit, then push all these tagged root commits into a central "library" 
repository. each tagged root commit stands alone and functions as a cvs 
module.

library repos are set up as remotes under git, and a simple script will import 
a set of snapshots into a brand new project as a baseline for customization.

the three shell scripts, and the two control files used by them, are included 
here:

==> vendor-tracking.sh <==
###########################################################
#
# Import a vendor snapshot into a "library" repository.
#
# To import a vendor snapshot, unzip the snapshot into
# a new folder.  Then go into the folder and type
#
#  xxx-import <library> <tag>
#
# This will create an empty git repo in the current
# snapshot folder and create a root commit in 
# it which holds the snapshot.  This root commit will be
# tagged as "tag" and then pushed into the "library" repo.
#
# No error checking is done.
#
###########################################################
xxx-import ()
{(
	repo=$1
	tag=$2

	export GIT_DIR=/tmp/$$.$tag

	rm -rf $GIT_DIR
	mkdir -p $GIT_DIR

	git init
	git add .
	git commit -s -m "Import $tag"
	git tag $tag master
	git push $repo refs/tags/$tag:refs/tags/$tag

	rm -rf $GIT_DIR
)}

###########################################################
#
# Use a vendor release in the current project.
#
# To use a copy of some vendor snapshot in the current
# project, go to the top level of your current project,
# then type
#
#  xxx-use <library> <tag> <path>
#
# The tagged snapshot will be fetched from the library
# and deposited into the <path> subdirectory, then committed
# to the project.
# 
# No error checking is done.
#
###########################################################
xxx-use ()
{(
	repo=$1
	tag=$2
	path=$3

	git fetch $repo refs/tags/$tag:refs/tags/$tag
	git read-tree --prefix=$path/ $tag
	git commit -s -m "Use $tag as $path"
)}

###########################################################
#
# Create a new project from a set of vendor components.
#
# To start a new project with a known array of vendor
# components, create a remotes file describing the libraries
# and an manifest file describing the components and how they
# will be used. Then, in a new folder, type
#
#  xxx-start <remotes> <manifest>
#
# This will create a new Git project in the current folder,
# add a remote for each library repository, then proceed to
# fetch and commit each vendor component into the new project.
# The new project is now ready to be customized.
# 
# No error checking is done.
#
###########################################################
xxx-start ()
{(
	remotes=$1
	manifest=$2

	git init

	cat $remotes |
	while read name url
	do
		git remote add $name $url
	done

	cat $manifest |
	while read path remote tag
	do
		xxx-use $remote $tag $path
	done

	git checkout -f master
)}


==> remotes <==
lib1	ssh://me@home.com/home/me/work/library
lib2	//filesrv/git/library

==> manifest <==
path/to/A		lib1	vendorA/componentA/version1
path/to/B		lib2	vendorB/componentB/version2






i would prefer to import and tag raw tree objects, since i get some of my 
snapshots as pre-assembled collections of (modified copies of) several 
components.  it would therefore be convenient to have several tags per 
snapshot pointing at the subtrees corresponding to each component.  however,  
fetch and push seem to prefer commits over trees.... 

as it stands, the vendor snapshots have no "history" to them.  each snapshot 
stands in isolation.  there was an interesting thread a few days ago about 
importing ancient kernel history as a series of trees, then "stitching 
together" a "synthetic history" from a collection of trees.  i would see 
something like that as the basis for "vendor tracking branches", which could 
be restitched if an older snapshot is received "out of order".


hope it helps
ray

Re: Can you do this with GIT?

[Ray Lehtiniemi]

On Friday 27 July 2007 13:49, Johannes Schindelin wrote:
> Hi,
>
> On Fri, 27 Jul 2007, Ray Lehtiniemi wrote:
> > On Friday 27 July 2007 13:05, Johannes Schindelin wrote:
> > > On Fri, 27 Jul 2007, CPD wrote:
> > > > We produce variations based on a (mostly) common codebase. In CVS I
> > > > set up "environment" modules for each platform, then when you are
> > > > working on that platform, you simply check out the correct
> > > > environment and build. Only the needed code and tools are exposed in
> > > > that environment (this is important as clients must NOT see each
> > > > other's code and most customers have some customization). I do this
> > > > by defining and renaming modules in the CVSROOT modules file.
> > >
> > > I would use branches for that.  A base branch with the common code, and
> > > the customisations in all the branches, which merge from the base
> > > branch.
> >
> > this would break down if there were client-specific modules in the base
> > branch, though... how could those be hidden from the other clients?
>
> Umm.  Don't put the client-specific modules in the base branch, then?  The
> base branch is the common code, the code that every client may look at.
> Nothing else.

yes, i don;t think there's any other way to do it with branches (and just be 
careful not to merge the private branches back! :-)


> Maybe I did not get the whole picture...  do you want your _clients_ to
> access your main repo with Git?

not in my case, anyway... although if you define "client" to 
include "subcontractor", then yes, i'd be interested in going down that 
road... sometimes there are pieces which they aren't licensed to see, but if 
we could somehow track their work in a separate repo and then easily merge it 
back into the fully licensed tree, that would have great value.

from the original message, i also keyed on "Only the needed code and tools are 
exposed in that environment".  this is something i'd like to see too... i'm 
working in a software eco-system with hundreds of standalone components, and 
would like the ability to pick and choose a small handful of those for any 
given project, without pulling in reams of history on the stuff i'm not 
using...

thanks
ray


>
> Ciao,
> Dscho