Handling non-git config files

Handling non-git config files

[Richard Lee]

Hi again git-list,

I have a workflow-related query.

I understand that git is for source code mangement. However in certains
applications like web applications in a live environment, it ends up
storing data related to the state of the application as well.

I myself am a web developer and for me git ends up storing data like the
root path of the web app. I would like to work on a test rig, commit and
push the changes to a central repo. Then pull the changes on to the live
server. Having different config files on the test and live deployments
make this workflow difficult as I don't know how to tell git to handle
the different config files. I have managed to do this with patches, but
I do not thing it is good in the long run.

I would put the config data in it's own location, but unfortunately I
have been given a product to work with and I cannot do that. This means
that config data that I do wish to commit is in the same file as data
that would vary from deployment to deployment.

Currently I only work on the live server with git. Firstly this is not
ideal to experiment on a live site. Secondly my colleagues now want to
learn git and you can't have several people performing git operations on
the same working directory.

So my quesstion is that is there any way to have several checked out
copies of a git repo each with their own slightly different config
files, yet still being able to perform git operations with respect to a
centralised repository as if they were identical?

I've thought about using a migration script for each target deployment
and then ignoring any changes related to the deployment. I've also
considered having each deployment as a seperate branch and then rebasing
changes back and forth. However this seems uneccesarily complicated and
I teaching git beginners about rebasing doesn't seem like a good idea.

The first solution give a me a corrolary question. I use tig for staging
changes. In the config file, the lines specific to the deployment can
end up in the same hunk as lines specific to the application that I do
want to keep. Can I stage partial hunks in tig? Or do I have to use git
add --interative?

Regards,

Richard

RE: Handling non-git config files

[Tim Mazid]

Hey Richard,
 
I am by no means an expert, or the sharpest knife in the drawer, for that matter, but let me see if I can understand what you're trying to do.


> I understand that git is for source code mangement. However in certains
> applications like web applications in a live environment, it ends up
> storing data related to the state of the application as well.

Do you mean that your app creates a file like 'users.dat' or similar that you do not wish to track with git? If so, you can simply not add it the repo, and add it to your .gitignore file.
 
 
> I myself am a web developer and for me git ends up storing data like the
> root path of the web app. I would like to work on a test rig, commit and
> push the changes to a central repo. Then pull the changes on to the live
> server. Having different config files on the test and live deployments
> make this workflow difficult as I don't know how to tell git to handle
> the different config files. I have managed to do this with patches, but
> I do not thing it is good in the long run.

Once again, do you need to track the file that stores this data with git?
One (probably not very elegant) solution might be to store all location independant data in one config file, to be tracked by git, and a script generated "local" config file ignored by git.
 
 
> I would put the config data in it's own location, but unfortunately I
> have been given a product to work with and I cannot do that. This means
> that config data that I do wish to commit is in the same file as data
> that would vary from deployment to deployment.

Well, that just killed my previous idea. But is there no possibility at all of editing the code to achieve that?
Unfortunately, I do not believe git can ignore half a file.
 
 
> Currently I only work on the live server with git. Firstly this is not
> ideal to experiment on a live site. Secondly my colleagues now want to
> learn git and you can't have several people performing git operations on
> the same working directory.

Well, you COULD have multiple people working in the same working directory, but that will present all the usual problems of multiple access to a single file (not very fun at all.)
Why not just have them clone their own copy of the repo?
 

> So my quesstion is that is there any way to have several checked out
> copies of a git repo each with their own slightly different config
> files, yet still being able to perform git operations with respect to a
> centralised repository as if they were identical?

By checked out do you mean a 'git clone'? If so, yes, you can have as many as you want. That is (I think) one of the major points of git.
 
 
> I've thought about using a migration script for each target deployment
> and then ignoring any changes related to the deployment. I've also
> considered having each deployment as a seperate branch and then rebasing
> changes back and forth. However this seems uneccesarily complicated and
> I teaching git beginners about rebasing doesn't seem like a good idea.

The branches for deployments could work. I see no need for rebasing, though. You merely add deployment-specific changes to those branches, and all other changes to your main branch, which you then merge into the deployment-specific branches.
Unless I've missed something, which is likely, that should work just fine.
 
 
> The first solution give a me a corrolary question. I use tig for staging
> changes. In the config file, the lines specific to the deployment can
> end up in the same hunk as lines specific to the application that I do
> want to keep. Can I stage partial hunks in tig? Or do I have to use git
> add --interative?

Unfortunately, I do not know tig and cannot comment on that. I do know, however, that git gui allows you to stage content by hunks, and even line by line, which I often find useful. Is there any reason you are unable/unwilling to use git gui?
 
 
All in all, quite a useless reply, but hopefully you will elaborate further as to the data file situation (which is your major concern, unless I'm mistaken, but requires a little more explanation as to why structural changes cannot be made), which can assist someone knowledgable work this out.
 
 
Regards,
Tim. 		 	   		  
_________________________________________________________________
Find a great deal on your next car. Get straight to the Point.
http://clk.atdmt.com/NMN/go/157637060/direct/01/

Re: Handling non-git config files

[Ian Hobson]

Richard Lee wrote:
> So my quesstion is that is there any way to have several checked out
> copies of a git repo each with their own slightly different config
> files, yet still being able to perform git operations with respect to a
> centralised repository as if they were identical?
>
>   
Hi Richard,

Yes. They are called branches :)

What I do is have a branch for each version that I need.

To fix a problem I checkout master, make the repair, and commit.

Then to deploy that change I perform three steps (for each production 
version).

git checkout <clientBranch>
git rebase master
rsync to the production server (ignoring .git and temp files)

All the differences between versions - config files, images, logos, etc 
- are all included in the GIT,
repo and I don't have to worry about them. To set up the branches, I 
simply checked out a new branch for each and applied the changes for 
that production version, and committed. It works very well in practise. 
(Do take care to checkout the version you want to work on before you 
start work, or you may have to lose your work to recover!).

Ian

Re: Handling non-git config files

[Jon Seymour]

[ Richard sent his own copy ]

I have used git in a deployment scenario and found it to be very useful.

The way I dealt with this problem was to treat the variants as a build
problem. During the build, I create a version of the deployment
artifacts that reflect a generic, uncustomized server. This gets
checked into a branch. To build the an environment specific variant, I
checkout the build result for the uncustomized server on a new branch,
customize it and commit that to the tip of the environment specfic
branch. The process continues down the hierarchy, so I take the
environment specific variant, customize it for particular servers and
commit that to a new branch. Each branch is represented in git as a
merge of the current build with the results of the previous build for
the previous for the same branch. [ Thereby allowing FF merges during
deployment ]

The end result is that I have a git repo with one branch for the
abstract server, one for each environment (all uncustomized or
partially customized) and one full customized branch for each physical
server. The whole repo can then be pushed to all environments (but not
checkout there). Deployment is then simply a question of pulling (or
checking out) the right tag of the right branch on each server at the
right time (e.g. after testing, during change windows etc).

Pulling has the advantage of preserving adhoc configuration changes in
deployed environments that have not been folded back into the build
process, though at some risk of merge conflicts &/or inconsistency. My
approach in the case of merge conflicts was to report the problem,
save the current state (in the git history) and resolve in favour of
the build product. Alternative strategies might be to abort the
deployment until the conflict is resolved, etc.

(I also used a separate aggregation technique to aggregate several
distinct repo into a single hub repo so that as to minimise the number
of git operations required to sync two nodes in the deployment
topology.)

Using git in this way was great - deployment of an incremental release
was super quick because the repos could be pre-deployed and transfers
typically only included files that had changed for the release - gone
are the big tar balls being transferred that contain mostly unchanged
stuff.

Re: Handling non-git config files

[rhlee]

Git-list,

Thanks to Jon, Ian and Tim for your replies.

I was reading Jon's reply and the workflow he uses seems to fit my purpose.
To migrate an application, you would branch it, set any local configuration
and commit. To bring over any changes you would just merge them over. This
would leave any deployment artifacts (a great term to describe local
changes) intact.

I have only been using branching and merging since November, when I was
instructed how to do so by git-list. I realise now that this is just basic
branching and merging. The workflow described will work with any SCM
software that supports branching and merging.

Richard
-- 
View this message in context: http://n2.nabble.com/Handling-non-git-config-files-tp4622419p4701369.html
Sent from the git mailing list archive at Nabble.com.