[RFC] separate .git from working directory

[RFC] separate .git from working directory

[Nguyen Thai Ngoc Duy]

Hi,
I was thinking about this while reading subproject thread. In a simple
case, I have a repo A located at ~/project-a and another repo B
located at ~/project-a/some/dir/project-b. With this setup, command
"find" and other directory-recursive commands will run horribly from
~/project-a when they go inside project-b/.git (no I don't want to
repack -d everytime I want to find something).
I propose to move project-b/.git outside and place a file, say
.gitdir, in project-b directory. git-sh-setup and setup_git_directory
are taught to recognize .gitdir, read it to find the actual GIT_DIR
recorded inside .gitdir. This way git commands inside project-b should
work fine while I can "find ~/project-a -name blah" or "grep -R blah"
quickly.
.gitdir format could be  a simple shell-like format with environment
variable assignments.
BTW, talking about subproject support, how about git be taught to
ignore directories which contain .gitdir so that git commands will not
waste time checking project-b directory?
-- 
Duy

Re: [RFC] separate .git from working directory

[Alex Riesen]

On 10/11/06, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:
> Hi,
> I was thinking about this while reading subproject thread. In a simple
> case, I have a repo A located at ~/project-a and another repo B
> located at ~/project-a/some/dir/project-b. With this setup, command
> "find" and other directory-recursive commands will run horribly from
> ~/project-a when they go inside project-b/.git (no I don't want to
> repack -d everytime I want to find something).
> I propose to move project-b/.git outside and place a file, say
> .gitdir, in project-b directory. git-sh-setup and setup_git_directory

You can even leave the name as is: it is impossible to misunderstand
a file for directory (except on AIX). Or put the information in extended
attributes.

Re: [RFC] separate .git from working directory

[Sean]

On Wed, 11 Oct 2006 20:23:50 +0700
"Nguyen Thai Ngoc Duy" <pclouds@gmail.com> wrote:

> I was thinking about this while reading subproject thread. In a simple
> case, I have a repo A located at ~/project-a and another repo B
> located at ~/project-a/some/dir/project-b. With this setup, command
> "find" and other directory-recursive commands will run horribly from
> ~/project-a when they go inside project-b/.git (no I don't want to
> repack -d everytime I want to find something).
> I propose to move project-b/.git outside and place a file, say
> .gitdir, in project-b directory. git-sh-setup and setup_git_directory
> are taught to recognize .gitdir, read it to find the actual GIT_DIR
> recorded inside .gitdir. This way git commands inside project-b should
> work fine while I can "find ~/project-a -name blah" or "grep -R blah"
> quickly.
> .gitdir format could be  a simple shell-like format with environment
> variable assignments.

Probably wouldn't be too hard to implement, but is it worth it?

You can export a GIT_DIR manually pretty easily if you want to move
the .git directory somewhere else.  Also you could make a "git find"
shell script named "gf" that does something like:

#/bin/sh
find "$@" ! -path '*/.git/*'

Which would let you type  "gf -name blah" and automatically ignore
the .git directory.

Sean

Re: [RFC] separate .git from working directory

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 886 bytes --]

hoi :)

On Wed, Oct 11, 2006 at 08:23:50PM +0700, Nguyen Thai Ngoc Duy wrote:
> I was thinking about this while reading subproject thread. In a simple
> case, I have a repo A located at ~/project-a and another repo B
> located at ~/project-a/some/dir/project-b. With this setup, command
> "find" and other directory-recursive commands will run horribly from
> ~/project-a when they go inside project-b/.git (no I don't want to
> repack -d everytime I want to find something).

no it won't.

For subprojects you really need a shared object repository.
In such a setup b/.git/objects would be a symlink to the parent
object directory.

On the other hand, you already have the same problem with
a/.git/objects.  If you really want to move .git outside of the
working directory you can always do so by using a symlink for
the entire .git directory.

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC] separate .git from working directory

[Nguyen Thai Ngoc Duy]

On 10/12/06, Martin Waitz <tali@admingilde.org> wrote:
> For subprojects you really need a shared object repository.
> In such a setup b/.git/objects would be a symlink to the parent
> object directory.
>
> On the other hand, you already have the same problem with
> a/.git/objects.  If you really want to move .git outside of the
> working directory you can always do so by using a symlink for
> the entire .git directory.

I thought about symlinks. However find will follow symlinks and
traverse .git directories again. My goal is to prevent looking into
.git directories

>
> --
> Martin Waitz
-- 
Duy

Re: [RFC] separate .git from working directory

[Nguyen Thai Ngoc Duy]

On 10/11/06, Sean <seanlkml@sympatico.ca> wrote:
> You can export a GIT_DIR manually pretty easily if you want to move
> the .git directory somewhere else.  Also you could make a "git find"
> shell script named "gf" that does something like:
>
> #/bin/sh
> find "$@" ! -path '*/.git/*'
>
> Which would let you type  "gf -name blah" and automatically ignore
> the .git directory.

It should work. However I would rather use "normal" find than
specialized ones. Imagine somedays I hate find and start to love grep,
I would have to find out --exclude option and create new "gg" script.
In worse cases where recursive commands don't support filtering, I
have no chance to filter out .git directories.

>
> Sean
-- 
Duy

Re: [RFC] separate .git from working directory

[Junio C Hamano]

"Nguyen Thai Ngoc Duy" <pclouds@gmail.com> writes:

> I thought about symlinks. However find will follow symlinks and
> traverse .git directories again. My goal is to prevent looking into
> .git directories

I do not think the user (i.e. the developer working in his git
tree or git forest that hosts subprojects) should have to bend
and move .git directories away from the directory structure he
works in just to please the tool (i.e. find).  In this case the
tool has an easy way to bend itself to match what the user
wants, which is:

	find ... \( -type d -name .git -prune \) -o ...

Re: [RFC] separate .git from working directory

[Sean]

On Wed, 11 Oct 2006 16:41:03 +0200
"Alex Riesen" <raa.lkml@gmail.com> wrote:

> You can even leave the name as is: it is impossible to misunderstand
> a file for directory (except on AIX). Or put the information in extended
> attributes.

Actually, if you just make .git a symlink to the real $GIT_DIR find won't
recurse into it.

Sean

Re: [RFC] separate .git from working directory

[Liu Yubao]

Nguyen Thai Ngoc Duy wrote:
> On 10/11/06, Sean <seanlkml@sympatico.ca> wrote:
>> You can export a GIT_DIR manually pretty easily if you want to move
>> the .git directory somewhere else.  Also you could make a "git find"
>> shell script named "gf" that does something like:
>>
>> #/bin/sh
>> find "$@" ! -path '*/.git/*'
>>
>> Which would let you type  "gf -name blah" and automatically ignore
>> the .git directory.
> 
> It should work. However I would rather use "normal" find than
> specialized ones. Imagine somedays I hate find and start to love grep,
> I would have to find out --exclude option and create new "gg" script.
> In worse cases where recursive commands don't support filtering, I
> have no chance to filter out .git directories.
> 
Subversion puts a .svn/ in every directory under control, I dislike this
feature, I have to use many wrapper scripts.

Maybe the svk way is worth considering, it maintains a map relation
between paths in repository and file system like this:

$ svk checkout --list
           depot path			path
==================================================================
//path/in/repos/hello/world.c           /path/in/fs/hello/world.c

The problem is users must maintain this map when move or delete
/path/in/fs/hello/world.c, it's a bit annoying.

see http://svkbook.elixus.org/nightly/en/svk-book.html#svk.ref.svk.c.checkout
for more information.

Re: [RFC] separate .git from working directory

[Liu Yubao]

Liu Yubao wrote:
> Maybe the svk way is worth considering, it maintains a map relation
> between paths in repository and file system like this:
> 
> $ svk checkout --list
>           depot path            path
> ==================================================================
> //path/in/repos/hello/world.c           /path/in/fs/hello/world.c
Sorry, I made a mistake, svk maintains a map of the top directory
of checkout, not each file:
            depot path             path
====================================================================
//path/in/repos/myproject                /path/to/myproject
> 
> The problem is users must maintain this map when move or delete
> /path/in/fs/hello/world.c, it's a bit annoying.
> 
> see 
> http://svkbook.elixus.org/nightly/en/svk-book.html#svk.ref.svk.c.checkout
> for more information.
> 

Re: [RFC] separate .git from working directory

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 342 bytes --]

hoi :)

On Thu, Oct 12, 2006 at 04:46:51AM +0700, Nguyen Thai Ngoc Duy wrote:
> I thought about symlinks. However find will follow symlinks and
> traverse .git directories again. My goal is to prevent looking into
> .git directories

only with -L.

By default, find will not recurse into a symlinked directory.

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC] separate .git from working directory

[Sergio Callegari]

Possibly a bit different from the RFC, however,

Other than making find happy, I see other potential advantages in supporting 
the two options of having .git be either

- a directory containing all the git stuff
- a single file with a pointer to the real directory containing the objects, 
references, branches, etc.

1) It might make the life easier on platforms where symlinks are not the 
easiest thing to do (are there any?)

2) it might make it easier to work with syncronization tools (some of you 
might know that I got burnt with them recently). One of the issues of 
syncronizatoin tools is that they typically recognize renames as the 
non-atomical sequence of creation+deletion. Hence imagine the following 
scenario. I have ProjectFoo with the .git dir in. I tell the syncronization 
tool to ignore things called .git (not to get burned again!). I decide to 
rename ProjectFoo into ProjectBar. When I sync, I get with a ProjectBar with 
no .git directory since .git was meant to be ignored and is consequently lost 
in the creation+deletion sequence. All objects are then lost at one of the 
two hosts participating in the syncronization. If .git was only a file with a 
pointer, it would at least be possible to recreate it by hand without 
depending on the other syncronization host.

3) it might make it possible to have all the git archives in a single place, 
where it is easy to program a script to scan all the archives and repack all 
of them periodically or to scan all of them and backup them periodically, 
etc.

Sergio

Re: [RFC] separate .git from working directory

[Alex Riesen]

On 10/12/06, Sergio Callegari <scallegari@arces.unibo.it> wrote:
> 1) It might make the life easier on platforms where symlinks are not the
> easiest thing to do (are there any?)

yes. The most widespread one, for a start.

Re: [RFC] separate .git from working directory

[Nguyen Thai Ngoc Duy]

Hi,
After reading a post in VCS comparison thread mentioning /etc example.
I think some of my patches might be useful for some users. They are
two patches, tp/gitlink~1 and tp/gitlink~2 in git-pclouds.git on
repo.or.cz.
The implementation is a bit differerent from what I proposed. The
.gitdir file (I named it .gitlink  though) contains an alternative
name for .git dir. It would be more obvious with the following
example:

The standard layout:

a/.git
a/file
a/b/.git
a/b/file

The new layout:

a/.git
a/.git-b (it is actually a/b/.git)
a/file
a/b/.gitlink (whose content is ".git-b")
a/b/file

With this I can move all .git directories to the top directory (or
just parent directories). In /etc example, it can create a less
cluttered directory structure. I do not submit these patches to git.
It needs more work IMO.
-- 
Duy