[RFC/PATCH] Implement poor-man's submodule support using commit hooks

[RFC/PATCH] Implement poor-man's submodule support using commit hooks

[Andy Parkins]

Make a file called .gitmodules.  In it, list the paths containing a
submodule.  Add that file to the index.

This patch adds a check to the pre-commit hook finds that file and pulls
the HEAD hash out of each of the listed submodule repositories.  That
hash is then listed to the .gitmodules file along with the submodule
name and .gitmodules is added back to the repository.

You've now got poor-man's submodule support.  Any commits to the
submodule will change the hash and hence the .gitmodules file will be
different and therefore will show up as "modified" to git.

It's not got any nice UI for checking out (obviously) or merging; but it
does at least record the state of a project. With a bit of manual work you
can easily check out the right commit in the submodule.  If there were a
post-checkout hook script, this could probably be automated.

To prevent git-prune in the submodule from removing references that the
supermodule refers to the post-commit hook reads the .gitmodules file
and creates a file in submodule/.git/refs/superrefs/ that refers to the
hash we've references.  git-prune in the submodule will find that
reference and hence won't remove it from under us.

Problems:
 - git-prune in the supermodule doesn't clean the supermodule refs in
   the submodule
 - no checkout support
 - no reset support
 - no merge support (other than what git provides for the .gitmodule
   file)
 - no check for dirty submodule before commit

Signed-off-by: Andy Parkins <andyparkins@gmail.com>
---
This is in replacement of the previous patch.

I've used the --show-cdup option of git-rev-parse to make it work even in
subdirectories.  I've also added git-prune protection by adding a post-commit
script to reference the hash in the submodule.

What'd you reckon?  Might be useful until real submodule support arrives.
If there is no .gitmodules file in the root, then git behaves as it always did.

I suppose if this were actually found to be really useful, it should go in
git-commit.sh itself, rather than the hooks.


 templates/hooks--post-commit |   29 +++++++++++++++++++++++++-
 templates/hooks--pre-commit  |   47 +++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 74 insertions(+), 2 deletions(-)

diff --git a/templates/hooks--post-commit b/templates/hooks--post-commit
index 8be6f34..551d928 100644
--- a/templates/hooks--post-commit
+++ b/templates/hooks--post-commit
@@ -5,4 +5,31 @@
 #
 # To enable this hook, make this file executable.
 
-: Nothing
+# Poor-man's Submodules
+# ---------------------
+# If we're here, then a commit has succeeded. If submodule support is enabled
+# then we need a way of telling the submodule that we now reference a hash
+# owned by it, so that it is not pruned.
+WORKINGTOP=$(git-rev-parse --show-cdup)
+GITMODULES="${WORKINGTOP}.gitmodules"
+if [ -f "$GITMODULES" ]; then
+	cat "$GITMODULES" |
+	while read subdir hash
+	do
+		SUBMODULEPATH="$WORKINGTOP$subdir/.git/refs/superrefs"
+
+		# XXX: check if the line is a comment
+
+		# check if the subdir is a repository
+		if [ ! -d "$WORKINGTOP$subdir/.git" ]; then
+			continue;
+		fi
+
+		# Write the hash to a file of the same name - this means that if we get
+		# multiple commits that refer to the submodule, we only get one file in
+		# the submodule, as the submodule hash is constant across supermodule
+		# commits
+		mkdir -p "$SUBMODULEPATH"
+		echo $hash > "$SUBMODULEPATH/$hash"
+	done
+fi
diff --git a/templates/hooks--pre-commit b/templates/hooks--pre-commit
index 723a9ef..7718369 100644
--- a/templates/hooks--pre-commit
+++ b/templates/hooks--pre-commit
@@ -67,5 +67,50 @@ perl -e '
 	}
     }
     exit($found_bad);
-'
+' || exit 1
+
+# Poor-man's Submodules
+# ---------------------
+# Enable poor-man's submodule support when .gitmodules is present
+# Simply create a .gitmodules file listing the paths in your repository
+# that contain other git repositories; each line will be replaced with the
+# path followed by the hash of the current HEAD of the submodule.
+# When the submodule changes hash this file will be different from the
+# version in the repository and a change will be noted - voila, submodules.
+# Of course there is no checkout support, but at least the current state
+# will be accurately recorded
+WORKINGTOP=$(git-rev-parse --show-cdup)
+GITMODULES="${WORKINGTOP}.gitmodules"
+if [ -f "$GITMODULES" ]; then
+	cat "$GITMODULES" |
+	while read subdir hash
+	do
+		# check if the line is a comment and output it anyway
+		if (expr "x$subdir" : "x#" >/dev/null) then
+			echo "$subdir $hash"
+			continue;
+		fi
+
+		# check if the subdir is a repository
+		if [ ! -d "$WORKINGTOP$subdir/.git" ]; then
+			echo "$subdir is not a git repository, so it can't be a submodule"
+			exit 1;
+		fi
+
+		# XXX: really need a check here and quit if the submodule is
+		# dirty
+
+		echo "$subdir $(GIT_DIR=$WORKINGTOP$subdir/.git git-rev-parse HEAD)"
+	done > newgitmodules
+	# Update
+	mv newgitmodules "$GITMODULES"
+
+	# This relies on the .gitmodules file having already been added to
+	# the repository - perhaps this should be automated?
+	git-update-index "$GITMODULES" ||
+	(
+	echo "FATAL: Submodule tracker file, $GITMODULES, is not tracked in this repository." >&2
+	exit 1
+	)
+fi
 
-- 
1.4.4.2.g0d2a

Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks

[Johannes Sixt]

Andy Parkins wrote:
> +WORKINGTOP=$(git-rev-parse --show-cdup)
> +GITMODULES="${WORKINGTOP}.gitmodules"
> +if [ -f "$GITMODULES" ]; then
> +       cat "$GITMODULES" |

useless-use-of-cat-syndrome

> +       while read subdir hash

Wouldn't it be better to have the order of subdir and hash swapped? That
way subdir may contain blanks, and it gives nicer alignment in the file
because of the constant length of the hashes.

-- Hannes

Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 478 bytes --]

hoi :)

On Wed, Dec 20, 2006 at 01:09:01PM +0000, Andy Parkins wrote:
> 
> You've now got poor-man's submodule support.  Any commits to the
> submodule will change the hash and hence the .gitmodules file will be
> different and therefore will show up as "modified" to git.

I started with something similiar, too.
You can have a look at http://git.admingilde.org/tali/git.git/module
which tries to implement submodules without changing the core.

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks

[Andy Parkins]

On Wednesday 2006 December 20 13:29, Johannes Sixt wrote:

> > +       cat "$GITMODULES" |
>
> useless-use-of-cat-syndrome

I tried 

 while
 do 
 done < file1 > file1

And that didn't work.  However, it was racy anyway using the same file, so I 
changed to as it is now, but forgot to switch back to input redirection.

diff --git a/git-commit.sh b/git-commit.sh
diff --git a/templates/hooks--pre-commit b/templates/hooks--pre-commit
index 7718369..74edfe2 100644
--- a/templates/hooks--pre-commit
+++ b/templates/hooks--pre-commit
@@ -82,7 +82,6 @@ perl -e '
 WORKINGTOP=$(git-rev-parse --show-cdup)
 GITMODULES="${WORKINGTOP}.gitmodules"
 if [ -f "$GITMODULES" ]; then
-   cat "$GITMODULES" |
    while read subdir hash
    do
        # check if the line is a comment and output it anyway
@@ -101,7 +100,7 @@ if [ -f "$GITMODULES" ]; then
        # dirty
 
        echo "$subdir $(GIT_DIR=$WORKINGTOP$subdir/.git git-rev-parse HEAD)"
-   done > newgitmodules
+   done < "$GITMODULES" > newgitmodules
    # Update
    mv newgitmodules "$GITMODULES"

> > +       while read subdir hash
>
> Wouldn't it be better to have the order of subdir and hash swapped? That
> way subdir may contain blanks, and it gives nicer alignment in the file
> because of the constant length of the hashes.

Unfortunately, it is the hash that is optional.  When you create the file, you 
don't list the hashes, you list the subdirectories.  I suppose I could make 
it so you have to give "000000" or something first?



Andy

-- 
Dr Andy Parkins, M Eng (hons), MIEE

Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks

[Andy Parkins]

On Wednesday 2006 December 20 13:36, Martin Waitz wrote:

> I started with something similiar, too.
> You can have a look at http://git.admingilde.org/tali/git.git/module
> which tries to implement submodules without changing the core.

I wasn't really trying to do away with the need for core changes; I just 
wanted something simple that I could use today.

If this is old news - no problem.  Drop it.


Andy

-- 
Dr Andy Parkins, M Eng (hons), MIEE

Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks

[Jakub Narebski]

Andy Parkins wrote:

> On Wednesday 2006 December 20 13:29, Johannes Sixt wrote:

>>> +       while read subdir hash
>>
>> Wouldn't it be better to have the order of subdir and hash swapped? That
>> way subdir may contain blanks, and it gives nicer alignment in the file
>> because of the constant length of the hashes.
> 
> Unfortunately, it is the hash that is optional.  When you create the file, you 
> don't list the hashes, you list the subdirectories.  I suppose I could make 
> it so you have to give "000000" or something first?

That's the convention git uses in git-diff, reflog, etc.
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks

[Johannes Sixt]

Andy Parkins wrote:
> +               # check if the subdir is a repository
> +               if [ ! -d "$WORKINGTOP$subdir/.git" ]; then
> +                       echo "$subdir is not a git repository, so it can't be a submodule"

You must redirect the message to >&2 otherwise it remains invisible (it
goes into file newgitmodules).

-- Hannes

Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks

[Andy Parkins]

On Wednesday 2006 December 20 14:07, Jakub Narebski wrote:

> > Unfortunately, it is the hash that is optional.  When you create the
> > file, you don't list the hashes, you list the subdirectories.  I suppose
> > I could make it so you have to give "000000" or something first?
>
> That's the convention git uses in git-diff, reflog, etc.

They are all outputs though; this is an input.  I did it that way round so 
that I could say:

cat .gitmodules <<EOF
submodule1
submodule2
EOF

And that was it.  Of course, this doesn't work when there are spaces in the 
filename, so I guess it's got to be 

cat .gitmodules <<EOF
000 submodule1
000 submodule2
EOF

I'll wait to see if there are more objections before posting an updated patch.


Andy

-- 
Dr Andy Parkins, M Eng (hons), MIEE

Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks

[Rogan Dawes]

Andy Parkins wrote:
> On Wednesday 2006 December 20 13:29, Johannes Sixt wrote:
> 
>>> +       cat "$GITMODULES" |
>> useless-use-of-cat-syndrome
> 
> I tried 
> 
>  while
>  do 
>  done < file1 > file1
> 
> And that didn't work. 

It wouldn't, since the redirection overwrites the file at the same time 
as trying to read from it. A better way is to redirect to a temp file, 
and rename it if the previous operation was successful.

e.g.

0 $ echo true > t
0 $ sed 's/true/false/g' < t > t~ && mv t~ t || rm t~
0 $ cat t
false

and

0 $ echo true > t
0 $ (sed 's/true/false/g' < t > t~; false) && mv t~ t || rm t~
0 $ cat t
true
0 $ ls t~
ls: t~: No such file or directory
2 $

Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks

[Andy Parkins]

On Wednesday 2006 December 20 13:36, Martin Waitz wrote:

> I started with something similiar, too.
> You can have a look at http://git.admingilde.org/tali/git.git/module
> which tries to implement submodules without changing the core.

I had a look at that; and my previous objection still applies: it blends two 
repositories.  I'd rather they were separate.

Having them separate is what makes my little hook scripts really simple; there 
is no special initialisation script needed, and the neither the supermodule 
nor the submodule needs special treatment; and there is no need for any 
alternates or new object type.  If I want to forget about submodule support I 
just delete the .gitmodules file and it's all back to normal.

I don't say you're wrong in your chosen method, and I don't say I'm right.  I 
wasn't really planning on going much further with these scripts, they 
actually represent all that I personally want from a submodule system, I am 
sure others would want far more sophistication.



Andy

-- 
Dr Andy Parkins, M Eng (hons), MIEE

Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks

[Andy Parkins]

On Wednesday 2006 December 20 14:33, Rogan Dawes wrote:

> It wouldn't, since the redirection overwrites the file at the same time

Of course you are correct; however it wasn't completely crazy.  Before now, 
I've replaced files in C by doing

 fd_old = open( "someexistingfile" );
 unlink( "someexistingfile" );
 fd_new = open( "someexistingfile" );

So I was really just trying it in case bash did a similar thing internally.

> as trying to read from it. A better way is to redirect to a temp file,
> and rename it if the previous operation was successful.

That's what I ended up doing; as you say it's better anyway because the error 
conditions don't mean that the .gitmodules file ends up half written.


Andy
-- 
Dr Andy Parkins, M Eng (hons), MIEE

Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks

[Randal L. Schwartz]

>>>>> "Andy" == Andy Parkins <andyparkins@gmail.com> writes:

Andy>  fd_old = open( "someexistingfile" );
Andy>  unlink( "someexistingfile" );
Andy>  fd_new = open( "someexistingfile" );

This is dangerous if the process aborts suddenly.

Might be better to do this:

mumble_command < thefile > thefile.$$
mv thefile.$$ thefile

The second move will be atomic.  The only problem will be if the process
aborts, which will leave stale temp files around.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.

Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 496 bytes --]

hoi :)

On Wed, Dec 20, 2006 at 02:35:48PM +0000, Andy Parkins wrote:
> I had a look at that; and my previous objection still applies: it blends two 
> repositories.  I'd rather they were separate.

yes, but one can see what is needed and where the real problems are.

And I really think we should only start adding submodule code to
mainline git when we are sure that we found an approach where we can see
that it works and that we don't have to change it later.

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC/PATCH] Implement poor-man's submodule support using commit hooks

[Johannes Schindelin]

Hi,

On Wed, 20 Dec 2006, Andy Parkins wrote:

>  Before now, I've replaced files in C by doing
> 
>  fd_old = open( "someexistingfile" );
>  unlink( "someexistingfile" );
>  fd_new = open( "someexistingfile" );

Lucky you! Obiously you are not stuck in the 10th circle called Windows.

Ciao,