Changing branches in supermodule does not affect submodule?

Changing branches in supermodule does not affect submodule?

[Peter Krefting]

Hi!

If I have a repository with submodules that is in a clean state, and switch 
branches in the super repository, the submodules are left in the state they 
were in before I switched branches (with 1.6.4, at least). Is this the 
expected behaviour?

--->8--8<---
#!/bin/bash
# Create supermodule.
mkdir super-$$
cd super-$$
git init

# Create submodule with commit.
mkdir sub
cd sub
git init
echo C > c.txt
echo D > d.txt
git add c.txt d.txt
git commit -m Created.
cd ..

# Create commit and branch in supermodule.
echo A > a.txt
echo B > b.txt
git add a.txt b.txt sub
git commit -m Created.
git tag branchpoint
git checkout -b branch1

# Create branch1 in submodule and commit.
cd sub
git checkout -b branch1
echo E > c.txt
echo F > d.txt
git add c.txt d.txt
git commit -m Branched.
cd ..

# Commit to a branch in the supermodule.
git add sub
git commit -m Sub-update.

# Status should now be clean.
git status

# Create a new branch in supermodule, from the original commit.
git checkout -b branch2 branchpoint

# I now expect the submodule to be up-to-date and the state clean.
git status
--->8--8<---

-- 
\\// Peter - http://www.softwolves.pp.se/

Re: Changing branches in supermodule does not affect submodule?

[Jens Lehmann]

Peter Krefting schrieb:
> If I have a repository with submodules that is in a clean state, and
> switch branches in the super repository, the submodules are left in the
> state they were in before I switched branches (with 1.6.4, at least). Is
> this the expected behaviour?

Yup, submodules have to be updated separatly.

When they are cloned from a remote with "git submodule add" followed by
a "git submodule init", just calling "git submodule update" every time
you want the submodule to be updated according to the state committed
in the superproject will do the trick (but keep in mind that all
referenced commits have to be accessible in the local clone of your
submodule, so you might have to do a fetch there once in a while).


Jens

Re: Changing branches in supermodule does not affect submodule?

[Peter Krefting]

Jens Lehmann:

> just calling "git submodule update" every time you want the submodule to 
> be updated according to the state committed in the superproject will do 
> the trick (but keep in mind that all referenced commits have to be 
> accessible in the local clone of your submodule, so you might have to do a 
> fetch there once in a while).

Is it possible to automate this from a hook or something else? Basically, I 
would like it to update all the submodules to the state recorded in the 
commit I move to, if they were in a clean state before I moved. I do not 
want it to change states if I do something like

   cd submodule
   # do some changes
   git commit
   cd ..
   git checkout -b newbranch

because there I want the commit I made to the submodule to be recorded on 
the new branch I created. But then it was in a dirty state before I created 
the branch anyway, so that shouldn't be a problem.

-- 
\\// Peter - http://www.softwolves.pp.se/

Re: Changing branches in supermodule does not affect submodule?

[Jens Lehmann]

Peter Krefting schrieb:
> Jens Lehmann:
> 
>> just calling "git submodule update" every time you want the submodule
>> to be updated according to the state committed in the superproject
>> will do the trick (but keep in mind that all referenced commits have
>> to be accessible in the local clone of your submodule, so you might
>> have to do a fetch there once in a while).

BTW: unless you use the -N or --no-fetch option, git submodule update
will do the fetch for you.


> Is it possible to automate this from a hook or something else?

Yep, you can use the post-checkout hook for that, just put a "git
submodule update" in it.

*But*: If you do a checkout in the superproject while the submodule
has new commits not contained in any branch (remember submodules
often have a detached head) you'll silently lose these commits!
Then only the reflog can help you ...

[RFC PATCH] implement sample post-checkout hook to checkout new/unchanged submodules

[Heiko Voigt]

Currently gits default behaviour is not to recursively update submodules
when they are unchanged in the working copy. This hook implements this
by comparing whether the current HEAD in the submodule is the same as
recorded in the commit we are coming from. It then initializes or
updates the submodule as necessary.

Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
---

On Wed, Oct 14, 2009 at 05:39:29PM +0200, Jens Lehmann wrote:
> Peter Krefting schrieb:
> > Jens Lehmann:
> > 
> >> just calling "git submodule update" every time you want the submodule
> >> to be updated according to the state committed in the superproject
> >> will do the trick (but keep in mind that all referenced commits have
> >> to be accessible in the local clone of your submodule, so you might
> >> have to do a fetch there once in a while).
> 
> BTW: unless you use the -N or --no-fetch option, git submodule update
> will do the fetch for you.
> 
> 
> > Is it possible to automate this from a hook or something else?
> 
> Yep, you can use the post-checkout hook for that, just put a "git
> submodule update" in it.

Incidentially I have just been working on such a hook. Here is a patch
that implements it as a sample hook.

I tested most cases, but I have not worked with it productively so it
might have strange results in some cases. One I already found is that
post-checkout is not called after a merge so you need to add a
submodule update there as well.

cheers Heiko


 templates/hooks--post-checkout.sample |   50 +++++++++++++++++++++++++++++++++
 1 files changed, 50 insertions(+), 0 deletions(-)
 create mode 100644 templates/hooks--post-checkout.sample

diff --git a/templates/hooks--post-checkout.sample b/templates/hooks--post-checkout.sample
new file mode 100644
index 0000000..9ffffa0
--- /dev/null
+++ b/templates/hooks--post-checkout.sample
@@ -0,0 +1,50 @@
+#!/bin/sh
+
+# if this is a file checkout we do nothing
+flag=$3
+if [ ! $flag ]
+then
+	exit 0
+fi
+
+# if this is the initial checkout also intialize submodules
+if [ $1 == "0000000000000000000000000000000000000000" ]; then
+	git submodule init
+	git submodule update
+	# after initial checkout we are done now
+	exit 0
+fi
+
+# update all submodules that where not modified beforehand
+# or did not exist
+prev_sha1=$1
+new_sha1=$2
+
+git ls-tree $new_sha1 | grep ^160000 |cut -f2 |
+while read submodule
+do
+	prev_submodule_sha1=$(git ls-tree $prev_sha1 |
+		grep "	$submodule$" | cut -f1 | cut -d' ' -f3)
+	curr_submodule_sha1=$(cd "$submodule"; git rev-parse HEAD \
+		2>/dev/null)
+
+	if [ -z "$prev_submodule_sha1" ]
+	then
+		git submodule init "$submodule"
+		git submodule update "$submodule"
+		continue
+	fi
+
+	if [ "$prev_submodule_sha1" = "$curr_submodule_sha1" ]
+	then
+		if [ -z "$(git config submodule."$submodule".url)" ]
+		then
+			git submodule init "$submodule"
+		fi
+
+		if [ "$(git diff $prev_sha1 $new_sha1 -- "$submodule")" ]
+		then
+			git submodule update "$submodule"
+		fi
+	fi
+done
-- 
1.6.5.rc1.12.gc72fe

Re: Changing branches in supermodule does not affect submodule?

[Junio C Hamano]

Peter Krefting <peter@softwolves.pp.se> writes:

> Basically, I would like it to update all the submodules to the state
> recorded in the commit I move to, if they were in a clean state before
> I moved. I do not want it to change states if I do something like
>
>   cd submodule
>   # do some changes
>   git commit
>   cd ..
>   git checkout -b newbranch
>
> because there I want the commit I made to the submodule to be recorded
> on the new branch I created. But then it was in a dirty state before I
> created the branch anyway, so that shouldn't be a problem.

My understanding is that the primary reason "git checkout $another_branch"
does not touch submodules (other than updating the commit object name
bound in the index in the superproject) is because we did not know what
people would want to happen when submodule support was introduced.

Now people have used submodules for a while, we can update this.  I think
modelling the behaviour after how normal blobs are updated upon branch
switching would make sense.

So let's step back a bit and review what happens to a normal blob upon
branch switching.  There are three cases for a path while switching
branches from A to B:

 #1 The path is the same between A and B.  In this case, whatever you have
    for path in the index and in the work tree stay the same.  Your local
    modification is carried across branch switching.

    This is already what we do with submodules.  In cases #2 and #3 below,
    path is different between A and B.

 #2 You do not have any change to the path in either the index nor in the
    work tree.  The path is updated to B's version in the index and also
    what you have in the work tree matches it.

    Note that if B lacks the path, it is _removed_ from both the index and
    the work tree.

 #3 You have a change to the path in the index or in the work tree.  The
    branch switch is refused if the index does not match what is in B.

    I think this also is already what we do with submodules.

With a reasonable definition of "have a change in the work tree" for a
submodule path, we can define a reasonable behaviour for updating the
submodules when a different branch is checked out in the superproject.

For a regular blob, having a change in the work tree is that the object
recorded in HEAD (in the superproject) is different from what you have in
the work tree.  What should the definition be for a submodule?

The HEAD (in the submodule), the index (in the submodule) and the work
tree (in the submodule) form the "state" of what you have in the path, and
that corresponds to the "work tree state".  So I think we can say that
there is no change in the work tree for a submodule at path P if all of
the following holds:

    - the HEAD in the submodule matches the commit recorded for the path P
      in the HEAD in the superproject;

    - the index in the submodule matches the HEAD in the submodule; and

    - the work tree in the submodule matches the index in the submodule.

With this definition, you should be able to patch "git checkout" to
recurse into the submodule directory for case #2.

As to the implementation, I'd suggest:

 - the check between the submodule HEAD and what the superproject records
   in the HEAD (the first condition above) be done inside the main process
   (because we already have a working check for #3 to refuse branch
   switching, I think it would be easier to implement this check by
   comparison between the submodule HEAD and the superproject index).

 - the check between the index and the HEAD in the submodule and the check
   between the index and the work tree in the submodule (the second and
   the third conditions above) be done in a subprocess that runs something
   like "git status" via the run_command() interface; and

 - the actual recursive checkout to happen in a subprocess that runs
   another instance of "git checkout" via the run_command() interface.

There is one caveat.  The potential removal of the work tree path in #2
may not be desirable for a submodule path, even if you do not have any
change to it (i.e. both the index and the work tree in the submodule match
the HEAD in the submodule), as you may have untracked files you would
rather want to keep in the submodule work tree.