[PATCH v3] submodule: Allow tracking of the newest revision of a branch in a submodule

[PATCH v3] submodule: Allow tracking of the newest revision of a branch in a submodule

[Fabian Franz]

Submodules currently only allow tracking a specific revision
and each update in a submodule leads to a new commit in the
master repository. However some users may want to always track
the newest revision of a specific (named) tag or branch or HEAD.
For example the user might want to track a staging branch in all
submodules.

To allow this the "--track|-t <branch>" parameter was added to
git-submodule.sh, which is added to .gitmodules config file as
well as "track" parameter. This creates a new local branch on
checkout, which is tracking the remote branch in case the local
branch does not yet exist.

Technically the gitlink code was changed to always compare
successful (so no changes) in case the sha1 is null. In that
case no new commit is created when there are changes in the
submodule.

The submodule code is adding the file with 0000* on
"add".

Signed-off-by: Fabian Franz <git@fabian-franz.de>
---
>
>Fabian Franz schrieb:
>> Submodules currently only allow tracking a specific revision
>> and each update in a submodule leads to a new commit in the
>> master repository. However some users may want to always track
>> the newest revision of a specific (named) tag or branch or HEAD.
>> For example the user might want to track a staging branch in all
>> submodules.
>
>Personally, I don't particularly like this feature (but then, nobody
>forces me to use it ;) In which situation do you need this?

I have a development workflow in my company where we are independently
working on many components. And for some things I just always want to 
have the newest revision without having to commit twice always.

>
>By tieing a project commit to a particular submodule commit the committer
>gives the guarantee: "I've tested this with this module version, and it
>works; all is ok." With this new feature, this guarantee vanishes, because
>the committer has no control over which version of the module will
>ultimately be used; it could be newer or it could be older.

I like this and because of that the --branch is optional. I also like that so much, that we have decided against Google Repo.

However I have both cases: Stable development, where I need one special version and "wild" development, where I always want the newest published one.
>
>I've reviewed the patch just from a shell code writer's point of view.

Okay, I added your suggestions.

Thanks for your feedback.

Best Wishes,

Fabian

 Documentation/git-submodule.txt |   10 +++++++++-
 git-submodule.sh                |   35 +++++++++++++++++++++++++++++++++--
 read-cache.c                    |    5 +++++
 3 files changed, 47 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index babaa9b..9c29678 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -9,7 +9,7 @@ git-submodule - Initialize, update or inspect submodules
 SYNOPSIS
 --------
 [verse]
-'git submodule' [--quiet] add [-b branch] [--] <repository> <path>
+'git submodule' [--quiet] add [-b branch] [-t|--track <branch>] [--] <repository> <path>
 'git submodule' [--quiet] status [--cached] [--] [<path>...]
 'git submodule' [--quiet] init [--] [<path>...]
 'git submodule' [--quiet] update [--init] [--] [<path>...]
@@ -118,6 +118,10 @@ update::
 If the submodule is not yet initialized, and you just want to use the
 setting as stored in .gitmodules, you can automatically initialize the
 submodule with the --init option.
++
+If you used --track or set the "track" option in .gitmodules this will
+automatically pull the newest updates from remote instead of tracking a
+specific revision.
 
 summary::
 	Show commit summary between the given commit (defaults to HEAD) and
@@ -159,6 +163,10 @@ OPTIONS
 --branch::
 	Branch of repository to add as submodule.
 
+-t::
+--track::
+	Branch/Tag/HEAD of repository to track in a submodule.
+
 --cached::
 	This option is only valid for status and summary commands.  These
 	commands typically use the commit found in the submodule HEAD, but
diff --git a/git-submodule.sh b/git-submodule.sh
index 2f47e06..16df528 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -5,7 +5,7 @@
 # Copyright (c) 2007 Lars Hjemli
 
 USAGE="[--quiet] [--cached] \
-[add <repo> [-b branch] <path>]|[status|init|update [-i|--init]|summary [-n|--summary-limit <n>] [<commit>]] \
+[add <repo> [-b branch] [--track|-t <branch>] <path>]|[status|init|update [-i|--init]|summary [-n|--summary-limit <n>] [<commit>]] \
 [--] [<path>...]|[foreach <command>]|[sync [--] [<path>...]]"
 OPTIONS_SPEC=
 . git-sh-setup
@@ -16,6 +16,7 @@ command=
 branch=
 quiet=
 cached=
+track=
 
 #
 # print stuff on stdout unless -q was specified
@@ -130,6 +131,11 @@ cmd_add()
 		-q|--quiet)
 			quiet=1
 			;;
+		-t|--track)
+			case "$2" in '') usage ;; esac
+			track=$2
+			shift
+			;;
 		--)
 			shift
 			break
@@ -197,12 +203,14 @@ cmd_add()
 		(unset GIT_DIR; cd "$path" && git checkout -f -q ${branch:+-b "$branch" "origin/$branch"}) ||
 		die "Unable to checkout submodule '$path'"
 	fi
+	test -n "$track" && printf '160000 0000000000000000000000000000000000000000\t%s\n' "$path" | git update-index --index-info
 
 	git add "$path" ||
 	die "Failed to add submodule '$path'"
 
 	git config -f .gitmodules submodule."$path".path "$path" &&
 	git config -f .gitmodules submodule."$path".url "$repo" &&
+	git config -f .gitmodules submodule."$path".track "$track" &&
 	git add .gitmodules ||
 	die "Failed to register submodule '$path'"
 }
@@ -277,6 +285,10 @@ cmd_init()
 		git config submodule."$name".url "$url" ||
 		die "Failed to register url for submodule path '$path'"
 
+		track=$(git config -f .gitmodules submodule."$name".track)
+		git config submodule."$name".track "$track" ||
+		die "Failed to register track for submodule path '$path'"
+
 		say "Submodule '$name' ($url) registered for path '$path'"
 	done
 }
@@ -345,11 +357,29 @@ cmd_update()
 			then
 				force="-f"
 			fi
+			pull=
+			if [ "$sha1" = "0000000000000000000000000000000000000000" ]
+			then
+				track=$(git config submodule."$name".track)
+				: ${track:="master"}
+				# if the local branch does not yet exist, create it
+				( unset GIT_DIR; cd "$path"; git-show-ref --heads --tags -q "$track" || git branch --track "$track" "origin/$track" )
+				sha1="$track"
+				pull=1
+			fi
+
 			(unset GIT_DIR; cd "$path" && git-fetch &&
 				git-checkout $force -q "$sha1") ||
 			die "Unable to checkout '$sha1' in submodule path '$path'"
 
 			say "Submodule path '$path': checked out '$sha1'"
+
+			if [ "$pull" = "1" ]
+			then
+				# Now pull new updates from origin
+				( unset GIT_DIR; cd "$path"; git-pull ) || die "Unable to pull in submodule path '$path'"
+			fi
+
 		fi
 	done
 }
@@ -596,7 +626,8 @@ cmd_status()
 		set_name_rev "$path" "$sha1"
 		if git diff-files --quiet -- "$path"
 		then
-			say " $sha1 $path$revname"
+			track=$(git config submodule."$name".track)
+			say " $sha1 $path$revname${track:+ (tracking "$track")}"
 		else
 			if test -z "$cached"
 			then
diff --git a/read-cache.c b/read-cache.c
index 8579663..0c14b68 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -137,6 +137,11 @@ static int ce_compare_gitlink(struct cache_entry *ce)
 	 */
 	if (resolve_gitlink_ref(ce->name, "HEAD", sha1) < 0)
 		return 0;
+
+	// To be able to track newest revision
+	if (is_null_sha1(ce->sha1))
+		return 0;
+
 	return hashcmp(sha1, ce->sha1);
 }
 
-- 
1.5.3.6

Re: [PATCH v3] submodule: Allow tracking of the newest revision of a branch in a submodule

[Junio C Hamano]

Fabian Franz <git@fabian-franz.de> writes:

> Submodules currently only allow tracking a specific revision
> and each update in a submodule leads to a new commit in the
> master repository. However some users may want to always track
> the newest revision of a specific (named) tag or branch or HEAD.
> For example the user might want to track a staging branch in all
> submodules.

I initially liked the direction this is going, but I think the above
rationale and the change to use 0{40} have impedance mismatch.  Your
change is not a good way to go about "some users may want".  I'll discuss
more on this below.

> To allow this the "--track|-t <branch>" parameter was added to
> git-submodule.sh, which is added to .gitmodules config file as
> well as "track" parameter. This creates a new local branch on
> checkout, which is tracking the remote branch in case the local
> branch does not yet exist.
>
> Technically the gitlink code was changed to always compare
> successful (so no changes) in case the sha1 is null. In that
> case no new commit is created when there are changes in the
> submodule.

"Technically" here sounds wrong.  I'd suggest dropping it, e.g. "Update
ce_compare_gitlink() so that it always reports a match if the commit
recorded in the index is a null SHA-1."

Because I also do not see a direct connection between "no new commit is
created" and "there are changes in the submodule", I think the last
sentence in the above paragraph of yours is misleading.  The user _can_
create A new commit in the superproject that records a different gitlink
from its parent commit, if (and only if) the user wishes to bind the
updated subproject's branch state to the new state of the superproject,
and that is done by adding the subproject status to the staging area with
"git add" (or use "git commit -a").

It seems to me that what you are really after is to let you change the
state of the subproject checkout in whatever way and have "git commit -a"
in the superproject ignore that change.

I wonder if you can just set "assume unchanged" bit for the subproject
gitlink in the index to achieve the same goal.

Or is there more to it?

> The submodule code is adding the file with 0000* on
> "add".
>
> Signed-off-by: Fabian Franz <git@fabian-franz.de>
> ---
>
> I like this and because of that the --branch is optional. I also like
> that so much, that we have decided against Google Repo.
>
> However I have both cases: Stable development, where I need one special
> version and "wild" development, where I always want the newest published
> one.

I do not think supporting both styles of development is a bad idea.

However, use of 0{40} in the index and the resulting commit object in the
superproject means that this is a project-wide decision, not your personal
preference.  It is not implausible that you would want to do a wild
expeeriment in your own clone of a project that uses the "Stable
development" approach (hence the upstream never would want to have 0{40}
gitlink in its commits).

For example, suppose the project uses "Stable development" approach, and
records the v1.0.0 of submodule at "sub/" in the superproject.  You are a
contributor to that project, and would want to help them futureproof the
superproject code to be forward compatible with the upcoming v1.2.0
release of the subproject.  What would you do?

 * have a clone of superproject, with v1.0.0 submodule bound at "sub/";

 * go to "sub/", fetch and checkout v1.2.0-rc2;

 * go up, build using the updated submodule, see many failures in
   supermodule build;

 * fix them up in a way that can work with both v1.0.0 and v1.2.0 of the
   submodule, while making commits in logical steps, in the supermodule.

And you do not want to record the fact that you used v1.2.0-rc2 of the
submodule at "sub/" in the commits you make in the supermodule, as you
would want to label these commits as "futureproof for upcoming submodule
v1.2.0".

But you cannot use your 0{40} trick, as sending that to the upstream of
the superproject would break their "Stable development" policy.

I wonder if you can just set "assume unchanged" bit for the subproject
gitlink in the index to achieve the same goal.  That would be a local
operation, the gitlink would still point at v1.0.0 version of submodule,
and "git commit -a" in the superproject won't make commits that flips
everybody else's copy to use v1.2.0-rc2 of submodule.

>>I've reviewed the patch just from a shell code writer's point of view.
>
> Okay, I added your suggestions.

In the commentary section in your v2 patch, you said "However I see
problems on remove".  Has that issue been addressed?

> @@ -118,6 +118,10 @@ update::
>  If the submodule is not yet initialized, and you just want to use the
>  setting as stored in .gitmodules, you can automatically initialize the
>  submodule with the --init option.
> ++
> +If you used --track or set the "track" option in .gitmodules this will
> +automatically pull the newest updates from remote instead of tracking a
> +specific revision.

"automatically pull" in the sense that it always goes to the remote, fetch
and merge?  That sounds horribly broken.  You can never work disconnected?

> @@ -159,6 +163,10 @@ OPTIONS
>  --branch::
>  	Branch of repository to add as submodule.
>  
> +-t::
> +--track::
> +	Branch/Tag/HEAD of repository to track in a submodule.
> +

How does the branch parameter to the --track option interact with the
branch parameter to the --branch option?  Does an end user typically set
them to the same branch?  Or would these parameters almost always point at
different branchesof the remote repository?  What are the reasons for the
end user to choose one parameter value for the --branch option and a
different parameter value for the --track option?

> diff --git a/git-submodule.sh b/git-submodule.sh
> index 2f47e06..16df528 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> ...
> @@ -197,12 +203,14 @@ cmd_add()
>  		(unset GIT_DIR; cd "$path" && git checkout -f -q ${branch:+-b "$branch" "origin/$branch"}) ||
>  		die "Unable to checkout submodule '$path'"
>  	fi
> +	test -n "$track" && printf '160000 0000000000000000000000000000000000000000\t%s\n' "$path" | git update-index --index-info
>  

You have many overlong lines due to the 0{40} constant string in your
patch.  Have a

	null_sha1=0000000000000000000000000000000000000000

at the beginning of the script, and rewrite the above like this:

	test -n "$track" &&
                printf '160000 %s\t%s\n' "$null_sha1" "$path" |
                git update-index --index-info

or even like this:

	if test -n "$track"
        then
                printf '160000 %s\t%s\n' "$null_sha1" "$path" |
                git update-index --index-info
	fi

Use of $null_sha1 throughout the script will make things easier to read
and at the same time make it less error prone as well for "git submodule"
developers.

> @@ -345,11 +357,29 @@ cmd_update()
>  			then
>  				force="-f"
>  			fi
> +			pull=
> +			if [ "$sha1" = "0000000000000000000000000000000000000000" ]
> +			then
> +				track=$(git config submodule."$name".track)
> +				: ${track:="master"}

In the v2 patch this used to point at "HEAD".  What made you change your
mind?

> +				# if the local branch does not yet exist, create it
> +				( unset GIT_DIR; cd "$path"; git-show-ref --heads --tags -q "$track" || git branch --track "$track" "origin/$track" )

No error checking?

	(
        	unset GIT_DIR;
                cd "$path" &&
                git show-ref --heads --tags -q "$track" ||
                git branch --track "$track" "origin/$track"
	) || barf

The ';' after unset is intentional; some shells reports failure when you
unset an unset variable.

> +				sha1="$track"
> +				pull=1

I tend to prefer booleans in shell scripts spelled like boolean, e.g.

	pull=yes

> +			fi
> +
>  			(unset GIT_DIR; cd "$path" && git-fetch &&
>  				git-checkout $force -q "$sha1") ||
>  			die "Unable to checkout '$sha1' in submodule path '$path'"
>  
>  			say "Submodule path '$path': checked out '$sha1'"
> +
> +			if [ "$pull" = "1" ]
> +			then
> +				# Now pull new updates from origin
> +				( unset GIT_DIR; cd "$path"; git-pull ) || die "Unable to pull in submodule path '$path'"

No error checking?


> +			fi
> +
>  		fi
>  	done
>  }
> @@ -596,7 +626,8 @@ cmd_status()
>  		set_name_rev "$path" "$sha1"
>  		if git diff-files --quiet -- "$path"
>  		then
> -			say " $sha1 $path$revname"
> +			track=$(git config submodule."$name".track)
> +			say " $sha1 $path$revname${track:+ (tracking "$track")}"
>  		else
>  			if test -z "$cached"
>  			then
> diff --git a/read-cache.c b/read-cache.c
> index 8579663..0c14b68 100644
> --- a/read-cache.c
> +++ b/read-cache.c
> @@ -137,6 +137,11 @@ static int ce_compare_gitlink(struct cache_entry *ce)
>  	 */
>  	if (resolve_gitlink_ref(ce->name, "HEAD", sha1) < 0)
>  		return 0;
> +
> +	// To be able to track newest revision
> +	if (is_null_sha1(ce->sha1))
> +		return 0;
> +

I think the comment is wrong, as it is not about newness at all.

	/* ignore changes in the submodule path */

would be more appropriate.

Re: [PATCH v3] submodule: Allow tracking of the newest revision of a branch in a submodule

[Fabian Franz]

> Fabian Franz <git@fabian-franz.de> writes:
> 
> > Submodules currently only allow tracking a specific revision
> > and each update in a submodule leads to a new commit in the
> > master repository. However some users may want to always track
> > the newest revision of a specific (named) tag or branch or HEAD.
> > For example the user might want to track a staging branch in all
> > submodules.
> 
> I initially liked the direction this is going, but I think the above
> rationale and the change to use 0{40} have impedance mismatch.  Your
> change is not a good way to go about "some users may want".  I'll discuss
> more on this below.

I do agree. I did like the idea of HEAD.gitlink also better and I even more like the assume-unchanged version, which works.

> It seems to me that what you are really after is to let you change the
> state of the subproject checkout in whatever way and have "git commit -a"
> in the superproject ignore that change.
> 
> I wonder if you can just set "assume unchanged" bit for the subproject
> gitlink in the index to achieve the same goal.
> 
> Or is there more to it?

Nope, that is it. I just did not knew that flag.

> > However I have both cases: Stable development, where I need one special
> > version and "wild" development, where I always want the newest published
> > one.
> 
> I do not think supporting both styles of development is a bad idea.
> 
> However, use of 0{40} in the index and the resulting commit object in the
> superproject means that this is a project-wide decision, not your personal
> preference.  It is not implausible that you would want to do a wild
> expeeriment in your own clone of a project that uses the "Stable
> development" approach (hence the upstream never would want to have 0{40}
> gitlink in its commits).

Yes, but at the same time I might want to record it permanently as a project decision or play at my own with it ...

So both styles should be supported.

> For example, suppose the project uses "Stable development" approach, and
> records the v1.0.0 of submodule at "sub/" in the superproject.  You are a
> contributor to that project, and would want to help them futureproof the
> superproject code to be forward compatible with the upcoming v1.2.0
> release of the subproject.  What would you do?
> 
>  * have a clone of superproject, with v1.0.0 submodule bound at "sub/";
> 
>  * go to "sub/", fetch and checkout v1.2.0-rc2;
> 
>  * go up, build using the updated submodule, see many failures in
>    supermodule build;
> 
>  * fix them up in a way that can work with both v1.0.0 and v1.2.0 of the
>    submodule, while making commits in logical steps, in the supermodule.
> 
> And you do not want to record the fact that you used v1.2.0-rc2 of the
> submodule at "sub/" in the commits you make in the supermodule, as you
> would want to label these commits as "futureproof for upcoming submodule
> v1.2.0".
> 
> But you cannot use your 0{40} trick, as sending that to the upstream of
> the superproject would break their "Stable development" policy.
> 
> I wonder if you can just set "assume unchanged" bit for the subproject
> gitlink in the index to achieve the same goal.  That would be a local
> operation, the gitlink would still point at v1.0.0 version of submodule,
> and "git commit -a" in the superproject won't make commits that flips
> everybody else's copy to use v1.2.0-rc2 of submodule.

Yes, that works. I tried it. I am now gonna change the patch to use this new approach and also re-think the workflow I want to support.
 
> > @@ -118,6 +118,10 @@ update::
> >  If the submodule is not yet initialized, and you just want to use the
> >  setting as stored in .gitmodules, you can automatically initialize the
> >  submodule with the --init option.
> > ++
> > +If you used --track or set the "track" option in .gitmodules this will
> > +automatically pull the newest updates from remote instead of tracking a
> > +specific revision.
> 
> "automatically pull" in the sense that it always goes to the remote, fetch
> and merge?  That sounds horribly broken.  You can never work disconnected?

Uhm, the same is already true for git submodule update. Before it checkouts the new branch it always does a fetch.

However I think what I really want (without) scripting via foreach is:

git checkout staging

In .gitmodules is from a personal (own branch) or project wide decision the fact documented that the submodules do track staging and one stable tag for example.

[module1]
track = staging
[module2]
track = staging
[module3]
track = stable-v1.0.0

And now I just do git submodule update and it fetches and afterwards checks out my local branch of staging and fast-forwards it.

However I do agree that in that workflow you always have to go online and that is not good.

But in my case I also want a developer to just be able to change to 

[module1]
track = feature_1
[module2]
track = feature_1
[module3]
track = stable-v1.0.0

to work in a feature branch in two submodules at once.

So I am gonna rethink this design.

> > @@ -159,6 +163,10 @@ OPTIONS
> >  --branch::
> >  	Branch of repository to add as submodule.
> >  
> > +-t::
> > +--track::
> > +	Branch/Tag/HEAD of repository to track in a submodule.
> > +
> 
> How does the branch parameter to the --track option interact with the
> branch parameter to the --branch option?  Does an end user typically set
> them to the same branch?  Or would these parameters almost always point at
> different branchesof the remote repository?  What are the reasons for the
> end user to choose one parameter value for the --branch option and a
> different parameter value for the --track option?

--branch is always checking out just the branch on the initial add and then the branch head is of course recorded as commit in the index.

However I found that option just helpful for creating a first initial commit already in a branch.

For later submodule init or updates this has no effect.

> > diff --git a/git-submodule.sh b/git-submodule.sh
> > index 2f47e06..16df528 100755
> > --- a/git-submodule.sh
> > +++ b/git-submodule.sh
> > ...
> > @@ -197,12 +203,14 @@ cmd_add()
> >  		(unset GIT_DIR; cd "$path" && git checkout -f -q ${branch:+-b
> "$branch" "origin/$branch"}) ||
> >  		die "Unable to checkout submodule '$path'"
> >  	fi
> > +	test -n "$track" && printf '160000
> 0000000000000000000000000000000000000000\t%s\n' "$path" | git update-index --index-info
> >  
> 
> You have many overlong lines due to the 0{40} constant string in your
> patch.  Have a
> 
> 	null_sha1=0000000000000000000000000000000000000000
> 
> at the beginning of the script, and rewrite the above like this:
> 
> 	test -n "$track" &&
>                 printf '160000 %s\t%s\n' "$null_sha1" "$path" |
>                 git update-index --index-info
> 
> or even like this:
> 
> 	if test -n "$track"
>         then
>                 printf '160000 %s\t%s\n' "$null_sha1" "$path" |
>                 git update-index --index-info
> 	fi
> 
> Use of $null_sha1 throughout the script will make things easier to read
> and at the same time make it less error prone as well for "git submodule"
> developers.

Okay, thanks that is a good practice.

> > @@ -345,11 +357,29 @@ cmd_update()
> >  			then
> >  				force="-f"
> >  			fi
> > +			pull=
> > +			if [ "$sha1" = "0000000000000000000000000000000000000000" ]
> > +			then
> > +				track=$(git config submodule."$name".track)
> > +				: ${track:="master"}
> 
> In the v2 patch this used to point at "HEAD".  What made you change your
> mind?

Because at the moment HEAD would be created as local branhc, which is horribly broken.

> > +				# if the local branch does not yet exist, create it
> > +				( unset GIT_DIR; cd "$path"; git-show-ref --heads --tags -q
> "$track" || git branch --track "$track" "origin/$track" )
> 
> No error checking?
> 
> 	(
>         	unset GIT_DIR;
>                 cd "$path" &&
>                 git show-ref --heads --tags -q "$track" ||
>                 git branch --track "$track" "origin/$track"
> 	) || barf
> 
> The ';' after unset is intentional; some shells reports failure when you
> unset an unset variable.

Okay, thanks. I didn't know that.

> 
> > +				sha1="$track"
> > +				pull=1
> 
> I tend to prefer booleans in shell scripts spelled like boolean, e.g.
> 
> 	pull=yes

Good idea.

> 
> > +			fi
> > +
> >  			(unset GIT_DIR; cd "$path" && git-fetch &&
> >  				git-checkout $force -q "$sha1") ||
> >  			die "Unable to checkout '$sha1' in submodule path '$path'"
> >  
> >  			say "Submodule path '$path': checked out '$sha1'"
> > +
> > +			if [ "$pull" = "1" ]
> > +			then
> > +				# Now pull new updates from origin
> > +				( unset GIT_DIR; cd "$path"; git-pull ) || die "Unable to pull in
> submodule path '$path'"
> 
> No error checking?

In v3 there should be ... And I even see error checking in the above ...

> 
> > +			fi
> > +
> >  		fi
> >  	done
> >  }
> > @@ -596,7 +626,8 @@ cmd_status()
> >  		set_name_rev "$path" "$sha1"
> >  		if git diff-files --quiet -- "$path"
> >  		then
> > -			say " $sha1 $path$revname"
> > +			track=$(git config submodule."$name".track)
> > +			say " $sha1 $path$revname${track:+ (tracking "$track")}"
> >  		else
> >  			if test -z "$cached"
> >  			then
> > diff --git a/read-cache.c b/read-cache.c
> > index 8579663..0c14b68 100644
> > --- a/read-cache.c
> > +++ b/read-cache.c
> > @@ -137,6 +137,11 @@ static int ce_compare_gitlink(struct cache_entry
> *ce)
> >  	 */
> >  	if (resolve_gitlink_ref(ce->name, "HEAD", sha1) < 0)
> >  		return 0;
> > +
> > +	// To be able to track newest revision
> > +	if (is_null_sha1(ce->sha1))
> > +		return 0;
> > +
> 
> I think the comment is wrong, as it is not about newness at all.
> 
> 	/* ignore changes in the submodule path */
> 
> would be more appropriate.

Yes, I do agree. I am now changing first to --assume-unchanged so this is made unnecessary.

I'll write a PATCH/RFC next.

Thank you for your detailed feedback,

Best Wishes,

Fabian

Re: [PATCH v3] submodule: Allow tracking of the newest revision of a branch in a submodule

[Lars Hjemli]

On Thu, Dec 11, 2008 at 21:42, Junio C Hamano <gitster@pobox.com> wrote:
> I wonder if you can just set "assume unchanged" bit for the subproject
> gitlink in the index to achieve the same goal.

Using assume-unchanged works, in the sense that the modification to
the submodule is not detected in the containing repo. But running `git
submodule update` will checkout the sha1 recorded in HEAD, and I
suspect Fabian wants something like the hypothetical command `git
submodule update -b [branch]` which could do `(cd sub && git fetch &&
git reset --hard origin/$branch)`.


>
> Or is there more to it?
>

Something like this (probably mangled) patch is needed for 'submodule
status' to behave sensibly when the assume-unchanged bit is turned on
for a submodule path.


diff --git a/git-submodule.sh b/git-submodule.sh
index 2f47e06..375dfbf 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -588,21 +588,21 @@ cmd_status()
        do
                name=$(module_name "$path") || exit
                url=$(git config submodule."$name".url)
+               orgsha1=$(git ls-tree HEAD "$path" | cut -d ' ' -f 3 | cut -f 1)
                if test -z "$url" || ! test -d "$path"/.git -o -f "$path"/.git
                then
                        say "-$sha1 $path"
                        continue;
                fi
+               if test -z "$cached"
+               then
+                       sha1=$(unset GIT_DIR; cd "$path" && git
rev-parse --verify HEAD)
+               fi
                set_name_rev "$path" "$sha1"
-               if git diff-files --quiet -- "$path"
+               if test "$sha1" = "$orgsha1"
                then
                        say " $sha1 $path$revname"
                else
-                       if test -z "$cached"
-                       then
-                               sha1=$(unset GIT_DIR; cd "$path" &&
git rev-parse --verify HEAD)
-                               set_name_rev "$path" "$sha1"
-                       fi
                        say "+$sha1 $path$revname"
                fi
        done

Re: [PATCH v3] submodule: Allow tracking of the newest revision of a branch in a submodule

[Junio C Hamano]

"Lars Hjemli" <hjemli@gmail.com> writes:

> On Thu, Dec 11, 2008 at 21:42, Junio C Hamano <gitster@pobox.com> wrote:
>> I wonder if you can just set "assume unchanged" bit for the subproject
>> gitlink in the index to achieve the same goal.
>
> Using assume-unchanged works, in the sense that the modification to
> the submodule is not detected in the containing repo. But running `git
> submodule update` will checkout the sha1 recorded in HEAD, and I
> suspect Fabian wants something like the hypothetical command `git
> submodule update -b [branch]` which could do `(cd sub && git fetch &&
> git reset --hard origin/$branch)`.

Yeah, that would *also* make sense, but I think that is orthogonal issue.

You can update the state of the checkouts of subproject repositories in
any way you want.  Doing so however makes "git commit -a" inconvenient to
use without assume-unchanged.  The magic 0{40} which Fabian's patch
addresses the same issue in a different way.

Although I would probably detach the head at that point, rather than
resetting whatever branch happens to be checked out:

	( cd sub && git fetch && git checkout origin/$branch^0 )

We also need to make sure that whatever we do we should not break
workflows that do not check out submodules that are uninteresting.  So
doing the above unconditionally to all the submodules is out.  In such a
sparsely populated superproject, "cd sub" would go to an empty directory,
and "git fetch" step would error out.

I did not read Fabian's patch too deeply, and do not remember what checks
it did before running "git pull".  Perhaps it pulled unconditionally?

Re: [PATCH v3] submodule: Allow tracking of the newest revision of a branch in a submodule

[Junio C Hamano]

"Fabian Franz" <FabianFranz@gmx.de> writes:

>> Fabian Franz <git@fabian-franz.de> writes:
>> 
>> > However I have both cases: Stable development, where I need one special
>> > version and "wild" development, where I always want the newest published
>> > one.
>> 
>> I do not think supporting both styles of development is a bad idea.
>> 
>> However, use of 0{40} in the index and the resulting commit object in the
>> superproject means that this is a project-wide decision, not your personal
>> preference.  It is not implausible that you would want to do a wild
>> expeeriment in your own clone of a project that uses the "Stable
>> development" approach (hence the upstream never would want to have 0{40}
>> gitlink in its commits).
>
> Yes, but at the same time I might want to record it permanently as a project decision or play at my own with it ...
>
> So both styles should be supported.

While I think they both _could_ have uses, I do not necessarily agree with
your "should be".  First of all, I am not sure project wide 0{40} really
makes sense.

By creating such a commit in your superproject, you are essentially
claiming that you will work with _any_ future version of the subproject,
which is rather absurd.

And using 0{40} in trees and in the index to mark it is not really
necessary, and here is why.

You could tell the participants that you do not care the exact version by
storing 0{40} in the trees and the index, but in order for you to tell
them the tip of which branch of the subproject to use, you need to give
that information (i.e. branch name) to them as well.  Obviously there is
not enough space to put that information in gitlink (we could make room
and I have another implementation in mind but that will be a more involved
change so for a moment let's not go there).  The infomation will come
somewhere out-of-band, not in trees nor in the index.  And at that point,
the presense of such an out-of-band information itself is a good enough
cue that such a path in the superproject is for the "wilder" style of
development with the submodule.

Such an out-of-band information is necessary to use submodules in
distributed development already (iow, the commit object name in gitlink is
not enough), and we already have a Porcelain convention for that.  The
canonical repository URL for each submodule path is distributed as part of
the superproject in .gitmodules.  I would imagine that the message from
the project that says "we expect you to use 'wilder' development style
with this submodule, and use the tip of frotz branch here", if it ever
makes sense, can be recorded in .gitmodules as well.

When updating (or initializing) a submodule, we can check .gitmodules, and
iff it is the "wilder" kind, we can set assume-unchanged in the index and
run "cd there && git fetch $remote $branch && git checkout FETCH_HEAD^0"
or whatever you did in your patch.

If the supermodule did not work well with the updated submodule in such a
checkout, at least you have one commit that you can reset your submodule
checkout to, if you do not wipe that information with 0{40} in the trees
and in the index.  The commit recorded in the gitlink can serve as the
"project wide" suggested version to use, even in "wilder" development
style that also suggests to use "tip of that branch".