Submodules and GIT_ALTERNATE_OBJECT_DIRECTORIES

Submodules and GIT_ALTERNATE_OBJECT_DIRECTORIES

[Michal Sojka]

Hi list,

I'd like to shorten the time needed by our continuous integration (CI)
tool to clone the source repositories. Currently the full clone takes
about 10 minutes (even from local server). Our main repository has
several submodules so the CI tool runs "git submodule update --init". My
idea was to use GIT_ALTERNATE_OBJECT_DIRECTORIES to cache objects from
several submodule repositories locally. However, this does not work
because GIT_ALTERNATE_OBJECT_DIRECTORIES is considered local to the
super-project and is not propagated to the "git clone" for submodules
(git-submodule.sh calls clear_local_git_env).

My question is why is GIT_ALTERNATE_OBJECT_DIRECTORIES considered local
to the repository? If I could modify the command-line I would use the
"git submodule update" with the --reference option, which is propagated
to clones of all submodules. Letting GIT_ALTERNATE_OBJECT_DIRECTORIES
propagate to the submodules should have the same effect as --reference
option. So why it is not propagated?

Thanks,
-Michal

Re: Submodules and GIT_ALTERNATE_OBJECT_DIRECTORIES

[Jens Lehmann]

Am 30.09.2014 um 15:25 schrieb Michal Sojka:
> I'd like to shorten the time needed by our continuous integration (CI)
> tool to clone the source repositories. Currently the full clone takes
> about 10 minutes (even from local server). Our main repository has
> several submodules so the CI tool runs "git submodule update --init". My
> idea was to use GIT_ALTERNATE_OBJECT_DIRECTORIES to cache objects from
> several submodule repositories locally. However, this does not work
> because GIT_ALTERNATE_OBJECT_DIRECTORIES is considered local to the
> super-project and is not propagated to the "git clone" for submodules
> (git-submodule.sh calls clear_local_git_env).
 >
> My question is why is GIT_ALTERNATE_OBJECT_DIRECTORIES considered local
> to the repository? If I could modify the command-line I would use the
> "git submodule update" with the --reference option, which is propagated
> to clones of all submodules. Letting GIT_ALTERNATE_OBJECT_DIRECTORIES
> propagate to the submodules should have the same effect as --reference
> option. So why it is not propagated?

Because then it would /always/ propagate? So while that would have the
same effect as using the --reference option, not using the --reference
option would behave differently, no?

Re: Submodules and GIT_ALTERNATE_OBJECT_DIRECTORIES

[Michal Sojka]

On Fri, Oct 03 2014, Jens Lehmann wrote:
> Am 30.09.2014 um 15:25 schrieb Michal Sojka:
>> I'd like to shorten the time needed by our continuous integration (CI)
>> tool to clone the source repositories. Currently the full clone takes
>> about 10 minutes (even from local server). Our main repository has
>> several submodules so the CI tool runs "git submodule update --init". My
>> idea was to use GIT_ALTERNATE_OBJECT_DIRECTORIES to cache objects from
>> several submodule repositories locally. However, this does not work
>> because GIT_ALTERNATE_OBJECT_DIRECTORIES is considered local to the
>> super-project and is not propagated to the "git clone" for submodules
>> (git-submodule.sh calls clear_local_git_env).
>  >
>> My question is why is GIT_ALTERNATE_OBJECT_DIRECTORIES considered local
>> to the repository? If I could modify the command-line I would use the
>> "git submodule update" with the --reference option, which is propagated
>> to clones of all submodules. Letting GIT_ALTERNATE_OBJECT_DIRECTORIES
>> propagate to the submodules should have the same effect as --reference
>> option. So why it is not propagated?
>
> Because then it would /always/ propagate? So while that would have the
> same effect as using the --reference option, not using the --reference
> option would behave differently, no?

That's a good reason, thanks. Fortunately, I found a way how to add
--reference to the submodule update command issued by the CI tool
(Bamboo). Instead of calling git directly, the CI tool calls my "git"
script, which modifies the command line in case of "submodule update"
and then calls the real git.

Best regards,
-Michal

Re: Submodules and GIT_ALTERNATE_OBJECT_DIRECTORIES

[Jens Lehmann]

Am 03.10.2014 um 13:56 schrieb Michal Sojka:
> On Fri, Oct 03 2014, Jens Lehmann wrote:
>> Am 30.09.2014 um 15:25 schrieb Michal Sojka:
>>> I'd like to shorten the time needed by our continuous integration (CI)
>>> tool to clone the source repositories. Currently the full clone takes
>>> about 10 minutes (even from local server). Our main repository has
>>> several submodules so the CI tool runs "git submodule update --init". My
>>> idea was to use GIT_ALTERNATE_OBJECT_DIRECTORIES to cache objects from
>>> several submodule repositories locally. However, this does not work
>>> because GIT_ALTERNATE_OBJECT_DIRECTORIES is considered local to the
>>> super-project and is not propagated to the "git clone" for submodules
>>> (git-submodule.sh calls clear_local_git_env).
>>   >
>>> My question is why is GIT_ALTERNATE_OBJECT_DIRECTORIES considered local
>>> to the repository? If I could modify the command-line I would use the
>>> "git submodule update" with the --reference option, which is propagated
>>> to clones of all submodules. Letting GIT_ALTERNATE_OBJECT_DIRECTORIES
>>> propagate to the submodules should have the same effect as --reference
>>> option. So why it is not propagated?
>>
>> Because then it would /always/ propagate? So while that would have the
>> same effect as using the --reference option, not using the --reference
>> option would behave differently, no?
>
> That's a good reason, thanks. Fortunately, I found a way how to add
> --reference to the submodule update command issued by the CI tool
> (Bamboo). Instead of calling git directly, the CI tool calls my "git"
> script, which modifies the command line in case of "submodule update"
> and then calls the real git.

Good to hear that! In the CI tool we use at $dayjob (Jenkins) we do
not use the automatic submodule update provided by the Git plugin
but call submodule update in the shell execution before building it.