Re: Saving space/network on common repos

[Craig Silverstein <csilvers@khanacademy.org>]

btw, just FYI, the scheme you lay out here doesn't actually work
as-is.  The problem is the config file, which has an entry like:
           worktree = ../../../mysubmodule
This depends on the config file living in
./git/modules/mysubmodule/config.  But the proposed scheme moves the
config file to mysubmodule/.git/config, and the relative path is
broken.

I'm not sure what the best solution is; the cleanest one requires a
pretty substantial rewrite of git-new-workdir (not that it's such a
giant piece of code), and separating out new_workdir from new_gitdir.
The smallest one involves having some way to suppress the final 'git
checkout -f' (which is the only thing in this script that needs the
worktree entry to resolve somewhere) to allow for post-script cleanup.

craig

On Wed, Dec 17, 2014 at 4:07 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Craig Silverstein wrote:
>> On Wed, Dec 17, 2014 at 2:32 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
>>> Craig Silverstein wrote:
>
>>>> Question 4) Is there a practical way to set up submodules so they can
>>>> use the same object-sharing framework that the main repo does?
>>>
>>> It's possible to do, but we haven't written a nice UI for it yet.
>>> (In other words, you can do this by cloning with --no-recurse-submodules
>>> and manually creating the submodule workdir in the appropriate place.
>>
>> Hmm, let me see if I understand you right -- you're suggesting that
>> when cloning my reference repo, I do
>>     git clone --no-recurse-submodules <my repo>
>>     for (path, url) in `parse-.gitmodules`: git clone url path
>> # this is psuedocode, obviously :-)
>>
>> and then when I want to create a new workdir, I do something like:
>>     cd reference_repo
>>     git new-workdir /var/workspace1
>>     for (path, url) in `parse-.gitmodules`: cd path && git new-workdir /var/workspace1/path
>>
>> ?  Basically, I'm going back to the old git way of having each
>> submodule have its own .git directory, rather than having it have a
>> .git file with a 'gitdir' entry.  Am I understanding this right?
>
> Basically.  The initial clone can still use --recurse-submodules.
> When you create a new workdir you'd create new workdirs for the
> submodules by hand.
>
> A 'git submodule foreach' command in the initial repo can take
> care of the `parse-.gitmodules` part.
>
> [...]
>> Also, it seems to me there's the possibility, with git-newdir, that if
>> several of the workspaces try to fetch at the same time they could
>> step on each others' toes.  Is that a problem?  I know there's a push
>> lock but I don't believe there's a fetch lock, and I could imagine git
>> getting unhappy if two fetches happened in the same repo at the same
>> time.
>
> That's a good question.  If concurrent fetches cause trouble then I'd
> consider it a bug (it's not too different from multiple concurrent
> pushes to the same repository, which is a very common thing to do),
> but I haven't looked carefully into whether such bugs exist.
>
> Hopefully someone else can chime in.
>
> Thanks,
> Jonathan