gc.linkedobjectdir discussion

gc.linkedobjectdir discussion

[Will Entriken]

Hello,

This post presents an idea for storing git objects into a hard-linked
shared directory which would allow performance gains and safety so
that users can set it and forget it.

SETUP:
Users run `git config --global gc.linkedobjectdir ~/.gitsharedobjects`

USAGE - CACHE MISS:
During a pull, git will look for the needed object in LINKEDOBJECTDIR,
similar to how it looks in GIT_ALTERNATE_OBJECT_DIRECTORIES. If it is
not found, the object is received as normal and then hard linked into
the object store at LINKEDOBJECTDIR.

USAGE - CACHE HIT:
During another pull or clone on a different working directory, git
will find the required object in LINKEDOBJECTDIR and hard link into
the active repo's object store.

USAGE - DELETING:
Whenever deleting an object, git will stat the file to see to see if
its link count is 2, if so, both copies are deleted.

PERIODIC MAINTENANCE:
Periodically, git will check all objects in LINKEDOBJECTDIR for a link
count of 1 and delete such files. This will happen if a repository
with linking is deleted with `rm -rf`.

This feature would be a benefit to organizations which do lots of
cloning (think Travis CI) or users that clone the same project
multiple times or have the same file used across different repos.

Please share your thoughts.

Thank you,
Will

Re: gc.linkedobjectdir discussion

[Junio C Hamano]

Will Entriken <fulldecent@gmail.com> writes:

> USAGE - CACHE HIT:
> During another pull or clone on a different working directory, git
> will find the required object in LINKEDOBJECTDIR and hard link into
> the active repo's object store.

What happens when the necessary objects are in a packfile, mixed
with other objects that this repository does not need but may be
necessary for other repositories?

> USAGE - DELETING:
> Whenever deleting an object, git will stat the file to see to see if
> its link count is 2, if so, both copies are deleted.

This is totally unworkable if you ever pack your repository.  And
packing loose objects is a lot more efficient way to reduce your
disk footprint than sharing the object store among a handful of
repositories.