Enforcing clone/fetch to use references.

Enforcing clone/fetch to use references.

[David Brown]

[resending with corrected subject]

Suppose I want to publish some changes to a tree.  I have a server
available where I can run a git daemon, but for one reason or another
I want to force people to use the another git repo as a reference.
The reason could be one of bandwidth, or someone who isn't comfortable
making all of the other source available.  Ideally, someone who
already has the other git repo cloned, and just adds mine as a remote
wouldn't notice the difference.

Is there a way to do this?  I've tried various ways of using
alternates to keep the blobs out of the repository I want to export,
but the daemon just follows the alternates.  If I remove the
alternates, I then seem to have a broken repository.  Most things I
try, at least carry objects for all of the files in the HEAD tree,
which most of the time is a large portion of the data.

If there isn't a way of doing this currently, is this something that
others would find useful?

Thanks,
David

-- 
Sent by an employee of the Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum.

Re: Enforcing clone/fetch to use references.

[Jeff King]

On Tue, Sep 21, 2010 at 01:44:56PM -0700, David Brown wrote:

> Suppose I want to publish some changes to a tree.  I have a server
> available where I can run a git daemon, but for one reason or another
> I want to force people to use the another git repo as a reference.
> The reason could be one of bandwidth, or someone who isn't comfortable
> making all of the other source available.  Ideally, someone who
> already has the other git repo cloned, and just adds mine as a remote
> wouldn't notice the difference.

I think the gentoo people were talking about doing something like this.
They wanted you to use some faster and/or restartable protocol to clone
initially, and so they wanted to reject initial clones. I'm not sure if
they are doing that, and how (from the thread below, I suspect they run
a patched git).

The simplest thing would be a pre-upload-pack hook. There was some
discussion of that in this thread:

  http://article.gmane.org/gmane.comp.version-control.git/137007

but there are some security implications. For a remote site, where the
user running upload-pack trusts the hook, it should be fine.

Once you have that hook, then you basically need to just check whether
they are requesting objects in your "you should have this in your
reference repository" set (probably by using merge-base to see if it is
contained in some "everything our alternate has" ref), and then just

  echo >&2 Sorry, here's how to clone properly...
  exit 1

which should go to the user over the sideband.

And that would cover both the "clone --reference" and "git remote add"
cases, as it is not looking at what the user invoked, but which objects
they're requesting.

-Peff

Re: Enforcing clone/fetch to use references.

[David Brown]

On Tue, Sep 21, 2010 at 05:31:35PM -0400, Jeff King wrote:
> On Tue, Sep 21, 2010 at 01:44:56PM -0700, David Brown wrote:
> 
> > Suppose I want to publish some changes to a tree.  I have a server
> > available where I can run a git daemon, but for one reason or another
> > I want to force people to use the another git repo as a reference.
> > The reason could be one of bandwidth, or someone who isn't comfortable
> > making all of the other source available.  Ideally, someone who
> > already has the other git repo cloned, and just adds mine as a remote
> > wouldn't notice the difference.
> 
> I think the gentoo people were talking about doing something like this.
> They wanted you to use some faster and/or restartable protocol to clone
> initially, and so they wanted to reject initial clones. I'm not sure if
> they are doing that, and how (from the thread below, I suspect they run
> a patched git).
> 
> The simplest thing would be a pre-upload-pack hook. There was some
> discussion of that in this thread:
> 
>   http://article.gmane.org/gmane.comp.version-control.git/137007

Arun, did you ever get a chance to rework the upload-pack hooks to
work only from git daemon?  If not, I'd like to take a look into
implementing this.

Thanks,
David

-- 
Sent by an employee of the Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum.

Re: Enforcing clone/fetch to use references.

[Arun Raghavan]

Hey David,

On 22 September 2010 03:42, David Brown <davidb@codeaurora.org> wrote:
> On Tue, Sep 21, 2010 at 05:31:35PM -0400, Jeff King wrote:
>> On Tue, Sep 21, 2010 at 01:44:56PM -0700, David Brown wrote:
>>
>> > Suppose I want to publish some changes to a tree.  I have a server
>> > available where I can run a git daemon, but for one reason or another
>> > I want to force people to use the another git repo as a reference.
>> > The reason could be one of bandwidth, or someone who isn't comfortable
>> > making all of the other source available.  Ideally, someone who
>> > already has the other git repo cloned, and just adds mine as a remote
>> > wouldn't notice the difference.
>>
>> I think the gentoo people were talking about doing something like this.
>> They wanted you to use some faster and/or restartable protocol to clone
>> initially, and so they wanted to reject initial clones. I'm not sure if
>> they are doing that, and how (from the thread below, I suspect they run
>> a patched git).
>>
>> The simplest thing would be a pre-upload-pack hook. There was some
>> discussion of that in this thread:
>>
>>   http://article.gmane.org/gmane.comp.version-control.git/137007
>
> Arun, did you ever get a chance to rework the upload-pack hooks to
> work only from git daemon?  If not, I'd like to take a look into
> implementing this.

I never did get a chance to finish work on this. The last IRC
discussion we had about this is at
http://dev.gentoo.org/~ford_prefect/git-hooks-discussion-log.txt

We still definitely do need this in Gentoo for when our git migration happens.

Cheers,
-- 
Arun Raghavan
http://arunraghavan.net/
(Ford_Prefect | Gentoo) & (arunsr | GNOME)