remote helper example with push/fetch capabilities

remote helper example with push/fetch capabilities

[Klein W]

Is there any example of a remote helper [0] with push and fetch capabilities?

The git-remote-testgit.sh example [1] only has import/export capabilities.

Also, what are the advantages and disadvantages of a remote helper
with push/fetch capabilities vs a remote helper with import/export
capabilities?

Thanks.

[0] https://www.kernel.org/pub/software/scm/git/docs/git-remote-helpers.html
[1] https://github.com/git/git/blob/master/git-remote-testgit.sh

Re: remote helper example with push/fetch capabilities

[Jonathan Nieder]

Hi,

Klein W wrote:

> Is there any example of a remote helper [0] with push and fetch capabilities?

Sure --- see remote-curl.c.

There's also the "connect" capability.  builtin/remote-ext.c and
builtin/remote-fd.c are examples using that one.

[...]
> Also, what are the advantages and disadvantages of a remote helper
> with push/fetch capabilities vs a remote helper with import/export
> capabilities?

It mainly has to do with what it is convenient for your helper to
produce.  If the helper would find it more convenient to write native
git objects (for example because the remote server speaks a
git-specific protocol, as in the case of remote-curl.c) then the
"fetch" capability will be more convenient.  If the helper wants to
make a batch of new objects then a fast-import stream can be a
convenient way to do this and the "import" capability takes care of
running fast-import to take care of that.

Thanks and hope that helps,
Jonathan

Re: remote helper example with push/fetch capabilities

[Klein W]

On Mon, Dec 15, 2014 at 3:47 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Sure --- see remote-curl.c.
>
> There's also the "connect" capability.  builtin/remote-ext.c and
> builtin/remote-fd.c are examples using that one.

Thanks.

>> Also, what are the advantages and disadvantages of a remote helper
>> with push/fetch capabilities vs a remote helper with import/export
>> capabilities?
>
> It mainly has to do with what it is convenient for your helper to
> produce.  If the helper would find it more convenient to write native
> git objects (for example because the remote server speaks a
> git-specific protocol, as in the case of remote-curl.c) then the
> "fetch" capability will be more convenient.  If the helper wants to
> make a batch of new objects then a fast-import stream can be a
> convenient way to do this and the "import" capability takes care of
> running fast-import to take care of that.

I'm trying to write a remote helper for hosting git remotes on Amazon
S3.  Do you have any intuition about which capabilities would work
best for this case?

Thanks.

Re: remote helper example with push/fetch capabilities

[Jonathan Nieder]

Klein W wrote:
> On Mon, Dec 15, 2014 at 3:47 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:

>> It mainly has to do with what it is convenient for your helper to
>> produce.  If the helper would find it more convenient to write native
>> git objects (for example because the remote server speaks a
>> git-specific protocol, as in the case of remote-curl.c) then the
>> "fetch" capability will be more convenient.  If the helper wants to
>> make a batch of new objects then a fast-import stream can be a
>> convenient way to do this and the "import" capability takes care of
>> running fast-import to take care of that.
>
> I'm trying to write a remote helper for hosting git remotes on Amazon
> S3.  Do you have any intuition about which capabilities would work
> best for this case?

fetch/push.  I'd suggest looking at the "dumb" HTTP code (fetch_dumb,
push_dav) in remote-curl.c to start.

Thanks,
Jonathan

Re: remote helper example with push/fetch capabilities

[Klein W]

On Mon, Dec 15, 2014 at 4:44 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
>> I'm trying to write a remote helper for hosting git remotes on Amazon
>> S3.  Do you have any intuition about which capabilities would work
>> best for this case?
>
> fetch/push.  I'd suggest looking at the "dumb" HTTP code (fetch_dumb,
> push_dav) in remote-curl.c to start.

This seems like a good starting point, but are there any simpler
examples that are not clouded by WebDAV/Curl stuff?  Perhaps a remote
helper that reads/writes to the filesystem directly?

Re: remote helper example with push/fetch capabilities

[Klein W]

Would someone be willing to extend the git-remote-testgit.sh example
[1] with push and fetch capabilities?  I am not familiar enough to do
it myself.

Thanks

[1] https://github.com/git/git/blob/master/git-remote-testgit.sh

Re: remote helper example with push/fetch capabilities

[Junio C Hamano]

Klein W <wineklein@gmail.com> writes:

> Is there any example of a remote helper [0] with push and fetch capabilities?
>
> The git-remote-testgit.sh example [1] only has import/export capabilities.
>
> Also, what are the advantages and disadvantages of a remote helper
> with push/fetch capabilities vs a remote helper with import/export
> capabilities?

A helper with push/fetch capabilities is responsible for (and more
importantly, "in control of") packdata creation, while helpers that
use import/export interface rely on fast-import, which is quite dumb
when it comes to storage efficiency of the resulting repository.
The former might be more preferrable from the efficiency point of
view.

BUT.

Unless your foreign SCM is Git itself, however, it is not practical
to write your own pack data generator correctly and efficiently to
support push/fetch capabilities anyway, so the choice is often made
not because of "advantages vs disadvantages" but because of what you
can write in practice.

Re: remote helper example with push/fetch capabilities

[Andrew Mackenzie]

I'm interested in doing something very similar (not to S3, but to a customer 
CMS repository), but have found it very difficult to find any documentation or 
good example code on git-remote-helpers.

If you can share any learnings or references I would really appreciate it!

thanks, Andrew