Non-http dumb protocols

Non-http dumb protocols

[Daniel Barkalow]

Do we actually support local-fetch and ssh-fetch/ssh-push any more? They 
seem to me to have been entirely supplanted with respect to functionality 
for a long time now, but the programs are still there. This complicates 
http-fetch, because only one -fetch can be linked into a binary as 
currently designed, and we may want to make http-fetch builtin at some 
point. Can we remove the other dumb protocols and merge fetch.c into 
http-fetch.c?

	-Daniel
*This .sig left intentionally blank*

Re: Non-http dumb protocols

[Junio C Hamano]

Daniel Barkalow <barkalow@iabervon.org> writes:

> Do we actually support local-fetch and ssh-fetch/ssh-push any more? They 
> seem to me to have been entirely supplanted with respect to functionality 
> for a long time now, but the programs are still there. This complicates 
> http-fetch, because only one -fetch can be linked into a binary as 
> currently designed, and we may want to make http-fetch builtin at some 
> point. Can we remove the other dumb protocols and merge fetch.c into 
> http-fetch.c?

I have a feeling that nobody uses commit walkers other than
http.  I do not see a point in ssh-fetch/push (we said it is
deprecated long time ago, and I was just too lazy to propose and
actually remove it when I did 1.5.2).  local-fetch _might_ be
useful as a debugging tool when you have trouble not related to
libcurl in http-fetch, but other than that I do not see a point
to it either.  

I am personally Ok with removing them on the sole basis of "they
are practically disused"; but there may be people on the list
with different opinions (at least here, they are not stupid nor
ugly).

However, from "technical" point of view, I do not think the
commit walker framework "fetch.c" is beyond salvaging, as you
make it sound, to make it a built-in.  The commit walker
framework was your work IIRC, so this may be preaching to the
choir, but my understanding is that it wants the following
functions defined in the transport backend module:

    extern int fetch(unsigned char *sha1);
    extern void prefetch(unsigned char *sha1);
    extern int fetch_ref(char *ref, unsigned char *sha1);

and exports the following symbols to be set:

    extern int get_tree;
    extern int get_history;
    extern int get_all;
    extern int get_verbosely;
    extern int get_recover;

I think you could define

	struct fetch_backend {
		int (*fetch_fn)(const unsigned char *);
		void (*prefetch_fn)(const unsigned char *);
		int (*fetch_ref_fn)(const char *, const unsigned char *);
		int get_tree;
		int get_history;
		int get_all;
		int get_verbosely;
		int get_recover;
	};

and have the backend "main()" pass a pointer to it to functions
pull(), pull_targets_stdin() and such, that are defined in
fetch.c.  That way, you can have multiple backend linked in.

To be linked with the rest of git, I think the global symbols
fetch.c has are named a little too generically, and they would
need to be fixed first.  It may make sense to do the "backend
descriptor" code restructure first _before_ linking anything
with the rest of git.  We can rename main() in *-fetch.c to
cmd_*_fetch() only for the ones that we choose to make built-in
when the conversion is ready.

Having said that, I have a feeling that many people do not build
any of the commit walkers, and especially the http walker,
because they have no need for dumb protocols, and libcurl-dev is
just another piece of dependency they do not have to have.  If
we were to do a built-in http-fetch, we also need to make sure 
it is done in such a way that people can choose to configure it
out.

Re: Non-http dumb protocols

[Julian Phillips]

On Sat, 30 Jun 2007, Junio C Hamano wrote:

> Daniel Barkalow <barkalow@iabervon.org> writes:
>
>> Do we actually support local-fetch and ssh-fetch/ssh-push any more? They
>> seem to me to have been entirely supplanted with respect to functionality
>> for a long time now, but the programs are still there. This complicates
>> http-fetch, because only one -fetch can be linked into a binary as
>> currently designed, and we may want to make http-fetch builtin at some
>> point. Can we remove the other dumb protocols and merge fetch.c into
>> http-fetch.c?


> To be linked with the rest of git, I think the global symbols
> fetch.c has are named a little too generically, and they would
> need to be fixed first.  It may make sense to do the "backend
> descriptor" code restructure first _before_ linking anything
> with the rest of git.  We can rename main() in *-fetch.c to
> cmd_*_fetch() only for the ones that we choose to make built-in
> when the conversion is ready.
>
> Having said that, I have a feeling that many people do not build
> any of the commit walkers, and especially the http walker,
> because they have no need for dumb protocols, and libcurl-dev is
> just another piece of dependency they do not have to have.  If
> we were to do a built-in http-fetch, we also need to make sure
> it is done in such a way that people can choose to configure it
> out.

As part of making fetch a builtin, I also made http-fetch a builtin so
that I reuse some of the code.  When I did this it kept the same optional
build status as the original separate program.  Since Daniel based his
work on mine, he should have this change too ... however I didn't do any 
renaming of symbols in builtin-http-fetch.c other than the minimum 
necessary to make it a builtin.

-- 
Julian

  ---
Taxes are going up so fast, the government is likely to price itself
out of the market.

Re: Non-http dumb protocols

[Johannes Schindelin]

Hi,

On Sat, 30 Jun 2007, Junio C Hamano wrote:

> Having said that, I have a feeling that many people do not build
> any of the commit walkers, and especially the http walker,
> because they have no need for dumb protocols, and libcurl-dev is
> just another piece of dependency they do not have to have.

Interestingly, I just was involved in a discussion on IRC, where somebody 
(out of quotat concerns) wants to use sftp to push to a bare repository, 
which is served via HTTP.

Unfortunately, it seems that all persons wanting to have some support for 
that, expect others to do the work for them.

However, there is a miniscule non-zero chance that eventually somebody 
might want to realise an sftp push protocol (where you basically need the 
ls-remote part of the fetcher, too, to determine what to pack and send). 
And to complete a dumb sftp fetch protocol, you'd need a commit walker,
so I'd like to have at least a minimal interface for commit walkers 
waiting for that saviour.

Ciao,
Dscho

Re: Non-http dumb protocols

[Daniel Barkalow]

On Sun, 1 Jul 2007, Johannes Schindelin wrote:

> Hi,
> 
> On Sat, 30 Jun 2007, Junio C Hamano wrote:
> 
> > Having said that, I have a feeling that many people do not build
> > any of the commit walkers, and especially the http walker,
> > because they have no need for dumb protocols, and libcurl-dev is
> > just another piece of dependency they do not have to have.
> 
> Interestingly, I just was involved in a discussion on IRC, where somebody 
> (out of quotat concerns) wants to use sftp to push to a bare repository, 
> which is served via HTTP.
> 
> Unfortunately, it seems that all persons wanting to have some support for 
> that, expect others to do the work for them.
> 
> However, there is a miniscule non-zero chance that eventually somebody 
> might want to realise an sftp push protocol (where you basically need the 
> ls-remote part of the fetcher, too, to determine what to pack and send). 
> And to complete a dumb sftp fetch protocol, you'd need a commit walker,
> so I'd like to have at least a minimal interface for commit walkers 
> waiting for that saviour.

Okay, I think I'll go for ditching ssh-fetch/-push, making the 
commit-walker code more modular, and possibly moving more of the smarts of 
http-fetch into the common code so that local-fetch is a better test for 
it and sftp fetch would be possible.

	-Daniel
*This .sig left intentionally blank*

Re: Non-http dumb protocols

[Johannes Schindelin]

Hi,

On Sun, 1 Jul 2007, Daniel Barkalow wrote:

> Okay, I think I'll go for ditching ssh-fetch/-push, making the 
> commit-walker code more modular, and possibly moving more of the smarts 
> of http-fetch into the common code so that local-fetch is a better test 
> for it and sftp fetch would be possible.

Thanks,
Dscho