[GSoC] Discuss: Implement support for reftables in ‘dumb’ HTTP transport

[GSoC] Discuss: Implement support for reftables in ‘dumb’ HTTP transport

[Aryan Gupta]

Hello everyone.

I hope you are doing well. I am brainstorming on the problem
statement "Implement support for reftables in ‘dumb’ HTTP transport"
where I encountered some doubts which I wanted to discuss with you.

Problem statement link: https://git.github.io/SoC-2024-Ideas/ (Last one)

1. Do we just need support to know about the default branch or the
entire reftable?

2. As far as I am able to understand, I just need to add support for
these reftable files to transfer through the backend incase of dumb
http protocol, is this correct or am I missing something? In simple
words what I understand is I just need to add support for the server
side nothing is to be done for the client side.

I am sorry if these sound absolutely basic but I'm genuinely eager to
learn and implement them.

Thank you.

Regards
Aryan Gupta

Re: [GSoC] Discuss: Implement support for reftables in ‘dumb’ HTTP transport

[Junio C Hamano]

Aryan Gupta <garyan447@gmail.com> writes:

> In simple
> words what I understand is I just need to add support for the server
> side nothing is to be done for the client side.

Hmph.  The "dumb" transport is kept primarily to allow folks, who
can only use web hosting that serves nothing but static files, to
publish their repositories, and requiring more things to be done on
the "dumb" server side sounds somewhat backwards to me.

To be quite honest, I personally doubt that this topic makes much
sense in this age and day---I've felt that the dumb HTTP walker
outlived its usefulness for the past 10 years already.  But perhaps
I am biased by the first-world access to the internet?

Thanks.

Re: [GSoC] Discuss: Implement support for reftables in ‘dumb’ HTTP transport

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 2123 bytes --]

On Wed, Mar 13, 2024 at 02:39:24PM -0700, Junio C Hamano wrote:
> Aryan Gupta <garyan447@gmail.com> writes:
> 
> > In simple
> > words what I understand is I just need to add support for the server
> > side nothing is to be done for the client side.
> 
> Hmph.  The "dumb" transport is kept primarily to allow folks, who
> can only use web hosting that serves nothing but static files, to
> publish their repositories, and requiring more things to be done on
> the "dumb" server side sounds somewhat backwards to me.
> 
> To be quite honest, I personally doubt that this topic makes much
> sense in this age and day---I've felt that the dumb HTTP walker
> outlived its usefulness for the past 10 years already.  But perhaps
> I am biased by the first-world access to the internet?

Even though I proposed this project in the first place I don't disagree
with you. The dumb HTTP protocol is indeed quite esoteric nowadays, and
if we want to put it officially into "maintenance" mode then I would not
mind to drop that proposed project.

The reason why I added this project is that I found it to be interesting
as a thought experiment. The reason why we have "info/refs" is to help
out clients of the dumb HTTP transport to figure out actual refs in the
repository because they typically live in many separate files. But what
I realized is that we don't actually need it with the reftable format
anymore because we basically already have a definitive list of all files
that a client needs to download to acquire all refs: "tables.list".

So theoretically speaking we can implement support for dumb HTTP with
reftables by having the client download "tables.list" and then fetch all
the "*.ref" files listed in it. Whether that is sensible may be a
different question.

Anyway. If we think that the dumb HTTP protocol extension is dumb then
I'd rather want to pull the plug on the proposed GSoC project now rather
than later to not lead down potential contributors the wrong path. But
in that case I think we might want to make the deprecated state of that
protocol official.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [GSoC] Discuss: Implement support for reftables in ‘dumb’ HTTP transport

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> The reason why I added this project is that I found it to be interesting
> as a thought experiment. The reason why we have "info/refs" is to help
> out clients of the dumb HTTP transport to figure out actual refs in the
> repository because they typically live in many separate files. But what
> I realized is that we don't actually need it with the reftable format
> anymore because we basically already have a definitive list of all files
> that a client needs to download to acquire all refs: "tables.list".

Yes, YES, YES!!!

> So theoretically speaking we can implement support for dumb HTTP with
> reftables by having the client download "tables.list" and then fetch all
> the "*.ref" files listed in it. Whether that is sensible may be a
> different question.

It is not even theoretical---I think it is the RIGHT way to do the
dumb HTTP walker if we had reftable from the beginning.  Having a
file at a known location where we can start "walking" from is
a very powerful thing [*1*].

If we do not have to change the server side at all, which I think
what you are saying, that removes the major part of the problem I
was having with this proposal.  I do not even have to worry about
speaking with the first-world bias and hurting folks with needs that
are only served by the dumb HTTP walker, either.

By the way, about two years ago, we have talked about making the
first step to deprecate dumb HTTP walker [*2*], but given that
nothing concrete materialized, we are ready to move further, yet.


[Footnote]

*1* With the filesystem backed refs, because there was no standard
    and widely supported way to ask a dumb HTTP server "what are the
    files and directories under this hierarchy?" and that was here
    the ugly "info/refs" hack came from.

*2* https://lore.kernel.org/git/f8639116d2d384a6d285c75830c52d8a8230ae6b.1647243509.git.ps@pks.im/