Status of dumb http transport?

Status of dumb http transport?

[Martin Langhoff]

-- 
 martin.langhoff@gmail.com
 martin@laptop.org -- Software Architect - OLPC
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff

Status of dumb http transport?

[Martin Langhoff]

Hi List!

What is the state of dumb http transport today, for fetching updates?
Is the client code smart enough to fetch indexes and use range
requests? If so, how does that fare for latency?

Background: I am looking at whether yum repositories' data (currently
in sqlite & xml) could benefit from being a git (or very gittish)
database -- with a bit of re-organizing to make it git-efficient of
course.

Not the data that would benefit, but rather, users pulling updates
from fast-moving repos (updates, updates-testing, rawhide...).

One of the constraints is that this has to be http, and work well
across a universe of mirrors (that won't install or configure
software) and the good bad and ugly world of http proxies. Yum can be
taught to use the git proto, but that won't gain widespread use
quickly -- http is and will be the mainstay for a long time.



m
-- 
 martin.langhoff@gmail.com
 martin@laptop.org -- Software Architect - OLPC
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff

Re: Status of dumb http transport?

[Jakub Narebski]

Martin Langhoff <martin.langhoff@gmail.com> writes:

> What is the state of dumb http transport today, for fetching updates?
> Is the client code smart enough to fetch indexes and use range
> requests? If so, how does that fare for latency?

A good description of "dumb" http transport can be found in Pro Git
book, in chapter 9.6 "Transfer Protocols", section "The Dumb Protocol"
http://progit.org/book/ch9-6.html
 
In short "dumb" http transport is realy dumb, it consults
.git/info/refs to know where to start from, and
.git/objects/info/packs to know what packfiles are available (both are
generated by git-update-server-info, usually run from update hook).

Starting from appropriate object, it tries to download loose object,
if posible, and if not then it downloads index files to know which
packfile to download.  It walks the chain down checking which objects
it has, and if there is one missing it downloads it.  It always
downloads _whole_ pack files.  It doesn't use range request... what's
more, as far as I know it doesn't know how to resume partially
downloaded packfile (hough it wouldn't re-download packfile it already
got).

> Background: I am looking at whether yum repositories' data (currently
> in sqlite & xml) could benefit from being a git (or very gittish)
> database -- with a bit of re-organizing to make it git-efficient of
> course.
> 
> Not the data that would benefit, but rather, users pulling updates
> from fast-moving repos (updates, updates-testing, rawhide...).
> 
> One of the constraints is that this has to be http, and work well
> across a universe of mirrors (that won't install or configure
> software) and the good bad and ugly world of http proxies. Yum can be
> taught to use the git proto, but that won't gain widespread use
> quickly -- http is and will be the mainstay for a long time.

Note that from some time we have "smart" http protocol, which is
almost git protocol proxied over http... with addition that http
protocol is stateless.  Default git-http-backend CGI script falls back
to "dumb" protocol if client doesn't support smart http transport.

-- 
Jakub Narebski
Poland
ShadeHawk on #git