git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jiang Xin <worldhello.net@gmail.com>
To: Git List <git@vger.kernel.org>,
	Junio C Hamano <gitster@pobox.com>,
	Eric Sunshine <sunshine@sunshineco.com>
Cc: Jiang Xin <zhiyou.jx@alibaba-inc.com>
Subject: [PATCH v3 0/4] support remote archive from stateless transport
Date: Wed,  4 Oct 2023 23:21:39 +0800	[thread overview]
Message-ID: <cover.1696432593.git.zhiyou.jx@alibaba-inc.com> (raw)
In-Reply-To: <xmqqil7yq6ms.fsf@gitster.g>

From: Jiang Xin <zhiyou.jx@alibaba-inc.com>

"git archive --remote=<remote>" learned to talk over the smart
http (aka stateless) transport.

range-diff v2...v3

1:  4497404900 = 1:  e660fc79b6 transport-helper: no connection restriction in connect_helper
-:  ---------- > 2:  e3dc18caa9 transport-helper: call do_take_over() in process_connect
2:  9bfaa1a904 ! 3:  01699822c3 transport-helper: run do_take_over in connect_helper
    @@ Metadata
     Author: Jiang Xin <zhiyou.jx@alibaba-inc.com>
     
      ## Commit message ##
    -    transport-helper: run do_take_over in connect_helper
    +    transport-helper: call do_take_over() in connect_helper
     
         After successfully connecting to the smart transport by calling
    -    "process_connect_service()" in "connect_helper()", run "do_take_over()"
    -    to replace the old vtable with a new one which has methods ready for
    -    the smart transport connection.
    +    process_connect_service() in connect_helper(), run do_take_over() to
    +    replace the old vtable with a new one which has methods ready for the
    +    smart transport connection.
     
    -    The subsequent commit introduces remote archive for a stateless-rpc
    -    connection. But without running "do_take_over()", it may fail to call
    -    "transport_disconnect()" in "run_remote_archiver()" of
    -    "builtin/archive.c". This is because for a stateless connection or a
    -    service like "git-upload-pack-archive", the remote helper may receive a
    -    SIGPIPE signal and exit early. To have a graceful disconnect method by
    -    calling "do_take_over()" will solve this issue.
    +    The connect_helper() function is used as the connect method of the
    +    vtable in "transport-helper.c", and it is called by transport_connect()
    +    in "transport.c" to setup a connection. The only place that we call
    +    transport_connect() so far is in "builtin/archive.c". Without running
    +    do_take_over(), it may fail to call transport_disconnect() in
    +    run_remote_archiver() of "builtin/archive.c". This is because for a
    +    stateless connection or a service like "git-upload-pack-archive", the
    +    remote helper may receive a SIGPIPE signal and exit early. To have a
    +    graceful disconnect method by calling do_take_over() will solve this
    +    issue.
    +
    +    The subsequent commit will introduce remote archive over a stateless-rpc
    +    connection.
     
         Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
     
3:  1e305394ee ! 4:  a38ac182d6 archive: support remote archive from stateless transport
    @@ Commit message
             capabilities when connecting to remote-helper, so do not send them
             in "remote-curl.c" for the "git-upload-archive" service.
     
    +    Helped-by: Eric Sunshine <sunshine@sunshineco.com>
         Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
     
      ## http-backend.c ##
    @@ http-backend.c: static void check_content_type(struct strbuf *hdr, const char *a
      static void service_rpc(struct strbuf *hdr, char *service_name)
      {
     -	const char *argv[] = {NULL, "--stateless-rpc", ".", NULL};
    -+	const char *argv[4];
    ++	struct strvec argv = STRVEC_INIT;
      	struct rpc_service *svc = select_service(hdr, service_name);
      	struct strbuf buf = STRBUF_INIT;
      
    -+	if (!strcmp(service_name, "git-upload-archive")) {
    -+		argv[1] = ".";
    -+		argv[2] = NULL;
    -+	} else {
    -+		argv[1] = "--stateless-rpc";
    -+		argv[2] = ".";
    -+		argv[3] = NULL;
    -+	}
    ++	strvec_push(&argv, svc->name);
    ++	if (strcmp(service_name, "git-upload-archive"))
    ++		strvec_push(&argv, "--stateless-rpc");
    ++	strvec_push(&argv, ".");
     +
      	strbuf_reset(&buf);
      	strbuf_addf(&buf, "application/x-git-%s-request", svc->name);
      	check_content_type(hdr, buf.buf);
    +@@ http-backend.c: static void service_rpc(struct strbuf *hdr, char *service_name)
    + 
    + 	end_headers(hdr);
    + 
    +-	argv[0] = svc->name;
    +-	run_service(argv, svc->buffer_input);
    ++	run_service(argv.v, svc->buffer_input);
    + 	strbuf_release(&buf);
    ++	strvec_clear(&argv);
    + }
    + 
    + static int dead;
     @@ http-backend.c: static struct service_cmd {
      	{"GET", "/objects/pack/pack-[0-9a-f]{64}\\.idx$", get_idx_file},
      
---

Jiang Xin (4):
  transport-helper: no connection restriction in connect_helper
  transport-helper: call do_take_over() in process_connect
  transport-helper: call do_take_over() in connect_helper
  archive: support remote archive from stateless transport

 http-backend.c         | 15 +++++++++++----
 remote-curl.c          | 14 +++++++++++---
 t/t5003-archive-zip.sh | 30 ++++++++++++++++++++++++++++++
 transport-helper.c     | 29 +++++++++++++----------------
 4 files changed, 65 insertions(+), 23 deletions(-)

-- 
2.40.1.50.gf560bcc116.dirty


  parent reply	other threads:[~2023-10-04 15:21 UTC|newest]

Thread overview: 61+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-09-19  6:41 [PATCH 1/2] transport-helper: no connection restriction in connect_helper Jiang Xin
2023-09-19  6:41 ` [PATCH 2/2] archive: support remote archive from stateless transport Jiang Xin
2023-09-19 17:18 ` [PATCH 1/2] transport-helper: no connection restriction in connect_helper Junio C Hamano
2023-09-20  0:20   ` Jiang Xin
2023-09-23 15:21   ` [PATCH v2 0/3] support remote archive from stateless transport Jiang Xin
2023-09-25 22:21     ` Junio C Hamano
2023-09-26  0:43       ` Jiang Xin
2023-09-23 15:21   ` [PATCH v2 1/3] transport-helper: no connection restriction in connect_helper Jiang Xin
2023-09-25 21:11     ` Junio C Hamano
2023-09-23 15:22   ` [PATCH v2 2/3] transport-helper: run do_take_over " Jiang Xin
2023-09-25 21:34     ` Junio C Hamano
2023-10-04 14:00       ` Jiang Xin
2023-10-04 15:21       ` Jiang Xin [this message]
2023-10-04 15:21         ` [PATCH v3 1/4] transport-helper: no connection restriction " Jiang Xin
2023-10-04 15:21         ` [PATCH v3 2/4] transport-helper: call do_take_over() in process_connect Jiang Xin
2023-10-04 18:29           ` Junio C Hamano
2023-10-04 15:21         ` [PATCH v3 3/4] transport-helper: call do_take_over() in connect_helper Jiang Xin
2023-10-04 15:21         ` [PATCH v3 4/4] archive: support remote archive from stateless transport Jiang Xin
2023-12-14 14:13         ` [PATCH v4 0/4] support remote archive via " Jiang Xin
2023-12-14 14:13           ` [PATCH v4 1/4] transport-helper: no connection restriction in connect_helper Jiang Xin
2024-01-12  7:42             ` Linus Arver
2024-01-12 21:50               ` Junio C Hamano
2024-01-16  9:04               ` Jiang Xin
2024-01-18 22:26                 ` Linus Arver
2024-01-19 10:56                   ` Jiang Xin
2024-01-20 20:25                     ` Linus Arver
2023-12-14 14:13           ` [PATCH v4 2/4] transport-helper: call do_take_over() in process_connect Jiang Xin
2023-12-14 14:13           ` [PATCH v4 3/4] transport-helper: call do_take_over() in connect_helper Jiang Xin
2024-01-12  7:56             ` Linus Arver
2024-01-16  9:41               ` Jiang Xin
2023-12-14 14:13           ` [PATCH v4 4/4] archive: support remote archive from stateless transport Jiang Xin
2024-01-12  8:12             ` Linus Arver
2024-01-16 13:39           ` [PATCH v5 0/6] support remote archive via " Jiang Xin
2024-01-16 13:39             ` [PATCH v5 1/6] transport-helper: no connection restriction in connect_helper Jiang Xin
2024-01-20 20:28               ` Linus Arver
2024-01-16 13:39             ` [PATCH v5 2/6] remote-curl: supports git-upload-archive service Jiang Xin
2024-01-20 20:30               ` Linus Arver
2024-01-16 13:39             ` [PATCH v5 3/6] transport-helper: protocol-v2 supports upload-archive Jiang Xin
2024-01-16 13:39             ` [PATCH v5 4/6] http-backend: new rpc-service for git-upload-archive Jiang Xin
2024-01-16 13:39             ` [PATCH v5 5/6] transport-helper: call do_take_over() in connect_helper Jiang Xin
2024-01-20 20:37               ` Linus Arver
2024-01-16 13:39             ` [PATCH v5 6/6] transport-helper: call do_take_over() in process_connect Jiang Xin
2024-01-20 20:43             ` [PATCH v5 0/6] support remote archive via stateless transport Linus Arver
2024-01-21  4:09               ` Jiang Xin
2024-01-21 13:15             ` [PATCH v6 " Jiang Xin
2024-01-21 13:15               ` [PATCH v6 1/6] transport-helper: no connection restriction in connect_helper Jiang Xin
2024-01-21 13:15               ` [PATCH v6 2/6] remote-curl: supports git-upload-archive service Jiang Xin
2024-01-21 13:15               ` [PATCH v6 3/6] transport-helper: protocol v2 supports upload-archive Jiang Xin
2024-01-21 13:15               ` [PATCH v6 4/6] http-backend: new rpc-service for git-upload-archive Jiang Xin
2024-01-21 13:15               ` [PATCH v6 5/6] transport-helper: call do_take_over() in connect_helper Jiang Xin
2024-01-21 13:15               ` [PATCH v6 6/6] transport-helper: call do_take_over() in process_connect Jiang Xin
2024-01-21 16:57               ` [PATCH v6 0/6] support remote archive via stateless transport Linus Arver
2024-01-22 15:54                 ` Junio C Hamano
2023-09-23 15:22   ` [PATCH v2 3/3] archive: support remote archive from " Jiang Xin
2023-09-24  6:52     ` Eric Sunshine
2023-09-24 23:39       ` Jiang Xin
2023-09-24 23:58         ` rsbecker
2023-09-25  0:15           ` Jiang Xin
2023-09-25  1:04             ` rsbecker
2023-09-24 13:41     ` Phillip Wood
2023-09-24 23:36       ` Jiang Xin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1696432593.git.zhiyou.jx@alibaba-inc.com \
    --to=worldhello.net@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=sunshine@sunshineco.com \
    --cc=zhiyou.jx@alibaba-inc.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).