From: Christian Couder <christian.couder@gmail.com>
To: git@vger.kernel.org
Cc: Junio C Hamano <gitster@pobox.com>, Jeff King <peff@peff.net>,
Ben Peart <Ben.Peart@microsoft.com>,
Nguyen Thai Ngoc Duy <pclouds@gmail.com>,
Mike Hommey <mh@glandium.org>,
Lars Schneider <larsxschneider@gmail.com>,
Eric Wong <e@80x24.org>,
Christian Couder <chriscool@tuxfamily.org>,
Ben Peart <benpeart@microsoft.com>
Subject: [RFC/PATCH v4 27/49] Documentation: add read-object-protocol.txt
Date: Tue, 20 Jun 2017 09:55:01 +0200 [thread overview]
Message-ID: <20170620075523.26961-28-chriscool@tuxfamily.org> (raw)
In-Reply-To: <20170620075523.26961-1-chriscool@tuxfamily.org>
From: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
Documentation/technical/read-object-protocol.txt | 102 +++++++++++++++++++++++
1 file changed, 102 insertions(+)
create mode 100644 Documentation/technical/read-object-protocol.txt
diff --git a/Documentation/technical/read-object-protocol.txt b/Documentation/technical/read-object-protocol.txt
new file mode 100644
index 0000000000..a893b46e7c
--- /dev/null
+++ b/Documentation/technical/read-object-protocol.txt
@@ -0,0 +1,102 @@
+Read Object Process
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The read-object process enables Git to read all missing blobs with a
+single process invocation for the entire life of a single Git command.
+This is achieved by using a packet format (pkt-line, see technical/
+protocol-common.txt) based protocol over standard input and standard
+output as follows. All packets, except for the "*CONTENT" packets and
+the "0000" flush packet, are considered text and therefore are
+terminated by a LF.
+
+Git starts the process when it encounters the first missing object that
+needs to be retrieved. After the process is started, Git sends a welcome
+message ("git-read-object-client"), a list of supported protocol version
+numbers, and a flush packet. Git expects to read a welcome response
+message ("git-read-object-server"), exactly one protocol version number
+from the previously sent list, and a flush packet. All further
+communication will be based on the selected version.
+
+The remaining protocol description below documents "version=1". Please
+note that "version=42" in the example below does not exist and is only
+there to illustrate how the protocol would look with more than one
+version.
+
+After the version negotiation Git sends a list of all capabilities that
+it supports and a flush packet. Git expects to read a list of desired
+capabilities, which must be a subset of the supported capabilities list,
+and a flush packet as response:
+------------------------
+packet: git> git-read-object-client
+packet: git> version=1
+packet: git> version=42
+packet: git> 0000
+packet: git< git-read-object-server
+packet: git< version=1
+packet: git< 0000
+packet: git> capability=get
+packet: git> capability=have
+packet: git> capability=put
+packet: git> capability=not-yet-invented
+packet: git> 0000
+packet: git< capability=get
+packet: git< 0000
+------------------------
+The only supported capability in version 1 is "get".
+
+Afterwards Git sends a list of "key=value" pairs terminated with a flush
+packet. The list will contain at least the command (based on the
+supported capabilities) and the sha1 of the object to retrieve. Please
+note, that the process must not send any response before it received the
+final flush packet.
+
+When the process receives the "get" command, it should make the requested
+object available in the git object store and then return success. Git will
+then check the object store again and this time find it and proceed.
+------------------------
+packet: git> command=get
+packet: git> sha1=0a214a649e1b3d5011e14a3dc227753f2bd2be05
+packet: git> 0000
+------------------------
+
+The process is expected to respond with a list of "key=value" pairs
+terminated with a flush packet. If the process does not experience
+problems then the list must contain a "success" status.
+------------------------
+packet: git< status=success
+packet: git< 0000
+------------------------
+
+In case the process cannot or does not want to process the content, it
+is expected to respond with an "error" status.
+------------------------
+packet: git< status=error
+packet: git< 0000
+------------------------
+
+In case the process cannot or does not want to process the content as
+well as any future content for the lifetime of the Git process, then it
+is expected to respond with an "abort" status at any point in the
+protocol.
+------------------------
+packet: git< status=abort
+packet: git< 0000
+------------------------
+
+Git neither stops nor restarts the process in case the "error"/"abort"
+status is set.
+
+If the process dies during the communication or does not adhere to the
+protocol then Git will stop the process and restart it with the next
+object that needs to be processed.
+
+After the read-object process has processed an object it is expected to
+wait for the next "key=value" list containing a command. Git will close
+the command pipe on exit. The process is expected to detect EOF and exit
+gracefully on its own. Git will wait until the process has stopped.
+
+A long running read-object process demo implementation can be found in
+`contrib/long-running-read-object/example.pl` located in the Git core
+repository. If you develop your own long running process then the
+`GIT_TRACE_PACKET` environment variables can be very helpful for
+debugging (see linkgit:git[1]).
--
2.13.1.565.gbfcd7a9048
next prev parent reply other threads:[~2017-06-20 7:56 UTC|newest]
Thread overview: 64+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-06-20 7:54 [RFC/PATCH v4 00/49] Add initial experimental external ODB support Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 01/49] builtin/clone: get rid of 'value' strbuf Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 02/49] t0021/rot13-filter: refactor packet reading functions Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 03/49] t0021/rot13-filter: improve 'if .. elsif .. else' style Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 04/49] Add Git/Packet.pm from parts of t0021/rot13-filter.pl Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 05/49] t0021/rot13-filter: use Git/Packet.pm Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 06/49] Git/Packet.pm: improve error message Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 07/49] Git/Packet.pm: add packet_initialize() Christian Couder
2017-06-23 18:55 ` Ben Peart
2017-06-20 7:54 ` [RFC/PATCH v4 08/49] Git/Packet: add capability functions Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 09/49] Add initial external odb support Christian Couder
2017-06-23 19:49 ` Ben Peart
2017-06-20 7:54 ` [RFC/PATCH v4 10/49] external odb foreach Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 11/49] t0400: add 'put' command to odb-helper script Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 12/49] external odb: add write support Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 13/49] external-odb: accept only blobs for now Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 14/49] t0400: add test for external odb write support Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 15/49] Add GIT_NO_EXTERNAL_ODB env variable Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 16/49] Add t0410 to test external ODB transfer Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 17/49] lib-httpd: pass config file to start_httpd() Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 18/49] lib-httpd: add upload.sh Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 19/49] lib-httpd: add list.sh Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 20/49] lib-httpd: add apache-e-odb.conf Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 21/49] odb-helper: add 'store_plain_objects' to 'struct odb_helper' Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 22/49] pack-objects: don't pack objects in external odbs Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 23/49] t0420: add test with HTTP external odb Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 24/49] odb-helper: start fault in implementation Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 25/49] external-odb: add external_odb_fault_in_object() Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 26/49] odb-helper: add script_mode Christian Couder
2017-06-20 7:55 ` Christian Couder [this message]
2017-06-20 7:55 ` [RFC/PATCH v4 28/49] contrib: add long-running-read-object/example.pl Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 29/49] Add t0410 to test read object mechanism Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 30/49] odb-helper: add read_object_process() Christian Couder
2017-07-10 15:57 ` Ben Peart
2017-06-20 7:55 ` [RFC/PATCH v4 31/49] external-odb: add external_odb_get_capabilities() Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 32/49] t04*: add 'get_cap' support to helpers Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 33/49] odb-helper: call odb_helper_lookup() with 'have' capability Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 34/49] odb-helper: fix odb_helper_fetch_object() for read_object Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 35/49] Add t0460 to test passing git objects Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 36/49] odb-helper: add read_packetized_git_object_to_fd() Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 37/49] odb-helper: add read_packetized_plain_object_to_fd() Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 38/49] Add t0470 to test passing plain objects Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 39/49] odb-helper: add write_object_process() Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 40/49] Add t0480 to test "have" capability and plain objects Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 41/49] external-odb: add external_odb_do_fetch_object() Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 42/49] odb-helper: advertise 'have' capability Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 43/49] odb-helper: advertise 'put' capability Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 44/49] odb-helper: add have_object_process() Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 45/49] clone: add initial param to write_remote_refs() Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 46/49] clone: add --initial-refspec option Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 47/49] clone: disable external odb before initial clone Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 48/49] Add test for 'clone --initial-refspec' Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 49/49] t: add t0430 to test cloning using bundles Christian Couder
2017-06-20 13:48 ` [RFC/PATCH v4 00/49] Add initial experimental external ODB support Christian Couder
2017-06-23 18:24 ` Ben Peart
2017-07-01 19:41 ` Christian Couder
2017-07-01 20:12 ` Christian Couder
2017-07-01 20:33 ` Junio C Hamano
2017-07-02 4:25 ` Christian Couder
2017-07-03 16:56 ` Junio C Hamano
2017-07-06 17:36 ` Ben Peart
2017-09-15 12:56 ` Christian Couder
2017-07-12 19:06 ` Jonathan Tan
2017-09-15 13:16 ` Christian Couder
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170620075523.26961-28-chriscool@tuxfamily.org \
--to=christian.couder@gmail.com \
--cc=Ben.Peart@microsoft.com \
--cc=benpeart@microsoft.com \
--cc=chriscool@tuxfamily.org \
--cc=e@80x24.org \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=larsxschneider@gmail.com \
--cc=mh@glandium.org \
--cc=pclouds@gmail.com \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).