From: "Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>
To: git@vger.kernel.org
Cc: "Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>
Subject: [PATCH v3 00/28] First class shallow clone
Date: Mon, 25 Nov 2013 10:55:26 +0700 [thread overview]
Message-ID: <1385351754-9954-1-git-send-email-pclouds@gmail.com> (raw)
Compared to v2 [1], v3 grows a bit. The biggest difference is
.git/shallow is not updated by default (except when you clone from
a shallow repository). When you send something, the "safe" refs that
do not need new shallow roots are accepted at the receiver end, the
others rejected.
To accept those other refs, either use "fetch --update-shallow" or
enable receive.shallowupdate on the receiver side of a push.
Filtering "safe" refs requires walking through some commits, so
it'll be more expensive than normal full clones. This is especially
true for the receiver of a push (see 07/28 and 17/28). I envision
shallow repos are used as upstream to archive old history, so this is
not a good news. Commit cache (or pack v4) might help. We might even
be able to move some work from receive-pack to send-pack to reduce
server load..
[1] http://mid.gmane.org/1374314290-5976-1-git-send-email-pclouds@gmail.com
Nguyễn Thái Ngọc Duy (28):
transport.h: remove send_pack prototype, already defined in send-pack.h
send-pack: forbid pushing from a shallow repository
clone: prevent --reference to a shallow repository
update-server-info: do not publish shallow clones
This part is just cleanup.
Advertise shallow graft information on the server end
connect.c: teach get_remote_heads to parse "shallow" lines
shallow.c: add remove_reachable_shallow_points()
shallow.c: add mark_new_shallow_refs()
shallow.c: extend setup_*_shallow() to accept extra shallow points
fetch-pack.c: move shallow update code out of fetch_pack()
fetch-pack.h: one statement per bitfield declaration
clone: support remote shallow repository
fetch: support fetching from a shallow repository
upload-pack: make sure deepening preserves shallow roots
fetch: add --update-shallow to get refs that require updating .git/shallow
Basic shallow fetch/clone support on git protocol
receive-pack: reorder some code in unpack()
Support pushing from a shallow clone
New var GIT_SHALLOW_FILE to propagate --shallow-file to subprocesses
connected.c: add new variant that runs with --shallow-file
receive-pack: allow pushing with new shallow roots
send-pack: support pushing to a shallow clone
remote-curl: pass ref SHA-1 to fetch-pack as well
Push support
Support fetch/clone over http
receive-pack: support pushing to a shallow clone via http
send-pack: support pushing from a shallow clone via http
smart-http support
git-clone.txt: remove shallow clone limitations
clone: use git protocol for cloning shallow repo locally
prune: clean .git/shallow after pruning objects
miscellaneous
Documentation/config.txt | 4 +
Documentation/fetch-options.txt | 14 +-
Documentation/git-clone.txt | 7 +-
Documentation/gitremote-helpers.txt | 7 +
Documentation/technical/pack-protocol.txt | 7 +-
builtin/clone.c | 18 +-
builtin/fetch-pack.c | 23 +-
builtin/fetch.c | 15 +-
builtin/gc.c | 1 +
builtin/prune.c | 4 +
builtin/receive-pack.c | 248 +++++++++++++++++----
builtin/send-pack.c | 5 +-
cache.h | 1 +
commit.h | 19 +-
connect.c | 14 +-
connected.c | 42 +++-
connected.h | 2 +
environment.c | 6 +
fetch-pack.c | 132 ++++++++++--
fetch-pack.h | 29 +--
git.c | 2 +-
remote-curl.c | 33 ++-
remote.h | 5 +-
send-pack.c | 20 +-
server-info.c | 9 +
shallow.c | 348 +++++++++++++++++++++++++++++-
t/t5536-fetch-shallow.sh (new +x) | 193 +++++++++++++++++
t/t5537-push-shallow.sh (new +x) | 184 ++++++++++++++++
t/t5601-clone.sh | 7 +
transport-helper.c | 6 +
transport.c | 22 +-
transport.h | 16 +-
upload-pack.c | 8 +-
33 files changed, 1323 insertions(+), 128 deletions(-)
create mode 100755 t/t5536-fetch-shallow.sh
create mode 100755 t/t5537-push-shallow.sh
--
1.8.2.83.gc99314b
next reply other threads:[~2013-11-25 3:51 UTC|newest]
Thread overview: 79+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-11-25 3:55 Nguyễn Thái Ngọc Duy [this message]
2013-11-25 3:55 ` [PATCH v3 01/28] transport.h: remove send_pack prototype, already defined in send-pack.h Nguyễn Thái Ngọc Duy
2013-11-25 3:55 ` [PATCH v3 02/28] send-pack: forbid pushing from a shallow repository Nguyễn Thái Ngọc Duy
2013-11-25 3:55 ` [PATCH v3 03/28] clone: prevent --reference to " Nguyễn Thái Ngọc Duy
2013-11-26 5:52 ` Eric Sunshine
2013-11-25 3:55 ` [PATCH v3 04/28] update-server-info: do not publish shallow clones Nguyễn Thái Ngọc Duy
2013-11-25 20:08 ` Junio C Hamano
2013-11-26 12:41 ` Duy Nguyen
2013-11-25 3:55 ` [PATCH v3 05/28] Advertise shallow graft information on the server end Nguyễn Thái Ngọc Duy
2013-11-25 3:55 ` [PATCH v3 06/28] connect.c: teach get_remote_heads to parse "shallow" lines Nguyễn Thái Ngọc Duy
2013-11-25 21:42 ` Junio C Hamano
2013-11-25 22:42 ` Junio C Hamano
2013-11-27 13:02 ` Duy Nguyen
2013-11-25 3:55 ` [PATCH v3 07/28] shallow.c: add remove_reachable_shallow_points() Nguyễn Thái Ngọc Duy
2013-11-25 21:53 ` Junio C Hamano
2013-11-25 3:55 ` [PATCH v3 08/28] shallow.c: add mark_new_shallow_refs() Nguyễn Thái Ngọc Duy
2013-11-25 22:20 ` Junio C Hamano
2013-11-26 13:18 ` Duy Nguyen
2013-11-26 22:20 ` Junio C Hamano
2013-11-25 3:55 ` [PATCH v3 09/28] shallow.c: extend setup_*_shallow() to accept extra shallow points Nguyễn Thái Ngọc Duy
2013-11-25 22:25 ` Junio C Hamano
2013-11-25 3:55 ` [PATCH v3 10/28] fetch-pack.c: move shallow update code out of fetch_pack() Nguyễn Thái Ngọc Duy
2013-11-25 3:55 ` [PATCH v3 11/28] fetch-pack.h: one statement per bitfield declaration Nguyễn Thái Ngọc Duy
2013-11-25 3:55 ` [PATCH v3 12/28] clone: support remote shallow repository Nguyễn Thái Ngọc Duy
2013-11-25 3:55 ` [PATCH v3 13/28] fetch: support fetching from a " Nguyễn Thái Ngọc Duy
2013-11-27 9:47 ` Eric Sunshine
2013-11-25 3:55 ` [PATCH v3 14/28] upload-pack: make sure deepening preserves shallow roots Nguyễn Thái Ngọc Duy
2013-11-25 3:55 ` [PATCH v3 15/28] fetch: add --update-shallow to get refs that require updating .git/shallow Nguyễn Thái Ngọc Duy
2013-11-27 1:53 ` Eric Sunshine
2013-11-27 12:54 ` Duy Nguyen
2013-11-27 19:04 ` Junio C Hamano
2013-11-25 3:55 ` [PATCH v3 16/28] receive-pack: reorder some code in unpack() Nguyễn Thái Ngọc Duy
2013-12-02 22:25 ` Junio C Hamano
2013-11-25 3:55 ` [PATCH v3 17/28] Support pushing from a shallow clone Nguyễn Thái Ngọc Duy
2013-11-26 20:38 ` Eric Sunshine
2013-11-25 3:55 ` [PATCH v3 18/28] New var GIT_SHALLOW_FILE to propagate --shallow-file to subprocesses Nguyễn Thái Ngọc Duy
2013-11-25 3:55 ` [PATCH v3 19/28] connected.c: add new variant that runs with --shallow-file Nguyễn Thái Ngọc Duy
2013-11-25 3:55 ` [PATCH v3 20/28] receive-pack: allow pushing with new shallow roots Nguyễn Thái Ngọc Duy
2013-11-25 3:55 ` [PATCH v3 21/28] send-pack: support pushing to a shallow clone Nguyễn Thái Ngọc Duy
2013-11-25 3:55 ` [PATCH v3 22/28] remote-curl: pass ref SHA-1 to fetch-pack as well Nguyễn Thái Ngọc Duy
2013-11-25 3:55 ` [PATCH v3 23/28] Support fetch/clone over http Nguyễn Thái Ngọc Duy
2013-11-25 3:55 ` [PATCH v3 24/28] receive-pack: support pushing to a shallow clone via http Nguyễn Thái Ngọc Duy
2013-11-25 3:55 ` [PATCH v3 25/28] send-pack: support pushing from " Nguyễn Thái Ngọc Duy
2013-11-25 3:55 ` [PATCH v3 26/28] git-clone.txt: remove shallow clone limitations Nguyễn Thái Ngọc Duy
2013-11-25 3:55 ` [PATCH v3 27/28] clone: use git protocol for cloning shallow repo locally Nguyễn Thái Ngọc Duy
2013-11-27 1:36 ` Eric Sunshine
2013-11-25 3:55 ` [PATCH v3 28/28] prune: clean .git/shallow after pruning objects Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 00/28] First class shallow clone Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 01/28] transport.h: remove send_pack prototype, already defined in send-pack.h Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 02/28] Replace struct extra_have_objects with struct sha1_array Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 03/28] send-pack: forbid pushing from a shallow repository Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 04/28] clone: prevent --reference to " Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 05/28] Make the sender advertise shallow commits to the receiver Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 06/28] connect.c: teach get_remote_heads to parse "shallow" lines Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 07/28] shallow.c: extend setup_*_shallow() to accept extra shallow commits Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 08/28] shallow.c: the 8 steps to select new commits for .git/shallow Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 09/28] shallow.c: steps 6 and 7 " Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 10/28] fetch-pack.c: move shallow update code out of fetch_pack() Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 11/28] fetch-pack.h: one statement per bitfield declaration Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 12/28] clone: support remote shallow repository Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 13/28] fetch: support fetching from a " Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 14/28] upload-pack: make sure deepening preserves shallow roots Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 15/28] fetch: add --update-shallow to accept refs that update .git/shallow Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 16/28] receive-pack: reorder some code in unpack() Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 17/28] receive/send-pack: support pushing from a shallow clone Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 18/28] New var GIT_SHALLOW_FILE to propagate --shallow-file to subprocesses Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 19/28] connected.c: add new variant that runs with --shallow-file Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 20/28] receive-pack: allow pushes that update .git/shallow Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 21/28] send-pack: support pushing to a shallow clone Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 22/28] remote-curl: pass ref SHA-1 to fetch-pack as well Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 23/28] Support shallow fetch/clone over smart-http Nguyễn Thái Ngọc Duy
2014-01-08 11:25 ` Jeff King
2014-01-08 12:13 ` [PATCH] t5537: fix incorrect expectation in test case 10 Nguyễn Thái Ngọc Duy
2014-01-09 21:57 ` Jeff King
2013-12-05 13:02 ` [PATCH v4 24/28] receive-pack: support pushing to a shallow clone via http Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 25/28] send-pack: support pushing from " Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 26/28] clone: use git protocol for cloning shallow repo locally Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 27/28] prune: clean .git/shallow after pruning objects Nguyễn Thái Ngọc Duy
2013-12-05 13:02 ` [PATCH v4 28/28] git-clone.txt: remove shallow clone limitations Nguyễn Thái Ngọc Duy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1385351754-9954-1-git-send-email-pclouds@gmail.com \
--to=pclouds@gmail.com \
--cc=git@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).