From: Stefan Beller <sbeller@google.com>
To: git@vger.kernel.org
Cc: Stefan Beller <sbeller@google.com>
Subject: [RFC PATCHv1 00/12] git submodule update in C with parallel cloning
Date: Thu, 15 Oct 2015 18:52:01 -0700 [thread overview]
Message-ID: <1444960333-16003-1-git-send-email-sbeller@google.com> (raw)
So eventually we want to have projects with lots of submodules such as Android
(which would have O(1000) submodules).
The very first thing a user does is cloning a project, so we want to impress
with speed there as well. git clone however is lazy and just calls
`git submodule update --init --recursive`. So we need to make that fast.
This series rewrites parts of git submodule update in C and in the second-last
patch it separates cloning and doing the other actions(checkout/rebase/merge etc)
by doing the cloning first and then the rest.
The last patch (which is broken in the first version of the series), then
proceeds to put the cloning of the submodules into the get_next_task callback
of the parallel process API.
That said, the first few patches introduce some churn in the behavior and tests
of Git, so maybe put your eyes there?
Thanks for any advice,
Stefan
Stefan Beller (12):
git submodule update: Announce skipping submodules on stderr
git submodule update: Announce uninitialized modules on stderr
git submodule update: Move branch calculation to where it's needed
git submodule update: Announce outcome of submodule operation to
stderr
git submodule update: Use its own list implementation.
git submodule update: Handle unmerged submodules in C
submodule config: keep update strategy around
git submodule update: check for "none" in C
git submodule update: Check url in C
git submodule update: Clone projects from within C
submodule--helper: Do not emit submodules to process directly.
WIP/broken Clone all outstanding submodules in parallel
builtin/submodule--helper.c | 221 ++++++++++++++++++++++++++++++++++++++++++++
git-submodule.sh | 38 +++-----
submodule-config.c | 11 +++
submodule-config.h | 1 +
t/t7400-submodule-basic.sh | 12 +--
t/t7406-submodule-update.sh | 12 +--
6 files changed, 256 insertions(+), 39 deletions(-)
--
2.5.0.277.gfdc362b.dirty
next reply other threads:[~2015-10-16 1:52 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-10-16 1:52 Stefan Beller [this message]
2015-10-16 1:52 ` [PATCH 01/12] git submodule update: Announce skipping submodules on stderr Stefan Beller
2015-10-16 20:37 ` Junio C Hamano
2015-10-16 20:47 ` Stefan Beller
2015-10-16 1:52 ` [PATCH 02/12] git submodule update: Announce uninitialized modules " Stefan Beller
2015-10-16 20:54 ` Junio C Hamano
2015-10-16 1:52 ` [PATCH 03/12] git submodule update: Move branch calculation to where it's needed Stefan Beller
2015-10-16 20:54 ` Junio C Hamano
2015-10-16 1:52 ` [PATCH 04/12] git submodule update: Announce outcome of submodule operation to stderr Stefan Beller
2015-10-16 1:52 ` [PATCH 05/12] git submodule update: Use its own list implementation Stefan Beller
2015-10-16 21:02 ` Junio C Hamano
2015-10-16 21:08 ` Stefan Beller
2015-10-16 1:52 ` [PATCH 06/12] git submodule update: Handle unmerged submodules in C Stefan Beller
2015-10-20 21:11 ` Junio C Hamano
2015-10-20 21:21 ` Stefan Beller
2015-10-16 1:52 ` [PATCH 07/12] submodule config: keep update strategy around Stefan Beller
2015-10-16 1:52 ` [PATCH 08/12] git submodule update: check for "none" in C Stefan Beller
2015-10-16 1:52 ` [PATCH 09/12] git submodule update: Check url " Stefan Beller
2015-10-16 1:52 ` [PATCH 10/12] git submodule update: Clone projects from within C Stefan Beller
2015-10-16 1:52 ` [PATCH 11/12] submodule--helper: Do not emit submodules to process directly Stefan Beller
2015-10-16 1:52 ` [PATCH 12/12] WIP/broken Clone all outstanding submodules in parallel Stefan Beller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1444960333-16003-1-git-send-email-sbeller@google.com \
--to=sbeller@google.com \
--cc=git@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).