[RFC/PATCH 0/4] parallel fetch for submodules

[Stefan Beller <sbeller@google.com>]

When I was looking at the branches of Jens for work done on submodules
not yet upstream I found a commit "WIP threaded submodule fetching[1],
and I was side tracked wanting to present a different approach to that.

The first patch is a bit unrelated as it relates to the rewrite of git-submodule.sh
but also has code in submodule.c and the following patches modify code just around
that, so I did not remove that patch from this series. It is the same I sent 
yesterday.

The next patch 2/4 presents a framework for parallel threaded work.
It allows to setup a worker pool of <n> threads and then have a queue
of tasks which are worked on by the threads. The patch is a the one which
I'd request most comments on as I think that can be reused in a variety of
situations (parallel checkout of files, parallel fetch of different remotes,
or such).

I consider the third patch farely boring as it adds argv_array_copy, so I
would not expect much discussion there.

The last patch 4/4 presents the new workdispatcher from 2/4 in use
with just one unsolved problem of how to handle the output of the
parallel commands to stdout and stderr. It may be useful to put
handling of parallel outputs into the work dispatcher.

[1] https://github.com/jlehmann/git-submod-enhancements/commit/47597753206d40e234a47392e258065c9489e2b3

This series applies on top of origin/sb/submodule-helper (d2c6c09ac819,
submodule: implement `module_list` as a builtin helper) and can also be found
at https://github.com/stefanbeller/git/tree/parallel-submodule-fetch

Stefan Beller (4):
  submodule: implement `module_name` as a builtin helper
  Add a workdispatcher to get work done in parallel
  argv_array: add argv_array_clone to clone an existing argv array
  submodule: add infrastructure to fetch submodules in parallel

 Makefile                    |   1 +
 argv-array.c                |  13 ++++
 argv-array.h                |   1 +
 builtin/fetch.c             |   3 +-
 builtin/submodule--helper.c |  23 ++++++
 git-submodule.sh            |  32 ++------
 submodule.c                 |  92 ++++++++++++++++------
 submodule.h                 |   3 +-
 workdispatcher.c            | 184 ++++++++++++++++++++++++++++++++++++++++++++
 workdispatcher.h            |  29 +++++++
 10 files changed, 332 insertions(+), 49 deletions(-)
 create mode 100644 workdispatcher.c
 create mode 100644 workdispatcher.h

-- 
2.5.0.239.g9728e1d.dirty