[PATCH 0/4] index-pack improvements

["Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>]

The first two patches are posted already, which help keep memory usage
down in the present of large blobs. The forth patch attempts to
resolve deltas in parallel.

$ time ./git index-pack --threads=1 --verify -v XXXX
Indexing objects: 100% (165375/165375), done.
Resolving deltas: 100% (124749/124749), done.

real    1m15.470s
user    1m14.899s
sys     0m0.552s

$ time ./git index-pack --threads=2 --verify -v XXXX
Indexing objects: 100% (165375/165375), done.
Resolving deltas: 100% (124749/124749), done.

real    0m41.339s
user    1m15.116s
sys     0m0.680s

$ time ./git index-pack --threads=3 --verify -v XXXX
Indexing objects: 100% (165375/165375), done.
Resolving deltas: 100% (124749/124749), done.

real    0m37.008s
user    1m35.742s
sys     0m0.803s

$ time ./git index-pack --verify -v XXXX   # four core machine
Indexing objects: 100% (165375/165375), done.
Resolving deltas: 100% (124749/124749), done.

real    0m33.701s
user    1m51.316s
sys     0m0.768s

$ time ./git index-pack --threads=8 --verify -v XXXX
Indexing objects: 100% (165375/165375), done.
Resolving deltas: 100% (124749/124749), done.

real    0m33.638s
user    1m51.783s
sys     0m0.773s

So there's improvement from user persepective, but overhead is too
high (user time from 1m14 to 1m51). Making threaded_second_pass to
process 512 consecutive objects each iteration to reduce contention on
work_mutex does not help. Any ideas?

Nguyễn Thái Ngọc Duy (4):
  index-pack --verify: skip sha-1 collision test
  index-pack: reduce memory usage when the pack has large blobs
  index-pack: move second pass code into separate function
  index-pack: support multithreaded delta resolving

 Documentation/config.txt         |    4 +
 Documentation/git-index-pack.txt |   10 ++
 Makefile                         |    2 +-
 builtin/index-pack.c             |  298 +++++++++++++++++++++++++++++++-------
 4 files changed, 260 insertions(+), 54 deletions(-)

-- 
1.7.8.36.g69ee2