[PATCH 0/9] multi-pack-index cleanups

[Derrick Stolee <dstolee@microsoft.com>]

This series is based on ds/multi-pack-index and
jk/for-each-object-iteration.

The multi-pack-index indexes objects across multiple pack-files. To
speed up object lookups and abbreviations, we do not place the pack-
files covered by the multi-pack-index into the packed_git linked list
or the packed_git_mru list. Existing test coverage focused on typical
uses and the main consumers of the multi-pack-index.

To better understand the implications of the multi-pack-index with
other scenarios, I ran the test suite after adding a step to 'git repack'
to write a multi-pack-index, and to default core.multiPackIndex to 'true'.
This commit is available as [1].

The following issues were discovered, and are fixed by this series:

1. The multi-pack-index did not distinguish between local and non-local
   pack-files.

2. A bad packed object was not inspected by object lookups in the multi-
   pack-index, so would loop infinitely trying to load the same object.

3. 'git count-objects --verbose' would not see the objects in the multi-
   pack-index and would report the multi-pack-index as garbage.

4. If the local object directory had a multi-pack-index but an alternate
   did not, then the multi-pack-index would be dropped.

5. If the multi-pack-index covered a pack-file that was paired with a
   reachability bitmap, then that bitmap would not be loaded.

Several issues were resolved simply by making a new 'all_packs' list in
the object store and replacing get_packed_git() calls with get_all_packs()
calls. The all_packs list is a linked list that starts with the pack-files
in multi-pack-indexes and then continues along the packed_git linked list.

Also: I simplified the usage reports in 'git multi-pack-index' to help
users who are entering parameters incorrectly.

[1] https://github.com/derrickstolee/git/commit/098dd1d515b592fb165a276241d7d68d1cde0036
    DO-NOT-MERGE: compute multi-pack-index on repack.
    I will send this commit as a separate patch so we can see the change
    I made and the one test I needed to fix (because it moves a pack-file,
    thereby making the multi-pack-index invalid).

[2] https://github.com/derrickstolee/git/pull/9
    A GitHub pull request containing this series.

Derrick Stolee (9):
  multi-pack-index: provide more helpful usage info
  multi-pack-index: store local property
  midx: mark bad packed objects
  midx: stop reporting garbage
  midx: fix bug that skips midx with alternates
  packfile: add all_packs list
  treewide: use get_all_packs
  midx: test a few commands that use get_all_packs
  pack-objects: consider packs in multi-pack-index

 builtin/count-objects.c     |  2 +-
 builtin/fsck.c              |  4 ++--
 builtin/gc.c                |  4 ++--
 builtin/multi-pack-index.c  | 16 +++++++-------
 builtin/pack-objects.c      | 42 +++++++++++++++++++++++++++++------
 builtin/pack-redundant.c    |  4 ++--
 fast-import.c               |  4 ++--
 http-backend.c              |  4 ++--
 midx.c                      | 32 ++++++++++++++++++---------
 midx.h                      |  7 ++++--
 object-store.h              |  6 +++++
 pack-bitmap.c               |  2 +-
 pack-objects.c              |  2 +-
 packfile.c                  | 40 ++++++++++++++++++++++++++++-----
 packfile.h                  |  1 +
 server-info.c               |  4 ++--
 t/helper/test-read-midx.c   |  2 +-
 t/t5319-multi-pack-index.sh | 44 ++++++++++++++++++++++++++++++++++---
 18 files changed, 168 insertions(+), 52 deletions(-)

-- 
2.18.0.118.gd4f65b8d14