git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jeff King <peff@peff.net>
To: git@vger.kernel.org
Subject: [PATCH v4 18/23] repack: consider bitmaps when performing repacks
Date: Sat, 21 Dec 2013 09:00:31 -0500	[thread overview]
Message-ID: <20131221140031.GR21145@sigill.intra.peff.net> (raw)
In-Reply-To: <20131221135651.GA20818@sigill.intra.peff.net>

From: Vicent Marti <tanoku@gmail.com>

Since `pack-objects` will write a `.bitmap` file next to the `.pack` and
`.idx` files, this commit teaches `git-repack` to consider the new
bitmap indexes (if they exist) when performing repack operations.

This implies moving old bitmap indexes out of the way if we are
repacking a repository that already has them, and moving the newly
generated bitmap indexes into the `objects/pack` directory, next to
their corresponding packfiles.

Since `git repack` is now capable of handling these `.bitmap` files,
a normal `git gc` run on a repository that has `pack.writebitmaps` set
to true in its config file will generate bitmap indexes as part of the
garbage collection process.

Alternatively, `git repack` can be called with the `-b` switch to
explicitly generate bitmap indexes if you are experimenting
and don't want them on all the time.

Signed-off-by: Vicent Marti <tanoku@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
---
 Documentation/git-repack.txt | 9 ++++++++-
 builtin/repack.c             | 9 ++++++++-
 2 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt
index 4c1aff6..dad186c 100644
--- a/Documentation/git-repack.txt
+++ b/Documentation/git-repack.txt
@@ -9,7 +9,7 @@ git-repack - Pack unpacked objects in a repository
 SYNOPSIS
 --------
 [verse]
-'git repack' [-a] [-A] [-d] [-f] [-F] [-l] [-n] [-q] [--window=<n>] [--depth=<n>]
+'git repack' [-a] [-A] [-d] [-f] [-F] [-l] [-n] [-q] [-b] [--window=<n>] [--depth=<n>]
 
 DESCRIPTION
 -----------
@@ -110,6 +110,13 @@ other objects in that pack they already have locally.
 	The default is unlimited, unless the config variable
 	`pack.packSizeLimit` is set.
 
+-b::
+--write-bitmap-index::
+	Write a reachability bitmap index as part of the repack. This
+	only makes sense when used with `-a` or `-A`, as the bitmaps
+	must be able to refer to all reachable objects. This option
+	overrides the setting of `pack.writebitmaps`.
+
 
 Configuration
 -------------
diff --git a/builtin/repack.c b/builtin/repack.c
index 8b7dfd0..239f278 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -94,7 +94,7 @@ static void get_non_kept_pack_filenames(struct string_list *fname_list)
 
 static void remove_redundant_pack(const char *dir_name, const char *base_name)
 {
-	const char *exts[] = {".pack", ".idx", ".keep"};
+	const char *exts[] = {".pack", ".idx", ".keep", ".bitmap"};
 	int i;
 	struct strbuf buf = STRBUF_INIT;
 	size_t plen;
@@ -121,6 +121,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
 	} exts[] = {
 		{".pack"},
 		{".idx"},
+		{".bitmap", 1},
 	};
 	struct child_process cmd;
 	struct string_list_item *item;
@@ -143,6 +144,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
 	int no_update_server_info = 0;
 	int quiet = 0;
 	int local = 0;
+	int write_bitmap = -1;
 
 	struct option builtin_repack_options[] = {
 		OPT_BIT('a', NULL, &pack_everything,
@@ -161,6 +163,8 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
 		OPT__QUIET(&quiet, N_("be quiet")),
 		OPT_BOOL('l', "local", &local,
 				N_("pass --local to git-pack-objects")),
+		OPT_BOOL('b', "write-bitmap-index", &write_bitmap,
+				N_("write bitmap index")),
 		OPT_STRING(0, "unpack-unreachable", &unpack_unreachable, N_("approxidate"),
 				N_("with -A, do not loosen objects older than this")),
 		OPT_INTEGER(0, "window", &window,
@@ -202,6 +206,9 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
 		argv_array_pushf(&cmd_args, "--no-reuse-delta");
 	if (no_reuse_object)
 		argv_array_pushf(&cmd_args, "--no-reuse-object");
+	if (write_bitmap >= 0)
+		argv_array_pushf(&cmd_args, "--%swrite-bitmap-index",
+				 write_bitmap ? "" : "no-");
 
 	if (pack_everything & ALL_INTO_ONE) {
 		get_non_kept_pack_filenames(&existing_packs);
-- 
1.8.5.1.399.g900e7cd

  parent reply	other threads:[~2013-12-21 14:00 UTC|newest]

Thread overview: 68+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-12-21 13:56 [PATCH v4 0/22] pack bitmaps Jeff King
2013-12-21 13:59 ` [PATCH v4 01/23] sha1write: make buffer const-correct Jeff King
2013-12-22  9:06   ` Christian Couder
2013-12-21 13:59 ` [PATCH v4 02/23] revindex: Export new APIs Jeff King
2013-12-21 13:59 ` [PATCH v4 03/23] pack-objects: Refactor the packing list Jeff King
2013-12-21 13:59 ` [PATCH v4 04/23] pack-objects: factor out name_hash Jeff King
2013-12-21 13:59 ` [PATCH v4 05/23] revision: allow setting custom limiter function Jeff King
2013-12-21 13:59 ` [PATCH v4 06/23] sha1_file: export `git_open_noatime` Jeff King
2013-12-21 13:59 ` [PATCH v4 07/23] compat: add endianness helpers Jeff King
2013-12-21 13:59 ` [PATCH v4 08/23] ewah: compressed bitmap implementation Jeff King
2014-01-23  2:05   ` Jonathan Nieder
2014-01-23 18:33     ` Jeff King
2014-01-23 18:35       ` [PATCH 1/2] compat: move unaligned helpers to bswap.h Jeff King
2014-01-23 19:41         ` Jonathan Nieder
2014-01-23 19:44           ` Jeff King
2014-01-23 19:56             ` Jonathan Nieder
2014-01-23 20:04               ` Jeff King
2014-01-23 20:08                 ` Jonathan Nieder
2014-01-23 20:09                   ` Jeff King
2014-01-23 18:35       ` [PATCH 2/2] ewah: support platforms that require aligned reads Jeff King
2014-01-23 19:52       ` [PATCH v4 08/23] ewah: compressed bitmap implementation Jonathan Nieder
2014-01-23 20:03         ` Jeff King
2014-01-23 20:12           ` Jonathan Nieder
2014-01-23 20:13             ` Jeff King
2014-01-23 20:23           ` Jonathan Nieder
2014-01-23 20:29             ` Jeff King
2014-01-23 20:38           ` Jeff King
2014-01-23 20:14       ` Shawn Pearce
2014-01-23 20:26         ` Jeff King
2014-01-23 21:53           ` brian m. carlson
2014-01-23 22:07             ` Jeff King
2014-01-23 22:17               ` Jonathan Nieder
2014-01-23 22:26                 ` Jeff King
2014-01-23 22:33                   ` Jonathan Nieder
2014-01-23 20:18       ` Jonathan Nieder
2014-01-23 21:20       ` [PATCH v2 0/3] unaligned reads from .bitmap files Jeff King
2014-01-23 21:23         ` [PATCH 1/3] block-sha1: factor out get_be and put_be wrappers Jeff King
2014-01-23 23:19           ` Jonathan Nieder
2014-01-23 21:26         ` [PATCH 2/3] read-cache: use get_be32 instead of hand-rolled ntoh_l Jeff King
2014-01-23 23:34           ` Jonathan Nieder
2014-01-24  2:22             ` Jeff King
2014-01-23 21:27         ` [PATCH 3/3] ewah: support platforms that require aligned reads Jeff King
2014-01-23 23:44           ` Jonathan Nieder
2014-01-23 23:49             ` Vicent Martí
2014-01-24  0:15               ` Jonathan Nieder
2014-01-23 23:17         ` [PATCH v2 0/3] unaligned reads from .bitmap files Jonathan Nieder
2013-12-21 13:59 ` [PATCH v4 09/23] documentation: add documentation for the bitmap format Jeff King
2013-12-21 14:00 ` [PATCH v4 10/23] pack-bitmap: add support for bitmap indexes Jeff King
2013-12-21 14:00 ` [PATCH v4 11/23] pack-objects: split add_object_entry Jeff King
2013-12-21 14:00 ` [PATCH v4 12/23] pack-objects: use bitmaps when packing objects Jeff King
2013-12-21 14:00 ` [PATCH v4 13/23] rev-list: add bitmap mode to speed up object lists Jeff King
2013-12-21 14:00 ` [PATCH v4 14/23] pack-objects: implement bitmap writing Jeff King
2013-12-21 14:00 ` [PATCH v4 15/23] repack: stop using magic number for ARRAY_SIZE(exts) Jeff King
2013-12-21 14:00 ` [PATCH v4 16/23] repack: turn exts array into array-of-struct Jeff King
2013-12-21 14:00 ` [PATCH v4 17/23] repack: handle optional files created by pack-objects Jeff King
2013-12-21 14:00 ` Jeff King [this message]
2013-12-21 14:00 ` [PATCH v4 19/23] count-objects: recognize .bitmap in garbage-checking Jeff King
2013-12-21 14:00 ` [PATCH v4 20/23] t: add basic bitmap functionality tests Jeff King
2013-12-21 14:00 ` [PATCH v4 21/23] t/perf: add tests for pack bitmaps Jeff King
2013-12-21 14:00 ` [PATCH v4 22/23] pack-bitmap: implement optional name_hash cache Jeff King
2013-12-21 14:00 ` [PATCH v4 23/23] compat/mingw.h: Fix the MinGW and msvc builds Jeff King
2013-12-25 22:08   ` Erik Faye-Lund
2013-12-28 10:00     ` Jeff King
2013-12-28 10:06       ` Vicent Martí
2013-12-28 15:58       ` Ramsay Jones
2013-12-21 14:03 ` [PATCH v4 0/22] pack bitmaps Jeff King
2013-12-21 14:05 ` Jeff King
2013-12-21 18:34 ` Thomas Rast

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20131221140031.GR21145@sigill.intra.peff.net \
    --to=peff@peff.net \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).