git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jeff King <peff@peff.net>
To: git@vger.kernel.org
Cc: "Vicent Martí" <vicent@github.com>
Subject: [PATCH v3 17/21] repack: consider bitmaps when performing repacks
Date: Thu, 14 Nov 2013 07:46:10 -0500	[thread overview]
Message-ID: <20131114124610.GQ10757@sigill.intra.peff.net> (raw)
In-Reply-To: <20131114124157.GA23784@sigill.intra.peff.net>

From: Vicent Marti <tanoku@gmail.com>

Since `pack-objects` will write a `.bitmap` file next to the `.pack` and
`.idx` files, this commit teaches `git-repack` to consider the new
bitmap indexes (if they exist) when performing repack operations.

This implies moving old bitmap indexes out of the way if we are
repacking a repository that already has them, and moving the newly
generated bitmap indexes into the `objects/pack` directory, next to
their corresponding packfiles.

Since `git repack` is now capable of handling these `.bitmap` files,
a normal `git gc` run on a repository that has `pack.writebitmaps` set
to true in its config file will generate bitmap indexes as part of the
garbage collection process.

Alternatively, `git repack` can be called with the `-b` switch to
explicitly generate bitmap indexes if you are experimenting
and don't want them on all the time.

Signed-off-by: Vicent Marti <tanoku@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
---
 Documentation/git-repack.txt | 9 ++++++++-
 builtin/repack.c             | 9 ++++++++-
 2 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt
index 509cf73..002cfd5 100644
--- a/Documentation/git-repack.txt
+++ b/Documentation/git-repack.txt
@@ -9,7 +9,7 @@ git-repack - Pack unpacked objects in a repository
 SYNOPSIS
 --------
 [verse]
-'git repack' [-a] [-A] [-d] [-f] [-F] [-l] [-n] [-q] [--window=<n>] [--depth=<n>]
+'git repack' [-a] [-A] [-d] [-f] [-F] [-l] [-n] [-q] [-b] [--window=<n>] [--depth=<n>]
 
 DESCRIPTION
 -----------
@@ -110,6 +110,13 @@ other objects in that pack they already have locally.
 	The default is unlimited, unless the config variable
 	`pack.packSizeLimit` is set.
 
+-b::
+--write-bitmap-index::
+	Write a reachability bitmap index as part of the repack. This
+	only makes sense when used with `-a` or `-A`, as the bitmaps
+	must be able to refer to all reachable objects. This option
+	overrides the setting of `pack.writebitmaps`.
+
 
 Configuration
 -------------
diff --git a/builtin/repack.c b/builtin/repack.c
index 8b7dfd0..239f278 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -94,7 +94,7 @@ static void get_non_kept_pack_filenames(struct string_list *fname_list)
 
 static void remove_redundant_pack(const char *dir_name, const char *base_name)
 {
-	const char *exts[] = {".pack", ".idx", ".keep"};
+	const char *exts[] = {".pack", ".idx", ".keep", ".bitmap"};
 	int i;
 	struct strbuf buf = STRBUF_INIT;
 	size_t plen;
@@ -121,6 +121,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
 	} exts[] = {
 		{".pack"},
 		{".idx"},
+		{".bitmap", 1},
 	};
 	struct child_process cmd;
 	struct string_list_item *item;
@@ -143,6 +144,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
 	int no_update_server_info = 0;
 	int quiet = 0;
 	int local = 0;
+	int write_bitmap = -1;
 
 	struct option builtin_repack_options[] = {
 		OPT_BIT('a', NULL, &pack_everything,
@@ -161,6 +163,8 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
 		OPT__QUIET(&quiet, N_("be quiet")),
 		OPT_BOOL('l', "local", &local,
 				N_("pass --local to git-pack-objects")),
+		OPT_BOOL('b', "write-bitmap-index", &write_bitmap,
+				N_("write bitmap index")),
 		OPT_STRING(0, "unpack-unreachable", &unpack_unreachable, N_("approxidate"),
 				N_("with -A, do not loosen objects older than this")),
 		OPT_INTEGER(0, "window", &window,
@@ -202,6 +206,9 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
 		argv_array_pushf(&cmd_args, "--no-reuse-delta");
 	if (no_reuse_object)
 		argv_array_pushf(&cmd_args, "--no-reuse-object");
+	if (write_bitmap >= 0)
+		argv_array_pushf(&cmd_args, "--%swrite-bitmap-index",
+				 write_bitmap ? "" : "no-");
 
 	if (pack_everything & ALL_INTO_ONE) {
 		get_non_kept_pack_filenames(&existing_packs);
-- 
1.8.5.rc0.443.g2df7f3f

  parent reply	other threads:[~2013-11-14 12:46 UTC|newest]

Thread overview: 55+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-11-14 12:41 [PATCH v3 0/21] pack bitmaps Jeff King
2013-11-14 12:42 ` [PATCH v3 01/21] sha1write: make buffer const-correct Jeff King
2013-11-14 12:42 ` [PATCH v3 02/21] revindex: Export new APIs Jeff King
2013-11-14 12:42 ` [PATCH v3 03/21] pack-objects: Refactor the packing list Jeff King
2013-11-14 12:43 ` [PATCH v3 04/21] pack-objects: factor out name_hash Jeff King
2013-11-14 12:43 ` [PATCH v3 05/21] revision: allow setting custom limiter function Jeff King
2013-11-14 12:43 ` [PATCH v3 06/21] sha1_file: export `git_open_noatime` Jeff King
2013-11-14 12:43 ` [PATCH v3 07/21] compat: add endianness helpers Jeff King
2013-11-14 12:43 ` [PATCH v3 08/21] ewah: compressed bitmap implementation Jeff King
2013-11-14 12:44 ` [PATCH v3 09/21] documentation: add documentation for the bitmap format Jeff King
2013-11-14 12:44 ` [PATCH v3 10/21] pack-bitmap: add support for bitmap indexes Jeff King
2013-11-24 21:36   ` Thomas Rast
2013-11-25 15:04     ` [PATCH] Document khash Thomas Rast
2013-11-28 10:35       ` Jeff King
2013-11-27  9:08     ` [PATCH v3 10/21] pack-bitmap: add support for bitmap indexes Karsten Blees
2013-11-28 10:38       ` Jeff King
2013-12-03 14:40         ` Karsten Blees
2013-12-03 18:21           ` Jeff King
2013-12-07 20:52             ` Karsten Blees
2013-11-29 21:21   ` Thomas Rast
2013-12-02 16:12     ` Jeff King
2013-12-02 20:36       ` Junio C Hamano
2013-12-02 20:47         ` Jeff King
2013-12-02 21:43           ` Junio C Hamano
2013-11-14 12:45 ` [PATCH v3 11/21] pack-objects: use bitmaps when packing objects Jeff King
2013-12-07 15:47   ` Thomas Rast
2013-12-21 13:15     ` Jeff King
2013-11-14 12:45 ` [PATCH v3 12/21] rev-list: add bitmap mode to speed up object lists Jeff King
2013-12-07 16:05   ` Thomas Rast
2013-11-14 12:45 ` [PATCH v3 13/21] pack-objects: implement bitmap writing Jeff King
2013-12-07 16:32   ` Thomas Rast
2013-12-21 13:17     ` Jeff King
2013-11-14 12:45 ` [PATCH v3 14/21] repack: stop using magic number for ARRAY_SIZE(exts) Jeff King
2013-12-07 16:34   ` Thomas Rast
2013-11-14 12:46 ` [PATCH v3 15/21] repack: turn exts array into array-of-struct Jeff King
2013-12-07 16:34   ` Thomas Rast
2013-11-14 12:46 ` [PATCH v3 16/21] repack: handle optional files created by pack-objects Jeff King
2013-12-07 16:35   ` Thomas Rast
2013-11-14 12:46 ` Jeff King [this message]
2013-12-07 16:37   ` [PATCH v3 17/21] repack: consider bitmaps when performing repacks Thomas Rast
2013-11-14 12:46 ` [PATCH v3 18/21] count-objects: recognize .bitmap in garbage-checking Jeff King
2013-12-07 16:38   ` Thomas Rast
2013-11-14 12:46 ` [PATCH v3 19/21] t: add basic bitmap functionality tests Jeff King
2013-12-07 16:43   ` Thomas Rast
2013-12-21 13:22     ` Jeff King
2013-11-14 12:48 ` [PATCH v3 20/21] t/perf: add tests for pack bitmaps Jeff King
2013-12-07 16:51   ` Thomas Rast
2013-12-21 13:40     ` Jeff King
2013-11-14 12:48 ` [PATCH v3 21/21] pack-bitmap: implement optional name_hash cache Jeff King
2013-12-07 16:59   ` Thomas Rast
2013-11-14 19:19 ` [PATCH v3 0/21] pack bitmaps Ramsay Jones
2013-11-14 21:33   ` Jeff King
2013-11-14 23:09     ` Ramsay Jones
2013-11-18 21:16       ` Ramsay Jones
2013-11-16 10:28     ` Thomas Rast

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20131114124610.GQ10757@sigill.intra.peff.net \
    --to=peff@peff.net \
    --cc=git@vger.kernel.org \
    --cc=vicent@github.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).