All of lore.kernel.org
 help / color / mirror / Atom feed
From: Junio C Hamano <gitster@pobox.com>
To: Doug Kelly <dougk.ff7@gmail.com>
Cc: git@vger.kernel.org, peff@peff.net
Subject: Re: [PATCH 2/2] gc: Remove garbage .idx files from pack dir
Date: Mon, 17 Aug 2015 09:35:11 -0700	[thread overview]
Message-ID: <xmqq7fotg9fk.fsf@gitster.dls.corp.google.com> (raw)
In-Reply-To: <1439488973-11522-2-git-send-email-dougk.ff7@gmail.com> (Doug Kelly's message of "Thu, 13 Aug 2015 13:02:53 -0500")

Doug Kelly <dougk.ff7@gmail.com> writes:

> Add a custom report_garbage handler to collect and remove garbage
> .idx files from the pack directory.

You need to explain "why" here.  Why do we want to remove them?  And
the definition of what is "garbage" depends on that exact reason why
we want to remove them.

You and I may remember the discussion we had that started with your
initial problem description right now.  IIRC, it had to do with
something with the performance when there are many lone .idx files
without corresponding .pack file in the repository, or something?

But those who will be reading "git log" output later will not know,
and we would forget, too.

As discussed elsewhere, the removal needs to be protected with grace
period, probably controlled via prune_expire.

Perhaps along this line on top of your patch you can squash but this
is not even compile tested yet, so take it with a grain of salt.

Thanks.

 builtin/gc.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index 4a459f3..a652773 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -49,8 +49,21 @@ static char *pidfile;
 static void clean_pack_garbage(void)
 {
 	int i;
-	for (i = 0; i < pack_garbage.nr; i++)
+	unsigned long expire = approxidate(prune_expire);
+
+	for (i = 0; i < pack_garbage.nr; i++) {
+		const char *path = pack_garbage.items[i].string;
+		struct stat st;
+
+		if (lstat(path, &st)) {
+			error("could not stat '%s'", path);
+			continue; /* don't risk */
+		}
+		if (st.st_mtime > expire)
+			continue; /* too young */
+
 		unlink_or_warn(pack_garbage.items[i].string);
+	}
 	string_list_clear(&pack_garbage, 0);
 }
 

  reply	other threads:[~2015-08-17 16:35 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-07-21 18:41 Question: .idx without .pack causes performance issues? Doug Kelly
2015-07-21 18:57 ` Junio C Hamano
2015-07-21 19:15   ` Junio C Hamano
2015-07-21 20:48     ` Junio C Hamano
2015-07-21 21:37       ` Doug Kelly
2015-08-03 22:17         ` Doug Kelly
2015-08-04  1:27           ` Junio C Hamano
2015-08-07 21:36             ` Doug Kelly
2015-08-07 22:27               ` Junio C Hamano
2015-08-13 18:02                 ` [PATCH 1/2] prepare_packed_git(): refactor garbage reporting in pack directory Doug Kelly
2015-08-13 18:02                   ` [PATCH 2/2] gc: Remove garbage .idx files from pack dir Doug Kelly
2015-08-17 16:35                     ` Junio C Hamano [this message]
2015-08-17 20:30                     ` Junio C Hamano
2015-08-13 18:46                   ` [PATCH 1/2] prepare_packed_git(): refactor garbage reporting in pack directory Eric Sunshine
2015-08-17 16:53                     ` Junio C Hamano
2015-10-28 17:48                       ` Junio C Hamano
2015-10-28 22:43                         ` Doug Kelly
2015-11-04  3:05                           ` [PATCH 1/3] " Doug Kelly
2015-11-04  3:05                             ` [PATCH 2/3] t5304: Add test for cleaning pack garbage Doug Kelly
2015-11-04  3:05                             ` [PATCH 3/3] gc: Remove garbage .idx files from pack dir Doug Kelly
2015-11-04  3:12                           ` [PATCH 1/2] prepare_packed_git(): refactor garbage reporting in pack directory Doug Kelly
2015-11-04 19:35                             ` Junio C Hamano
2015-11-04 19:56                               ` Doug Kelly
2015-11-04 20:02                                 ` Jeff King
2015-11-04 20:08                                   ` Doug Kelly
2015-11-04 20:15                                     ` Jeff King
2015-12-30  7:37                                     ` Jeff King
2016-01-13 17:14                                       ` Doug Kelly
2016-01-13 20:08                                         ` Junio C Hamano
2016-01-13 20:19                                           ` Doug Kelly
2016-01-13 20:23                                             ` Jeff King
2015-11-04 19:56                               ` Jeff King
     [not found]     ` <CABYiQpn7r2Vcf=S5RaWHBN85eBYGPV_e02+BY=4L98qfUzDT1Q@mail.gmail.com>
2015-11-11 14:58       ` Fwd: Question: .idx without .pack causes performance issues? Thomas Berg
2015-07-21 19:49   ` Doug Kelly

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=xmqq7fotg9fk.fsf@gitster.dls.corp.google.com \
    --to=gitster@pobox.com \
    --cc=dougk.ff7@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.