git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Shawn O. Pearce" <spearce@spearce.org>
To: Nicolas Pitre <nico@cam.org>, Stephan Hennig <mailing_list@arcor.de>
Cc: Andreas Ericsson <ae@op5.se>,
	Johannes Schindelin <Johannes.Schindelin@gmx.de>,
	Jakub Narebski <jnareb@gmail.com>,
	Junio C Hamano <gitster@pobox.com>,
	git@vger.kernel.org
Subject: [RFC PATCH] index-pack: Issue a warning if deltaBaseCacheLimit is too small
Date: Thu, 17 Jul 2008 22:02:51 +0000	[thread overview]
Message-ID: <20080717220251.GA3072@spearce.org> (raw)
In-Reply-To: <20080717213550.GA2798@spearce.org>

Its rare that we should exceed deltaBaseCacheLimit while resolving
delta compressed objects.  By default this limit is 16M, and most
chains are under 50 objects in length.  This affords about 327K per
object in the chain, which is quite large by source code standards.

If we have to recreate a prior delta base because we evicted it to
stay within the deltaBaseCacheLimit we can warn the user that their
configured limit is perhaps too low for this repository data set.
If the user keeps seeing the warning they can research it in the
documentation, and consider setting it higher on this repository,
or just globally on their system.

Suggested-by: Stephan Hennig <mailing_list@arcor.de>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 index-pack.c |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)

diff --git a/index-pack.c b/index-pack.c
index ac20a46..97533d6 100644
--- a/index-pack.c
+++ b/index-pack.c
@@ -53,6 +53,7 @@ static struct object_entry *objects;
 static struct delta_entry *deltas;
 static struct base_data *base_cache;
 static size_t base_cache_used;
+static int oom_warning;
 static int nr_objects;
 static int nr_deltas;
 static int nr_resolved_deltas;
@@ -481,6 +482,13 @@ static void *get_base_data(struct base_data *c)
 	if (!c->data) {
 		struct object_entry *obj = c->obj;
 
+		if (!oom_warning && verbose) {
+			if (progress)
+				fputc('\n', stderr);
+			warning("One or more delta chains are larger than deltaBaseCache.");
+			oom_warning = 1;
+		}
+
 		if (obj->type == OBJ_REF_DELTA || obj->type == OBJ_OFS_DELTA) {
 			void *base = get_base_data(c->base);
 			void *raw = get_data_from_pack(obj);
-- 
1.5.6.3.569.ga9185

  reply	other threads:[~2008-07-17 22:03 UTC|newest]

Thread overview: 51+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-07-10 14:40 git pull is slow Stephan Hennig
2008-07-10 15:13 ` Martin Langhoff
2008-07-10 15:28 ` Petr Baudis
2008-07-10 15:30 ` Johannes Sixt
2008-07-10 15:45   ` Stephan Hennig
2008-07-10 15:50     ` Petr Baudis
2008-07-10 17:44       ` Stephan Hennig
2008-07-11 12:25 ` Stephan Hennig
2008-07-11 13:34   ` Andreas Ericsson
2008-07-11 14:04     ` Johannes Schindelin
2008-07-12 12:32       ` Stephan Hennig
2008-07-12 17:05         ` Johannes Schindelin
2008-07-13  1:15           ` Shawn O. Pearce
2008-07-13 13:59             ` Johannes Schindelin
2008-07-13 22:11               ` Shawn O. Pearce
2008-07-14  2:07             ` [PATCH 0/4] Honor core.deltaBaseCacheLimit during index-pack Shawn O. Pearce
2008-07-14  2:27               ` Nicolas Pitre
2008-07-14  3:12                 ` Shawn O. Pearce
2008-07-14 11:44                   ` Johannes Schindelin
2008-07-14 11:54                     ` Jakub Narebski
2008-07-14 12:10                       ` Johannes Schindelin
2008-07-14 12:16                       ` Andreas Ericsson
2008-07-14 12:25                         ` Johannes Schindelin
2008-07-14 12:51                           ` Andreas Ericsson
2008-07-14 12:58                             ` Johannes Schindelin
2008-07-15  2:21                             ` Nicolas Pitre
2008-07-15  2:47                               ` Shawn O. Pearce
2008-07-15  3:06                                 ` Nicolas Pitre
2008-07-17 16:06                                 ` Stephan Hennig
2008-07-17 16:25                                   ` Nicolas Pitre
2008-07-17 21:35                                     ` Shawn O. Pearce
2008-07-17 22:02                                       ` Shawn O. Pearce [this message]
2008-07-17 23:45                                         ` [RFC PATCH] index-pack: Issue a warning if deltaBaseCacheLimit is too small Nicolas Pitre
2008-07-15  4:19                       ` [PATCH 0/4] Honor core.deltaBaseCacheLimit during index-pack Shawn O. Pearce
2008-07-14  2:07             ` [PATCH 1/4] index-pack: Refactor base arguments of resolve_delta into a struct Shawn O. Pearce
2008-07-15  2:40               ` Nicolas Pitre
2008-07-14  2:07             ` [PATCH 2/4] index-pack: Chain the struct base_data on the stack for traversal Shawn O. Pearce
2008-07-15  2:48               ` Nicolas Pitre
2008-07-14  2:07             ` [PATCH 3/4] index-pack: Track the object_entry that creates each base_data Shawn O. Pearce
2008-07-14 10:15               ` Johannes Schindelin
2008-07-15  2:50               ` Nicolas Pitre
2008-07-15  3:20                 ` Shawn O. Pearce
2008-07-15  3:42                   ` Nicolas Pitre
2008-07-14  2:07             ` [PATCH 4/4] index-pack: Honor core.deltaBaseCacheLimit when resolving deltas Shawn O. Pearce
2008-07-15  3:05               ` Nicolas Pitre
2008-07-15  3:18                 ` Shawn O. Pearce
2008-07-15  4:45                   ` [PATCH v2] " Shawn O. Pearce
2008-07-15  5:05                     ` Nicolas Pitre
2008-07-15 18:48                     ` Junio C Hamano
2008-07-13  9:01           ` git pull is slow Stephan Hennig
2008-07-11 12:55 ` Stephan Hennig

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20080717220251.GA3072@spearce.org \
    --to=spearce@spearce.org \
    --cc=Johannes.Schindelin@gmx.de \
    --cc=ae@op5.se \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jnareb@gmail.com \
    --cc=mailing_list@arcor.de \
    --cc=nico@cam.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).