git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Johan Herland <johan@herland.net>
To: Junio C Hamano <gitster@pobox.com>
Cc: Shawn Pearce <spearce@spearce.org>,
	Johan Herland <johan@herland.net>,
	git@vger.kernel.org
Subject: [PATCHv4 07/10] pack-objects: Allow --max-pack-size to be used together with --stdout
Date: Mon, 23 May 2011 02:52:00 +0200	[thread overview]
Message-ID: <1306111923-16859-8-git-send-email-johan@herland.net> (raw)
In-Reply-To: <1306111923-16859-1-git-send-email-johan@herland.net>

Currently we refuse combining --max-pack-size with --stdout since there's
no way to make multiple packs when the pack is written to stdout. However,
we want to be able to limit the maximum size of the pack created by
--stdout (and abort pack-objects if we are unable to meet that limit).

Therefore, when used together with --stdout, we reinterpret --max-pack-size
to indicate the maximum pack size which - if exceeded - will cause
pack-objects to abort with an error message.

Signed-off-by: Johan Herland <johan@herland.net>
---
 Documentation/git-pack-objects.txt |    3 ++
 builtin/pack-objects.c             |    9 ++++---
 t/t5300-pack-object.sh             |   43 ++++++++++++++++++++++++++++++++++++
 3 files changed, 51 insertions(+), 4 deletions(-)

diff --git a/Documentation/git-pack-objects.txt b/Documentation/git-pack-objects.txt
index e43904e..24cf975 100644
--- a/Documentation/git-pack-objects.txt
+++ b/Documentation/git-pack-objects.txt
@@ -112,6 +112,9 @@ base-name::
 	If specified,  multiple packfiles may be created.
 	The default is unlimited, unless the config variable
 	`pack.packSizeLimit` is set.
++
+When used together with --stdout, the command will fail with an error
+message if the pack output exceeds the given limit.
 
 --max-commit-count=<n>::
 	This option is only useful together with --stdout.
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index f0fc187..e226053 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -229,7 +229,7 @@ static unsigned long write_object(struct sha1file *f,
 
 	if (!entry->delta)
 		usable_delta = 0;	/* no delta */
-	else if (!pack_size_limit)
+	else if (!pack_size_limit || pack_to_stdout)
 	       usable_delta = 1;	/* unlimited packfile */
 	else if (entry->delta->idx.offset == (off_t)-1)
 		usable_delta = 0;	/* base was written to another pack */
@@ -478,6 +478,9 @@ static void write_pack_file(void)
 		 * If so, rewrite it like in fast-import
 		 */
 		if (pack_to_stdout) {
+			if (nr_written != nr_remaining)
+				die("unable to make pack within the pack size"
+				    " limit (%lu bytes)", pack_size_limit);
 			sha1close(f, sha1, CSUM_CLOSE);
 		} else if (nr_written == nr_remaining) {
 			sha1close(f, sha1, CSUM_FSYNC);
@@ -2327,9 +2330,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 
 	if (!pack_to_stdout && !pack_size_limit)
 		pack_size_limit = pack_size_limit_cfg;
-	if (pack_to_stdout && pack_size_limit)
-		die("--max-pack-size cannot be used to build a pack for transfer.");
-	if (pack_size_limit && pack_size_limit < 1024*1024) {
+	if (!pack_to_stdout && pack_size_limit && pack_size_limit < 1024*1024) {
 		warning("minimum pack size limit is 1 MiB");
 		pack_size_limit = 1024*1024;
 	}
diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh
index 80df631..46c1214 100755
--- a/t/t5300-pack-object.sh
+++ b/t/t5300-pack-object.sh
@@ -396,6 +396,49 @@ test_expect_success 'verify resulting packs' '
 	git verify-pack test-11-*.pack
 '
 
+test_expect_success '--stdout ignores pack.packSizeLimit' '
+	git pack-objects --stdout <obj-list >test-12.pack &&
+	git index-pack --strict test-12.pack
+'
+
+test_expect_success 'verify resulting pack' '
+	git verify-pack test-12.pack
+'
+
+test_expect_success 'honor --max-pack-size' '
+	git config --unset pack.packSizeLimit &&
+	packname_13=$(git pack-objects --max-pack-size=3m test-13 <obj-list) &&
+	test 2 = $(ls test-13-*.pack | wc -l)
+'
+
+test_expect_success 'verify resulting packs' '
+	git verify-pack test-13-*.pack
+'
+
+test_expect_success 'tolerate --max-pack-size smaller than biggest object' '
+	packname_14=$(git pack-objects --max-pack-size=1 test-14 <obj-list) &&
+	test 5 = $(ls test-14-*.pack | wc -l)
+'
+
+test_expect_success 'verify resulting packs' '
+	git verify-pack test-14-*.pack
+'
+
+test_expect_success '--stdout works with large enough --max-pack-size' '
+	git pack-objects --stdout --max-pack-size=10m <obj-list >test-15.pack &&
+	git index-pack --strict test-15.pack
+'
+
+test_expect_success 'verify resulting pack' '
+	git verify-pack test-15.pack
+'
+
+test_expect_success '--stdout fails when pack exceeds --max-pack-size' '
+	test_must_fail git pack-objects --stdout --max-pack-size=1 <obj-list >test-16.pack 2>errs &&
+	test_must_fail git index-pack --strict test-16.pack &&
+	grep -q "pack size limit" errs
+'
+
 test_expect_success 'make a few more commits' '
 	git reset --hard $commit &&
 	echo "change" > file &&
-- 
1.7.5.rc1.3.g4d7b

  parent reply	other threads:[~2011-05-23  0:52 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-05-23  0:51 [PATCHv4 00/10] Push limits Johan Herland
2011-05-23  0:51 ` [PATCHv4 01/10] Update technical docs to reflect side-band-64k capability in receive-pack Johan Herland
2011-05-23  0:51 ` [PATCHv4 02/10] send-pack: Attempt to retrieve remote status even if pack-objects fails Johan Herland
2011-05-23 20:06   ` Junio C Hamano
2011-05-23 22:58     ` Johan Herland
2011-05-23  0:51 ` [PATCHv4 03/10] Tighten rules for matching server capabilities in server_supports() Johan Herland
2011-05-23  0:51 ` [PATCHv4 04/10] receive-pack: Prepare for addition of the new 'limit-*' family of capabilities Johan Herland
2011-05-23 20:21   ` Junio C Hamano
2011-05-24  0:16     ` Johan Herland
2011-05-23  0:51 ` [PATCHv4 05/10] pack-objects: Teach new option --max-commit-count, limiting #commits in pack Johan Herland
2011-05-23 23:17   ` Junio C Hamano
2011-05-24  0:18     ` Johan Herland
2011-05-23  0:51 ` [PATCHv4 06/10] send-pack/receive-pack: Allow server to refuse pushes with too many commits Johan Herland
2011-05-23 23:39   ` Junio C Hamano
2011-05-24  1:11     ` Johan Herland
2011-05-23  0:52 ` Johan Herland [this message]
2011-05-24  0:09   ` [PATCHv4 07/10] pack-objects: Allow --max-pack-size to be used together with --stdout Junio C Hamano
2011-05-24  1:15     ` Johan Herland
2011-05-23  0:52 ` [PATCHv4 08/10] send-pack/receive-pack: Allow server to refuse pushing too large packs Johan Herland
2011-05-24  0:12   ` Junio C Hamano
2011-05-23  0:52 ` [PATCHv4 09/10] pack-objects: Estimate pack size; abort early if pack size limit is exceeded Johan Herland
2011-05-23 16:11   ` Shawn Pearce
2011-05-23 17:07     ` Johan Herland
2011-05-24  0:18   ` Junio C Hamano
2011-05-24  1:17     ` Johan Herland
2011-05-23  0:52 ` [PATCHv4 10/10] receive-pack: Allow server to refuse pushes with too many objects Johan Herland

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1306111923-16859-8-git-send-email-johan@herland.net \
    --to=johan@herland.net \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=spearce@spearce.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).