git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Johan Herland <johan@herland.net>
To: Junio C Hamano <gitster@pobox.com>
Cc: Shawn Pearce <spearce@spearce.org>,
	Johan Herland <johan@herland.net>,
	git@vger.kernel.org
Subject: [PATCHv3 3/9] pack-objects: Allow --max-pack-size to be used together with --stdout
Date: Sun, 15 May 2011 23:37:14 +0200	[thread overview]
Message-ID: <1305495440-30836-4-git-send-email-johan@herland.net> (raw)
In-Reply-To: <1305495440-30836-1-git-send-email-johan@herland.net>

Currently we refuse combining --max-pack-size with --stdout since there's
no way to make multiple packs when the pack is written to stdout. However,
we want to be able to limit the maximum size of the pack created by
--stdout (and abort pack-objects if we are unable to meet that limit).

Therefore, when used together with --stdout, we reinterpret --max-pack-size
to indicate the maximum pack size which - if exceeded - will cause
pack-objects to abort with an error message.

Signed-off-by: Johan Herland <johan@herland.net>
---
 Documentation/git-pack-objects.txt |    3 ++
 builtin/pack-objects.c             |    9 ++++---
 t/t5300-pack-object.sh             |   43 ++++++++++++++++++++++++++++++++++++
 3 files changed, 51 insertions(+), 4 deletions(-)

diff --git a/Documentation/git-pack-objects.txt b/Documentation/git-pack-objects.txt
index 20c8551..ca97463 100644
--- a/Documentation/git-pack-objects.txt
+++ b/Documentation/git-pack-objects.txt
@@ -112,6 +112,9 @@ base-name::
 	If specified,  multiple packfiles may be created.
 	The default is unlimited, unless the config variable
 	`pack.packSizeLimit` is set.
++
+When used together with --stdout, the command will fail with an error
+message if the pack output exceeds the given limit.
 
 --honor-pack-keep::
 	This flag causes an object already in a local pack that
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index f402a84..69f1c51 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -229,7 +229,7 @@ static unsigned long write_object(struct sha1file *f,
 
 	if (!entry->delta)
 		usable_delta = 0;	/* no delta */
-	else if (!pack_size_limit)
+	else if (!pack_size_limit || pack_to_stdout)
 	       usable_delta = 1;	/* unlimited packfile */
 	else if (entry->delta->idx.offset == (off_t)-1)
 		usable_delta = 0;	/* base was written to another pack */
@@ -478,6 +478,9 @@ static void write_pack_file(void)
 		 * If so, rewrite it like in fast-import
 		 */
 		if (pack_to_stdout) {
+			if (nr_written != nr_remaining)
+				die("unable to make pack within the pack size"
+				    " limit (%lu bytes)", pack_size_limit);
 			sha1close(f, sha1, CSUM_CLOSE);
 		} else if (nr_written == nr_remaining) {
 			sha1close(f, sha1, CSUM_FSYNC);
@@ -2315,9 +2318,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 
 	if (!pack_to_stdout && !pack_size_limit)
 		pack_size_limit = pack_size_limit_cfg;
-	if (pack_to_stdout && pack_size_limit)
-		die("--max-pack-size cannot be used to build a pack for transfer.");
-	if (pack_size_limit && pack_size_limit < 1024*1024) {
+	if (!pack_to_stdout && pack_size_limit && pack_size_limit < 1024*1024) {
 		warning("minimum pack size limit is 1 MiB");
 		pack_size_limit = 1024*1024;
 	}
diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh
index 602806d..00f1bd8 100755
--- a/t/t5300-pack-object.sh
+++ b/t/t5300-pack-object.sh
@@ -396,6 +396,49 @@ test_expect_success 'verify resulting packs' '
 	git verify-pack test-11-*.pack
 '
 
+test_expect_success '--stdout ignores pack.packSizeLimit' '
+	git pack-objects --stdout <obj-list >test-12.pack &&
+	git index-pack --strict test-12.pack
+'
+
+test_expect_success 'verify resulting pack' '
+	git verify-pack test-12.pack
+'
+
+test_expect_success 'honor --max-pack-size' '
+	git config --unset pack.packSizeLimit &&
+	packname_13=$(git pack-objects --max-pack-size=3m test-13 <obj-list) &&
+	test 2 = $(ls test-13-*.pack | wc -l)
+'
+
+test_expect_success 'verify resulting packs' '
+	git verify-pack test-13-*.pack
+'
+
+test_expect_success 'tolerate --max-pack-size smaller than biggest object' '
+	packname_14=$(git pack-objects --max-pack-size=1 test-14 <obj-list) &&
+	test 5 = $(ls test-14-*.pack | wc -l)
+'
+
+test_expect_success 'verify resulting packs' '
+	git verify-pack test-14-*.pack
+'
+
+test_expect_success '--stdout works with large enough --max-pack-size' '
+	git pack-objects --stdout --max-pack-size=10m <obj-list >test-15.pack &&
+	git index-pack --strict test-15.pack
+'
+
+test_expect_success 'verify resulting pack' '
+	git verify-pack test-15.pack
+'
+
+test_expect_success '--stdout fails when pack exceeds --max-pack-size' '
+	test_must_fail git pack-objects --stdout --max-pack-size=1 <obj-list >test-16.pack 2>errs &&
+	test_must_fail git index-pack --strict test-16.pack &&
+	grep -q "pack size limit" errs
+'
+
 #
 # WARNING!
 #
-- 
1.7.5.rc1.3.g4d7b

  parent reply	other threads:[~2011-05-15 21:38 UTC|newest]

Thread overview: 53+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-05-13 16:54 [PATCH 2/2] receive-pack: Add receive.denyObjectLimit to refuse push with too many objects Johan Herland
2011-05-13 17:09 ` Junio C Hamano
2011-05-14  1:43   ` Johan Herland
2011-05-14  2:03     ` [PATCHv2 2/2] receive-pack: Add receive.objectCountLimit " Johan Herland
2011-05-14  2:30       ` Shawn Pearce
2011-05-14 13:17         ` Johan Herland
2011-05-14 22:17           ` Shawn Pearce
2011-05-15 17:42             ` Johan Herland
2011-05-15 21:37               ` [PATCHv3 0/9] Push limits Johan Herland
2011-05-15 21:37                 ` [PATCHv3 1/9] Update technical docs to reflect side-band-64k capability in receive-pack Johan Herland
2011-05-15 21:37                 ` [PATCHv3 2/9] send-pack: Attempt to retrieve remote status even if pack-objects fails Johan Herland
2011-05-16  4:07                   ` Jeff King
2011-05-16  6:13                     ` Jeff King
2011-05-16  6:39                       ` Jeff King
2011-05-16  6:46                         ` [PATCH 1/3] connect: treat generic proxy processes like ssh processes Jeff King
2011-05-16 19:57                           ` Johannes Sixt
2011-05-16 23:12                             ` Junio C Hamano
2011-05-17  5:54                             ` Jeff King
2011-05-17 20:14                               ` Johannes Sixt
2011-05-18  8:57                                 ` Jeff King
2011-05-16  6:52                         ` [PATCH 2/3] connect: let callers know if connection is a socket Jeff King
2011-05-16  6:52                         ` [PATCH 3/3] send-pack: avoid deadlock on git:// push with failed pack-objects Jeff King
2011-05-16 20:02                           ` Johannes Sixt
2011-05-17  5:56                             ` Jeff King
2011-05-18 20:24                               ` [PATCH] Windows: add a wrapper for the shutdown() system call Johannes Sixt
2011-05-15 21:37                 ` Johan Herland [this message]
2011-05-15 22:06                   ` [PATCHv3 3/9] pack-objects: Allow --max-pack-size to be used together with --stdout Shawn Pearce
2011-05-16  1:39                     ` Johan Herland
2011-05-16  6:12                   ` Junio C Hamano
2011-05-16  9:27                     ` Johan Herland
2011-05-15 21:37                 ` [PATCHv3 4/9] pack-objects: Teach new option --max-object-count, similar to --max-pack-size Johan Herland
2011-05-15 22:07                   ` Shawn Pearce
2011-05-15 22:31                     ` Johan Herland
2011-05-15 23:48                       ` Shawn Pearce
2011-05-16  6:25                     ` Junio C Hamano
2011-05-16  9:49                       ` Johan Herland
2011-05-15 21:37                 ` [PATCHv3 5/9] pack-objects: Teach new option --max-commit-count, limiting #commits in pack Johan Herland
2011-05-15 21:37                 ` [PATCHv3 6/9] receive-pack: Prepare for addition of the new 'limit-*' family of capabilities Johan Herland
2011-05-16  6:50                   ` Junio C Hamano
2011-05-16  9:53                     ` Johan Herland
2011-05-16 22:02                     ` Sverre Rabbelier
2011-05-16 22:07                       ` Junio C Hamano
2011-05-16 22:09                         ` Sverre Rabbelier
2011-05-16 22:12                           ` Junio C Hamano
2011-05-16 22:16                             ` Sverre Rabbelier
2011-05-15 21:37                 ` [PATCHv3 7/9] send-pack/receive-pack: Allow server to refuse pushes with too many objects Johan Herland
2011-05-15 21:37                 ` [PATCHv3 8/9] send-pack/receive-pack: Allow server to refuse pushing too large packs Johan Herland
2011-05-15 21:37                 ` [PATCHv3 9/9] send-pack/receive-pack: Allow server to refuse pushes with too many commits Johan Herland
2011-05-15 21:52                 ` [PATCHv3 0/9] Push limits Ævar Arnfjörð Bjarmason
2011-05-14 17:50         ` [PATCHv2 2/2] receive-pack: Add receive.objectCountLimit to refuse push with too many objects Junio C Hamano
2011-05-14 22:27           ` Shawn Pearce
2011-05-13 18:20 ` [PATCH 2/2] receive-pack: Add receive.denyObjectLimit " Johannes Sixt
2011-05-14  1:49   ` Johan Herland

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1305495440-30836-4-git-send-email-johan@herland.net \
    --to=johan@herland.net \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=spearce@spearce.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).