From: Jeff King <peff@peff.net>
To: git@vger.kernel.org
Cc: Ramkumar Ramachandra <artagnon@gmail.com>,
Duy Nguyen <pclouds@gmail.com>, Brandon Casey <drafnel@gmail.com>,
Junio C Hamano <gitster@pobox.com>
Subject: [PATCH 04/10] cat-file: teach --batch to stream blob objects
Date: Wed, 10 Jul 2013 07:38:24 -0400 [thread overview]
Message-ID: <20130710113824.GD21963@sigill.intra.peff.net> (raw)
In-Reply-To: <20130710113447.GA20113@sigill.intra.peff.net>
The regular "git cat-file -p" and "git cat-file blob" code
paths already learned to stream large blobs. Let's do the
same here.
Note that this means we look up the type and size before
making a decision of whether to load the object into memory
or stream (just like the "-p" code path does). That can lead
to extra work, but it should be dwarfed by the cost of
actually accessing the object itself. In my measurements,
there was a 1-2% slowdown when using "--batch" on a large
number of objects.
Signed-off-by: Jeff King <peff@peff.net>
---
builtin/cat-file.c | 41 ++++++++++++++++++++++++++++-------------
1 file changed, 28 insertions(+), 13 deletions(-)
diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index 045cee7..70dd8c8 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -117,12 +117,36 @@ static int batch_one_object(const char *obj_name, int print_contents)
return 0;
}
+static void print_object_or_die(int fd, const unsigned char *sha1,
+ enum object_type type, unsigned long size)
+{
+ if (type == OBJ_BLOB) {
+ if (stream_blob_to_fd(fd, sha1, NULL, 0) < 0)
+ die("unable to stream %s to stdout", sha1_to_hex(sha1));
+ }
+ else {
+ enum object_type rtype;
+ unsigned long rsize;
+ void *contents;
+
+ contents = read_sha1_file(sha1, &rtype, &rsize);
+ if (!contents)
+ die("object %s disappeared", sha1_to_hex(sha1));
+ if (rtype != type)
+ die("object %s changed type!?", sha1_to_hex(sha1));
+ if (rsize != size)
+ die("object %s change size!?", sha1_to_hex(sha1));
+
+ write_or_die(fd, contents, size);
+ free(contents);
+ }
+}
+
static int batch_one_object(const char *obj_name, int print_contents)
{
unsigned char sha1[20];
enum object_type type = 0;
unsigned long size;
- void *contents = NULL;
if (!obj_name)
return 1;
@@ -133,16 +157,10 @@ static int batch_one_object(const char *obj_name, int print_contents)
return 0;
}
- if (print_contents == BATCH)
- contents = read_sha1_file(sha1, &type, &size);
- else
- type = sha1_object_info(sha1, &size);
-
+ type = sha1_object_info(sha1, &size);
if (type <= 0) {
printf("%s missing\n", obj_name);
fflush(stdout);
- if (print_contents == BATCH)
- free(contents);
return 0;
}
@@ -150,12 +168,9 @@ static int batch_one_object(const char *obj_name, int print_contents)
fflush(stdout);
if (print_contents == BATCH) {
- write_or_die(1, contents, size);
- printf("\n");
- fflush(stdout);
- free(contents);
+ print_object_or_die(1, sha1, type, size);
+ write_or_die(1, "\n", 1);
}
-
return 0;
}
--
1.8.3.rc3.24.gec82cb9
next prev parent reply other threads:[~2013-07-10 11:38 UTC|newest]
Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-07-07 10:01 [RFC/PATCH 0/4] cat-file --batch-disk-sizes Jeff King
2013-07-07 10:03 ` [PATCH 1/4] zero-initialize object_info structs Jeff King
2013-07-07 17:34 ` Junio C Hamano
2013-07-07 10:04 ` [PATCH 2/4] teach sha1_object_info_extended a "disk_size" query Jeff King
2013-07-07 10:09 ` [PATCH 3/4] cat-file: add --batch-disk-sizes option Jeff King
2013-07-07 17:49 ` Junio C Hamano
2013-07-07 18:19 ` Jeff King
2013-07-08 11:04 ` Duy Nguyen
2013-07-08 12:00 ` Ramkumar Ramachandra
2013-07-08 13:13 ` Duy Nguyen
2013-07-08 13:37 ` Ramkumar Ramachandra
2013-07-09 2:55 ` Duy Nguyen
2013-07-09 10:32 ` Ramkumar Ramachandra
2013-07-10 11:16 ` Jeff King
2013-07-08 16:40 ` Junio C Hamano
2013-07-10 11:04 ` Jeff King
2013-07-11 16:35 ` Junio C Hamano
2013-07-07 21:15 ` brian m. carlson
2013-07-10 10:57 ` Jeff King
2013-07-07 10:14 ` [PATCH 4/4] pack-revindex: radix-sort the revindex Jeff King
2013-07-07 23:52 ` Shawn Pearce
2013-07-08 7:57 ` Jeff King
2013-07-08 15:38 ` Shawn Pearce
2013-07-08 20:50 ` Brandon Casey
2013-07-08 21:35 ` Brandon Casey
2013-07-10 10:57 ` Jeff King
2013-07-10 10:52 ` Jeff King
2013-07-10 11:34 ` [PATCHv2 00/10] cat-file formats/on-disk sizes Jeff King
2013-07-10 11:35 ` [PATCH 01/10] zero-initialize object_info structs Jeff King
2013-07-10 11:35 ` [PATCH 02/10] teach sha1_object_info_extended a "disk_size" query Jeff King
2013-07-10 11:36 ` [PATCH 03/10] t1006: modernize output comparisons Jeff King
2013-07-10 11:38 ` Jeff King [this message]
2013-07-10 11:38 ` [PATCH 05/10] cat-file: refactor --batch option parsing Jeff King
2013-07-10 11:45 ` [PATCH 06/10] cat-file: add --batch-check=<format> Jeff King
2013-07-10 11:57 ` Eric Sunshine
2013-07-10 14:51 ` Ramkumar Ramachandra
2013-07-11 11:24 ` Jeff King
2013-07-10 11:46 ` [PATCH 07/10] cat-file: add %(objectsize:disk) format atom Jeff King
2013-07-10 11:48 ` [PATCH 08/10] cat-file: split --batch input lines on whitespace Jeff King
2013-07-10 15:29 ` Ramkumar Ramachandra
2013-07-11 11:36 ` Jeff King
2013-07-11 17:42 ` Junio C Hamano
2013-07-11 20:45 ` [PATCHv3 " Jeff King
2013-07-10 11:50 ` [PATCH 09/10] pack-revindex: use unsigned to store number of objects Jeff King
2013-07-10 11:55 ` [PATCH 10/10] pack-revindex: radix-sort the revindex Jeff King
2013-07-10 12:00 ` Jeff King
2013-07-10 13:17 ` Ramkumar Ramachandra
2013-07-11 11:03 ` Jeff King
2013-07-10 17:10 ` Brandon Casey
2013-07-11 11:17 ` Jeff King
2013-07-11 12:16 ` [PATCHv3 " Jeff King
2013-07-11 21:12 ` Brandon Casey
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130710113824.GD21963@sigill.intra.peff.net \
--to=peff@peff.net \
--cc=artagnon@gmail.com \
--cc=drafnel@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=pclouds@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).