git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jonathan Nieder <jrnieder@gmail.com>
To: git@vger.kernel.org
Cc: Ramkumar Ramachandra <artagnon@gmail.com>,
	Sverre Rabbelier <srabbelier@gmail.com>,
	David Barr <david.barr@cordelta.com>
Subject: [PATCH 2/8] vcs-svn: Internal fast_export_save_blob helper
Date: Sat, 20 Nov 2010 13:25:09 -0600	[thread overview]
Message-ID: <20101120192509.GC17823@burratino> (raw)
In-Reply-To: <20101120192153.GA17823@burratino>

Introduce fast_export_save_blob, meant to be used after printing

	cat-blob :n

to the fast-import stream.  It reads a response from fd 3 in cat-file
--batch format.  To avoid deadlock, it uses file descriptor-level
calls (no stdio) and reads only one character at a time (though the
latter restriction could and should be relaxed somehow --- O_NONBLOCK,
perhaps).

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
The effect of this one is almost completely undone by a later patch. :)

Probably it would be better to use stdio with O_NONBLOCK, after all
(David, sorry I forgot about our conversations on this before).  This
series uses file descriptors to be conservative, because I do not have
much of a desire to test for proper O_NONBLOCK support on the relevant
platforms yet.

 vcs-svn/fast_export.c |   87 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 87 insertions(+), 0 deletions(-)

diff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c
index 6cfa256..3feef66 100644
--- a/vcs-svn/fast_export.c
+++ b/vcs-svn/fast_export.c
@@ -8,8 +8,10 @@
 #include "line_buffer.h"
 #include "repo_tree.h"
 #include "string_pool.h"
+#include "strbuf.h"
 
 #define MAX_GITSVN_LINE_LEN 4096
+#define REPORT_FILENO 3
 
 static uint32_t first_commit_done;
 
@@ -63,6 +65,91 @@ void fast_export_commit(uint32_t revision, uint32_t author, char *log,
 	printf("progress Imported commit %"PRIu32".\n\n", revision);
 }
 
+static int ends_with(const char *s, size_t len, const char *suffix)
+{
+	const size_t suffixlen = strlen(suffix);
+	if (len < suffixlen)
+		return 0;
+	return !memcmp(s + len - suffixlen, suffix, suffixlen);
+}
+
+static int parse_cat_response_line(const char *header, size_t *len)
+{
+	size_t headerlen = strlen(header);
+	const char *type;
+	const char *end;
+
+	if (ends_with(header, headerlen, " missing"))
+		return error("cat-blob reports missing blob: %s", header);
+	type = memmem(header, headerlen, " blob ", strlen(" blob "));
+	if (!type)
+		return error("cat-blob header has wrong object type: %s", header);
+	*len = strtoumax(type + strlen(" blob "), (char **) &end, 10);
+	if (end == type + strlen(" blob "))
+		return error("cat-blob header does not contain length: %s", header);
+	if (*end)
+		return error("cat-blob header contains garbage after length: %s", header);
+	return 0;
+}
+
+static struct strbuf response_line = STRBUF_INIT;
+static const char *get_response_line(void)
+{
+	/*
+	 * NEEDSWORK: Does not actually need to read one byte at a time.
+	 * Some platforms have O_NONBLOCK.  On others we could read 8 chars
+	 * at a time until a potential appearance of " blob ".
+	 */
+	strbuf_reset(&response_line);
+	for (;;) {
+		char buf[1];
+		if (xread(REPORT_FILENO, buf, 1) < 0) {
+			error("cannot read cat-blob result: %s", strerror(errno));
+			return NULL;
+		}
+		if (*buf == '\n')
+			return response_line.buf;
+		strbuf_addch(&response_line, *buf);
+	}
+}
+
+static int copy_bytes(FILE *out, size_t len)
+{
+	char buf[4096];
+	ssize_t nread;
+	for (; len; len -= nread) {
+		nread = xread(REPORT_FILENO, buf,
+					len < sizeof(buf) ? len : sizeof(buf));
+		if (nread < 0)
+			return error("cannot copy cat-blob result: %s",
+					strerror(errno));
+		if (!nread)
+			return error("0-length read...");
+		if (fwrite(buf, 1, nread, out) != nread)
+			return error("cannot write cat-blob results: %s",
+					strerror(errno));
+	}
+}
+
+static int fast_export_save_blob(FILE *out)
+{
+	size_t len = len;
+	const char *header, *tail;
+
+	header = get_response_line();
+	if (!header || parse_cat_response_line(header, &len))
+		return -1;
+	copy_bytes(out, len);
+
+	/* Discard trailing newline. */
+	tail = get_response_line();
+	if (!tail)
+		return -1;
+	if (*tail)
+		return error("line following cat-blob response contains garbage: %s", tail);
+	return 0;
+}
+
 void fast_export_blob(uint32_t mode, uint32_t mark, uint32_t len)
 {
 	if (mode == REPO_MODE_LNK) {
-- 
1.7.2.3

  parent reply	other threads:[~2010-11-20 19:25 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-11-18  5:00 [PATCH 0/2] svn-fe: recognize v3 dumps Jonathan Nieder
2010-11-18  5:02 ` [PATCH 1/2] vcs-svn: Error out for " Jonathan Nieder
2010-11-18  5:03 ` [PATCH 2/2] vcs-svn: Allow simple v3 dumps (no deltas yet) Jonathan Nieder
2010-11-20  0:45 ` [RFC/PATCH 0/15] svn-fe: support for property deltas (but not text " Jonathan Nieder
2010-11-20  0:46   ` [PATCH 01/15] vcs-svn: Check for errors from open() Jonathan Nieder
2010-11-20  0:46   ` [PATCH 02/15] vcs-svn: Eliminate node_ctx.srcRev global Jonathan Nieder
2010-11-20  0:46   ` [PATCH 03/15] vcs-svn: Eliminate node_ctx.mark global Jonathan Nieder
2010-11-20  0:47   ` [PATCH 04/15] vcs-svn: Unclutter handle_node by introducing have_props var Jonathan Nieder
2010-11-20  0:48   ` [PATCH 05/15] vcs-svn: Use mark to indicate nodes with included text Jonathan Nieder
2010-11-20  0:49   ` [PATCH 06/15] vcs-svn: handle_node: Handle deletion case early Jonathan Nieder
2010-11-20  0:49   ` [PATCH 07/15] vcs-svn: Replace = Delete + Add Jonathan Nieder
2010-11-20  0:51   ` [PATCH 08/15] vcs-svn: Combine repo_replace and repo_modify functions Jonathan Nieder
2010-11-20  0:52   ` [PATCH 09/15] vcs-svn: Delay read of per-path properties Jonathan Nieder
2010-11-20  0:52   ` [PATCH 10/15] vcs-svn: Reject path nodes without Node-action Jonathan Nieder
2010-11-20 14:53     ` Jonathan Nieder
2010-11-20  0:53   ` [PATCH 11/15] vcs-svn: More dump format sanity checks Jonathan Nieder
2010-11-30 19:48     ` Jonathan Nieder
     [not found]       ` <20101205091605.GA4332@burratino>
2010-12-05  9:32         ` [PATCH 2/2] vcs-svn: fix intermittent repo_tree corruption Jonathan Nieder
2010-12-05  9:33       ` [PATCH jn/svn-fe-maint 0/2] " Jonathan Nieder
2010-12-05  9:35         ` [PATCH 1/2] treap: make treap_insert return inserted node Jonathan Nieder
2010-12-06 22:19     ` [PATCH jn/svn-fe] vcs-svn: Allow change nodes for root of tree (/) Jonathan Nieder
2010-12-06 23:12       ` Jonathan Nieder
2010-11-20  0:53   ` [PATCH 12/15] vcs-svn: Make source easier to read on small screens Jonathan Nieder
2010-11-20  0:54   ` [PATCH 13/15] vcs-svn: Split off function for handling of individual properties Jonathan Nieder
2010-11-20  0:54   ` [PATCH 14/15] vcs-svn: Sharpen parsing of property lines Jonathan Nieder
2010-11-20  0:57   ` [PATCH 15/15] vcs-svn: Implement Prop-delta handling Jonathan Nieder
2010-11-20 19:21   ` [WIP/PATCH 0/8] svn-fe: support for text deltas Jonathan Nieder
2010-11-20 19:22     ` [PATCH 1/8] svn-fe: Prepare for strbuf use Jonathan Nieder
2010-11-20 19:25     ` Jonathan Nieder [this message]
2010-11-20 19:25     ` [PATCH 3/8] vcs-svn: Introduce repo_read_path to check the content at a path Jonathan Nieder
2011-03-06 12:29       ` Jonathan Nieder
2010-11-20 19:26     ` [PATCH 4/8] vcs-svn: Introduce fd_buffer routines Jonathan Nieder
2010-11-20 19:27     ` [PATCH 5/8] vcs-svn: Read delta preimage from file descriptor Jonathan Nieder
2010-11-20 19:28     ` [PATCH 6/8] vcs-svn: Let caller set up sliding window for delta preimage Jonathan Nieder
2010-11-20 19:31       ` Jonathan Nieder
2010-11-20 19:29     ` [PATCH 7/8] vcs-svn: Teach line_buffer about temporary files Jonathan Nieder
2010-11-20 19:29     ` [PATCH 8/8] vcs-svn: Implement text-delta handling Jonathan Nieder
2010-12-04 17:34       ` [PATCH 10/8] vcs-svn: Consume whole preimage when applying deltas Jonathan Nieder
2010-11-20 19:30     ` [PATCH 9/8] svn-fe: Test script for handling of dumps with --deltas Jonathan Nieder
2010-12-04 17:29       ` Jonathan Nieder

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20101120192509.GC17823@burratino \
    --to=jrnieder@gmail.com \
    --cc=artagnon@gmail.com \
    --cc=david.barr@cordelta.com \
    --cc=git@vger.kernel.org \
    --cc=srabbelier@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).