From: Jonathan Nieder <jrnieder@gmail.com>
To: git@vger.kernel.org
Cc: Ramkumar Ramachandra <artagnon@gmail.com>,
Sverre Rabbelier <srabbelier@gmail.com>,
David Barr <david.barr@cordelta.com>
Subject: [PATCH 10/8] vcs-svn: Consume whole preimage when applying deltas
Date: Sat, 4 Dec 2010 11:34:49 -0600 [thread overview]
Message-ID: <20101204173449.GB28299@burratino> (raw)
In-Reply-To: <20101120192928.GI17823@burratino>
Date: Sun Nov 21 20:24:12 2010 -0600
Some deltas do not consume the entire preimage, so we have to
consume it explicitly.
Noticed during imports from the ASF repo.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Jonathan Nieder wrote:
> +++ b/vcs-svn/fast_export.c
> @@ -102,33 +115,39 @@ static const char *get_response_line(void)
[...]
> + if (svndiff0_apply(input, len, &preimage, out))
> + die("cannot apply delta");
> + if (old_mark) {
> + /* Discard trailing newline from cat-blob-fd. */
> + const char *tail = get_response_line();
> + if (!tail || *tail)
> + die("missing newline after cat-blob response");
As mentioned before, this error
fatal: missing newline after cat-blob response
and
error: Preimage ends early
were sometimes triggering. Here's a fix for squashing. Sorry for
the nonsensical code.
t/t9010-svn-fe.sh | 108 +++++++++++++++++++++++++++++++++++++++++++++++++
vcs-svn/fast_export.c | 16 ++++---
2 files changed, 117 insertions(+), 7 deletions(-)
diff --git a/t/t9010-svn-fe.sh b/t/t9010-svn-fe.sh
index 6c8b803..1776a38 100755
--- a/t/t9010-svn-fe.sh
+++ b/t/t9010-svn-fe.sh
@@ -678,6 +678,114 @@ test_expect_success 'deltas for typechange' '
test_cmp expect actual
'
+test_expect_success PIPE 'deltas need not consume the whole preimage' '
+ reinit_git &&
+ cat >expect <<-\EOF &&
+ OBJID
+ :120000 100644 OBJID OBJID T postimage
+ OBJID
+ :100644 120000 OBJID OBJID T postimage
+ OBJID
+ :000000 100644 OBJID OBJID A postimage
+ EOF
+ echo "first preimage" >expect.1 &&
+ printf target >expect.2 &&
+ printf lnk >expect.3 &&
+ rm -f backflow &&
+ {
+ printf "SVNQ%b%b%b" "QQ\017\001\017" "\0217" "first preimage\n" |
+ q_to_nul
+ } >delta.1 &&
+ {
+ properties svn:special "*" &&
+ echo PROPS-END
+ } >symlink.props &&
+ {
+ printf "SVNQ%b%b%b" "Q\002\013\004\012" "\0201\001\001\0211" "lnk target" |
+ q_to_nul
+ } >delta.2 &&
+ {
+ printf "SVNQ%b%b" "Q\004\003\004Q" "\001Q\002\002" |
+ q_to_nul
+ } >delta.3 &&
+ {
+ cat <<-\EOF &&
+ SVN-fs-dump-format-version: 3
+
+ Revision-number: 1
+ Prop-content-length: 10
+ Content-length: 10
+
+ PROPS-END
+
+ Node-path: postimage
+ Node-kind: file
+ Node-action: add
+ Text-delta: true
+ Prop-content-length: 10
+ EOF
+ echo Text-content-length: $(wc -c <delta.1) &&
+ echo Content-length: $((10 + $(wc -c <delta.1))) &&
+ echo &&
+ echo PROPS-END &&
+ cat delta.1 &&
+ cat <<-\EOF &&
+
+ Revision-number: 2
+ Prop-content-length: 10
+ Content-length: 10
+
+ PROPS-END
+
+ Node-path: postimage
+ Node-kind: file
+ Node-action: change
+ Text-delta: true
+ EOF
+ echo Prop-content-length: $(wc -c <symlink.props) &&
+ echo Text-content-length: $(wc -c <delta.2) &&
+ echo Content-length: $(($(wc -c <symlink.props) + $(wc -c <delta.2))) &&
+ echo &&
+ cat symlink.props &&
+ cat delta.2 &&
+ cat <<-\EOF &&
+
+ Revision-number: 3
+ Prop-content-length: 10
+ Content-length: 10
+
+ PROPS-END
+
+ Node-path: postimage
+ Node-kind: file
+ Node-action: change
+ Text-delta: true
+ Prop-content-length: 10
+ EOF
+ echo Text-content-length: $(wc -c <delta.3) &&
+ echo Content-length: $((10 + $(wc -c <delta.3))) &&
+ echo &&
+ echo PROPS-END &&
+ cat delta.3 &&
+ echo
+ } >deltapartial.dump &&
+ mkfifo backflow &&
+ test-svn-fe deltapartial.dump 3<backflow |
+ git fast-import --cat-blob-fd=3 3>backflow &&
+ {
+ git rev-list HEAD |
+ git diff-tree --root --stdin |
+ sed "s/$_x40/OBJID/g"
+ } >actual &&
+ test_cmp expect actual &&
+ git show HEAD:postimage >actual.3 &&
+ git show HEAD^:postimage >actual.2 &&
+ git show HEAD^^:postimage >actual.1 &&
+ test_cmp expect.1 actual.1 &&
+ test_cmp expect.2 actual.2 &&
+ test_cmp expect.3 actual.3
+'
+
test_expect_success 't9135/svn.dump' '
svnadmin create simple-svn &&
svnadmin load simple-svn <"$TEST_DIRECTORY/t9135/svn.dump" &&
diff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c
index ceb1fc5..f8a41e7 100644
--- a/vcs-svn/fast_export.c
+++ b/vcs-svn/fast_export.c
@@ -119,6 +119,7 @@ static long apply_delta(uint32_t mark, off_t len, struct line_buffer *input,
uint32_t old_mark, uint32_t old_mode)
{
long ret;
+ off_t preimage_len = 0;
struct view preimage = {REPORT_FILENO, 0, STRBUF_INIT};
FILE *out;
@@ -126,22 +127,23 @@ static long apply_delta(uint32_t mark, off_t len, struct line_buffer *input,
die("cannot open temporary file for blob retrieval");
if (old_mark) {
const char *response;
- off_t dummy;
printf("cat-blob :%"PRIu32"\n", old_mark);
fflush(stdout);
response = get_response_line();
- /* Not necessary, just for robustness */
- if (parse_cat_response_line(response, &dummy))
+ if (parse_cat_response_line(response, &preimage_len))
die("invalid cat-blob response: %s", response);
}
- if (old_mode == REPO_MODE_LNK)
+ if (old_mode == REPO_MODE_LNK) {
strbuf_addstr(&preimage.buf, "link ");
+ preimage_len += strlen("link ");
+ }
if (svndiff0_apply(input, len, &preimage, out))
die("cannot apply delta");
if (old_mark) {
- /* Discard trailing newline from cat-blob-fd. */
- const char *tail = get_response_line();
- if (!tail || *tail)
+ /* Read the remainder of preimage and trailing newline. */
+ if (move_window(&preimage, preimage_len, 1))
+ die("cannot seek to end of input");
+ if (preimage.buf.buf[0] != '\n')
die("missing newline after cat-blob response");
}
ret = buffer_tmpfile_prepare_to_read(&postimage);
--
1.7.2.4
next prev parent reply other threads:[~2010-12-04 17:35 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-11-18 5:00 [PATCH 0/2] svn-fe: recognize v3 dumps Jonathan Nieder
2010-11-18 5:02 ` [PATCH 1/2] vcs-svn: Error out for " Jonathan Nieder
2010-11-18 5:03 ` [PATCH 2/2] vcs-svn: Allow simple v3 dumps (no deltas yet) Jonathan Nieder
2010-11-20 0:45 ` [RFC/PATCH 0/15] svn-fe: support for property deltas (but not text " Jonathan Nieder
2010-11-20 0:46 ` [PATCH 01/15] vcs-svn: Check for errors from open() Jonathan Nieder
2010-11-20 0:46 ` [PATCH 02/15] vcs-svn: Eliminate node_ctx.srcRev global Jonathan Nieder
2010-11-20 0:46 ` [PATCH 03/15] vcs-svn: Eliminate node_ctx.mark global Jonathan Nieder
2010-11-20 0:47 ` [PATCH 04/15] vcs-svn: Unclutter handle_node by introducing have_props var Jonathan Nieder
2010-11-20 0:48 ` [PATCH 05/15] vcs-svn: Use mark to indicate nodes with included text Jonathan Nieder
2010-11-20 0:49 ` [PATCH 06/15] vcs-svn: handle_node: Handle deletion case early Jonathan Nieder
2010-11-20 0:49 ` [PATCH 07/15] vcs-svn: Replace = Delete + Add Jonathan Nieder
2010-11-20 0:51 ` [PATCH 08/15] vcs-svn: Combine repo_replace and repo_modify functions Jonathan Nieder
2010-11-20 0:52 ` [PATCH 09/15] vcs-svn: Delay read of per-path properties Jonathan Nieder
2010-11-20 0:52 ` [PATCH 10/15] vcs-svn: Reject path nodes without Node-action Jonathan Nieder
2010-11-20 14:53 ` Jonathan Nieder
2010-11-20 0:53 ` [PATCH 11/15] vcs-svn: More dump format sanity checks Jonathan Nieder
2010-11-30 19:48 ` Jonathan Nieder
[not found] ` <20101205091605.GA4332@burratino>
2010-12-05 9:32 ` [PATCH 2/2] vcs-svn: fix intermittent repo_tree corruption Jonathan Nieder
2010-12-05 9:33 ` [PATCH jn/svn-fe-maint 0/2] " Jonathan Nieder
2010-12-05 9:35 ` [PATCH 1/2] treap: make treap_insert return inserted node Jonathan Nieder
2010-12-06 22:19 ` [PATCH jn/svn-fe] vcs-svn: Allow change nodes for root of tree (/) Jonathan Nieder
2010-12-06 23:12 ` Jonathan Nieder
2010-11-20 0:53 ` [PATCH 12/15] vcs-svn: Make source easier to read on small screens Jonathan Nieder
2010-11-20 0:54 ` [PATCH 13/15] vcs-svn: Split off function for handling of individual properties Jonathan Nieder
2010-11-20 0:54 ` [PATCH 14/15] vcs-svn: Sharpen parsing of property lines Jonathan Nieder
2010-11-20 0:57 ` [PATCH 15/15] vcs-svn: Implement Prop-delta handling Jonathan Nieder
2010-11-20 19:21 ` [WIP/PATCH 0/8] svn-fe: support for text deltas Jonathan Nieder
2010-11-20 19:22 ` [PATCH 1/8] svn-fe: Prepare for strbuf use Jonathan Nieder
2010-11-20 19:25 ` [PATCH 2/8] vcs-svn: Internal fast_export_save_blob helper Jonathan Nieder
2010-11-20 19:25 ` [PATCH 3/8] vcs-svn: Introduce repo_read_path to check the content at a path Jonathan Nieder
2011-03-06 12:29 ` Jonathan Nieder
2010-11-20 19:26 ` [PATCH 4/8] vcs-svn: Introduce fd_buffer routines Jonathan Nieder
2010-11-20 19:27 ` [PATCH 5/8] vcs-svn: Read delta preimage from file descriptor Jonathan Nieder
2010-11-20 19:28 ` [PATCH 6/8] vcs-svn: Let caller set up sliding window for delta preimage Jonathan Nieder
2010-11-20 19:31 ` Jonathan Nieder
2010-11-20 19:29 ` [PATCH 7/8] vcs-svn: Teach line_buffer about temporary files Jonathan Nieder
2010-11-20 19:29 ` [PATCH 8/8] vcs-svn: Implement text-delta handling Jonathan Nieder
2010-12-04 17:34 ` Jonathan Nieder [this message]
2010-11-20 19:30 ` [PATCH 9/8] svn-fe: Test script for handling of dumps with --deltas Jonathan Nieder
2010-12-04 17:29 ` Jonathan Nieder
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20101204173449.GB28299@burratino \
--to=jrnieder@gmail.com \
--cc=artagnon@gmail.com \
--cc=david.barr@cordelta.com \
--cc=git@vger.kernel.org \
--cc=srabbelier@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.