git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Dmitry Ivankov <divanorama@gmail.com>
To: git@vger.kernel.org
Cc: Jonathan Nieder <jrnieder@gmail.com>,
	"Shawn O. Pearce" <spearce@spearce.org>,
	David Barr <davidbarr@google.com>,
	Dmitry Ivankov <divanorama@gmail.com>
Subject: [PATCH 0/3] fix data corruption in fast-import
Date: Fri, 12 Aug 2011 16:32:47 +0600	[thread overview]
Message-ID: <1313145170-24471-1-git-send-email-divanorama@gmail.com> (raw)

Finally the bug reported first in [1] is solved and has a small testcase.
Preliminary attempts can be found in [2] for curious.
And the actual "3/3: fix" comes from [3].

Brief introduction. While testing huge imports produced by svn-fe I've found
a "failed to unpack delta" error in fast-import, which is actually caused by
"sha1 mismatch" error in a packfile, and this one is caused by a bug of producing
wrong deltas for tree objects in fast-import. 

Looks like that only 'M 040000 sha1_or_mark path' commands could trigger it. 
They were introduced in tags/v1.7.3-rc0~75^2
(30 Jun 2010  334fba65.. Teach fast-import to import subtrees named by tree id)
This series should resolve the bug for any copy/rename/set/delete trees scenario
anyway.

I've tested it on a gcc svn repository import - went fine, trees match the gcc
git mirror on github. One more test is ~700k commits from kde repository - fine
too.

1/3 just extracts a sha1 calculation function for 2/3
2/3 adds a die() for "corrupted" delta data and a testcase that triggers it
3/3 is the fix

[1] http://thread.gmane.org/gmane.comp.version-control.git/176753
[2] http://thread.gmane.org/gmane.comp.version-control.git/178007
[3] http://thread.gmane.org/gmane.comp.version-control.git/176753/focus=178053

Dmitry Ivankov (3):
  fast-import: extract object preparation function
  fast-import: add a check for tree delta base sha1
  fast-import: prevent producing bad delta

 fast-import.c          |   85 +++++++++++++++++++++++++++++++++++++++---------
 t/t9300-fast-import.sh |   38 +++++++++++++++++++++
 2 files changed, 107 insertions(+), 16 deletions(-)

-- 
1.7.3.4

             reply	other threads:[~2011-08-12 10:32 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-08-12 10:32 Dmitry Ivankov [this message]
2011-08-12 10:32 ` [PATCH 1/3] fast-import: extract object preparation function Dmitry Ivankov
2011-08-12 10:32 ` [PATCH 2/3] fast-import: add a check for tree delta base sha1 Dmitry Ivankov
2011-08-13 21:02   ` Jonathan Nieder
2011-08-12 10:32 ` [PATCH 3/3] fast-import: prevent producing bad delta Dmitry Ivankov
2011-08-14 18:32 ` [PATCH v2 0/2] fix data corruption in fast-import Dmitry Ivankov
2011-08-14 18:32 ` [PATCH v2 1/2] fast-import: add a test for tree delta base corruption Dmitry Ivankov
2011-08-14 18:32 ` [PATCH v2 2/2] fast-import: prevent producing bad delta Dmitry Ivankov
2011-08-20  1:09   ` [PATCH v3] fast-import: do not write bad delta for replaced subtrees Jonathan Nieder
2011-08-20  9:08     ` Andreas Schwab
2011-08-20 15:43       ` Jonathan Nieder
2011-08-20 17:22         ` [PATCH v4] " Dmitry Ivankov
2011-08-20 17:48           ` Jonathan Nieder
2011-08-20 18:28             ` Dmitry Ivankov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1313145170-24471-1-git-send-email-divanorama@gmail.com \
    --to=divanorama@gmail.com \
    --cc=davidbarr@google.com \
    --cc=git@vger.kernel.org \
    --cc=jrnieder@gmail.com \
    --cc=spearce@spearce.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).