All of lore.kernel.org
 help / color / mirror / Atom feed
From: Dmitry Ivankov <divanorama@gmail.com>
To: git@vger.kernel.org
Cc: Jonathan Nieder <jrnieder@gmail.com>,
	"Shawn O. Pearce" <spearce@spearce.org>,
	David Barr <davidbarr@google.com>,
	Dmitry Ivankov <divanorama@gmail.com>
Subject: [PATCH 0/3] fix data corruption in fast-import
Date: Fri, 12 Aug 2011 16:32:47 +0600	[thread overview]
Message-ID: <1313145170-24471-1-git-send-email-divanorama@gmail.com> (raw)

Finally the bug reported first in [1] is solved and has a small testcase.
Preliminary attempts can be found in [2] for curious.
And the actual "3/3: fix" comes from [3].

Brief introduction. While testing huge imports produced by svn-fe I've found
a "failed to unpack delta" error in fast-import, which is actually caused by
"sha1 mismatch" error in a packfile, and this one is caused by a bug of producing
wrong deltas for tree objects in fast-import. 

Looks like that only 'M 040000 sha1_or_mark path' commands could trigger it. 
They were introduced in tags/v1.7.3-rc0~75^2
(30 Jun 2010  334fba65.. Teach fast-import to import subtrees named by tree id)
This series should resolve the bug for any copy/rename/set/delete trees scenario
anyway.

I've tested it on a gcc svn repository import - went fine, trees match the gcc
git mirror on github. One more test is ~700k commits from kde repository - fine
too.

1/3 just extracts a sha1 calculation function for 2/3
2/3 adds a die() for "corrupted" delta data and a testcase that triggers it
3/3 is the fix

[1] http://thread.gmane.org/gmane.comp.version-control.git/176753
[2] http://thread.gmane.org/gmane.comp.version-control.git/178007
[3] http://thread.gmane.org/gmane.comp.version-control.git/176753/focus=178053

Dmitry Ivankov (3):
  fast-import: extract object preparation function
  fast-import: add a check for tree delta base sha1
  fast-import: prevent producing bad delta

 fast-import.c          |   85 +++++++++++++++++++++++++++++++++++++++---------
 t/t9300-fast-import.sh |   38 +++++++++++++++++++++
 2 files changed, 107 insertions(+), 16 deletions(-)

-- 
1.7.3.4

             reply	other threads:[~2011-08-12 10:32 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-08-12 10:32 Dmitry Ivankov [this message]
2011-08-12 10:32 ` [PATCH 1/3] fast-import: extract object preparation function Dmitry Ivankov
2011-08-12 10:32 ` [PATCH 2/3] fast-import: add a check for tree delta base sha1 Dmitry Ivankov
2011-08-13 21:02   ` Jonathan Nieder
2011-08-12 10:32 ` [PATCH 3/3] fast-import: prevent producing bad delta Dmitry Ivankov
2011-08-14 18:32 ` [PATCH v2 0/2] fix data corruption in fast-import Dmitry Ivankov
2011-08-14 18:32 ` [PATCH v2 1/2] fast-import: add a test for tree delta base corruption Dmitry Ivankov
2011-08-14 18:32 ` [PATCH v2 2/2] fast-import: prevent producing bad delta Dmitry Ivankov
2011-08-20  1:09   ` [PATCH v3] fast-import: do not write bad delta for replaced subtrees Jonathan Nieder
2011-08-20  9:08     ` Andreas Schwab
2011-08-20 15:43       ` Jonathan Nieder
2011-08-20 17:22         ` [PATCH v4] " Dmitry Ivankov
2011-08-20 17:48           ` Jonathan Nieder
2011-08-20 18:28             ` Dmitry Ivankov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1313145170-24471-1-git-send-email-divanorama@gmail.com \
    --to=divanorama@gmail.com \
    --cc=davidbarr@google.com \
    --cc=git@vger.kernel.org \
    --cc=jrnieder@gmail.com \
    --cc=spearce@spearce.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.