[BUG] git-svn returning "Incomplete data: Delta source ended unexpectedly"

[BUG] git-svn returning "Incomplete data: Delta source ended unexpectedly"

[Ævar Arnfjörð Bjarmason]

I run a SVN -> Git mirror which fetches & rebases with git-svn and
pushes the result to GitHub:

    http://github.com/avar/openstreetmap-mirror/blob/master/josm-mirror.sh

Something about what that script is doing corrupted the git-svn
repository, likely due to a git-svn bug. Now when I do git svn fetch
on 1.7.1:

    $ git svn fetch
    Incomplete data: Delta source ended unexpectedly at
/usr/lib/git-core/git-svn line 5061

I re-cloned the repository and started running my mirror again from
that. But it would be nice if we could track this bug down.

Here's a copy of the repository:

    http://v.nix.is/~avar/josm.old.tar.gz

Unpacking it and running git svn fetch will yield the same error on
another box I have with 1.7.1.

Re: [BUG] git-svn returning "Incomplete data: Delta source ended unexpectedly"

[Eric Wong]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> I run a SVN -> Git mirror which fetches & rebases with git-svn and
> pushes the result to GitHub:
> 
>     http://github.com/avar/openstreetmap-mirror/blob/master/josm-mirror.sh
> 
> Something about what that script is doing corrupted the git-svn
> repository, likely due to a git-svn bug. Now when I do git svn fetch
> on 1.7.1:
> 
>     $ git svn fetch
>     Incomplete data: Delta source ended unexpectedly at
> /usr/lib/git-core/git-svn line 5061

The geticons.pl file on the git side doesn't match what SVN expects
when doing a delta.

> I re-cloned the repository and started running my mirror again from
> that. But it would be nice if we could track this bug down.
> 
> Here's a copy of the repository:
> 
>     http://v.nix.is/~avar/josm.old.tar.gz
> 
> Unpacking it and running git svn fetch will yield the same error on
> another box I have with 1.7.1.

Something went wrong with your mirror script (the way you're doing
externals is probably screwing it up, badly).

Compare the "git svn log" output of your in-progress repository
with the "svn log http://josm.openstreetmap.de/svn/trunk"

The revision numbers/commit messages don't sync up at all.

-- 
Eric Wong

Re: [BUG] git-svn returning "Incomplete data: Delta source ended unexpectedly"

[Ævar Arnfjörð Bjarmason]

On Wed, Jul 14, 2010 at 21:09, Eric Wong <normalperson@yhbt.net> wrote:

> Something went wrong with your mirror script (the way you're doing
> externals is probably screwing it up, badly).
>
> Compare the "git svn log" output of your in-progress repository
> with the "svn log http://josm.openstreetmap.de/svn/trunk"
>
> The revision numbers/commit messages don't sync up at all.

Sorry for the false alarm, I didn't check that. Thanks for looking at
it, and sorry for wasting your time.

Re: [BUG] git-svn returning "Incomplete data: Delta source ended unexpectedly"

[Nicolas Noble]

Hello,

  I'm bumping this thread, as our git-svn mirror at work broke down
with the same exact problem. Now, rebuilding it wasn't exactly an
option, and I rather went the "restore backup" way, but my management
feels a bit nervous about git now since this issue hit us, and I
wasn't able to fix it in any way.

  Basically, at our company, we're having subversion to track our
software codebase, but I'm trying hard to make people move to git. My
biggest (quite successful) attempt so far is to have a git-svn
"gateway" machine that people can clone in order to work on their own.
They don't have the right to push back, as it would obviously break
with the way git-svn works, but I take care of rebasing their commits
if they need to, in order to push them back onto the subversion. So we
have two sets of developers: one working with git, and one still with
svn. I'm having a crontab job that's using git-svn fetch every 2
minutes in order to get the gateway in sync at all times; the git-svn
rebase part doesn't really apply, as the point is to create local
branches from the subversion snapshots.

  Now, yesterday, that exact issue described below hit us. But unlike
what's described below, both the git and svn history match perfectly.

  Unfortunately, I can't upload the git repository anywhere, but as
far as I could see, there's indeed some blobs that don't match. But
for the life of me, I have absolutely no idea how to fix this without
going the backup way. I've tried to roll-back the remote branch to the
commit before, by hand-editing .git/refs/remotes/trunk,
.git/packed-refs, .git/info/refs and .git/svn/.metadata, running git
gc --prune in order to make sure I get rid of the corrupted blobs, and
re-running git svn fetch, but that didn't help. Probably because the
corruption happened even before, but then, I have no idea how to find
and fix this. I tried going back several days before, but no luck. And
the worst part is that I tried to skip over that commit, just to see,
using git-svn fetch --revision=xxxx and pointing at some revisions
after that, but ALL of the revisions I tried did the same thing, which
would tend to indicate that... everything is corrupted ? There's
something fundamentally wrong here, where I'm actually trusting
git-svn fetch to do the job properly and work at all time, yet it's
not.

  Again, I can still restore backups, but my management would feel
much better if this issue was fixable instead. And the bad part is
that by backup, I mean I was running another git-svn mirror in
parallel on another machine, but for some reason, they diverged in
history. There's one commit message long time ago that didn't get
fetch from the subversion repository the same way, and from it, the
two history diverged. The delta of that commit itself is fine, but the
commit message got truncated. Which is of course going to have me
breaking any branches people created, and I'll have a very fun time
rebasing everything now :-)

  I don't know exactly what kind of information I could provide in
order to troubleshoot, without disclosing our sourcecode, but if you'd
like to investigate this, I'd be happy to help and provide as much as
I can, as long as you describe what you'd need me to do.

  Thanks!

  -- Nicolas Noble

On Wed, Jul 14, 2010 at 3:16 PM, Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> On Wed, Jul 14, 2010 at 21:09, Eric Wong <normalperson@yhbt.net> wrote:
>
>> Something went wrong with your mirror script (the way you're doing
>> externals is probably screwing it up, badly).
>>
>> Compare the "git svn log" output of your in-progress repository
>> with the "svn log http://josm.openstreetmap.de/svn/trunk"
>>
>> The revision numbers/commit messages don't sync up at all.
>
> Sorry for the false alarm, I didn't check that. Thanks for looking at
> it, and sorry for wasting your time.
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>