git-svn goes into infinite loop rebuilding rev_map

git-svn goes into infinite loop rebuilding rev_map

[Daniel Berlin]

Starting last night, for no particularly obvious reason, git-svn fetch
(or git-svn rebase or any command that does naything interesting) on
gcc.gnu.org's repo causes git-svn to do the following:

-bash-3.00$ git-svn fetch
Rebuilding .git/svn/trunk/.rev_map.138bc75d-0d04-0410-961f-82ee72b054a4 ...
Done rebuilding .git/svn/trunk/.rev_map.138bc75d-0d04-0410-961f-82ee72b054a4
Rebuilding .git/svn/trunk/.rev_map.138bc75d-0d04-0410-961f-82ee72b054a4 ...
<repeat forever>


I checked the obvious things. There is no permissions problem, strace
doesn't show the writes failing, etc.
git-fsck shows nothing as being obviously wrong.


This just suddenly started happening with no obvious reason (we fire
off git-svn fetch and rebase on the repo when cvs commits happen).

git-log shows the last processed rev was:

commit e66b310fb9a3ddc7ed5143db3c468711c8a36d08
Author: uros <uros@138bc75d-0d04-0410-961f-82ee72b054a4>
Date:   Thu May 15 14:38:23 2008 +0000

        * config/i386/sse.md (*vec_concatv2sf_sse4_1): Add "m" constraint
        to alternative 4 of operand 2.


    git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@135364
138bc75d-0d04-0410-961f-82ee72b054a4


I'm happy to provide anything i can to get the problem fixed, but not
being a git expert, i have no idea what is up here.

Re: git-svn goes into infinite loop rebuilding rev_map

[Kevin Ballard]

On May 16, 2008, at 1:15 PM, Daniel Berlin wrote:

> Starting last night, for no particularly obvious reason, git-svn fetch
> (or git-svn rebase or any command that does naything interesting) on
> gcc.gnu.org's repo causes git-svn to do the following:
>
> -bash-3.00$ git-svn fetch
> Rebuilding .git/svn/trunk/.rev_map. 
> 138bc75d-0d04-0410-961f-82ee72b054a4 ...
> Done rebuilding .git/svn/trunk/.rev_map. 
> 138bc75d-0d04-0410-961f-82ee72b054a4
> Rebuilding .git/svn/trunk/.rev_map. 
> 138bc75d-0d04-0410-961f-82ee72b054a4 ...
> <repeat forever>
>
>
> I checked the obvious things. There is no permissions problem, strace
> doesn't show the writes failing, etc.
> git-fsck shows nothing as being obviously wrong.

Are you sure? I just glanced at the code, and it avoids rebuilding i  
the map file exists and is not empty.

> This just suddenly started happening with no obvious reason (we fire
> off git-svn fetch and rebase on the repo when cvs commits happen).
>
> git-log shows the last processed rev was:
>
> commit e66b310fb9a3ddc7ed5143db3c468711c8a36d08
> Author: uros <uros@138bc75d-0d04-0410-961f-82ee72b054a4>
> Date:   Thu May 15 14:38:23 2008 +0000
>
>        * config/i386/sse.md (*vec_concatv2sf_sse4_1): Add "m"  
> constraint
>        to alternative 4 of operand 2.
>
>
>    git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@135364
> 138bc75d-0d04-0410-961f-82ee72b054a4
>
>
> I'm happy to provide anything i can to get the problem fixed, but not
> being a git expert, i have no idea what is up here.

What version of git do you have? And you might want to CC Eric Wong <normalperson@yhbt.net 
 > on this.

-Kevin Ballard

-- 
Kevin Ballard
http://kevin.sb.org
kevin@sb.org
http://www.tildesoft.com

Re: git-svn goes into infinite loop rebuilding rev_map

[Daniel Berlin]

On Fri, May 16, 2008 at 1:48 PM, Kevin Ballard <kevin@sb.org> wrote:
> On May 16, 2008, at 1:15 PM, Daniel Berlin wrote:
>
>> Starting last night, for no particularly obvious reason, git-svn fetch
>> (or git-svn rebase or any command that does naything interesting) on
>> gcc.gnu.org's repo causes git-svn to do the following:
>>
>> -bash-3.00$ git-svn fetch
>> Rebuilding .git/svn/trunk/.rev_map.138bc75d-0d04-0410-961f-82ee72b054a4
>> ...
>> Done rebuilding
>> .git/svn/trunk/.rev_map.138bc75d-0d04-0410-961f-82ee72b054a4
>> Rebuilding .git/svn/trunk/.rev_map.138bc75d-0d04-0410-961f-82ee72b054a4
>> ...
>> <repeat forever>
>>
>>
>> I checked the obvious things. There is no permissions problem, strace
>> doesn't show the writes failing, etc.
>> git-fsck shows nothing as being obviously wrong.
>
> Are you sure? I just glanced at the code, and it avoids rebuilding i the map
> file exists and is not empty.
Absolutely positive.

You can see the strace at http://www.dberlin.org/git.strace.bz2
Note there are no eperms or anything of the sort.


>
>> This just suddenly started happening with no obvious reason (we fire
>> off git-svn fetch and rebase on the repo when cvs commits happen).
>>
>> git-log shows the last processed rev was:
>>
>> commit e66b310fb9a3ddc7ed5143db3c468711c8a36d08
>> Author: uros <uros@138bc75d-0d04-0410-961f-82ee72b054a4>
>> Date:   Thu May 15 14:38:23 2008 +0000
>>
>>       * config/i386/sse.md (*vec_concatv2sf_sse4_1): Add "m" constraint
>>       to alternative 4 of operand 2.
>>
>>
>>   git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@135364
>> 138bc75d-0d04-0410-961f-82ee72b054a4
>>
>>
>> I'm happy to provide anything i can to get the problem fixed, but not
>> being a git expert, i have no idea what is up here.
>
> What version of git do you have? And you might want to CC Eric Wong
> <normalperson@yhbt.net> on this.
>
It was 1.5.5
After we noticed this problem, i tried updating it to the absolute
latest from the repo as of this morning (didn't help).
So it is now 1.5.5.1.308.g1fbb5


> -Kevin Ballard
>
> --
> Kevin Ballard
> http://kevin.sb.org
> kevin@sb.org
> http://www.tildesoft.com
>
>
>

Re: git-svn goes into infinite loop rebuilding rev_map

[Daniel Berlin]

On Fri, May 16, 2008 at 2:02 PM, Daniel Berlin <dberlin@dberlin.org> wrote:
> On Fri, May 16, 2008 at 1:48 PM, Kevin Ballard <kevin@sb.org> wrote:
>> On May 16, 2008, at 1:15 PM, Daniel Berlin wrote:
>>
>>> Starting last night, for no particularly obvious reason, git-svn fetch
>>> (or git-svn rebase or any command that does naything interesting) on
>>> gcc.gnu.org's repo causes git-svn to do the following:
>>>

I finally got a chance to debug this a bit, and it appears to occur
because rewriteRoot is not taken into account during rebuilding of
metadata.


In this loop in rebuild:
 while (<$log>) {
                if ( m{^commit ($::sha1)$} ) {
                        $c = $1;
                        next;
                }
                next unless s{^\s*(git-svn-id:)}{$1};
                my ($url, $rev, $uuid) = ::extract_metadata($_);
                remove_username($url);

                # ignore merges (from set-tree)
                next if (!defined $rev || !$uuid);
                print "uuid:$uuid, svn_uuid:$svn_uuid, full_url:
$full_url, url: $url!\n";
                my $rwr = $self->rewrite_root;

                # if we merged or otherwise started elsewhere, this is
                # how we break out of it

                if (($uuid ne $svn_uuid) ||
                    ($full_url && $url && ($url ne $full_url))) {
                        next;
                }

                $self->rev_map_set($rev, $c);
                print "r$rev = $c\n";
        }

We end up with
uuid:138bc75d-0d04-0410-961f-82ee72b054a4,
svn_uuid:138bc75d-0d04-0410-961f-82ee72b054a4, full_url:
file:///svn/gcc/trunk, url: svn+ssh://gcc.gnu.org/svn/gcc/trunk

This is because we fetch from the local fs but rewriteRoot it into a
path that works for remote users who want to commit.

However, in that loop that $full_url ne $url so it skips setting the
rev in the rev map, under the assumption it came from elsewhere.

Since it does this for every rev, it never actually puts anything in
the rev map (which means it never creates it).
So it decides to rebuild it again since the file existence fails.
Lather, Rinse, Repeat

The correct fix (i'm not good enough at perl/git-svn to know how to
write this code) is to take rewriteRoot into account.
In addition, it might make sense to assert that when we rebuild the
rev_map, it actually exists when we are done :).

HTH,
Dan