git-rebase dirty index and email address bug?

git-rebase dirty index and email address bug?

[John Goerzen]

Hi folks,

I have checked out a remote git repo.  I have hacked on my local copy
for the last two days.  I have performed no merges or anything elese
odd.  Today I git fetched the remote, and then I tried to git rebase
upstream/master.  I had committed all outstanding changes first.  My
index was clean.  It displayed:

First, rewinding head to replay your work on top of it...
HEAD is now at 77f1a60... Merge branch 'trunk'
Dirty index: cannot apply patches (dirty: public/dispatch.fcgi)

Tthat particular file was one that I added.  It did not exist upstream
either at my initial checkout, nor was it added upstream at any point
since.

git status shows:

# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#       new file:   public/dispatch.fcgi
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#       .dotest/

So I tried the git reset command on public/dispatch.fcgi, and also
then deleted that file.  I then tried git-rebase --continue, which
displayed:

sed: can't read .dotest/info: No such file or directory
sed: can't read .dotest/info: No such file or directory
sed: can't read .dotest/info: No such file or directory
Patch does not have a valid e-mail address.

I have no idea what sed is complaining about, but that last line is
interesting.  The upstream git repo I am monitoring was itself
converted from an svn repo using, presumably, git-svn or
git-svnimport.  It lies at
git://gitorious.org/redmine-git/mainline.git

Most patches have an author like this:

Author: jplang <jplang@e93f8b46-1217-0410-a6f0-8f06a7374b81>

Could one of the git tools be trying to parse this for some odd
reason?

Or is something else going on here?

Re: git-rebase dirty index and email address bug?

[John Goerzen]

On 2008-03-01, John Goerzen <jgoerzen@complete.org> wrote:
> Hi folks,
>
> I have checked out a remote git repo.  I have hacked on my local copy
> for the last two days.  I have performed no merges or anything elese
> odd.  Today I git fetched the remote, and then I tried to git rebase
> upstream/master.  I had committed all outstanding changes first.  My
> index was clean.  It displayed:

Two things to add:

1) This was a clean branch prior to rebase (git status was empty)

2) Adding -i to the rebase command, then making no changes to the
list, caused the rebase to work perfectly.

Re: git-rebase dirty index and email address bug?

[Jeff King]

On Sat, Mar 01, 2008 at 03:29:37PM -0600, John Goerzen wrote:

> Two things to add:
> 
> 1) This was a clean branch prior to rebase (git status was empty)
> 
> 2) Adding -i to the rebase command, then making no changes to the
> list, caused the rebase to work perfectly.

Yes, "rebase -i" works somewhat differently than an ordinary rebase. It
uses a series of cherry-picks rather than "format-patch | am". And the
bug is, I think, in "git-am".

It probably would also have worked using "git rebase -m" which uses
merge strategies to rebase.

And yes, it is a bit crazy that there are so many ways to rebase. The
original, "format-patch | am" is faster than the other schemes, but
doesn't handle renames (as "rebase -m" does); nor is it flexible enough
to do what "rebase -i" does.

-Peff

Re: git-rebase dirty index and email address bug?

[John Goerzen]

On Sunday 02 March 2008 1:30:16 am Jeff King wrote:
> On Sat, Mar 01, 2008 at 03:29:37PM -0600, John Goerzen wrote:
> > Two things to add:
> >
> > 1) This was a clean branch prior to rebase (git status was empty)
> >
> > 2) Adding -i to the rebase command, then making no changes to the
> > list, caused the rebase to work perfectly.
>
> Yes, "rebase -i" works somewhat differently than an ordinary rebase. It
> uses a series of cherry-picks rather than "format-patch | am". And the
> bug is, I think, in "git-am".
>
> It probably would also have worked using "git rebase -m" which uses
> merge strategies to rebase.

I did try that.  The resulting error message was slightly different, but not 
significantly so, if I remember correctly.

-- John

Re: git-rebase dirty index and email address bug?

[Jeff King]

On Sat, Mar 01, 2008 at 02:40:24PM -0600, John Goerzen wrote:

> odd.  Today I git fetched the remote, and then I tried to git rebase
> upstream/master.  I had committed all outstanding changes first.  My
> index was clean.  It displayed:
> 
> First, rewinding head to replay your work on top of it...
> HEAD is now at 77f1a60... Merge branch 'trunk'
> Dirty index: cannot apply patches (dirty: public/dispatch.fcgi)

That seems quite strange, since git-rebase did a "git-reset --hard",
which should have cleared your index anyway. And then it runs "git
format-patch | git am", and "am" is responsible for the "Dirty index"
message. So perhaps there is something in your commit that is triggering
a bug when piped in this way.

I tried cloning the "mainline.git" repository you mentioned and doing a
few simple tests, but was unable to reproduce. Can you make available
the commits that you are trying to rebase?

> So I tried the git reset command on public/dispatch.fcgi, and also
> then deleted that file.  I then tried git-rebase --continue, which
> displayed:
> 
> sed: can't read .dotest/info: No such file or directory
> sed: can't read .dotest/info: No such file or directory
> sed: can't read .dotest/info: No such file or directory
> Patch does not have a valid e-mail address.

Hrm. I don't think it has anything to do with your email address being
invalid, but rather that we somehow failed to create the 'info' file
containing information about the commit.

> converted from an svn repo using, presumably, git-svn or
> git-svnimport.  It lies at
> git://gitorious.org/redmine-git/mainline.git
> 
> Most patches have an author like this:
> 
> Author: jplang <jplang@e93f8b46-1217-0410-a6f0-8f06a7374b81>
> 
> Could one of the git tools be trying to parse this for some odd
> reason?

No, that should parse fine. I think there is something more fundamental
going wrong.

-Peff

Re: git-rebase dirty index and email address bug?

[John Goerzen]

On Sunday 02 March 2008 1:26:22 am Jeff King wrote:

> I tried cloning the "mainline.git" repository you mentioned and doing a
> few simple tests, but was unable to reproduce. Can you make available
> the commits that you are trying to rebase?

That repo contains some local configuration and password information.  
However, I would be happy to make it available to you or any other Git 
developer privately.  Could you contact me off-list if you'd like me to send 
it to you?  The .git directory in question is about 4.5MB, and I'd be happy 
to email it.

-- John

Re: git-rebase dirty index and email address bug?

[Jeff King]

On Sun, Mar 02, 2008 at 09:28:14PM -0600, John Goerzen wrote:

> That repo contains some local configuration and password information.  
> However, I would be happy to make it available to you or any other Git 
> developer privately.  Could you contact me off-list if you'd like me to send 
> it to you?  The .git directory in question is about 4.5MB, and I'd be happy 
> to email it.

I was able to turn this into a much smaller test case.

What is happening is that rebase works something like:
  1. reset to "onto" commit
  2. replay each commit between "upstream" and "branch"
but there is a bug in step 1 that is leaving the bogus
"public/dispatch.fcgi" file in the index and working tree. So step 2
correctly complains that the file is already there and uncommitted.

The bug in step 1 seems to be very hard to trigger, and requires that
the "branch" commit and the "onto" commit differ such that:

  - there is a file on the branch that was a directory in "onto"

  - there is a file on the branch that is new since "onto", and that
    sorts lexically after the file mentioned above

So you can see that it is rare for it to come up. :)

I'm not sure of the correct fix for the bug, but I have posted a request
for help under the subject "bug in unpack_trees", so hopefully somebody
more wise than I will have something to say.

-Peff

Re: git-rebase dirty index and email address bug?

[John Goerzen]

On 2008-03-04, Jeff King <peff@peff.net> wrote:
>> to email it.
>
> I was able to turn this into a much smaller test case.
>

Many thanks for helping to track this down, Jeff!

-- John