git-fast-import segfaults

git-fast-import segfaults

[cpettitt]

I'm seeing the following errors when I run git-fast-import (on Intel
OSX) with some data from a git-p4 import:

[cpettitt@gish scratch2]$ rm -rf .git; git init; git-fast-import < ~/writer.out
Initialized empty Git repository in .git/
git-fast-import(23021) malloc: *** error for object 0x500e50: double free
git-fast-import(23021) malloc: *** set a breakpoint in szone_error to debug
git-fast-import(23021) malloc: ***  Deallocation of a pointer not
malloced: 0x501a80; This could be a double free(), or free() called
with the middle of an allocated block; Try setting environment
variable MallocHelp to see tools to help debug
git-fast-import(23021) malloc: *** error for object 0x5020a0: double free
git-fast-import(23021) malloc: *** set a breakpoint in szone_error to debug
git-fast-import(23021) malloc: ***  Deallocation of a pointer not
malloced: 0x5007e0; This could be a double free(), or free() called
with the middle of an allocated block; Try setting environment
variable MallocHelp to see tools to help debug
git-fast-import(23021) malloc: ***  Deallocation of a pointer not
malloced: 0x5006e0; This could be a double free(), or free() called
with the middle of an allocated block; Try setting environment
variable MallocHelp to see tools to help debug
git-fast-import(23021) malloc: ***  Deallocation of a pointer not
malloced: 0x501e10; This could be a double free(), or free() called
with the middle of an allocated block; Try setting environment
variable MallocHelp to see tools to help debug
git-fast-import(23021) malloc: ***  Deallocation of a pointer not
malloced: 0x502190; This could be a double free(), or free() called
with the middle of an allocated block; Try setting environment
variable MallocHelp to see tools to help debug
git-fast-import(23021) malloc: *** error for object 0x500280: double free
git-fast-import(23021) malloc: *** set a breakpoint in szone_error to debug
git-fast-import(23021) malloc: ***  Deallocation of a pointer not
malloced: 0x5009c0; This could be a double free(), or free() called
with the middle of an allocated block; Try setting environment
variable MallocHelp to see tools to help debug
git-fast-import(23021) malloc: *** error for object 0x500b00:
incorrect checksum for freed object - object was probably modified
after being freed, break at szone_error to debug
git-fast-import(23021) malloc: *** set a breakpoint in szone_error to debug
Segmentation fault

I start getting free errors at fast-import.c:1577:

                        rc = rc_free;
                        if (rc)
                                rc_free = rc->next;
                         else {
                                rc = cmd_hist.next;
                                cmd_hist.next = rc->next;
                                cmd_hist.next->prev = &cmd_hist;
                                free(rc->buf); // <-- error is emitted
in free here
                        }


I believe these errors started showing up in commit
b449f4cfc972929b638b90d375b8960c37790618. I did a bisect on
fast-import.c and this was the first commit for that file that
exhibits this bug with the input.

I thought I would check with the list to see if this is a known issue
before I spend time trying to dig into it.

Re: git-fast-import segfaults

[Shawn O. Pearce]

cpettitt <cpettitt@gmail.com> wrote:
> I'm seeing the following errors when I run git-fast-import (on Intel
> OSX) with some data from a git-p4 import:
...
> I believe these errors started showing up in commit
> b449f4cfc972929b638b90d375b8960c37790618. I did a bisect on
> fast-import.c and this was the first commit for that file that
> exhibits this bug with the input.
> 
> I thought I would check with the list to see if this is a known issue
> before I spend time trying to dig into it.

It is a known issue.  Someone else has reported the same thing,
and bisecting pointed at the same commit.  But they weren't able
to supply their input data for debugging by Pierre or myself as it
was a private project and they haven't had a chance to attempt to
debug it on their own.

Any light you can shed on the problem would be most appreciated.

-- 
Shawn.

[PATCH] Fix regression in fast-import.c due to strbufs.

[Pierre Habouzit]

[-- Attachment #1: Type: text/plain, Size: 2219 bytes --]

Without this strbuf_release, it yields a double free later, the command is
in fact stashed, and this is not a memory leak.

Signed-off-by: Pierre Habouzit <madcoder@debian.org>
---

  On Fri, Oct 26, 2007 at 06:53:01AM +0000, Shawn O. Pearce wrote:
  > cpettitt <cpettitt@gmail.com> wrote:
  > > I'm seeing the following errors when I run git-fast-import (on Intel
  > > OSX) with some data from a git-p4 import:
  > ....
  > > I believe these errors started showing up in commit
  > > b449f4cfc972929b638b90d375b8960c37790618. I did a bisect on
  > > fast-import.c and this was the first commit for that file that
  > > exhibits this bug with the input.
  > > 
  > > I thought I would check with the list to see if this is a known issue
  > > before I spend time trying to dig into it.
  > 
  > It is a known issue.  Someone else has reported the same thing,
  > and bisecting pointed at the same commit.  But they weren't able
  > to supply their input data for debugging by Pierre or myself as it
  > was a private project and they haven't had a chance to attempt to
  > debug it on their own.
  > 
  > Any light you can shed on the problem would be most appreciated.

  Wait, I believe I found the problem thanks to the "free" that fails.

  Could you please try that patch ? looking at the diff again, and
  knowing the issue is with an rc->buf (which are old command_buf
  stashed buffers) it looks like I migrated cmd_data improperly.

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org
 fast-import.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/fast-import.c b/fast-import.c
index 6f888f6..f93d7d6 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1616,6 +1616,7 @@ static void cmd_data(struct strbuf *sb)
 		char *term = xstrdup(command_buf.buf + 5 + 2);
 		size_t term_len = command_buf.len - 5 - 2;
 
+		strbuf_detach(&command_buf, NULL);
 		for (;;) {
 			if (strbuf_getline(&command_buf, stdin, '\n') == EOF)
 				die("EOF in data (terminator '%s' not found)", term);
-- 
1.5.3.4.1358.gfae55-dirty


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] Fix regression in fast-import.c due to strbufs.

[cpettitt]

On 10/26/07, Pierre Habouzit <madcoder@debian.org> wrote:
> Without this strbuf_release, it yields a double free later, the command is
> in fact stashed, and this is not a memory leak.
>
> Signed-off-by: Pierre Habouzit <madcoder@debian.org>

Pierre,

You nailed it! No more double frees and no segfault.

Thanks,
Chris

Re: [PATCH] Fix regression in fast-import.c due to strbufs.

[Pierre Habouzit]

[-- Attachment #1: Type: text/plain, Size: 920 bytes --]

Dear Shun, the fast-import bug your reported has a fix.

Dear Junio, please merge the patch in the mail I'm answering to[0] into
master as it fixes a crash in fast-import.

Cheers,

  [0]  Message-Id: <20071026075912.GA25365@artemis.corp>

diff --git a/fast-import.c b/fast-import.c
index 6f888f6..f93d7d6 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1616,6 +1616,7 @@ static void cmd_data(struct strbuf *sb)
 		char *term = xstrdup(command_buf.buf + 5 + 2);
 		size_t term_len = command_buf.len - 5 - 2;
 
+		strbuf_detach(&command_buf, NULL);
 		for (;;) {
 			if (strbuf_getline(&command_buf, stdin, '\n') == EOF)
 				die("EOF in data (terminator '%s' not found)", term);
-- 
1.5.3.4.1358.gfae55-dirty




-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] Fix regression in fast-import.c due to strbufs.

[Shun Kei Leung]

Hi Pierre,

Thanks. You are the man! It works perfectly now.

Regards,
Kevin

Re: [PATCH] Fix regression in fast-import.c due to strbufs.

[Pierre Habouzit]

[-- Attachment #1: Type: text/plain, Size: 403 bytes --]

On Mon, Oct 29, 2007 at 02:59:02AM +0000, Shun Kei Leung wrote:
> Hi Pierre,
> 
> Thanks. You are the man! It works perfectly now.

  Actually, I also was the one breaking it in the first place, but
you're welcome :)

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]