Bug - crash on large commit

Bug - crash on large commit

[P Fudd]

Hi...

I'm a new user, and thinking that git is better than sliced bread.  So,
today I had to set up a new computer (MacOSX), and thought: "hey, I could
put the whole system into git!"

Well, ok, technically yes, you can do that.  Go into the root directory,
type 'git init', then 'git add .', then 'git commit -a'.

Bug 1: git complained about a symbolic link to nowhere and halted...
several hours into the 'add'.  If I want to store a broken link, let me;
maybe warn me.

Bug 2: git died with an out-of-memory error on the commit:
------
# git commit -a
[master (root-commit) 62b52e2] The initial checkin of the whole hard disk.
 Committer: System Administrator <root@Mac-Pro.local>
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly:

    git config --global user.name "Your Name"
    git config --global user.email you@example.com

If the identity used for this commit is wrong, you can fix it with:

    git commit --amend --author='Your Name <you@example.com>'

git(4667) malloc: *** mmap(size=1964838912) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
git(4667) malloc: *** mmap(size=1964838912) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
fatal: Out of memory, malloc failed
[Mac-Pro:/] shadministrator# git version
git version 1.7.0.2
-----

Anyhoo, it's not really the right tool for the job, but Apple hasn't got a
package management system to speak of (yes, I know about bom files --
don't speak of it), and their filesystem loses bits now and then, so I
wanted the crypto checksums and the ability to spot and roll back changes.
 I suppose TimeMachine will have to do.

Thanks for a great program!

Re: Bug - crash on large commit

[Erik Faye-Lund]

On Tue, Mar 23, 2010 at 12:36 AM, P Fudd <fink@ch.pkts.ca> wrote:
> Bug 2: git died with an out-of-memory error on the commit:
> ------
> # git commit -a
> [master (root-commit) 62b52e2] The initial checkin of the whole hard disk.
>  Committer: System Administrator <root@Mac-Pro.local>
> Your name and email address were configured automatically based
> on your username and hostname. Please check that they are accurate.
> You can suppress this message by setting them explicitly:
>
>    git config --global user.name "Your Name"
>    git config --global user.email you@example.com
>
> If the identity used for this commit is wrong, you can fix it with:
>
>    git commit --amend --author='Your Name <you@example.com>'
>
> git(4667) malloc: *** mmap(size=1964838912) failed (error code=12)
> *** error: can't allocate region
> *** set a breakpoint in malloc_error_break to debug
> git(4667) malloc: *** mmap(size=1964838912) failed (error code=12)
> *** error: can't allocate region
> *** set a breakpoint in malloc_error_break to debug
> fatal: Out of memory, malloc failed
> [Mac-Pro:/] shadministrator# git version
> git version 1.7.0.2

Is your git built as a 32-bit program? If so, it can't handle files
larger than ~1-2 gb, which looks like it might be the case here...

-- 
Erik "kusma" Faye-Lund

Re: Bug - crash on large commit

[Jeff King]

On Mon, Mar 22, 2010 at 04:36:25PM -0700, P Fudd wrote:

> Bug 1: git complained about a symbolic link to nowhere and halted...
> several hours into the 'add'.  If I want to store a broken link, let me;
> maybe warn me.

Git does let you store broken links. Without seeing the actual error
message you got, it's hard to say what actually happened. But you should
be able to:

  $ mkdir foo && cd foo && git init
  $ ln -s bogus link
  $ git add .
  $ git commit -m 'add bogus link'
  $ git show
  [...]
  diff --git a/link b/link
  new file mode 120000
  index 0000000..882b604
  --- /dev/null
  +++ b/link
  @@ -0,0 +1 @@
  +bogus
  \ No newline at end of file

> Bug 2: git died with an out-of-memory error on the commit:
> ------
> # git commit -a
> [master (root-commit) 62b52e2] The initial checkin of the whole hard disk.
>  Committer: System Administrator <root@Mac-Pro.local>
> Your name and email address were configured automatically based
> on your username and hostname. Please check that they are accurate.
> You can suppress this message by setting them explicitly:
> 
>     git config --global user.name "Your Name"
>     git config --global user.email you@example.com
> 
> If the identity used for this commit is wrong, you can fix it with:
> 
>     git commit --amend --author='Your Name <you@example.com>'
> 
> git(4667) malloc: *** mmap(size=1964838912) failed (error code=12)
> *** error: can't allocate region
> *** set a breakpoint in malloc_error_break to debug

Note that it actually did make the commit, which means that the problem
you saw happened while git was generating a diffstat summary of the
commit.

It's hard to say exactly what it was trying to mmap without seeing a
stack trace. But note that it is trying to malloc almost 2 gigabytes.
In general, git assumes that it can fit any file you throw at it into
memory during a diff. So presumably you have some gigantic file, and
that is the problem.

Which, as you noted, means git is not really suitable for just throwing
your whole hard disk at it. The topic of better handling gigantic files
comes up occasionally on the list, but I don't know that anyone is
working on anything concrete right now.

Avery Pennarun's "bup" tool may be closer to what you want, but I've
never used it myself:

  http://github.com/apenwarr/bup

-Peff