From: Shawn Pearce <spearce@spearce.org>
To: Dongsheng Song <dongsheng.song@gmail.com>
Cc: git@vger.kernel.org
Subject: Re: git-repack: Outof memory
Date: Sat, 16 Sep 2006 22:25:35 -0400 [thread overview]
Message-ID: <20060917022534.GB7512@spearce.org> (raw)
In-Reply-To: <450CA561.9030602@gmail.com>
Dongsheng Song <dongsheng.song@gmail.com> wrote:
> I'm import from subversion. The problem appears to be git-repack phase using too many memory:
>
> $ git-repack -a -f -d --window=64 --depth=64
> Generating pack...
> Done counting 123497 objects.
> Deltifying 123497 objects.
> 24% (29677/123497) done
>
> $ top
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 3572 www-data 18 0 2591m 1.9g 528 R 13 94.8 81:48.98 git-pack-object
*ouch* You are probably running out of address space if you are on
a 32 bit architecture. A 2.5 GiB virtual address space is pretty
close to the maximum allowed on most OSes.
The code that I'm sitting on but haven't yet completed rebasing
onto current Git would probably help here.
Do you have any existing .pack files in .git/objects/pack? How big
are they and their corresponding .idx files?
git-repack will need to mmap every .pack and .idx in
.git/objects/pack, plus it needs working memory for each object
(123,497 of 'em) but as I recall its pretty frugal on its per-object
allocation. It can easily work with as many as 2 million objects on
a 32 bit system, assuming the .pack and .idx files aren't too large.
--
Shawn.
next prev parent reply other threads:[~2006-09-17 2:25 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2006-09-17 1:31 git-repack: Outof memory Dongsheng Song
2006-09-17 2:25 ` Shawn Pearce [this message]
[not found] ` <4b3406f0609161946x52a89dd3q6d74127c376270e7@mail.gmail.com>
[not found] ` <20060917031853.GC7512@spearce.org>
[not found] ` <4b3406f0609170003t63a2f9f4pf2c42faec9746a73@mail.gmail.com>
[not found] ` <4b3406f0609170543p68d96b9x9ba0c5a74d9e89e8@mail.gmail.com>
2006-09-18 0:23 ` Shawn Pearce
2006-09-18 2:47 ` Junio C Hamano
2006-09-18 6:50 ` Shawn Pearce
2006-09-17 21:09 ` Nicolas Pitre
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20060917022534.GB7512@spearce.org \
--to=spearce@spearce.org \
--cc=dongsheng.song@gmail.com \
--cc=git@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.