From: Shawn Pearce <spearce@spearce.org>
To: Jon Smirl <jonsmirl@gmail.com>
Cc: git <git@vger.kernel.org>
Subject: Re: GPF in index-pack
Date: Sat, 5 Aug 2006 23:32:22 -0400 [thread overview]
Message-ID: <20060806033222.GD20565@spearce.org> (raw)
In-Reply-To: <9e4733910608051958h6144add7td46de70dd8b45a5e@mail.gmail.com>
Jon Smirl <jonsmirl@gmail.com> wrote:
> On 8/5/06, Shawn Pearce <spearce@spearce.org> wrote:
> >Jon Smirl <jonsmirl@gmail.com> wrote:
> >> Process size is 2.6GB when the seg fault happen. That's a lot of
> >> memory to build a pack index over 1M objects.
> >>
> >> I'm running a 3:1 process address space split. I wonder why it didn't
> >> grow all the way to 3GB. I still have RAM and swap available.
> >
> >Was the pack you are trying to index built with that fast-import.c
> >I sent last night? Its possible its doing something weird that
> >pack-index can't handle, such as insert a duplicate object into
> >the same pack...
>
> built with fast-import.
>
> >How big is the pack file? I'd expect pack-index to be using
> >something around 24 MB of memory (24 bytes/entry) but maybe its
> >hanging onto a lot of data (memory leak?) as it decompresses the
> >entries to compute the checksums.
>
> It is 934MB in size with 985,000 entries.
>
> Why does resolve_delta in index-pack.c need to be recursive? Is there
> a better way to code that routine? If it mmaps the file that uses 1GB
> address space, why does it need another 1.5GB to build an index?
Probably the easiest way to code the routine. Delta depth is
bounded; in the fast-import.c that I sent out last night I hardcoded
it to 10, which is (I believe) the default for GIT. So long as that
routine is recursive only along a single delta chain the recursion
depth won't be very high and shouldn't be the problem.
> I had a prior 400MB pack file built with fast-import that I was able
> to index ok.
Dumb luck? Maybe that had no duplicates while this one does?
--
Shawn.
next prev parent reply other threads:[~2006-08-06 3:32 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <9e4733910608051805j1192d910hf55393f1dbe1e472@mail.gmail.com>
2006-08-06 2:44 ` GPF in index-pack Jon Smirl
2006-08-06 2:48 ` Shawn Pearce
2006-08-06 2:58 ` Jon Smirl
2006-08-06 3:32 ` Shawn Pearce [this message]
2006-08-06 4:00 ` Jon Smirl
2006-08-06 4:08 ` Shawn Pearce
2006-08-06 4:16 ` Jon Smirl
2006-08-06 4:22 ` Jon Smirl
2006-08-06 13:46 ` Sergey Vlasov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20060806033222.GD20565@spearce.org \
--to=spearce@spearce.org \
--cc=git@vger.kernel.org \
--cc=jonsmirl@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).