From: Shawn Pearce <spearce@spearce.org>
To: Junio C Hamano <gitster@pobox.com>
Cc: Kartik Agaram <ak@akkartik.com>, git@vger.kernel.org
Subject: Re: how to keep git-fetch from running out of memory?
Date: Sun, 15 May 2011 13:37:27 -0700 [thread overview]
Message-ID: <BANLkTi=+k_qh_RzuL73NPnEEcpA=0B8X_Q@mail.gmail.com> (raw)
In-Reply-To: <7vboz3pv7x.fsf@alter.siamese.dyndns.org>
On Sun, May 15, 2011 at 13:25, Junio C Hamano <gitster@pobox.com> wrote:
> Shawn Pearce <spearce@spearce.org> writes:
>
>> However, that may still be insufficient. A big object must still be
>> allocated in memory in order to compute its SHA-1.
>
> I wonder if we can stream this?
If its a non-delta, yes. We already do this in JGit for blobs over
50M. (Trees, commits and tags are not streamed because we might need
to do fsck on them and the fsck code for these object types requires
the entire object in a single buffer.)
Its probably not a huge change to index-pack.c. But I haven't dug
around in there in a while. I wish I could say I will look at this on
Monday, but I don't have the time.
--
Shawn.
prev parent reply other threads:[~2011-05-15 20:37 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-05-15 5:24 how to keep git-fetch from running out of memory? Kartik Agaram
2011-05-15 19:13 ` Shawn Pearce
2011-05-15 20:25 ` Junio C Hamano
2011-05-15 20:37 ` Shawn Pearce [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='BANLkTi=+k_qh_RzuL73NPnEEcpA=0B8X_Q@mail.gmail.com' \
--to=spearce@spearce.org \
--cc=ak@akkartik.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).