git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Ken Pratt" <ken@kenpratt.net>
To: git@vger.kernel.org
Subject: pack operation is thrashing my server
Date: Sun, 10 Aug 2008 12:47:37 -0700	[thread overview]
Message-ID: <a6b6acf60808101247r4fea978ft6d2cdc53e1f99c0e@mail.gmail.com> (raw)

Hi,

I'm having memory issues when trying to clone a remote git repository.

I'm running: "git clone git+ssh://user@foo.bar.com/var/git/foo"

The remote repository is bare, and is 180MB in size (says du), with
1824 objects. The remote (VPS) server is running git version 1.5.6.4
on Arch Linux on a x86_64 Opteron with 256MB of dedicated RAM.

The clone command fires off some packing operations that bring the
server to its knees:

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
21782 kenpratt  20   0  444m 212m  272 D    3 83.0   0:04.98 git-pack-object

The clone also seems to hang forever. Progress stays at 0% for hours,
and it never progresses past compressing the first object.

I've tried very conservative pack settings:

[pack]
        threads = 1
        windowmemory = 64M
        deltacachesize = 1M
        deltacachelimit = 1M

[pack]
        threads = 1
        windowmemory = 16M
        deltacachesize = 16M
        deltacachelimit = 0

I've tries many variations like those, but nothing seems to help.

A "git repack -a -d" only takes 5 seconds to run on the same
repository on my laptop (a non-bare copy), and seems to peak at ~160MB
of RAM usage.

Any tips/help would be greatly appreciated. This repository is still
small -- it will eventually grow to multiple GB in size, as it is a
mix of small text files and binaries ranging in size from 2MB to
200MB. Is it not feasible to clone repositories of that size that are
hosted on a server with 256MB of RAM?

Thanks!

Ken

             reply	other threads:[~2008-08-10 19:48 UTC|newest]

Thread overview: 80+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-08-10 19:47 Ken Pratt [this message]
2008-08-10 23:06 ` pack operation is thrashing my server Martin Langhoff
2008-08-10 23:12   ` Ken Pratt
2008-08-10 23:30     ` Martin Langhoff
2008-08-10 23:34       ` Ken Pratt
2008-08-11  3:04 ` Shawn O. Pearce
2008-08-11  7:43   ` Ken Pratt
2008-08-11 15:01     ` Shawn O. Pearce
2008-08-11 15:40       ` Avery Pennarun
2008-08-11 15:59         ` Shawn O. Pearce
2008-08-11 19:13       ` Ken Pratt
2008-08-11 19:10     ` Andi Kleen
2008-08-11 19:15       ` Ken Pratt
2008-08-13  2:38         ` Nicolas Pitre
2008-08-13  2:50           ` Andi Kleen
2008-08-13  2:57             ` Shawn O. Pearce
2008-08-11 19:22       ` Shawn O. Pearce
2008-08-11 19:29         ` Ken Pratt
2008-08-11 19:34           ` Shawn O. Pearce
2008-08-11 20:10             ` Andi Kleen
2008-08-13  3:12       ` Geert Bosch
2008-08-13  3:15         ` Shawn O. Pearce
2008-08-13  3:58           ` Geert Bosch
2008-08-13 14:37             ` Nicolas Pitre
2008-08-13 14:56               ` Jakub Narebski
2008-08-13 15:04                 ` Shawn O. Pearce
2008-08-13 15:26                   ` David Tweed
2008-08-13 23:54                     ` Martin Langhoff
2008-08-14  9:04                       ` David Tweed
2008-08-13 16:10                   ` Johan Herland
2008-08-13 17:38                     ` Ken Pratt
2008-08-13 17:57                       ` Nicolas Pitre
2008-08-13 14:35         ` Nicolas Pitre
2008-08-13 14:59           ` Shawn O. Pearce
2008-08-13 15:43             ` Nicolas Pitre
2008-08-13 15:50               ` Shawn O. Pearce
2008-08-13 17:04                 ` Nicolas Pitre
2008-08-13 17:19                   ` Shawn O. Pearce
2008-08-14  6:33                   ` Andreas Ericsson
2008-08-14 10:04                     ` Thomas Rast
2008-08-14 10:15                       ` Andreas Ericsson
2008-08-14 22:33                         ` Shawn O. Pearce
2008-08-15  1:46                           ` Nicolas Pitre
2008-08-14 14:01                     ` Nicolas Pitre
2008-08-14 17:21                   ` Linus Torvalds
2008-08-14 17:58                     ` Linus Torvalds
2008-08-14 19:04                       ` Nicolas Pitre
2008-08-14 19:44                         ` Linus Torvalds
2008-08-14 21:30                           ` Andi Kleen
2008-08-15 16:15                             ` Linus Torvalds
2008-08-14 21:50                           ` Nicolas Pitre
2008-08-14 23:14                             ` Linus Torvalds
2008-08-14 23:39                               ` Björn Steinbrink
2008-08-15  0:06                                 ` Linus Torvalds
2008-08-15  0:25                                   ` Linus Torvalds
2008-08-16 12:47                                   ` Björn Steinbrink
2008-08-16  0:34                               ` Linus Torvalds
2008-09-07  1:03                                 ` Junio C Hamano
2008-09-07  1:46                                   ` Linus Torvalds
2008-09-07  2:33                                     ` Junio C Hamano
2008-09-07 17:11                                       ` Nicolas Pitre
2008-09-07 17:41                                         ` Junio C Hamano
2008-09-07  2:50                                     ` Jon Smirl
2008-09-07  3:07                                       ` Linus Torvalds
2008-09-07  3:43                                         ` Jon Smirl
2008-09-07  4:50                                           ` Linus Torvalds
2008-09-07 13:58                                             ` Jon Smirl
2008-09-07 17:08                                               ` Nicolas Pitre
2008-09-07 20:33                                                 ` Jon Smirl
2008-09-08 14:17                                                   ` Nicolas Pitre
2008-09-08 15:12                                                     ` Jon Smirl
2008-09-08 16:01                                                       ` Jon Smirl
2008-09-07  8:18                                         ` Andreas Ericsson
2008-09-07  7:45                                     ` Mike Hommey
2008-08-14 18:38                     ` Nicolas Pitre
2008-08-14 18:55                       ` Linus Torvalds
2008-08-13 16:01           ` Geert Bosch
2008-08-13 17:13             ` Dana How
2008-08-13 17:26             ` Nicolas Pitre
2008-08-13 12:43 ` Jakub Narebski

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=a6b6acf60808101247r4fea978ft6d2cdc53e1f99c0e@mail.gmail.com \
    --to=ken@kenpratt.net \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).