Re: Big project, slow access!

["R. Tyler Ballance" <tyler@slide.com>]

[-- Attachment #1: Type: text/plain, Size: 1766 bytes --]


On Fri, 18 Sep 2009, Toan Pham wrote:

> Hi,
> 
> I use git to maintain a project that is at least 8 gigs in size.
> The project is a Linux from Scratch repository that includes source
> codes to approximately 2000 open source projects,
> gcc tool-chain, 1000+ configurations for different software packages,
> source code for different kernel versions,
> and many linux distributions/flavors resulted from this LFS build environment.
> 
> The git's object repository is now 4.6 gigs and consists of approx.
> 610,000 files and folders.
> The speed of git is now terribly slow.  Each time I use basic commands
> like 'git status' or 'git diff',
> it would take at least 5 minutes for git to give me back a result.
> Again, the machine that i run git on is a P4 3.2 gig-hertz with HT.

Howdy Toan, we have a similarly large repository ~405k files, the .git
folder fully packed is ~6GB. 

The advise to fully-pack your repository is likely going to have the
greatest impact on your performance in the short term, in the long term
however you might want to consider using git-filter-branch(1) or other
tools available to separate our the components of your current Git
reposotory into a series of repos.

The performance hit you're seeing likely has nothing to do with your
processor speed either, but rather your disk search speed (i'm waiting
for a new fancy SSD to help alleviate my issues ;))

> would  someone please recommend on how i can optimize git's performance?
> Git is so slow, are there better ways to manage a project like this?

Rethink how your project is laid out, and whether certain binaries files
need to sit in the tree, or can be build on a need-by-need basis.



Cheers
-R. Tyler Ballance
Slide, Inc.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]