Re: Big project, slow access!

[Toan Pham <tpham3783@gmail.com>]

Dear Tyler


Thank you for your valuable feedback.

I'll research into git-filter-branch and also dividing a big project
into several sub-repositories.
This seems to increase the performance very much; however, there is a
draw-back that I am a little bit
concern with.  When we use several sub-repos option, we would probably
do manual book-keeping as to
which repo commits are compatible/built-able with other repo. commits.
 How did you manage to track
dependencies and their versions between different depos?

>>i'm waiting for a new fancy SSD to help alleviate my issues.

Please report the performance increase after you tested on your SS Drive.








On Fri, Sep 18, 2009 at 5:32 PM, R. Tyler Ballance <tyler@slide.com> wrote:
>
> On Fri, 18 Sep 2009, Toan Pham wrote:
>
>> Hi,
>>
>> I use git to maintain a project that is at least 8 gigs in size.
>> The project is a Linux from Scratch repository that includes source
>> codes to approximately 2000 open source projects,
>> gcc tool-chain, 1000+ configurations for different software packages,
>> source code for different kernel versions,
>> and many linux distributions/flavors resulted from this LFS build environment.
>>
>> The git's object repository is now 4.6 gigs and consists of approx.
>> 610,000 files and folders.
>> The speed of git is now terribly slow.  Each time I use basic commands
>> like 'git status' or 'git diff',
>> it would take at least 5 minutes for git to give me back a result.
>> Again, the machine that i run git on is a P4 3.2 gig-hertz with HT.
>
> Howdy Toan, we have a similarly large repository ~405k files, the .git
> folder fully packed is ~6GB.
>
> The advise to fully-pack your repository is likely going to have the
> greatest impact on your performance in the short term, in the long term
> however you might want to consider using git-filter-branch(1) or other
> tools available to separate our the components of your current Git
> reposotory into a series of repos.
>
> The performance hit you're seeing likely has nothing to do with your
> processor speed either, but rather your disk search speed (i'm waiting
> for a new fancy SSD to help alleviate my issues ;))
>
>> would  someone please recommend on how i can optimize git's performance?
>> Git is so slow, are there better ways to manage a project like this?
>
> Rethink how your project is laid out, and whether certain binaries files
> need to sit in the tree, or can be build on a need-by-need basis.
>
>
>
> Cheers
> -R. Tyler Ballance
> Slide, Inc.
>