Re: Leaving large binaries out of the packfile

["Shawn O. Pearce" <spearce@spearce.org>]

Paolo Bonzini <bonzini@gnu.org> wrote:
> On 06/10/2010 08:04 PM, Shawn O. Pearce wrote:
>> Joshua Jensen<jjensen@workspacewhiz.com>  wrote:
>>> Sometimes, 'git gc' runs out of memory.  I have to discover which file
>>> is causing the problem, so I can add it to .gitattributes with a
>>> '-delta' flag.  Mostly, though, the repacking takes forever, and I dread
>>> running the operation.
>>
>> If you have the list of big objects, you can put them into their
>> own pack file manually.  Feed their SHA-1 names on stdin to git
>> pack-objects, and save the resulting pack under .git/objects/pack.
>
> Do you know any simpler way than
>
> git log --pretty=format:%H | while read x; do
>   git ls-tree $x -- ChangeLog | awk '{print $3}'
> done | sort -u
>
> to do this?  I thought it would be nice to add --sha1-only to  
> git-ls-tree, but maybe I'm missing some other trick.

Maybe

  git rev-list --objects HEAD | grep ' ChangeLog'

pack-objects wants the output of rev-list --objects as input, file
name and all.  So its just a matter of selecting the right lines
from its output.

>> Assuming the pack was called pack-DEADC0FFEE.pack, create a file
>> called pack-DEADC0FFEE.keep in the same directory.  This will stop
>> Git from trying to repack the contents of that pack file.
>>
>> Now run `git gc` to remove those huge objects from the pack file
>> that contains all of the other stuff.
>
> That obviously wouldn't help if these large binaries are updated often,  
> however.

No, it doesn't.  But you still could do this on a periodic basis.
That way you only drag around a handful of recently created large
binaries during a typical `git gc`, and not the entire project's
history of them.

-- 
Shawn.