Re: git clone dies (large git repository)

[Junio C Hamano <junkio@cox.net>]

"Troy Telford" <ttelford@linuxnetworx.com> writes:

> I'm thinking of it as an option for git-repack-- that the user can set
> the maximum size of any individual pack, and after that limit is
> reached, a  new pack file is started.  (ie. --max-size 2GB) and will
> end up with two  packs, each 2GB in size.

The way I would suggest you do it is not by size but by distance
from the latest.  If you want to split the kernel history for
example, you repack up to 2.6.14 for example, and then repack
the remainder.  That way, you can optimize for size for older
(presumably less frequently used) data while optimizing for
speed for more reent stuff.

There is no wrapper support for the above splitting in
git-repack.  The low-level plumbing tools can be used this way
for example:

	name=`
                git rev-list --objects $list_old_tags_here |
                git pack-objects --window=50 --depth=50 --non-empty .tmp-pack
        ` &&
        mv -f .tmp-pack-$name.{pack,idx} .git/objects/pack/

	name=`
		git revlist --objects --all --not $list_old_tags_here |
		git pack-objects --non-empty .tmp-pack
	` &&
        mv -f .tmp-pack-$name.{pack,idx} .git/objects/pack/

If you are splitting into more than two, you would instead have
more than one $list_old_tags_here list, and iterate them
through, something like:

	pack_between () {
        	already_done="$1"
                do_this_time="$2"
                w=${3-10}
		name=`
			git rev-list --objects \
                        	$do_this_time \
                                --not $already_done |
			git pack-objects --window=$w --depth=$w \
                        	--non-empty .tmp-pack
                ` &&
                mv -f .tmp-pack-$name.{pack,idx} .git/objects/pack/
	}

	pack_between "" "$prehistoric_tag_list" 100
	pack_between "$prehistoric_tag_list" "$more_recent_tag_list" 50
	pack_between "$more_recent_tag_list" --all

All untested, of course, so do not play with it in your precious
repository you do not have any other copy, but hopefully you get
an idea ;-).