Re: git packs

[Nicolas Pitre <nico@cam.org>]

On Sun, 11 Nov 2007, bob wrote:

> I applied the patch and these commands:
> 
> cd rmwHtmlOld
> rm -fr .git
> git init
> git config core.compression 0
> git add .

Note that I did "git config core.compression 0" simply to disable 
zlib compression altogether when creating the test repo just so it gets 
created faster.  even then, auto-generating and cloning a 8GB test 
repository isn't particularly quick.

> I then got the same error as before, "Bus error".  Rats!

Do you get that with a 32-bit or 64-bit build of Git?

> Then I modified your script since I do not have seq or
> your test-genrandom.

test-genrandom is built with Git.  It is just not installed anywhere.

> I substituted:
> 
> dd count=XX  if=/dev/random of=file_$i
> 
> where XX is adjusted to meet dd's requirements.  Also,

Again I used test-genrandom instead of /dev/random or /dev/urandom 
simply because the former is much faster.

> I found after searching for a while, that the following
> works just like your seq command:
> 
> xyzzy="1 2 3 4"
> for i in $xyzzy
> do
> ...
> done
> 
> Your script then ran flawlessly.

However 'seq -w 1 2 63' should be replaced  with "01 03 05 07 09 11 13 
15" and so on up to 63, and 'seq -w 2 2 64' is "02 04 06 08 10 12 16" 
and so on.

> I looked through index-pack.c some more, but it is
> very hard to figure it out without doing a lot of research
> since there doesn't seem to be anything that describes
> the layout of a pack.  The link towards the end of the user's
> manual doesn't work for me.

Look at Documentation/technical/pack-format.txt in the Git source tree.

> The difference between your test and my data is that
> instead of having a few large files, I have 11,500 files
> of varying sizes.  On average though, the file size is
> about 370k.

Are you saying that the test repo with big files works for you but not 
your own data set?

Would you please recap what your problem is?

With my one line patch you should not get the "serious inflate 
inconsistency" error anymore.  The bus error must be another issue.


Nicolas