bitmap creation failed

bitmap creation failed

[gjarms]

Hi Git Experts,

We have been exploring various ways to improve git cloning time, one among
them is using bitmap which is suppose to save time "counting objects".  but
i have problem creating bitmap since the repository contains 100's of pack
files. the bitmap file is not created when i use "git gc".

I have the following entries in my .gitconfig.

[pack]
        packSizeLimit = 10m
        writebitmaps = on
        writeBitmapHashCache = on

If i just dont use "packSizeLimit = 10m", then bitmap is created just by
running git gc

Can you please make me understand ?, What i understood is that the bitmap is
created when there is a single pack file, but if i split it into multiple
pack file, the bitmap generation fails with the warning
"warning: disabling bitmap writing, as some objects are not being packed".

Regards,
Arumuga



--
View this message in context: http://git.661346.n2.nabble.com/bitmap-creation-failed-tp7657450.html
Sent from the git mailing list archive at Nabble.com.

Re: bitmap creation failed

[Stefan Beller]

On Thu, Sep 1, 2016 at 11:16 AM, gjarms <gjarms@gmail.com> wrote:
> Hi Git Experts,
>
> We have been exploring various ways to improve git cloning time, one among
> them is using bitmap which is suppose to save time "counting objects".  but
> i have problem creating bitmap since the repository contains 100's of pack
> files. the bitmap file is not created when i use "git gc".
>
> I have the following entries in my .gitconfig.
>
> [pack]
>         packSizeLimit = 10m
>         writebitmaps = on
>         writeBitmapHashCache = on
>
> If i just dont use "packSizeLimit = 10m", then bitmap is created just by
> running git gc
>
> Can you please make me understand ?, What i understood is that the bitmap is
> created when there is a single pack file, but if i split it into multiple
> pack file, the bitmap generation fails with the warning
> "warning: disabling bitmap writing, as some objects are not being packed".

So I guess your single pack is larger than 10m, so it tries to create
multiple packs,
and that is not supported as bitmaps only operate on one pack.

Stefan

>
> Regards,
> Arumuga
>
>
>
> --
> View this message in context: http://git.661346.n2.nabble.com/bitmap-creation-failed-tp7657450.html
> Sent from the git mailing list archive at Nabble.com.

Re: bitmap creation failed

[Jeff King]

On Thu, Sep 01, 2016 at 11:16:49AM -0700, gjarms wrote:

> Hi Git Experts,
> 
> We have been exploring various ways to improve git cloning time, one among
> them is using bitmap which is suppose to save time "counting objects".  but
> i have problem creating bitmap since the repository contains 100's of pack
> files. the bitmap file is not created when i use "git gc".
> 
> I have the following entries in my .gitconfig.
> 
> [pack]
>         packSizeLimit = 10m
>         writebitmaps = on
>         writeBitmapHashCache = on
> 
> If i just dont use "packSizeLimit = 10m", then bitmap is created just by
> running git gc
> 
> Can you please make me understand ?, What i understood is that the bitmap is
> created when there is a single pack file, but if i split it into multiple
> pack file, the bitmap generation fails with the warning
> "warning: disabling bitmap writing, as some objects are not being packed".

That's weird. I'd expect it to say:

  disabling bitmap writing, packs are split due to pack.packSizeLimit

At least since v2.8.3, which has 9cea46cdda. Before that it wouldn't
have printed anything, and just silently turned off bitmaps.

The "some objects are not being packed" warning should only come when
want_object_in_pack() says we don't want an object. That's generally
because the object is either found in a shared alternates repository, or
is in a .keep pack. Though in the latter case, unless you've set
repack.packKeptObjects manually, we'll pack it anyway when bitmaps are
in effect (since ee34a2bead, in v2.0.0).

That confusion aside, you almost certainly should not be setting
packSizeLimit, and definitely not to something so low. Git will not
store cross-pack deltas, so you miss out on tons of delta opportunities.
As a result:

  1. Your on-disk repository size will balloon. So you'll have a hundred
     10m packs rather than one 200mb pack.

  2. Your clone times will also grow, as git will try to find new deltas
     between the objects in various packs independently for each clone.

-Peff

Re: bitmap creation failed

[Jeff King]

On Fri, Sep 02, 2016 at 12:04:54PM +0530, Arumuga wrote:

> So I understand now, the following.
> 
> 1. reducing the pack file size will increase the clone time
> 2. Single pack file is expected to better use bitmap feature.
> 
> Am i correct ?

Yes, on both.

-Peff