Re: [PATCH 0/17] Sliding window mmap for packfiles.

[Andy Whitcroft <apw@shadowen.org>]

Shawn Pearce wrote:
> Francis Moreau <francis.moro@gmail.com> wrote:
>> On 12/24/06, Shawn Pearce <spearce@spearce.org> wrote:
>>> However with this series even a 32 bit OS which only permits
>>> processes to have at most 2 GiB of address space (2 GiB split
>>> between kernel space and userspace) can access packfiles up
>>> to 4 GiB in size.  That seems to be the split most OSes wind
>>> up using, if they didn't push it out to 3.2 GiB like Linux
>>> and Solaris have done.
>>>
>> Does it still needed for 64 bit OS ?
> 
> Not really.  Almost any reasonable 64 bit OS which is also running
> a Git compiled for 64 bit userspace would be able to mmap multiple
> 4 GiB packfiles without this series.
>  
>> if not, can the overhead (if there is a significant one) implied by
>> your rework be avoid for such cases ?
> 
> The overhead is rather low.  I did try hard to make it only a handful
> of machine instructions worth of additional work, and even then I
> tried to ammortize those over relatively large blocks of data to
> reduce the impact.  But yes, there is an overhead over the current
> shipping version of Git.
> 
> However at least some of the overhead can be avoided by setting
> core.packedGitWindowSize and core.packedGitLimit to higher values.
> This will allow the implementation to mmap() larger windows of the
> packfiles and retain a greater number of windows in memory at once.
> 
> If core.packedGitWindowSize is larger than your largest packfile
> then most of the code will just "shutoff" and won't get in the way.
> Its default is 32 MiB (see Documentation/config.txt).
> 
> I think the additional overhead added by this series is neglible
> and worth the more graceful degredation it allows when virtual
> address space becomes limited.

You now change the default size based on NO_MMAP, could you not just
bump the window size to 4GiB on 64 bit?

-apw