Re: win2k/cygwin cannot handle even moderately sized packs

["Alex Riesen" <raa.lkml@gmail.com>]

> This looks to me as if you have NO_MMAP=1 set in your Makefile (which I
> also do automatically when compiling on cygwin).

Kind of. I use mmap (from cygwin) in specially selected places.
I remember posting the patches once.

> The old problem: Windows does not have fork.

As if it have anything non-fubar at all...

> <digression> And before somebody starts cygwin bashing: don't. It is not
> cygwin's problem, it is Windows'.

I didn't bash cygwin. I just pity the whole effort (and myself, atm).

> And I think that a mmap() of 332MB would not fail.

It does not in isolated environment. It just fails in my particular context.
And before anyone suggests: yes, CreateFileMapping, and VirtualAlloc were
tried. They do return errors suggesting the same reason (ENOMEM).

> Long time ago (to be precise, July 18th), Linus suggested (in Message-Id:
> <Pine.LNX.4.64.0607180837260.3386@evo.osdl.org>) to find out which mmap()
> calls are _not_ used before a fork(), and not emulate them by malloc().
>
> I never came around to do that, but maybe others do?

I'm trying to find some time to make Shawn's sliding window work.
It looks promising (patches, not the time).


On 11/13/06, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
> On Tue, 7 Nov 2006, Alex Riesen wrote:
>
> > For me, it fails even on 332Mb pack:
> >
> > $ git reset --hard 61bb7fcb
> > fatal: packfile .git/objects/pack/pack-ad37...pack cannot be mapped.
> >
> > Instrumenting the code reveals that it fails on 348876870 bytes.
> > Strangely enough, a cygwin program which just reads that pack
> > many times without freeing the mem goes up to 1395507480 (1330Mb).
> >
> > If I replace the malloc (cygwin) with native VirtualAlloc(MEM_COMMIT)
> > it reports that "Not enough storage is available to process this command",
> > which is just ENOMEM, I think.
>
> This looks to me as if you have NO_MMAP=1 set in your Makefile (which I
> also do automatically when compiling on cygwin).
>
> The old problem: Windows does not have fork.
>
> <digression> And before somebody starts cygwin bashing: don't. It is not
> cygwin's problem, it is Windows'. The cygwin people worked long and hard
> on something truly useful, and it helps me _every_ time I have to work on
> a Windows platform (which _is_ utter crap). Some problems of Windows are
> so unhideable though, that even cygwin cannot work around them.
> </digression>
>
> Cygwin provides a mmap(), which works remarkably well even with the
> emulated fork(), but one thing is not possible: since mmap()ed files
> have to be _reopened_ after a fork(), and git uses the
> open-temp-file-then-delete-it-but-continue-to-use-it paradigm quite often,
> we work around it by setting NO_MMAP=1. Again, this is _not_ Cygwin's
> fault!
>
> And I think that a mmap() of 332MB would not fail.
>
> Long time ago (to be precise, July 18th), Linus suggested (in Message-Id:
> <Pine.LNX.4.64.0607180837260.3386@evo.osdl.org>) to find out which mmap()
> calls are _not_ used before a fork(), and not emulate them by malloc().
>
> I never came around to do that, but maybe others do?
>
> Ciao,
> Dscho
>