[PATCH 0/5] Various fixes for Windows

[PATCH 0/5] Various fixes for Windows

[Johannes Sixt]

Here are some unrelated patches that I had accumulated over the past
weeks that improve the git experience on Windows.

The first three are maint-worthy and have been cooked long enough.
IMO it is safe to put them on maint directly.

Erik Faye-Lund (1):
  mingw: use _commit to implement fsync

Ian McLean (1):
  Fix "Out of memory? mmap failed" for files larger than 4GB on Windows

Johannes Sixt (1):
  Recent MinGW has a C99 implementation of snprintf functions

René Scharfe (1):
  Fix checkout of large files to network shares on Windows XP

bert Dvornik (1):
  start_command: close cmd->err descriptor when fork/spawn fails

 Makefile              |    1 -
 compat/mingw.c        |   17 +++++++++++++++++
 compat/mingw.h        |    5 ++++-
 compat/win32mmap.c    |    6 +++---
 run-command.c         |    2 ++
 t/t5516-fetch-push.sh |    2 +-
 t/t5705-clone-2gb.sh  |   12 +++++++++---
 7 files changed, 36 insertions(+), 9 deletions(-)

[PATCH 1/5 maint] Fix "Out of memory? mmap failed" for files larger than 4GB on Windows

[Johannes Sixt]

From: Ian McLean <ian.mclean@gmail.com>

The git_mmap implementation was broken for file sizes that wouldn't fit
into a size_t (32 bits).  This was caused by intermediate variables that
were only 32 bits wide when they should be 64 bits.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
---
 This patch is obviously correct. I have used it in production to verify
 that it doesn't break repositories that are smaller than 4GB.

 compat/win32mmap.c |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/compat/win32mmap.c b/compat/win32mmap.c
index 1c5a149..b58aa69 100644
--- a/compat/win32mmap.c
+++ b/compat/win32mmap.c
@@ -4,19 +4,19 @@ void *git_mmap(void *start, size_t length, int prot, int flags, int fd, off_t of
 {
 	HANDLE hmap;
 	void *temp;
-	size_t len;
+	off_t len;
 	struct stat st;
 	uint64_t o = offset;
 	uint32_t l = o & 0xFFFFFFFF;
 	uint32_t h = (o >> 32) & 0xFFFFFFFF;
 
 	if (!fstat(fd, &st))
-		len = xsize_t(st.st_size);
+		len = st.st_size;
 	else
 		die("mmap: could not determine filesize");
 
 	if ((length + offset) > len)
-		length = len - offset;
+		length = xsize_t(len - offset);
 
 	if (!(flags & MAP_PRIVATE))
 		die("Invalid usage of mmap when built with USE_WIN32_MMAP");
-- 
1.7.1.64.ga1799.dirty

[PATCH 2/5 maint] start_command: close cmd->err descriptor when fork/spawn fails

[Johannes Sixt]

From: bert Dvornik <dvornik+git@gmail.com>

Fix the problem where the cmd->err passed into start_command wasn't
being properly closed when certain types of errors occurr.  (Compare
the affected code with the clean shutdown code later in the function.)

On Windows, this problem would be triggered if mingw_spawnvpe()
failed, which would happen if the command to be executed was malformed
(e.g. a text file that didn't start with a #! line).  If cmd->err was
a pipe, the failure to close it could result in a hang while the other
side was waiting (forever) for either input or pipe close, e.g. while
trying to shove the output into the side band.  On msysGit, this
problem was causing a hang in t5516-fetch-push.

[J6t: With a slight adjustment of the test case, the hang is also
observed on Linux.]

Signed-off-by: bert Dvornik <dvornik+git@gmail.com>
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
---
 run-command.c         |    2 ++
 t/t5516-fetch-push.sh |    2 +-
 2 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/run-command.c b/run-command.c
index eb5c575..c7793f5 100644
--- a/run-command.c
+++ b/run-command.c
@@ -383,6 +383,8 @@ fail_pipe:
 			close(cmd->out);
 		if (need_err)
 			close_pair(fderr);
+		else if (cmd->err)
+			close(cmd->err);
 		errno = failed_errno;
 		return -1;
 	}
diff --git a/t/t5516-fetch-push.sh b/t/t5516-fetch-push.sh
index 2de98e6..6a37a4d 100755
--- a/t/t5516-fetch-push.sh
+++ b/t/t5516-fetch-push.sh
@@ -528,7 +528,7 @@ test_expect_success 'push does not update local refs on failure' '
 	mk_test heads/master &&
 	mk_child child &&
 	mkdir testrepo/.git/hooks &&
-	echo exit 1 >testrepo/.git/hooks/pre-receive &&
+	echo "#!/no/frobnication/today" >testrepo/.git/hooks/pre-receive &&
 	chmod +x testrepo/.git/hooks/pre-receive &&
 	(cd child &&
 		git pull .. master
-- 
1.7.1.64.ga1799.dirty

[PATCH 3/5 maint] Fix checkout of large files to network shares on Windows XP

[Johannes Sixt]

From: René Scharfe <rene.scharfe@lsrfire.ath.cx>

Bigger writes to network drives on Windows XP fail.  Cap them at 31MB to
allow them to succeed.  Callers need to be prepared for write() calls
that do less work than requested anyway.

On local drives, write() calls are translated to WriteFile() calls with
a cap of 64KB on Windows XP and 256KB on Vista.  Thus a cap of 31MB won't
affect the number of WriteFile() calls which do the actual work.  There's
still room for some other version of Windows to use a chunk size of 1MB
without increasing the number of system calls.

Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
---
 Again, I've used this patch in production with smaller repositories,
 and it worked flawlessly.

 compat/mingw.c       |   17 +++++++++++++++++
 compat/mingw.h       |    3 +++
 t/t5705-clone-2gb.sh |   12 +++++++++---
 3 files changed, 29 insertions(+), 3 deletions(-)

diff --git a/compat/mingw.c b/compat/mingw.c
index f90a114..9a8e336 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -140,6 +140,23 @@ int mingw_open (const char *filename, int oflags, ...)
 	return fd;
 }
 
+#undef write
+ssize_t mingw_write(int fd, const void *buf, size_t count)
+{
+	/*
+	 * While write() calls to a file on a local disk are translated
+	 * into WriteFile() calls with a maximum size of 64KB on Windows
+	 * XP and 256KB on Vista, no such cap is placed on writes to
+	 * files over the network on Windows XP.  Unfortunately, there
+	 * seems to be a limit of 32MB-28KB on X64 and 64MB-32KB on x86;
+	 * bigger writes fail on Windows XP.
+	 * So we cap to a nice 31MB here to avoid write failures over
+	 * the net without changing the number of WriteFile() calls in
+	 * the local case.
+	 */
+	return write(fd, buf, min(count, 31 * 1024 * 1024));
+}
+
 #undef fopen
 FILE *mingw_fopen (const char *filename, const char *otype)
 {
diff --git a/compat/mingw.h b/compat/mingw.h
index a6b8778..f465566 100644
--- a/compat/mingw.h
+++ b/compat/mingw.h
@@ -170,6 +170,9 @@ int link(const char *oldpath, const char *newpath);
 int mingw_open (const char *filename, int oflags, ...);
 #define open mingw_open
 
+ssize_t mingw_write(int fd, const void *buf, size_t count);
+#define write mingw_write
+
 FILE *mingw_fopen (const char *filename, const char *otype);
 #define fopen mingw_fopen
 
diff --git a/t/t5705-clone-2gb.sh b/t/t5705-clone-2gb.sh
index adfaae8..8afbdd4 100755
--- a/t/t5705-clone-2gb.sh
+++ b/t/t5705-clone-2gb.sh
@@ -12,7 +12,7 @@ test_expect_success 'setup' '
 
 	git config pack.compression 0 &&
 	git config pack.depth 0 &&
-	blobsize=$((20*1024*1024)) &&
+	blobsize=$((100*1024*1024)) &&
 	blobcount=$((2*1024*1024*1024/$blobsize+1)) &&
 	i=1 &&
 	(while test $i -le $blobcount
@@ -36,9 +36,15 @@ test_expect_success 'setup' '
 
 '
 
-test_expect_success 'clone' '
+test_expect_success 'clone - bare' '
 
-	git clone --bare --no-hardlinks . clone
+	git clone --bare --no-hardlinks . clone-bare
+
+'
+
+test_expect_success 'clone - with worktree, file:// protocol' '
+
+	git clone file://. clone-wt
 
 '
 
-- 
1.7.1.64.ga1799.dirty

[PATCH 4/5] mingw: use _commit to implement fsync

[Johannes Sixt]

From: Erik Faye-Lund <kusmabite@googlemail.com>

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
---
 compat/mingw.h |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/compat/mingw.h b/compat/mingw.h
index 7c2ab64..a6b8778 100644
--- a/compat/mingw.h
+++ b/compat/mingw.h
@@ -80,7 +80,7 @@ static inline int fork(void)
 static inline unsigned int alarm(unsigned int seconds)
 { return 0; }
 static inline int fsync(int fd)
-{ return 0; }
+{ return _commit(fd); }
 static inline int getppid(void)
 { return 1; }
 static inline void sync(void)
-- 
1.7.1.64.ga1799.dirty

[PATCH 5/5] Recent MinGW has a C99 implementation of snprintf functions

[Johannes Sixt]

Date: Mon, 20 Jul 2009 22:15:07 +0200

Starting with MinGW 3.14, released end of 2007, a working snprintf
is available. This means we do not need our own replacement that works
around the broken implementation in Microsoft's C runtime.

People who build git in an old MinGW environment are expected to set
SNPRINTF_RETURNS_BOGUS in their config.mak. msysgit is sufficiently
recent, of course.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
---
 The Date: line is deliberate to show how long I carry this patch. :-)

 Makefile |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

diff --git a/Makefile b/Makefile
index 910f471..c36c5b0 100644
--- a/Makefile
+++ b/Makefile
@@ -1026,7 +1026,6 @@ ifneq (,$(findstring MINGW,$(uname_S)))
 	NO_STRTOUMAX = YesPlease
 	NO_MKDTEMP = YesPlease
 	NO_MKSTEMPS = YesPlease
-	SNPRINTF_RETURNS_BOGUS = YesPlease
 	NO_SVN_TESTS = YesPlease
 	NO_PERL_MAKEMAKER = YesPlease
 	RUNTIME_PREFIX = YesPlease
-- 
1.7.1.64.ga1799.dirty

Re: [PATCH 0/5] Various fixes for Windows

[Junio C Hamano]

Thanks.