[PATCH v6 00/16] daemon-win32

[PATCH v6 00/16] daemon-win32

[Erik Faye-Lund]

Here's hopefully the last iteration of this series. The previous version
only got a single complain about a typo in the subject of patch 14/15, so
it seems like most controversies have been settled.

This time the series has been rebased to junio's master branch, as the
'as/daemon-multi-listen' topic has been merged there.

The series can also be pulled from the 'win32-daemon-v6' tag in
git://repo.or.cz/git/kusma.git

Erik Faye-Lund (12):
  inet_ntop: fix a couple of old-style decls
  mingw: use real pid
  mingw: support waitpid with pid > 0 and WNOHANG
  mingw: add kill emulation
  daemon: use run-command api for async serving
  daemon: use full buffered mode for stderr
  daemon: get remote host address from root-process
  mingw: import poll-emulation from gnulib
  mingw: use poll-emulation from gnulib
  daemon: use socklen_t
  daemon: make --inetd and --detach incompatible
  daemon: opt-out on features that require posix

Martin Storsjö (1):
  Improve the mingw getaddrinfo stub to handle more use cases

Mike Pape (3):
  mingw: add network-wrappers for daemon
  mingw: implement syslog
  compat: add inet_pton and inet_ntop prototypes

 Documentation/git-daemon.txt |    3 +-
 Makefile                     |   23 ++-
 compat/inet_ntop.c           |   22 +--
 compat/inet_pton.c           |    8 +-
 compat/mingw.c               |  232 +++++++++++------
 compat/mingw.h               |   41 ++--
 compat/win32/sys/poll.c      |  596 ++++++++++++++++++++++++++++++++++++++++++
 compat/win32/sys/poll.h      |   53 ++++
 compat/win32/syslog.c        |   72 +++++
 compat/win32/syslog.h        |   20 ++
 daemon.c                     |  250 ++++++++++--------
 git-compat-util.h            |   11 +-
 12 files changed, 1103 insertions(+), 228 deletions(-)
 create mode 100644 compat/win32/sys/poll.c
 create mode 100644 compat/win32/sys/poll.h
 create mode 100644 compat/win32/syslog.c
 create mode 100644 compat/win32/syslog.h

-- 
1.7.3.2.161.gd6e00

[PATCH v6 01/16] mingw: add network-wrappers for daemon

[Erik Faye-Lund]

From: Mike Pape <dotzenlabs@gmail.com>

git-daemon requires some socket-functionality that is not yet
supported in the Windows-port. This patch adds said functionality,
and makes sure WSAStartup gets called by socket(), since it is the
first network-call in git-daemon.

Signed-off-by: Mike Pape <dotzenlabs@gmail.com>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
---
 compat/mingw.c |   44 +++++++++++++++++++++++++++++++++++++++++++-
 compat/mingw.h |   16 ++++++++++++++++
 2 files changed, 59 insertions(+), 1 deletions(-)

diff --git a/compat/mingw.c b/compat/mingw.c
index 6590f33..701a555 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -1175,7 +1175,10 @@ int mingw_getnameinfo(const struct sockaddr *sa, socklen_t salen,
 int mingw_socket(int domain, int type, int protocol)
 {
 	int sockfd;
-	SOCKET s = WSASocket(domain, type, protocol, NULL, 0, 0);
+	SOCKET s;
+
+	ensure_socket_initialization();
+	s = WSASocket(domain, type, protocol, NULL, 0, 0);
 	if (s == INVALID_SOCKET) {
 		/*
 		 * WSAGetLastError() values are regular BSD error codes
@@ -1205,6 +1208,45 @@ int mingw_connect(int sockfd, struct sockaddr *sa, size_t sz)
 	return connect(s, sa, sz);
 }
 
+#undef bind
+int mingw_bind(int sockfd, struct sockaddr *sa, size_t sz)
+{
+	SOCKET s = (SOCKET)_get_osfhandle(sockfd);
+	return bind(s, sa, sz);
+}
+
+#undef setsockopt
+int mingw_setsockopt(int sockfd, int lvl, int optname, void *optval, int optlen)
+{
+	SOCKET s = (SOCKET)_get_osfhandle(sockfd);
+	return setsockopt(s, lvl, optname, (const char*)optval, optlen);
+}
+
+#undef listen
+int mingw_listen(int sockfd, int backlog)
+{
+	SOCKET s = (SOCKET)_get_osfhandle(sockfd);
+	return listen(s, backlog);
+}
+
+#undef accept
+int mingw_accept(int sockfd1, struct sockaddr *sa, socklen_t *sz)
+{
+	int sockfd2;
+
+	SOCKET s1 = (SOCKET)_get_osfhandle(sockfd1);
+	SOCKET s2 = accept(s1, sa, sz);
+
+	/* convert into a file descriptor */
+	if ((sockfd2 = _open_osfhandle(s2, O_RDWR|O_BINARY)) < 0) {
+		int err = errno;
+		closesocket(s2);
+		return error("unable to make a socket file descriptor: %s",
+			strerror(err));
+	}
+	return sockfd2;
+}
+
 #undef rename
 int mingw_rename(const char *pold, const char *pnew)
 {
diff --git a/compat/mingw.h b/compat/mingw.h
index 83e35e8..a5bde82 100644
--- a/compat/mingw.h
+++ b/compat/mingw.h
@@ -7,6 +7,7 @@
 
 typedef int pid_t;
 typedef int uid_t;
+typedef int socklen_t;
 #define hstrerror strerror
 
 #define S_IFLNK    0120000 /* Symbolic link */
@@ -47,6 +48,9 @@ typedef int uid_t;
 #define F_SETFD 2
 #define FD_CLOEXEC 0x1
 
+#define EAFNOSUPPORT WSAEAFNOSUPPORT
+#define ECONNABORTED WSAECONNABORTED
+
 struct passwd {
 	char *pw_name;
 	char *pw_gecos;
@@ -225,6 +229,18 @@ int mingw_socket(int domain, int type, int protocol);
 int mingw_connect(int sockfd, struct sockaddr *sa, size_t sz);
 #define connect mingw_connect
 
+int mingw_bind(int sockfd, struct sockaddr *sa, size_t sz);
+#define bind mingw_bind
+
+int mingw_setsockopt(int sockfd, int lvl, int optname, void *optval, int optlen);
+#define setsockopt mingw_setsockopt
+
+int mingw_listen(int sockfd, int backlog);
+#define listen mingw_listen
+
+int mingw_accept(int sockfd, struct sockaddr *sa, socklen_t *sz);
+#define accept mingw_accept
+
 int mingw_rename(const char*, const char*);
 #define rename mingw_rename
 
-- 
1.7.3.2.161.gd6e00

[PATCH v6 02/16] mingw: implement syslog

[Erik Faye-Lund]

From: Mike Pape <dotzenlabs@gmail.com>

Syslog does not usually exist on Windows, so implement our own using
Window's ReportEvent mechanism.

Strings containing "%1" gets expanded into them selves by ReportEvent,
resulting in an unreadable string. "%2" and above is not a problem.
Unfortunately, on Windows an IPv6 address can contain "%1", so expand
"%1" to "% 1" before reporting. "%%1" is also a problem for ReportEvent,
but that string cannot occur in an IPv6 address.

Signed-off-by: Mike Pape <dotzenlabs@gmail.com>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
---
 Makefile              |    5 ++-
 compat/win32/syslog.c |   72 +++++++++++++++++++++++++++++++++++++++++++++++++
 compat/win32/syslog.h |   20 +++++++++++++
 daemon.c              |    2 -
 git-compat-util.h     |    1 +
 5 files changed, 96 insertions(+), 4 deletions(-)
 create mode 100644 compat/win32/syslog.c
 create mode 100644 compat/win32/syslog.h

diff --git a/Makefile b/Makefile
index 1f1ce04..d9d9419 100644
--- a/Makefile
+++ b/Makefile
@@ -496,6 +496,7 @@ LIB_H += compat/bswap.h
 LIB_H += compat/cygwin.h
 LIB_H += compat/mingw.h
 LIB_H += compat/win32/pthread.h
+LIB_H += compat/win32/syslog.h
 LIB_H += csum-file.h
 LIB_H += decorate.h
 LIB_H += delta.h
@@ -1081,7 +1082,7 @@ ifeq ($(uname_S),Windows)
 	AR = compat/vcbuild/scripts/lib.pl
 	CFLAGS =
 	BASIC_CFLAGS = -nologo -I. -I../zlib -Icompat/vcbuild -Icompat/vcbuild/include -DWIN32 -D_CONSOLE -DHAVE_STRING_H -D_CRT_SECURE_NO_WARNINGS -D_CRT_NONSTDC_NO_DEPRECATE
-	COMPAT_OBJS = compat/msvc.o compat/fnmatch/fnmatch.o compat/winansi.o compat/win32/pthread.o
+	COMPAT_OBJS = compat/msvc.o compat/fnmatch/fnmatch.o compat/winansi.o compat/win32/pthread.o compat/win32/syslog.o
 	COMPAT_CFLAGS = -D__USE_MINGW_ACCESS -DNOGDI -DHAVE_STRING_H -DHAVE_ALLOCA_H -Icompat -Icompat/fnmatch -Icompat/regex -Icompat/fnmatch -Icompat/win32 -DSTRIP_EXTENSION=\".exe\"
 	BASIC_LDFLAGS = -IGNORE:4217 -IGNORE:4049 -NOLOGO -SUBSYSTEM:CONSOLE -NODEFAULTLIB:MSVCRT.lib
 	EXTLIBS = advapi32.lib shell32.lib wininet.lib ws2_32.lib
@@ -1131,7 +1132,7 @@ ifneq (,$(findstring MINGW,$(uname_S)))
 	COMPAT_CFLAGS += -D__USE_MINGW_ACCESS -DNOGDI -Icompat -Icompat/fnmatch -Icompat/win32
 	COMPAT_CFLAGS += -DSTRIP_EXTENSION=\".exe\"
 	COMPAT_OBJS += compat/mingw.o compat/fnmatch/fnmatch.o compat/winansi.o \
-		compat/win32/pthread.o
+		compat/win32/pthread.o compat/win32/syslog.o
 	EXTLIBS += -lws2_32
 	PTHREAD_LIBS =
 	X = .exe
diff --git a/compat/win32/syslog.c b/compat/win32/syslog.c
new file mode 100644
index 0000000..42b95a9
--- /dev/null
+++ b/compat/win32/syslog.c
@@ -0,0 +1,72 @@
+#include "../../git-compat-util.h"
+#include "../../strbuf.h"
+
+static HANDLE ms_eventlog;
+
+void openlog(const char *ident, int logopt, int facility)
+{
+	if (ms_eventlog)
+		return;
+
+	ms_eventlog = RegisterEventSourceA(NULL, ident);
+
+	if (!ms_eventlog)
+		warning("RegisterEventSource() failed: %lu", GetLastError());
+}
+
+void syslog(int priority, const char *fmt, ...)
+{
+	struct strbuf sb = STRBUF_INIT;
+	struct strbuf_expand_dict_entry dict[] = {
+		{"1", "% 1"},
+		{NULL, NULL}
+	};
+	WORD logtype;
+	char *str;
+	int str_len;
+	va_list ap;
+
+	if (!ms_eventlog)
+		return;
+
+	va_start(ap, fmt);
+	str_len = vsnprintf(NULL, 0, fmt, ap);
+	va_end(ap);
+
+	if (str_len < 0) {
+		warning("vsnprintf failed: '%s'", strerror(errno));
+		return;
+	}
+
+	str = malloc(str_len + 1);
+	va_start(ap, fmt);
+	vsnprintf(str, str_len + 1, fmt, ap);
+	va_end(ap);
+	strbuf_expand(&sb, str, strbuf_expand_dict_cb, &dict);
+	free(str);
+
+	switch (priority) {
+	case LOG_EMERG:
+	case LOG_ALERT:
+	case LOG_CRIT:
+	case LOG_ERR:
+		logtype = EVENTLOG_ERROR_TYPE;
+		break;
+
+	case LOG_WARNING:
+		logtype = EVENTLOG_WARNING_TYPE;
+		break;
+
+	case LOG_NOTICE:
+	case LOG_INFO:
+	case LOG_DEBUG:
+	default:
+		logtype = EVENTLOG_INFORMATION_TYPE;
+		break;
+	}
+
+	ReportEventA(ms_eventlog, logtype, 0, 0, NULL, 1, 0,
+	    (const char **)&sb.buf, NULL);
+
+	strbuf_release(&sb);
+}
diff --git a/compat/win32/syslog.h b/compat/win32/syslog.h
new file mode 100644
index 0000000..70daa7c
--- /dev/null
+++ b/compat/win32/syslog.h
@@ -0,0 +1,20 @@
+#ifndef SYSLOG_H
+#define SYSLOG_H
+
+#define LOG_PID     0x01
+
+#define LOG_EMERG   0
+#define LOG_ALERT   1
+#define LOG_CRIT    2
+#define LOG_ERR     3
+#define LOG_WARNING 4
+#define LOG_NOTICE  5
+#define LOG_INFO    6
+#define LOG_DEBUG   7
+
+#define LOG_DAEMON  (3<<3)
+
+void openlog(const char *ident, int logopt, int facility);
+void syslog(int priority, const char *fmt, ...);
+
+#endif /* SYSLOG_H */
diff --git a/daemon.c b/daemon.c
index 7ccd097..535ae88 100644
--- a/daemon.c
+++ b/daemon.c
@@ -5,8 +5,6 @@
 #include "strbuf.h"
 #include "string-list.h"
 
-#include <syslog.h>
-
 #ifndef HOST_NAME_MAX
 #define HOST_NAME_MAX 256
 #endif
diff --git a/git-compat-util.h b/git-compat-util.h
index 2af8d3e..e192831 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -104,6 +104,7 @@
 #include <assert.h>
 #include <regex.h>
 #include <utime.h>
+#include <syslog.h>
 #ifndef __MINGW32__
 #include <sys/wait.h>
 #include <sys/poll.h>
-- 
1.7.3.2.161.gd6e00

[PATCH v6 03/16] compat: add inet_pton and inet_ntop prototypes

[Erik Faye-Lund]

From: Mike Pape <dotzenlabs@gmail.com>

Windows doesn't have inet_pton and inet_ntop, so
add prototypes in git-compat-util.h for them.

At the same time include git-compat-util.h in
the sources for these functions, so they use the
network-wrappers from there on Windows.

Signed-off-by: Mike Pape <dotzenlabs@gmail.com>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
---
 Makefile           |    2 ++
 compat/inet_ntop.c |    6 +++---
 compat/inet_pton.c |    8 +++++---
 git-compat-util.h  |    8 ++++++++
 4 files changed, 18 insertions(+), 6 deletions(-)

diff --git a/Makefile b/Makefile
index d9d9419..2aa067a 100644
--- a/Makefile
+++ b/Makefile
@@ -1398,9 +1398,11 @@ endif
 endif
 ifdef NO_INET_NTOP
 	LIB_OBJS += compat/inet_ntop.o
+	BASIC_CFLAGS += -DNO_INET_NTOP
 endif
 ifdef NO_INET_PTON
 	LIB_OBJS += compat/inet_pton.o
+	BASIC_CFLAGS += -DNO_INET_PTON
 endif
 
 ifdef NO_ICONV
diff --git a/compat/inet_ntop.c b/compat/inet_ntop.c
index f444982..e5b46a0 100644
--- a/compat/inet_ntop.c
+++ b/compat/inet_ntop.c
@@ -17,9 +17,9 @@
 
 #include <errno.h>
 #include <sys/types.h>
-#include <sys/socket.h>
-#include <netinet/in.h>
-#include <arpa/inet.h>
+
+#include "../git-compat-util.h"
+
 #include <stdio.h>
 #include <string.h>
 
diff --git a/compat/inet_pton.c b/compat/inet_pton.c
index 4078fc0..2ec995e 100644
--- a/compat/inet_pton.c
+++ b/compat/inet_pton.c
@@ -17,9 +17,9 @@
 
 #include <errno.h>
 #include <sys/types.h>
-#include <sys/socket.h>
-#include <netinet/in.h>
-#include <arpa/inet.h>
+
+#include "../git-compat-util.h"
+
 #include <stdio.h>
 #include <string.h>
 
@@ -41,7 +41,9 @@
  */
 
 static int inet_pton4(const char *src, unsigned char *dst);
+#ifndef NO_IPV6
 static int inet_pton6(const char *src, unsigned char *dst);
+#endif
 
 /* int
  * inet_pton4(src, dst)
diff --git a/git-compat-util.h b/git-compat-util.h
index e192831..56dce85 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -387,6 +387,14 @@ static inline void *gitmempcpy(void *dest, const void *src, size_t n)
 }
 #endif
 
+#ifdef NO_INET_PTON
+int inet_pton(int af, const char *src, void *dst);
+#endif
+
+#ifdef NO_INET_NTOP
+const char *inet_ntop(int af, const void *src, char *dst, size_t size);
+#endif
+
 extern void release_pack_memory(size_t, int);
 
 typedef void (*try_to_free_t)(size_t);
-- 
1.7.3.2.161.gd6e00

[PATCH v6 04/16] inet_ntop: fix a couple of old-style decls

[Erik Faye-Lund]

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
---
 compat/inet_ntop.c |   16 +++-------------
 1 files changed, 3 insertions(+), 13 deletions(-)

diff --git a/compat/inet_ntop.c b/compat/inet_ntop.c
index e5b46a0..ea249c6 100644
--- a/compat/inet_ntop.c
+++ b/compat/inet_ntop.c
@@ -50,10 +50,7 @@
  *	Paul Vixie, 1996.
  */
 static const char *
-inet_ntop4(src, dst, size)
-	const u_char *src;
-	char *dst;
-	size_t size;
+inet_ntop4(const u_char *src, char *dst, size_t size)
 {
 	static const char fmt[] = "%u.%u.%u.%u";
 	char tmp[sizeof "255.255.255.255"];
@@ -78,10 +75,7 @@ inet_ntop4(src, dst, size)
  *	Paul Vixie, 1996.
  */
 static const char *
-inet_ntop6(src, dst, size)
-	const u_char *src;
-	char *dst;
-	size_t size;
+inet_ntop6(const u_char *src, char *dst, size_t size)
 {
 	/*
 	 * Note that int32_t and int16_t need only be "at least" large enough
@@ -178,11 +172,7 @@ inet_ntop6(src, dst, size)
  *	Paul Vixie, 1996.
  */
 const char *
-inet_ntop(af, src, dst, size)
-	int af;
-	const void *src;
-	char *dst;
-	size_t size;
+inet_ntop(int af, const void *src, char *dst, size_t size)
 {
 	switch (af) {
 	case AF_INET:
-- 
1.7.3.2.161.gd6e00

[PATCH v6 05/16] mingw: use real pid

[Erik Faye-Lund]

The Windows port have so far been using process handles in place
of PID. However, this is not work consistent with what getpid
returns.

PIDs are system-global identifiers, but process handles are local
to a process. Using PIDs instead of process handles allows, for
instance, a user to kill a hung process with the Task Manager,
something that would have been impossible with process handles.

Change the code to use the real PID, and use OpenProcess to get a
process-handle. Store the PID and the process handle in a table
protected by a critical section, so we can safely close the
process handle later.

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
---
 compat/mingw.c |   70 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 compat/mingw.h |   10 ++-----
 2 files changed, 72 insertions(+), 8 deletions(-)

diff --git a/compat/mingw.c b/compat/mingw.c
index 701a555..e2e3c54 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -702,6 +702,13 @@ static int env_compare(const void *a, const void *b)
 	return strcasecmp(*ea, *eb);
 }
 
+struct {
+	pid_t pid;
+	HANDLE proc;
+} *pinfo;
+static int num_pinfo;
+CRITICAL_SECTION pinfo_cs;
+
 static pid_t mingw_spawnve_fd(const char *cmd, const char **argv, char **env,
 			      const char *dir,
 			      int prepend_cmd, int fhin, int fhout, int fherr)
@@ -794,7 +801,23 @@ static pid_t mingw_spawnve_fd(const char *cmd, const char **argv, char **env,
 		return -1;
 	}
 	CloseHandle(pi.hThread);
-	return (pid_t)pi.hProcess;
+
+	/*
+	 * The process ID is the human-readable identifier of the process
+	 * that we want to present in log and error messages. The handle
+	 * is not useful for this purpose. But we cannot close it, either,
+	 * because it is not possible to turn a process ID into a process
+	 * handle after the process terminated.
+	 * Keep the handle in a list for waitpid.
+	 */
+	EnterCriticalSection(&pinfo_cs);
+	num_pinfo++;
+	pinfo = xrealloc(pinfo, sizeof(*pinfo) * num_pinfo);
+	pinfo[num_pinfo - 1].pid = pi.dwProcessId;
+	pinfo[num_pinfo - 1].proc = pi.hProcess;
+	LeaveCriticalSection(&pinfo_cs);
+
+	return (pid_t)pi.dwProcessId;
 }
 
 static pid_t mingw_spawnve(const char *cmd, const char **argv, char **env,
@@ -1518,6 +1541,51 @@ char *getpass(const char *prompt)
 	return strbuf_detach(&buf, NULL);
 }
 
+pid_t waitpid(pid_t pid, int *status, unsigned options)
+{
+	HANDLE h = OpenProcess(SYNCHRONIZE | PROCESS_QUERY_INFORMATION,
+	    FALSE, pid);
+	if (!h) {
+		errno = ECHILD;
+		return -1;
+	}
+
+	if (options == 0) {
+		int i;
+		if (WaitForSingleObject(h, INFINITE) != WAIT_OBJECT_0) {
+			CloseHandle(h);
+			return 0;
+		}
+
+		if (status)
+			GetExitCodeProcess(h, (LPDWORD)status);
+
+		EnterCriticalSection(&pinfo_cs);
+
+		for (i = 0; i < num_pinfo; ++i)
+			if (pinfo[i].pid == pid)
+				break;
+
+		if (i < num_pinfo) {
+			CloseHandle(pinfo[i].proc);
+			memmove(pinfo + i, pinfo + i + 1,
+			    sizeof(*pinfo) * (num_pinfo - i - 1));
+			num_pinfo--;
+			pinfo = xrealloc(pinfo,
+			    sizeof(*pinfo) * num_pinfo);
+		}
+
+		LeaveCriticalSection(&pinfo_cs);
+
+		CloseHandle(h);
+		return pid;
+	}
+	CloseHandle(h);
+
+	errno = EINVAL;
+	return -1;
+}
+
 #ifndef NO_MINGW_REPLACE_READDIR
 /* MinGW readdir implementation to avoid extra lstats for Git */
 struct mingw_DIR
diff --git a/compat/mingw.h b/compat/mingw.h
index a5bde82..7c4eeea 100644
--- a/compat/mingw.h
+++ b/compat/mingw.h
@@ -140,13 +140,7 @@ static inline int mingw_unlink(const char *pathname)
 }
 #define unlink mingw_unlink
 
-static inline pid_t waitpid(pid_t pid, int *status, unsigned options)
-{
-	if (options == 0)
-		return _cwait(status, pid, 0);
-	errno = EINVAL;
-	return -1;
-}
+pid_t waitpid(pid_t pid, int *status, unsigned options);
 
 #ifndef NO_OPENSSL
 #include <openssl/ssl.h>
@@ -321,11 +315,13 @@ void free_environ(char **env);
 static int mingw_main(); \
 int main(int argc, const char **argv) \
 { \
+	extern CRITICAL_SECTION pinfo_cs; \
 	_fmode = _O_BINARY; \
 	_setmode(_fileno(stdin), _O_BINARY); \
 	_setmode(_fileno(stdout), _O_BINARY); \
 	_setmode(_fileno(stderr), _O_BINARY); \
 	argv[0] = xstrdup(_pgmptr); \
+	InitializeCriticalSection(&pinfo_cs); \
 	return mingw_main(argc, argv); \
 } \
 static int mingw_main(c,v)
-- 
1.7.3.2.161.gd6e00

[PATCH v6 06/16] mingw: support waitpid with pid > 0 and WNOHANG

[Erik Faye-Lund]

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
---
 compat/mingw.c |    6 ++++++
 compat/mingw.h |    1 +
 2 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/compat/mingw.c b/compat/mingw.c
index e2e3c54..2e7c644 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -1550,6 +1550,12 @@ pid_t waitpid(pid_t pid, int *status, unsigned options)
 		return -1;
 	}
 
+	if (pid > 0 && options & WNOHANG) {
+		if (WAIT_OBJECT_0 != WaitForSingleObject((HANDLE)pid, 0))
+			return 0;
+		options &= ~WNOHANG;
+	}
+
 	if (options == 0) {
 		int i;
 		if (WaitForSingleObject(h, INFINITE) != WAIT_OBJECT_0) {
diff --git a/compat/mingw.h b/compat/mingw.h
index 7c4eeea..379d7bf 100644
--- a/compat/mingw.h
+++ b/compat/mingw.h
@@ -140,6 +140,7 @@ static inline int mingw_unlink(const char *pathname)
 }
 #define unlink mingw_unlink
 
+#define WNOHANG 1
 pid_t waitpid(pid_t pid, int *status, unsigned options);
 
 #ifndef NO_OPENSSL
-- 
1.7.3.2.161.gd6e00

[PATCH v6 07/16] mingw: add kill emulation

[Erik Faye-Lund]

This is a quite limited kill-emulation; it can only handle
SIGTERM on positive pids. However, it's enough for git-daemon.

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
---
 compat/mingw.c |   19 +++++++++++++++++++
 compat/mingw.h |    3 +++
 2 files changed, 22 insertions(+), 0 deletions(-)

diff --git a/compat/mingw.c b/compat/mingw.c
index 2e7c644..21d1c2c 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -932,6 +932,25 @@ void mingw_execv(const char *cmd, char *const *argv)
 	mingw_execve(cmd, argv, environ);
 }
 
+int mingw_kill(pid_t pid, int sig)
+{
+	if (pid > 0 && sig == SIGTERM) {
+		HANDLE h = OpenProcess(PROCESS_TERMINATE, FALSE, pid);
+
+		if (TerminateProcess(h, -1)) {
+			CloseHandle(h);
+			return 0;
+		}
+
+		errno = err_win_to_posix(GetLastError());
+		CloseHandle(h);
+		return -1;
+	}
+
+	errno = EINVAL;
+	return -1;
+}
+
 static char **copy_environ(void)
 {
 	char **env;
diff --git a/compat/mingw.h b/compat/mingw.h
index 379d7bf..51fca2f 100644
--- a/compat/mingw.h
+++ b/compat/mingw.h
@@ -143,6 +143,9 @@ static inline int mingw_unlink(const char *pathname)
 #define WNOHANG 1
 pid_t waitpid(pid_t pid, int *status, unsigned options);
 
+#define kill mingw_kill
+int mingw_kill(pid_t pid, int sig);
+
 #ifndef NO_OPENSSL
 #include <openssl/ssl.h>
 static inline int mingw_SSL_set_fd(SSL *ssl, int fd)
-- 
1.7.3.2.161.gd6e00

[PATCH v6 08/16] daemon: use run-command api for async serving

[Erik Faye-Lund]

fork() is only available on POSIX, so to support git-daemon
on Windows we have to use something else.

Instead we invent the flag --serve, which is a stripped down
version of --inetd-mode. We use start_command() to call
git-daemon with this flag appended to serve clients.

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
---
 daemon.c |   87 +++++++++++++++++++++++++++++++------------------------------
 1 files changed, 44 insertions(+), 43 deletions(-)

diff --git a/daemon.c b/daemon.c
index 535ae88..6eee570 100644
--- a/daemon.c
+++ b/daemon.c
@@ -614,17 +614,17 @@ static unsigned int live_children;
 
 static struct child {
 	struct child *next;
-	pid_t pid;
+	struct child_process cld;
 	struct sockaddr_storage address;
 } *firstborn;
 
-static void add_child(pid_t pid, struct sockaddr *addr, int addrlen)
+static void add_child(struct child_process *cld, struct sockaddr *addr, int addrlen)
 {
 	struct child *newborn, **cradle;
 
 	newborn = xcalloc(1, sizeof(*newborn));
 	live_children++;
-	newborn->pid = pid;
+	memcpy(&newborn->cld, cld, sizeof(*cld));
 	memcpy(&newborn->address, addr, addrlen);
 	for (cradle = &firstborn; *cradle; cradle = &(*cradle)->next)
 		if (!addrcmp(&(*cradle)->address, &newborn->address))
@@ -633,19 +633,6 @@ static void add_child(pid_t pid, struct sockaddr *addr, int addrlen)
 	*cradle = newborn;
 }
 
-static void remove_child(pid_t pid)
-{
-	struct child **cradle, *blanket;
-
-	for (cradle = &firstborn; (blanket = *cradle); cradle = &blanket->next)
-		if (blanket->pid == pid) {
-			*cradle = blanket->next;
-			live_children--;
-			free(blanket);
-			break;
-		}
-}
-
 /*
  * This gets called if the number of connections grows
  * past "max_connections".
@@ -661,7 +648,7 @@ static void kill_some_child(void)
 
 	for (; (next = blanket->next); blanket = next)
 		if (!addrcmp(&blanket->address, &next->address)) {
-			kill(blanket->pid, SIGTERM);
+			kill(blanket->cld.pid, SIGTERM);
 			break;
 		}
 }
@@ -671,18 +658,26 @@ static void check_dead_children(void)
 	int status;
 	pid_t pid;
 
-	while ((pid = waitpid(-1, &status, WNOHANG)) > 0) {
-		const char *dead = "";
-		remove_child(pid);
-		if (!WIFEXITED(status) || (WEXITSTATUS(status) > 0))
-			dead = " (with error)";
-		loginfo("[%"PRIuMAX"] Disconnected%s", (uintmax_t)pid, dead);
-	}
+	struct child **cradle, *blanket;
+	for (cradle = &firstborn; (blanket = *cradle);)
+		if ((pid = waitpid(blanket->cld.pid, &status, WNOHANG)) > 1) {
+			const char *dead = "";
+			if (status)
+				dead = " (with error)";
+			loginfo("[%"PRIuMAX"] Disconnected%s", (uintmax_t)pid, dead);
+
+			/* remove the child */
+			*cradle = blanket->next;
+			live_children--;
+			free(blanket);
+		} else
+			cradle = &blanket->next;
 }
 
+static char **cld_argv;
 static void handle(int incoming, struct sockaddr *addr, int addrlen)
 {
-	pid_t pid;
+	struct child_process cld = { 0 };
 
 	if (max_connections && live_children >= max_connections) {
 		kill_some_child();
@@ -695,22 +690,15 @@ static void handle(int incoming, struct sockaddr *addr, int addrlen)
 		}
 	}
 
-	if ((pid = fork())) {
-		close(incoming);
-		if (pid < 0) {
-			logerror("Couldn't fork %s", strerror(errno));
-			return;
-		}
-
-		add_child(pid, addr, addrlen);
-		return;
-	}
+	cld.argv = (const char **)cld_argv;
+	cld.in = incoming;
+	cld.out = dup(incoming);
 
-	dup2(incoming, 0);
-	dup2(incoming, 1);
+	if (start_command(&cld))
+		logerror("unable to fork");
+	else
+		add_child(&cld, addr, addrlen);
 	close(incoming);
-
-	exit(execute(addr));
 }
 
 static void child_handler(int signo)
@@ -991,7 +979,7 @@ int main(int argc, char **argv)
 {
 	int listen_port = 0;
 	struct string_list listen_addr = STRING_LIST_INIT_NODUP;
-	int inetd_mode = 0;
+	int serve_mode = 0, inetd_mode = 0;
 	const char *pid_file = NULL, *user_name = NULL, *group_name = NULL;
 	int detach = 0;
 	struct passwd *pass = NULL;
@@ -1017,6 +1005,10 @@ int main(int argc, char **argv)
 				continue;
 			}
 		}
+		if (!strcmp(arg, "--serve")) {
+			serve_mode = 1;
+			continue;
+		}
 		if (!strcmp(arg, "--inetd")) {
 			inetd_mode = 1;
 			log_syslog = 1;
@@ -1162,13 +1154,15 @@ int main(int argc, char **argv)
 		    base_path);
 
 	if (inetd_mode) {
+		if (!freopen("/dev/null", "w", stderr))
+			die_errno("failed to redirect stderr to /dev/null");
+	}
+
+	if (inetd_mode || serve_mode) {
 		struct sockaddr_storage ss;
 		struct sockaddr *peer = (struct sockaddr *)&ss;
 		socklen_t slen = sizeof(ss);
 
-		if (!freopen("/dev/null", "w", stderr))
-			die_errno("failed to redirect stderr to /dev/null");
-
 		if (getpeername(0, peer, &slen))
 			peer = NULL;
 
@@ -1185,5 +1179,12 @@ int main(int argc, char **argv)
 	if (pid_file)
 		store_pid(pid_file);
 
+	/* prepare argv for serving-processes */
+	cld_argv = xmalloc(sizeof (char *) * (argc + 2));
+	for (i = 0; i < argc; ++i)
+		cld_argv[i] = argv[i];
+	cld_argv[argc] = "--serve";
+	cld_argv[argc+1] = NULL;
+
 	return serve(&listen_addr, listen_port, pass, gid);
 }
-- 
1.7.3.2.161.gd6e00

[PATCH v6 09/16] daemon: use full buffered mode for stderr

[Erik Faye-Lund]

Windows doesn't support line buffered mode for file
streams, so let's just use full buffered mode with
a big buffer ("4096 should be enough for everyone")
and add explicit flushing.

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
---
 daemon.c |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/daemon.c b/daemon.c
index 6eee570..9a4884a 100644
--- a/daemon.c
+++ b/daemon.c
@@ -67,12 +67,14 @@ static void logreport(int priority, const char *err, va_list params)
 		syslog(priority, "%s", buf);
 	} else {
 		/*
-		 * Since stderr is set to linebuffered mode, the
+		 * Since stderr is set to buffered mode, the
 		 * logging of different processes will not overlap
+		 * unless they overflow the (rather big) buffers.
 		 */
 		fprintf(stderr, "[%"PRIuMAX"] ", (uintmax_t)getpid());
 		vfprintf(stderr, err, params);
 		fputc('\n', stderr);
+		fflush(stderr);
 	}
 }
 
@@ -1117,7 +1119,7 @@ int main(int argc, char **argv)
 		set_die_routine(daemon_die);
 	} else
 		/* avoid splitting a message in the middle */
-		setvbuf(stderr, NULL, _IOLBF, 0);
+		setvbuf(stderr, NULL, _IOFBF, 4096);
 
 	if (inetd_mode && (group_name || user_name))
 		die("--user and --group are incompatible with --inetd");
-- 
1.7.3.2.161.gd6e00

[PATCH v6 10/16] Improve the mingw getaddrinfo stub to handle more use cases

[Erik Faye-Lund]

From: Martin Storsjö <martin@martin.st>

Allow the node parameter to be null, which is used for getting
the default bind address.

Also allow the hints parameter to be null, to improve standard
conformance of the stub implementation a little.

Signed-off-by: Martin Storsjo <martin@martin.st>
---
 compat/mingw.c |   28 +++++++++++++++++++++-------
 1 files changed, 21 insertions(+), 7 deletions(-)

diff --git a/compat/mingw.c b/compat/mingw.c
index 21d1c2c..d88c0d0 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -1035,19 +1035,22 @@ static int WSAAPI getaddrinfo_stub(const char *node, const char *service,
 				   const struct addrinfo *hints,
 				   struct addrinfo **res)
 {
-	struct hostent *h = gethostbyname(node);
+	struct hostent *h = NULL;
 	struct addrinfo *ai;
 	struct sockaddr_in *sin;
 
-	if (!h)
-		return WSAGetLastError();
+	if (node) {
+		h = gethostbyname(node);
+		if (!h)
+			return WSAGetLastError();
+	}
 
 	ai = xmalloc(sizeof(struct addrinfo));
 	*res = ai;
 	ai->ai_flags = 0;
 	ai->ai_family = AF_INET;
-	ai->ai_socktype = hints->ai_socktype;
-	switch (hints->ai_socktype) {
+	ai->ai_socktype = hints ? hints->ai_socktype : 0;
+	switch (ai->ai_socktype) {
 	case SOCK_STREAM:
 		ai->ai_protocol = IPPROTO_TCP;
 		break;
@@ -1059,14 +1062,25 @@ static int WSAAPI getaddrinfo_stub(const char *node, const char *service,
 		break;
 	}
 	ai->ai_addrlen = sizeof(struct sockaddr_in);
-	ai->ai_canonname = strdup(h->h_name);
+	if (hints && (hints->ai_flags & AI_CANONNAME))
+		ai->ai_canonname = h ? strdup(h->h_name) : NULL;
+	else
+		ai->ai_canonname = NULL;
 
 	sin = xmalloc(ai->ai_addrlen);
 	memset(sin, 0, ai->ai_addrlen);
 	sin->sin_family = AF_INET;
+	/* Note: getaddrinfo is supposed to allow service to be a string,
+	 * which should be looked up using getservbyname. This is
+	 * currently not implemented */
 	if (service)
 		sin->sin_port = htons(atoi(service));
-	sin->sin_addr = *(struct in_addr *)h->h_addr;
+	if (h)
+		sin->sin_addr = *(struct in_addr *)h->h_addr;
+	else if (hints && (hints->ai_flags & AI_PASSIVE))
+		sin->sin_addr.s_addr = INADDR_ANY;
+	else
+		sin->sin_addr.s_addr = INADDR_LOOPBACK;
 	ai->ai_addr = (struct sockaddr *)sin;
 	ai->ai_next = 0;
 	return 0;
-- 
1.7.3.2.161.gd6e00

[PATCH v6 11/16] daemon: get remote host address from root-process

[Erik Faye-Lund]

Get remote host in the process that accept() and pass it through
the REMOTE_ADDR environment variable to the handler-process.
Introduce the REMOTE_PORT environmen variable for the port.

Use these variables for reporting instead of doing
getpeername(0, ...), which doesn't work on Windows.

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
---
 daemon.c |   67 +++++++++++++++++++++++++++----------------------------------
 1 files changed, 30 insertions(+), 37 deletions(-)

diff --git a/daemon.c b/daemon.c
index 9a4884a..3d8f6e6 100644
--- a/daemon.c
+++ b/daemon.c
@@ -516,37 +516,14 @@ static void parse_host_arg(char *extra_args, int buflen)
 }
 
 
-static int execute(struct sockaddr *addr)
+static int execute(void)
 {
 	static char line[1000];
 	int pktlen, len, i;
+	char *addr = getenv("REMOTE_ADDR"), *port = getenv("REMOTE_PORT");
 
-	if (addr) {
-		char addrbuf[256] = "";
-		int port = -1;
-
-		if (addr->sa_family == AF_INET) {
-			struct sockaddr_in *sin_addr = (void *) addr;
-			inet_ntop(addr->sa_family, &sin_addr->sin_addr, addrbuf, sizeof(addrbuf));
-			port = ntohs(sin_addr->sin_port);
-#ifndef NO_IPV6
-		} else if (addr && addr->sa_family == AF_INET6) {
-			struct sockaddr_in6 *sin6_addr = (void *) addr;
-
-			char *buf = addrbuf;
-			*buf++ = '['; *buf = '\0'; /* stpcpy() is cool */
-			inet_ntop(AF_INET6, &sin6_addr->sin6_addr, buf, sizeof(addrbuf) - 1);
-			strcat(buf, "]");
-
-			port = ntohs(sin6_addr->sin6_port);
-#endif
-		}
-		loginfo("Connection from %s:%d", addrbuf, port);
-		setenv("REMOTE_ADDR", addrbuf, 1);
-	}
-	else {
-		unsetenv("REMOTE_ADDR");
-	}
+	if (addr)
+		loginfo("Connection from %s:%s", addr, port);
 
 	alarm(init_timeout ? init_timeout : timeout);
 	pktlen = packet_read_line(0, line, sizeof(line));
@@ -680,6 +657,8 @@ static char **cld_argv;
 static void handle(int incoming, struct sockaddr *addr, int addrlen)
 {
 	struct child_process cld = { 0 };
+	char addrbuf[300] = "REMOTE_ADDR=", portbuf[300];
+	char *env[] = { addrbuf, portbuf, NULL };
 
 	if (max_connections && live_children >= max_connections) {
 		kill_some_child();
@@ -692,6 +671,28 @@ static void handle(int incoming, struct sockaddr *addr, int addrlen)
 		}
 	}
 
+	if (addr->sa_family == AF_INET) {
+		struct sockaddr_in *sin_addr = (void *) addr;
+		inet_ntop(addr->sa_family, &sin_addr->sin_addr, addrbuf + 12,
+		    sizeof(addrbuf) - 12);
+		snprintf(portbuf, sizeof(portbuf), "REMOTE_PORT=%d",
+		    ntohs(sin_addr->sin_port));
+#ifndef NO_IPV6
+	} else if (addr && addr->sa_family == AF_INET6) {
+		struct sockaddr_in6 *sin6_addr = (void *) addr;
+
+		char *buf = addrbuf + 12;
+		*buf++ = '['; *buf = '\0'; /* stpcpy() is cool */
+		inet_ntop(AF_INET6, &sin6_addr->sin6_addr, buf,
+		    sizeof(addrbuf) - 13);
+		strcat(buf, "]");
+
+		snprintf(portbuf, sizeof(portbuf), "REMOTE_PORT=%d",
+		    ntohs(sin6_addr->sin6_port));
+#endif
+	}
+
+	cld.env = (const char **)env;
 	cld.argv = (const char **)cld_argv;
 	cld.in = incoming;
 	cld.out = dup(incoming);
@@ -1160,16 +1161,8 @@ int main(int argc, char **argv)
 			die_errno("failed to redirect stderr to /dev/null");
 	}
 
-	if (inetd_mode || serve_mode) {
-		struct sockaddr_storage ss;
-		struct sockaddr *peer = (struct sockaddr *)&ss;
-		socklen_t slen = sizeof(ss);
-
-		if (getpeername(0, peer, &slen))
-			peer = NULL;
-
-		return execute(peer);
-	}
+	if (inetd_mode || serve_mode)
+		return execute();
 
 	if (detach) {
 		daemonize();
-- 
1.7.3.2.161.gd6e00

[PATCH v6 12/16] mingw: import poll-emulation from gnulib

[Erik Faye-Lund]

copy lib/poll.c and lib/poll.in.h verbatim from commit 0a05120 in
git://git.savannah.gnu.org/gnulib.git to compat/win32/sys/poll.[ch]

To upgrade this code in the future, branch out from this commit, copy
new versions of the files above on top, and merge back the result.

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
---
 compat/win32/sys/poll.c |  597 +++++++++++++++++++++++++++++++++++++++++++++++
 compat/win32/sys/poll.h |   53 +++++
 2 files changed, 650 insertions(+), 0 deletions(-)
 create mode 100644 compat/win32/sys/poll.c
 create mode 100644 compat/win32/sys/poll.h

diff --git a/compat/win32/sys/poll.c b/compat/win32/sys/poll.c
new file mode 100644
index 0000000..7c52cb6
--- /dev/null
+++ b/compat/win32/sys/poll.c
@@ -0,0 +1,597 @@
+/* Emulation for poll(2)
+   Contributed by Paolo Bonzini.
+
+   Copyright 2001-2003, 2006-2010 Free Software Foundation, Inc.
+
+   This file is part of gnulib.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 2, or (at your option)
+   any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License along
+   with this program; if not, write to the Free Software Foundation,
+   Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.  */
+
+/* Tell gcc not to warn about the (nfd < 0) tests, below.  */
+#if (__GNUC__ == 4 && 3 <= __GNUC_MINOR__) || 4 < __GNUC__
+# pragma GCC diagnostic ignored "-Wtype-limits"
+#endif
+
+#include <config.h>
+#include <alloca.h>
+
+#include <sys/types.h>
+#include "poll.h"
+#include <errno.h>
+#include <limits.h>
+#include <assert.h>
+
+#if (defined _WIN32 || defined __WIN32__) && ! defined __CYGWIN__
+# define WIN32_NATIVE
+# include <winsock2.h>
+# include <windows.h>
+# include <io.h>
+# include <stdio.h>
+# include <conio.h>
+#else
+# include <sys/time.h>
+# include <sys/socket.h>
+# include <sys/select.h>
+# include <unistd.h>
+#endif
+
+#ifdef HAVE_SYS_IOCTL_H
+# include <sys/ioctl.h>
+#endif
+#ifdef HAVE_SYS_FILIO_H
+# include <sys/filio.h>
+#endif
+
+#include <time.h>
+
+#ifndef INFTIM
+# define INFTIM (-1)
+#endif
+
+/* BeOS does not have MSG_PEEK.  */
+#ifndef MSG_PEEK
+# define MSG_PEEK 0
+#endif
+
+#ifdef WIN32_NATIVE
+
+#define IsConsoleHandle(h) (((long) (h) & 3) == 3)
+
+static BOOL
+IsSocketHandle (HANDLE h)
+{
+  WSANETWORKEVENTS ev;
+
+  if (IsConsoleHandle (h))
+    return FALSE;
+
+  /* Under Wine, it seems that getsockopt returns 0 for pipes too.
+     WSAEnumNetworkEvents instead distinguishes the two correctly.  */
+  ev.lNetworkEvents = 0xDEADBEEF;
+  WSAEnumNetworkEvents ((SOCKET) h, NULL, &ev);
+  return ev.lNetworkEvents != 0xDEADBEEF;
+}
+
+/* Declare data structures for ntdll functions.  */
+typedef struct _FILE_PIPE_LOCAL_INFORMATION {
+  ULONG NamedPipeType;
+  ULONG NamedPipeConfiguration;
+  ULONG MaximumInstances;
+  ULONG CurrentInstances;
+  ULONG InboundQuota;
+  ULONG ReadDataAvailable;
+  ULONG OutboundQuota;
+  ULONG WriteQuotaAvailable;
+  ULONG NamedPipeState;
+  ULONG NamedPipeEnd;
+} FILE_PIPE_LOCAL_INFORMATION, *PFILE_PIPE_LOCAL_INFORMATION;
+
+typedef struct _IO_STATUS_BLOCK
+{
+  union {
+    DWORD Status;
+    PVOID Pointer;
+  } u;
+  ULONG_PTR Information;
+} IO_STATUS_BLOCK, *PIO_STATUS_BLOCK;
+
+typedef enum _FILE_INFORMATION_CLASS {
+  FilePipeLocalInformation = 24
+} FILE_INFORMATION_CLASS, *PFILE_INFORMATION_CLASS;
+
+typedef DWORD (WINAPI *PNtQueryInformationFile)
+         (HANDLE, IO_STATUS_BLOCK *, VOID *, ULONG, FILE_INFORMATION_CLASS);
+
+# ifndef PIPE_BUF
+#  define PIPE_BUF      512
+# endif
+
+/* Compute revents values for file handle H.  If some events cannot happen
+   for the handle, eliminate them from *P_SOUGHT.  */
+
+static int
+win32_compute_revents (HANDLE h, int *p_sought)
+{
+  int i, ret, happened;
+  INPUT_RECORD *irbuffer;
+  DWORD avail, nbuffer;
+  BOOL bRet;
+  IO_STATUS_BLOCK iosb;
+  FILE_PIPE_LOCAL_INFORMATION fpli;
+  static PNtQueryInformationFile NtQueryInformationFile;
+  static BOOL once_only;
+
+  switch (GetFileType (h))
+    {
+    case FILE_TYPE_PIPE:
+      if (!once_only)
+        {
+          NtQueryInformationFile = (PNtQueryInformationFile)
+            GetProcAddress (GetModuleHandle ("ntdll.dll"),
+                            "NtQueryInformationFile");
+          once_only = TRUE;
+        }
+
+      happened = 0;
+      if (PeekNamedPipe (h, NULL, 0, NULL, &avail, NULL) != 0)
+        {
+          if (avail)
+            happened |= *p_sought & (POLLIN | POLLRDNORM);
+        }
+      else if (GetLastError () == ERROR_BROKEN_PIPE)
+        happened |= POLLHUP;
+
+      else
+        {
+          /* It was the write-end of the pipe.  Check if it is writable.
+             If NtQueryInformationFile fails, optimistically assume the pipe is
+             writable.  This could happen on Win9x, where NtQueryInformationFile
+             is not available, or if we inherit a pipe that doesn't permit
+             FILE_READ_ATTRIBUTES access on the write end (I think this should
+             not happen since WinXP SP2; WINE seems fine too).  Otherwise,
+             ensure that enough space is available for atomic writes.  */
+          memset (&iosb, 0, sizeof (iosb));
+          memset (&fpli, 0, sizeof (fpli));
+
+          if (!NtQueryInformationFile
+              || NtQueryInformationFile (h, &iosb, &fpli, sizeof (fpli),
+                                         FilePipeLocalInformation)
+              || fpli.WriteQuotaAvailable >= PIPE_BUF
+              || (fpli.OutboundQuota < PIPE_BUF &&
+                  fpli.WriteQuotaAvailable == fpli.OutboundQuota))
+            happened |= *p_sought & (POLLOUT | POLLWRNORM | POLLWRBAND);
+        }
+      return happened;
+
+    case FILE_TYPE_CHAR:
+      ret = WaitForSingleObject (h, 0);
+      if (!IsConsoleHandle (h))
+        return ret == WAIT_OBJECT_0 ? *p_sought & ~(POLLPRI | POLLRDBAND) : 0;
+
+      nbuffer = avail = 0;
+      bRet = GetNumberOfConsoleInputEvents (h, &nbuffer);
+      if (bRet)
+        {
+          /* Input buffer.  */
+          *p_sought &= POLLIN | POLLRDNORM;
+          if (nbuffer == 0)
+            return POLLHUP;
+          if (!*p_sought)
+            return 0;
+
+          irbuffer = (INPUT_RECORD *) alloca (nbuffer * sizeof (INPUT_RECORD));
+          bRet = PeekConsoleInput (h, irbuffer, nbuffer, &avail);
+          if (!bRet || avail == 0)
+            return POLLHUP;
+
+          for (i = 0; i < avail; i++)
+            if (irbuffer[i].EventType == KEY_EVENT)
+              return *p_sought;
+          return 0;
+        }
+      else
+        {
+          /* Screen buffer.  */
+          *p_sought &= POLLOUT | POLLWRNORM | POLLWRBAND;
+          return *p_sought;
+        }
+
+    default:
+      ret = WaitForSingleObject (h, 0);
+      if (ret == WAIT_OBJECT_0)
+        return *p_sought & ~(POLLPRI | POLLRDBAND);
+
+      return *p_sought & (POLLOUT | POLLWRNORM | POLLWRBAND);
+    }
+}
+
+/* Convert fd_sets returned by select into revents values.  */
+
+static int
+win32_compute_revents_socket (SOCKET h, int sought, long lNetworkEvents)
+{
+  int happened = 0;
+
+  if ((lNetworkEvents & (FD_READ | FD_ACCEPT | FD_CLOSE)) == FD_ACCEPT)
+    happened |= (POLLIN | POLLRDNORM) & sought;
+
+  else if (lNetworkEvents & (FD_READ | FD_ACCEPT | FD_CLOSE))
+    {
+      int r, error;
+
+      char data[64];
+      WSASetLastError (0);
+      r = recv (h, data, sizeof (data), MSG_PEEK);
+      error = WSAGetLastError ();
+      WSASetLastError (0);
+
+      if (r > 0 || error == WSAENOTCONN)
+        happened |= (POLLIN | POLLRDNORM) & sought;
+
+      /* Distinguish hung-up sockets from other errors.  */
+      else if (r == 0 || error == WSAESHUTDOWN || error == WSAECONNRESET
+               || error == WSAECONNABORTED || error == WSAENETRESET)
+        happened |= POLLHUP;
+
+      else
+        happened |= POLLERR;
+    }
+
+  if (lNetworkEvents & (FD_WRITE | FD_CONNECT))
+    happened |= (POLLOUT | POLLWRNORM | POLLWRBAND) & sought;
+
+  if (lNetworkEvents & FD_OOB)
+    happened |= (POLLPRI | POLLRDBAND) & sought;
+
+  return happened;
+}
+
+#else /* !MinGW */
+
+/* Convert select(2) returned fd_sets into poll(2) revents values.  */
+static int
+compute_revents (int fd, int sought, fd_set *rfds, fd_set *wfds, fd_set *efds)
+{
+  int happened = 0;
+  if (FD_ISSET (fd, rfds))
+    {
+      int r;
+      int socket_errno;
+
+# if defined __MACH__ && defined __APPLE__
+      /* There is a bug in Mac OS X that causes it to ignore MSG_PEEK
+         for some kinds of descriptors.  Detect if this descriptor is a
+         connected socket, a server socket, or something else using a
+         0-byte recv, and use ioctl(2) to detect POLLHUP.  */
+      r = recv (fd, NULL, 0, MSG_PEEK);
+      socket_errno = (r < 0) ? errno : 0;
+      if (r == 0 || socket_errno == ENOTSOCK)
+        ioctl (fd, FIONREAD, &r);
+# else
+      char data[64];
+      r = recv (fd, data, sizeof (data), MSG_PEEK);
+      socket_errno = (r < 0) ? errno : 0;
+# endif
+      if (r == 0)
+        happened |= POLLHUP;
+
+      /* If the event happened on an unconnected server socket,
+         that's fine. */
+      else if (r > 0 || ( /* (r == -1) && */ socket_errno == ENOTCONN))
+        happened |= (POLLIN | POLLRDNORM) & sought;
+
+      /* Distinguish hung-up sockets from other errors.  */
+      else if (socket_errno == ESHUTDOWN || socket_errno == ECONNRESET
+               || socket_errno == ECONNABORTED || socket_errno == ENETRESET)
+        happened |= POLLHUP;
+
+      else
+        happened |= POLLERR;
+    }
+
+  if (FD_ISSET (fd, wfds))
+    happened |= (POLLOUT | POLLWRNORM | POLLWRBAND) & sought;
+
+  if (FD_ISSET (fd, efds))
+    happened |= (POLLPRI | POLLRDBAND) & sought;
+
+  return happened;
+}
+#endif /* !MinGW */
+
+int
+poll (pfd, nfd, timeout)
+     struct pollfd *pfd;
+     nfds_t nfd;
+     int timeout;
+{
+#ifndef WIN32_NATIVE
+  fd_set rfds, wfds, efds;
+  struct timeval tv;
+  struct timeval *ptv;
+  int maxfd, rc;
+  nfds_t i;
+
+# ifdef _SC_OPEN_MAX
+  static int sc_open_max = -1;
+
+  if (nfd < 0
+      || (nfd > sc_open_max
+          && (sc_open_max != -1
+              || nfd > (sc_open_max = sysconf (_SC_OPEN_MAX)))))
+    {
+      errno = EINVAL;
+      return -1;
+    }
+# else /* !_SC_OPEN_MAX */
+#  ifdef OPEN_MAX
+  if (nfd < 0 || nfd > OPEN_MAX)
+    {
+      errno = EINVAL;
+      return -1;
+    }
+#  endif /* OPEN_MAX -- else, no check is needed */
+# endif /* !_SC_OPEN_MAX */
+
+  /* EFAULT is not necessary to implement, but let's do it in the
+     simplest case. */
+  if (!pfd)
+    {
+      errno = EFAULT;
+      return -1;
+    }
+
+  /* convert timeout number into a timeval structure */
+  if (timeout == 0)
+    {
+      ptv = &tv;
+      ptv->tv_sec = 0;
+      ptv->tv_usec = 0;
+    }
+  else if (timeout > 0)
+    {
+      ptv = &tv;
+      ptv->tv_sec = timeout / 1000;
+      ptv->tv_usec = (timeout % 1000) * 1000;
+    }
+  else if (timeout == INFTIM)
+    /* wait forever */
+    ptv = NULL;
+  else
+    {
+      errno = EINVAL;
+      return -1;
+    }
+
+  /* create fd sets and determine max fd */
+  maxfd = -1;
+  FD_ZERO (&rfds);
+  FD_ZERO (&wfds);
+  FD_ZERO (&efds);
+  for (i = 0; i < nfd; i++)
+    {
+      if (pfd[i].fd < 0)
+        continue;
+
+      if (pfd[i].events & (POLLIN | POLLRDNORM))
+        FD_SET (pfd[i].fd, &rfds);
+
+      /* see select(2): "the only exceptional condition detectable
+         is out-of-band data received on a socket", hence we push
+         POLLWRBAND events onto wfds instead of efds. */
+      if (pfd[i].events & (POLLOUT | POLLWRNORM | POLLWRBAND))
+        FD_SET (pfd[i].fd, &wfds);
+      if (pfd[i].events & (POLLPRI | POLLRDBAND))
+        FD_SET (pfd[i].fd, &efds);
+      if (pfd[i].fd >= maxfd
+          && (pfd[i].events & (POLLIN | POLLOUT | POLLPRI
+                               | POLLRDNORM | POLLRDBAND
+                               | POLLWRNORM | POLLWRBAND)))
+        {
+          maxfd = pfd[i].fd;
+          if (maxfd > FD_SETSIZE)
+            {
+              errno = EOVERFLOW;
+              return -1;
+            }
+        }
+    }
+
+  /* examine fd sets */
+  rc = select (maxfd + 1, &rfds, &wfds, &efds, ptv);
+  if (rc < 0)
+    return rc;
+
+  /* establish results */
+  rc = 0;
+  for (i = 0; i < nfd; i++)
+    if (pfd[i].fd < 0)
+      pfd[i].revents = 0;
+    else
+      {
+        int happened = compute_revents (pfd[i].fd, pfd[i].events,
+                                        &rfds, &wfds, &efds);
+        if (happened)
+          {
+            pfd[i].revents = happened;
+            rc++;
+          }
+      }
+
+  return rc;
+#else
+  static struct timeval tv0;
+  static HANDLE hEvent;
+  WSANETWORKEVENTS ev;
+  HANDLE h, handle_array[FD_SETSIZE + 2];
+  DWORD ret, wait_timeout, nhandles;
+  fd_set rfds, wfds, xfds;
+  BOOL poll_again;
+  MSG msg;
+  int rc = 0;
+  nfds_t i;
+
+  if (nfd < 0 || timeout < -1)
+    {
+      errno = EINVAL;
+      return -1;
+    }
+
+  if (!hEvent)
+    hEvent = CreateEvent (NULL, FALSE, FALSE, NULL);
+
+  handle_array[0] = hEvent;
+  nhandles = 1;
+  FD_ZERO (&rfds);
+  FD_ZERO (&wfds);
+  FD_ZERO (&xfds);
+
+  /* Classify socket handles and create fd sets. */
+  for (i = 0; i < nfd; i++)
+    {
+      int sought = pfd[i].events;
+      pfd[i].revents = 0;
+      if (pfd[i].fd < 0)
+        continue;
+      if (!(sought & (POLLIN | POLLRDNORM | POLLOUT | POLLWRNORM | POLLWRBAND
+                      | POLLPRI | POLLRDBAND)))
+        continue;
+
+      h = (HANDLE) _get_osfhandle (pfd[i].fd);
+      assert (h != NULL);
+      if (IsSocketHandle (h))
+        {
+          int requested = FD_CLOSE;
+
+          /* see above; socket handles are mapped onto select.  */
+          if (sought & (POLLIN | POLLRDNORM))
+            {
+              requested |= FD_READ | FD_ACCEPT;
+              FD_SET ((SOCKET) h, &rfds);
+            }
+          if (sought & (POLLOUT | POLLWRNORM | POLLWRBAND))
+            {
+              requested |= FD_WRITE | FD_CONNECT;
+              FD_SET ((SOCKET) h, &wfds);
+            }
+          if (sought & (POLLPRI | POLLRDBAND))
+            {
+              requested |= FD_OOB;
+              FD_SET ((SOCKET) h, &xfds);
+            }
+
+          if (requested)
+            WSAEventSelect ((SOCKET) h, hEvent, requested);
+        }
+      else
+        {
+          /* Poll now.  If we get an event, do not poll again.  Also,
+             screen buffer handles are waitable, and they'll block until
+             a character is available.  win32_compute_revents eliminates
+             bits for the "wrong" direction. */
+          pfd[i].revents = win32_compute_revents (h, &sought);
+          if (sought)
+            handle_array[nhandles++] = h;
+          if (pfd[i].revents)
+            timeout = 0;
+        }
+    }
+
+  if (select (0, &rfds, &wfds, &xfds, &tv0) > 0)
+    {
+      /* Do MsgWaitForMultipleObjects anyway to dispatch messages, but
+         no need to call select again.  */
+      poll_again = FALSE;
+      wait_timeout = 0;
+    }
+  else
+    {
+      poll_again = TRUE;
+      if (timeout == INFTIM)
+        wait_timeout = INFINITE;
+      else
+        wait_timeout = timeout;
+    }
+
+  for (;;)
+    {
+      ret = MsgWaitForMultipleObjects (nhandles, handle_array, FALSE,
+                                       wait_timeout, QS_ALLINPUT);
+
+      if (ret == WAIT_OBJECT_0 + nhandles)
+        {
+          /* new input of some other kind */
+          BOOL bRet;
+          while ((bRet = PeekMessage (&msg, NULL, 0, 0, PM_REMOVE)) != 0)
+            {
+              TranslateMessage (&msg);
+              DispatchMessage (&msg);
+            }
+        }
+      else
+        break;
+    }
+
+  if (poll_again)
+    select (0, &rfds, &wfds, &xfds, &tv0);
+
+  /* Place a sentinel at the end of the array.  */
+  handle_array[nhandles] = NULL;
+  nhandles = 1;
+  for (i = 0; i < nfd; i++)
+    {
+      int happened;
+
+      if (pfd[i].fd < 0)
+        continue;
+      if (!(pfd[i].events & (POLLIN | POLLRDNORM |
+                             POLLOUT | POLLWRNORM | POLLWRBAND)))
+        continue;
+
+      h = (HANDLE) _get_osfhandle (pfd[i].fd);
+      if (h != handle_array[nhandles])
+        {
+          /* It's a socket.  */
+          WSAEnumNetworkEvents ((SOCKET) h, NULL, &ev);
+          WSAEventSelect ((SOCKET) h, 0, 0);
+
+          /* If we're lucky, WSAEnumNetworkEvents already provided a way
+             to distinguish FD_READ and FD_ACCEPT; this saves a recv later.  */
+          if (FD_ISSET ((SOCKET) h, &rfds)
+              && !(ev.lNetworkEvents & (FD_READ | FD_ACCEPT)))
+            ev.lNetworkEvents |= FD_READ | FD_ACCEPT;
+          if (FD_ISSET ((SOCKET) h, &wfds))
+            ev.lNetworkEvents |= FD_WRITE | FD_CONNECT;
+          if (FD_ISSET ((SOCKET) h, &xfds))
+            ev.lNetworkEvents |= FD_OOB;
+
+          happened = win32_compute_revents_socket ((SOCKET) h, pfd[i].events,
+                                                   ev.lNetworkEvents);
+        }
+      else
+        {
+          /* Not a socket.  */
+          int sought = pfd[i].events;
+          happened = win32_compute_revents (h, &sought);
+          nhandles++;
+        }
+
+       if ((pfd[i].revents |= happened) != 0)
+        rc++;
+    }
+
+  return rc;
+#endif
+}
diff --git a/compat/win32/sys/poll.h b/compat/win32/sys/poll.h
new file mode 100644
index 0000000..b7aa59d
--- /dev/null
+++ b/compat/win32/sys/poll.h
@@ -0,0 +1,53 @@
+/* Header for poll(2) emulation
+   Contributed by Paolo Bonzini.
+
+   Copyright 2001, 2002, 2003, 2007, 2009, 2010 Free Software Foundation, Inc.
+
+   This file is part of gnulib.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 2, or (at your option)
+   any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License along
+   with this program; if not, write to the Free Software Foundation,
+   Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.  */
+
+#ifndef _GL_POLL_H
+#define _GL_POLL_H
+
+/* fake a poll(2) environment */
+#define POLLIN      0x0001      /* any readable data available   */
+#define POLLPRI     0x0002      /* OOB/Urgent readable data      */
+#define POLLOUT     0x0004      /* file descriptor is writeable  */
+#define POLLERR     0x0008      /* some poll error occurred      */
+#define POLLHUP     0x0010      /* file descriptor was "hung up" */
+#define POLLNVAL    0x0020      /* requested events "invalid"    */
+#define POLLRDNORM  0x0040
+#define POLLRDBAND  0x0080
+#define POLLWRNORM  0x0100
+#define POLLWRBAND  0x0200
+
+struct pollfd
+{
+  int fd;                       /* which file descriptor to poll */
+  short events;                 /* events we are interested in   */
+  short revents;                /* events found on return        */
+};
+
+typedef unsigned long nfds_t;
+
+extern int poll (struct pollfd *pfd, nfds_t nfd, int timeout);
+
+/* Define INFTIM only if doing so conforms to POSIX.  */
+#if !defined (_POSIX_C_SOURCE) && !defined (_XOPEN_SOURCE)
+#define INFTIM (-1)
+#endif
+
+#endif /* _GL_POLL_H */
-- 
1.7.3.2.161.gd6e00

[PATCH v6 13/16] mingw: use poll-emulation from gnulib

[Erik Faye-Lund]

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
---
 Makefile                |    6 +++-
 compat/mingw.c          |   65 -----------------------------------------------
 compat/mingw.h          |   11 --------
 compat/win32/sys/poll.c |    3 +-
 git-compat-util.h       |    2 +-
 5 files changed, 6 insertions(+), 81 deletions(-)

diff --git a/Makefile b/Makefile
index 2aa067a..46034bf 100644
--- a/Makefile
+++ b/Makefile
@@ -497,6 +497,7 @@ LIB_H += compat/cygwin.h
 LIB_H += compat/mingw.h
 LIB_H += compat/win32/pthread.h
 LIB_H += compat/win32/syslog.h
+LIB_H += compat/win32/sys/poll.h
 LIB_H += csum-file.h
 LIB_H += decorate.h
 LIB_H += delta.h
@@ -1082,7 +1083,7 @@ ifeq ($(uname_S),Windows)
 	AR = compat/vcbuild/scripts/lib.pl
 	CFLAGS =
 	BASIC_CFLAGS = -nologo -I. -I../zlib -Icompat/vcbuild -Icompat/vcbuild/include -DWIN32 -D_CONSOLE -DHAVE_STRING_H -D_CRT_SECURE_NO_WARNINGS -D_CRT_NONSTDC_NO_DEPRECATE
-	COMPAT_OBJS = compat/msvc.o compat/fnmatch/fnmatch.o compat/winansi.o compat/win32/pthread.o compat/win32/syslog.o
+	COMPAT_OBJS = compat/msvc.o compat/fnmatch/fnmatch.o compat/winansi.o compat/win32/pthread.o compat/win32/syslog.o compat/win32/sys/poll.o
 	COMPAT_CFLAGS = -D__USE_MINGW_ACCESS -DNOGDI -DHAVE_STRING_H -DHAVE_ALLOCA_H -Icompat -Icompat/fnmatch -Icompat/regex -Icompat/fnmatch -Icompat/win32 -DSTRIP_EXTENSION=\".exe\"
 	BASIC_LDFLAGS = -IGNORE:4217 -IGNORE:4049 -NOLOGO -SUBSYSTEM:CONSOLE -NODEFAULTLIB:MSVCRT.lib
 	EXTLIBS = advapi32.lib shell32.lib wininet.lib ws2_32.lib
@@ -1132,7 +1133,8 @@ ifneq (,$(findstring MINGW,$(uname_S)))
 	COMPAT_CFLAGS += -D__USE_MINGW_ACCESS -DNOGDI -Icompat -Icompat/fnmatch -Icompat/win32
 	COMPAT_CFLAGS += -DSTRIP_EXTENSION=\".exe\"
 	COMPAT_OBJS += compat/mingw.o compat/fnmatch/fnmatch.o compat/winansi.o \
-		compat/win32/pthread.o compat/win32/syslog.o
+		compat/win32/pthread.o compat/win32/syslog.o \
+		compat/win32/sys/poll.o
 	EXTLIBS += -lws2_32
 	PTHREAD_LIBS =
 	X = .exe
diff --git a/compat/mingw.c b/compat/mingw.c
index d88c0d0..b780200 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -408,71 +408,6 @@ int pipe(int filedes[2])
 	return 0;
 }
 
-int poll(struct pollfd *ufds, unsigned int nfds, int timeout)
-{
-	int i, pending;
-
-	if (timeout >= 0) {
-		if (nfds == 0) {
-			Sleep(timeout);
-			return 0;
-		}
-		return errno = EINVAL, error("poll timeout not supported");
-	}
-
-	/* When there is only one fd to wait for, then we pretend that
-	 * input is available and let the actual wait happen when the
-	 * caller invokes read().
-	 */
-	if (nfds == 1) {
-		if (!(ufds[0].events & POLLIN))
-			return errno = EINVAL, error("POLLIN not set");
-		ufds[0].revents = POLLIN;
-		return 0;
-	}
-
-repeat:
-	pending = 0;
-	for (i = 0; i < nfds; i++) {
-		DWORD avail = 0;
-		HANDLE h = (HANDLE) _get_osfhandle(ufds[i].fd);
-		if (h == INVALID_HANDLE_VALUE)
-			return -1;	/* errno was set */
-
-		if (!(ufds[i].events & POLLIN))
-			return errno = EINVAL, error("POLLIN not set");
-
-		/* this emulation works only for pipes */
-		if (!PeekNamedPipe(h, NULL, 0, NULL, &avail, NULL)) {
-			int err = GetLastError();
-			if (err == ERROR_BROKEN_PIPE) {
-				ufds[i].revents = POLLHUP;
-				pending++;
-			} else {
-				errno = EINVAL;
-				return error("PeekNamedPipe failed,"
-					" GetLastError: %u", err);
-			}
-		} else if (avail) {
-			ufds[i].revents = POLLIN;
-			pending++;
-		} else
-			ufds[i].revents = 0;
-	}
-	if (!pending) {
-		/* The only times that we spin here is when the process
-		 * that is connected through the pipes is waiting for
-		 * its own input data to become available. But since
-		 * the process (pack-objects) is itself CPU intensive,
-		 * it will happily pick up the time slice that we are
-		 * relinquishing here.
-		 */
-		Sleep(0);
-		goto repeat;
-	}
-	return 0;
-}
-
 struct tm *gmtime_r(const time_t *timep, struct tm *result)
 {
 	/* gmtime() in MSVCRT.DLL is thread-safe, but not reentrant */
diff --git a/compat/mingw.h b/compat/mingw.h
index 51fca2f..99a7467 100644
--- a/compat/mingw.h
+++ b/compat/mingw.h
@@ -59,16 +59,6 @@ struct passwd {
 
 extern char *getpass(const char *prompt);
 
-#ifndef POLLIN
-struct pollfd {
-	int fd;           /* file descriptor */
-	short events;     /* requested events */
-	short revents;    /* returned events */
-};
-#define POLLIN 1
-#define POLLHUP 2
-#endif
-
 typedef void (__cdecl *sig_handler_t)(int);
 struct sigaction {
 	sig_handler_t sa_handler;
@@ -175,7 +165,6 @@ int pipe(int filedes[2]);
 unsigned int sleep (unsigned int seconds);
 int mkstemp(char *template);
 int gettimeofday(struct timeval *tv, void *tz);
-int poll(struct pollfd *ufds, unsigned int nfds, int timeout);
 struct tm *gmtime_r(const time_t *timep, struct tm *result);
 struct tm *localtime_r(const time_t *timep, struct tm *result);
 int getpagesize(void);	/* defined in MinGW's libgcc.a */
diff --git a/compat/win32/sys/poll.c b/compat/win32/sys/poll.c
index 7c52cb6..c1ca0d2 100644
--- a/compat/win32/sys/poll.c
+++ b/compat/win32/sys/poll.c
@@ -24,8 +24,7 @@
 # pragma GCC diagnostic ignored "-Wtype-limits"
 #endif
 
-#include <config.h>
-#include <alloca.h>
+#include <malloc.h>
 
 #include <sys/types.h>
 #include "poll.h"
diff --git a/git-compat-util.h b/git-compat-util.h
index 56dce85..d0a1e48 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -105,9 +105,9 @@
 #include <regex.h>
 #include <utime.h>
 #include <syslog.h>
+#include <sys/poll.h>
 #ifndef __MINGW32__
 #include <sys/wait.h>
-#include <sys/poll.h>
 #include <sys/socket.h>
 #include <sys/ioctl.h>
 #include <termios.h>
-- 
1.7.3.2.161.gd6e00

[PATCH v6 14/16] daemon: use socklen_t

[Erik Faye-Lund]

Windows's accept()-function takes the last argument as an int, but glibc
takes an unsigned int. Use socklen_t to get rid of a warning. This is
basically a revert of 7fa0908, but we have already been depending on
socklen_t existing since June 2006 (commit 5b276ee4). I guess this means
that socklen_t IS defined on OSX after all - at least in recent headers.

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
---
 daemon.c |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/daemon.c b/daemon.c
index 3d8f6e6..5b30a2e 100644
--- a/daemon.c
+++ b/daemon.c
@@ -597,7 +597,7 @@ static struct child {
 	struct sockaddr_storage address;
 } *firstborn;
 
-static void add_child(struct child_process *cld, struct sockaddr *addr, int addrlen)
+static void add_child(struct child_process *cld, struct sockaddr *addr, socklen_t addrlen)
 {
 	struct child *newborn, **cradle;
 
@@ -654,7 +654,7 @@ static void check_dead_children(void)
 }
 
 static char **cld_argv;
-static void handle(int incoming, struct sockaddr *addr, int addrlen)
+static void handle(int incoming, struct sockaddr *addr, socklen_t addrlen)
 {
 	struct child_process cld = { 0 };
 	char addrbuf[300] = "REMOTE_ADDR=", portbuf[300];
@@ -904,7 +904,7 @@ static int service_loop(struct socketlist *socklist)
 		for (i = 0; i < socklist->nr; i++) {
 			if (pfd[i].revents & POLLIN) {
 				struct sockaddr_storage ss;
-				unsigned int sslen = sizeof(ss);
+				socklen_t sslen = sizeof(ss);
 				int incoming = accept(pfd[i].fd, (struct sockaddr *)&ss, &sslen);
 				if (incoming < 0) {
 					switch (errno) {
-- 
1.7.3.2.161.gd6e00

[PATCH v6 15/16] daemon: make --inetd and --detach incompatible

[Erik Faye-Lund]

Since --inetd makes main return with the result of execute() before
daemonize is gets called, these two options are already incompatible.

Document it, and add an error if attempted.

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
---
 Documentation/git-daemon.txt |    3 ++-
 daemon.c                     |    8 ++++----
 2 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/Documentation/git-daemon.txt b/Documentation/git-daemon.txt
index 5054f79..d15cb6a 100644
--- a/Documentation/git-daemon.txt
+++ b/Documentation/git-daemon.txt
@@ -78,7 +78,8 @@ OPTIONS
 
 --inetd::
 	Have the server run as an inetd service. Implies --syslog.
-	Incompatible with --port, --listen, --user and --group options.
+	Incompatible with --detach, --port, --listen, --user and --group
+	options.
 
 --listen=<host_or_ipaddr>::
 	Listen on a specific IP address or hostname.  IP addresses can
diff --git a/daemon.c b/daemon.c
index 5b30a2e..485bec5 100644
--- a/daemon.c
+++ b/daemon.c
@@ -23,10 +23,10 @@ static const char daemon_usage[] =
 "           [--strict-paths] [--base-path=<path>] [--base-path-relaxed]\n"
 "           [--user-path | --user-path=<path>]\n"
 "           [--interpolated-path=<path>]\n"
-"           [--reuseaddr] [--detach] [--pid-file=<file>]\n"
+"           [--reuseaddr] [--pid-file=<file>]\n"
 "           [--(enable|disable|allow-override|forbid-override)=<service>]\n"
 "           [--inetd | [--listen=<host_or_ipaddr>] [--port=<n>]\n"
-"                      [--user=<user> [--group=<group>]]\n"
+"                      [--detach] [--user=<user> [--group=<group>]]\n"
 "           [<directory>...]";
 
 /* List of acceptable pathname prefixes */
@@ -1122,8 +1122,8 @@ int main(int argc, char **argv)
 		/* avoid splitting a message in the middle */
 		setvbuf(stderr, NULL, _IOFBF, 4096);
 
-	if (inetd_mode && (group_name || user_name))
-		die("--user and --group are incompatible with --inetd");
+	if (inetd_mode && (detach || group_name || user_name))
+		die("--detach, --user and --group are incompatible with --inetd");
 
 	if (inetd_mode && (listen_port || (listen_addr.nr > 0)))
 		die("--listen= and --port= are incompatible with --inetd");
-- 
1.7.3.2.161.gd6e00

[PATCH v6 16/16] daemon: opt-out on features that require posix

[Erik Faye-Lund]

Windows does not supply the POSIX-functions fork(), setuuid(), setgid(),
setsid() and initgroups(). Error out if --user or --detach is specified
when if so.

MinGW doesn't have prototypes and headers for inet_ntop and inet_pton,
so include our implementation instead. MSVC does, so avoid doing so
there.

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
---
 Makefile |   14 ++++++---
 daemon.c |   88 +++++++++++++++++++++++++++++++++++++++++++++-----------------
 2 files changed, 73 insertions(+), 29 deletions(-)

diff --git a/Makefile b/Makefile
index 46034bf..53986b1 100644
--- a/Makefile
+++ b/Makefile
@@ -401,6 +401,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
 PROGRAM_OBJS += imap-send.o
 PROGRAM_OBJS += shell.o
@@ -1066,7 +1067,6 @@ ifeq ($(uname_S),Windows)
 	NO_SVN_TESTS = YesPlease
 	NO_PERL_MAKEMAKER = YesPlease
 	RUNTIME_PREFIX = YesPlease
-	NO_POSIX_ONLY_PROGRAMS = YesPlease
 	NO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease
 	NO_NSEC = YesPlease
 	USE_WIN32_MMAP = YesPlease
@@ -1077,6 +1077,7 @@ ifeq ($(uname_S),Windows)
 	NO_CURL = YesPlease
 	NO_PYTHON = YesPlease
 	BLK_SHA1 = YesPlease
+	NO_POSIX_GOODIES = UnfortunatelyYes
 	NATIVE_CRLF = YesPlease
 
 	CC = compat/vcbuild/scripts/clink.pl
@@ -1119,7 +1120,6 @@ ifneq (,$(findstring MINGW,$(uname_S)))
 	NO_SVN_TESTS = YesPlease
 	NO_PERL_MAKEMAKER = YesPlease
 	RUNTIME_PREFIX = YesPlease
-	NO_POSIX_ONLY_PROGRAMS = YesPlease
 	NO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease
 	NO_NSEC = YesPlease
 	USE_WIN32_MMAP = YesPlease
@@ -1130,6 +1130,9 @@ ifneq (,$(findstring MINGW,$(uname_S)))
 	NO_PYTHON = YesPlease
 	BLK_SHA1 = YesPlease
 	ETAGS_TARGET = ETAGS
+	NO_INET_PTON = YesPlease
+	NO_INET_NTOP = YesPlease
+	NO_POSIX_GOODIES = UnfortunatelyYes
 	COMPAT_CFLAGS += -D__USE_MINGW_ACCESS -DNOGDI -Icompat -Icompat/fnmatch -Icompat/win32
 	COMPAT_CFLAGS += -DSTRIP_EXTENSION=\".exe\"
 	COMPAT_OBJS += compat/mingw.o compat/fnmatch/fnmatch.o compat/winansi.o \
@@ -1249,9 +1252,6 @@ ifdef ZLIB_PATH
 endif
 EXTLIBS += -lz
 
-ifndef NO_POSIX_ONLY_PROGRAMS
-	PROGRAM_OBJS += daemon.o
-endif
 ifndef NO_OPENSSL
 	OPENSSL_LIBSSL = -lssl
 	ifdef OPENSSLDIR
@@ -1419,6 +1419,10 @@ ifdef NO_DEFLATE_BOUND
 	BASIC_CFLAGS += -DNO_DEFLATE_BOUND
 endif
 
+ifdef NO_POSIX_GOODIES
+	BASIC_CFLAGS += -DNO_POSIX_GOODIES
+endif
+
 ifdef BLK_SHA1
 	SHA1_HEADER = "block-sha1/sha1.h"
 	LIB_OBJS += block-sha1/sha1.o
diff --git a/daemon.c b/daemon.c
index 485bec5..299201f 100644
--- a/daemon.c
+++ b/daemon.c
@@ -934,6 +934,62 @@ static void sanitize_stdfds(void)
 		close(fd);
 }
 
+#ifdef NO_POSIX_GOODIES
+
+struct credentials;
+
+static void drop_privileges(struct credentials *cred)
+{
+	/* nothing */
+}
+
+static void daemonize(void)
+{
+	die("--detach not supported on this platform");
+}
+
+static struct credentials *prepare_credentials(const char *user_name,
+    const char *group_name)
+{
+	die("--user not supported on this platform");
+}
+
+#else
+
+struct credentials {
+	struct passwd *pass;
+	gid_t gid;
+};
+
+static void drop_privileges(struct credentials *cred)
+{
+	if (cred && (initgroups(cred->pass->pw_name, cred->gid) ||
+	    setgid (cred->gid) || setuid(cred->pass->pw_uid)))
+		die("cannot drop privileges");
+}
+
+static struct credentials *prepare_credentials(const char *user_name,
+    const char *group_name)
+{
+	static struct credentials c;
+
+	c.pass = getpwnam(user_name);
+	if (!c.pass)
+		die("user not found - %s", user_name);
+
+	if (!group_name)
+		c.gid = c.pass->pw_gid;
+	else {
+		struct group *group = getgrnam(group_name);
+		if (!group)
+			die("group not found - %s", group_name);
+
+		c.gid = group->gr_gid;
+	}
+
+	return &c;
+}
+
 static void daemonize(void)
 {
 	switch (fork()) {
@@ -951,6 +1007,7 @@ static void daemonize(void)
 	close(2);
 	sanitize_stdfds();
 }
+#endif
 
 static void store_pid(const char *path)
 {
@@ -961,7 +1018,8 @@ static void store_pid(const char *path)
 		die_errno("failed to write pid file '%s'", path);
 }
 
-static int serve(struct string_list *listen_addr, int listen_port, struct passwd *pass, gid_t gid)
+static int serve(struct string_list *listen_addr, int listen_port,
+    struct credentials *cred)
 {
 	struct socketlist socklist = { NULL, 0, 0 };
 
@@ -970,10 +1028,7 @@ static int serve(struct string_list *listen_addr, int listen_port, struct passwd
 		die("unable to allocate any listen sockets on port %u",
 		    listen_port);
 
-	if (pass && gid &&
-	    (initgroups(pass->pw_name, gid) || setgid (gid) ||
-	     setuid(pass->pw_uid)))
-		die("cannot drop privileges");
+	drop_privileges(cred);
 
 	return service_loop(&socklist);
 }
@@ -985,9 +1040,7 @@ int main(int argc, char **argv)
 	int serve_mode = 0, inetd_mode = 0;
 	const char *pid_file = NULL, *user_name = NULL, *group_name = NULL;
 	int detach = 0;
-	struct passwd *pass = NULL;
-	struct group *group;
-	gid_t gid = 0;
+	struct credentials *cred = NULL;
 	int i;
 
 	git_extract_argv0_path(argv[0]);
@@ -1133,21 +1186,8 @@ int main(int argc, char **argv)
 	if (group_name && !user_name)
 		die("--group supplied without --user");
 
-	if (user_name) {
-		pass = getpwnam(user_name);
-		if (!pass)
-			die("user not found - %s", user_name);
-
-		if (!group_name)
-			gid = pass->pw_gid;
-		else {
-			group = getgrnam(group_name);
-			if (!group)
-				die("group not found - %s", group_name);
-
-			gid = group->gr_gid;
-		}
-	}
+	if (user_name)
+		cred = prepare_credentials(user_name, group_name);
 
 	if (strict_paths && (!ok_paths || !*ok_paths))
 		die("option --strict-paths requires a whitelist");
@@ -1181,5 +1221,5 @@ int main(int argc, char **argv)
 	cld_argv[argc] = "--serve";
 	cld_argv[argc+1] = NULL;
 
-	return serve(&listen_addr, listen_port, pass, gid);
+	return serve(&listen_addr, listen_port, cred);
 }
-- 
1.7.3.2.161.gd6e00

Re: [PATCH v6 00/16] daemon-win32

[Pat Thoyts]

Erik Faye-Lund <kusmabite@gmail.com> writes:

>Here's hopefully the last iteration of this series. The previous version
>only got a single complain about a typo in the subject of patch 14/15, so
>it seems like most controversies have been settled.

I pulled this win32-daemon branch into my msysgit build tree and built
it. I get the following warnings:

    CC daemon.o
daemon.c: In function 'service_loop':
daemon.c:674: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
daemon.c:676: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
daemon.c:681: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
daemon.c:919: note: initialized from here
daemon.c:679: warning: dereferencing pointer 'sin_addr' does break strict-aliasing rules
daemon.c:675: note: initialized from here
daemon.c:691: warning: dereferencing pointer 'sin6_addr' does break strict-aliasing rules
daemon.c:682: note: initialized from here

Otherwise it builds clean. The daemon running on Windows7 seems to be
working fine for both ipv4 and ipv6 connections (I tried both).

However, monitoring the resource usage in procexp it looks like there is
a handle leak. Each 'git ls-remote' over ipv6 is gaining 16 handles that
do not appear to be released. They're all process handles for dead
processes it looks like, so possibly there is a missing waitpid() or
something similar for the 'git daemon -serve' subprocess. Doing this
over ipv4 leaks 2 handles per request.

-- 
Pat Thoyts                            http://www.patthoyts.tk/
PGP fingerprint 2C 6E 98 07 2C 59 C8 97  10 CE 11 E6 04 E0 B9 DD

Re: [PATCH v6 00/16] daemon-win32

[Erik Faye-Lund]

On Wed, Nov 3, 2010 at 10:11 PM, Pat Thoyts
<patthoyts@users.sourceforge.net> wrote:
> Erik Faye-Lund <kusmabite@gmail.com> writes:
>
>>Here's hopefully the last iteration of this series. The previous version
>>only got a single complain about a typo in the subject of patch 14/15, so
>>it seems like most controversies have been settled.
>
> I pulled this win32-daemon branch into my msysgit build tree and built
> it. I get the following warnings:
>
>    CC daemon.o
> daemon.c: In function 'service_loop':
> daemon.c:674: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
> daemon.c:676: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
> daemon.c:681: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
> daemon.c:919: note: initialized from here
> daemon.c:679: warning: dereferencing pointer 'sin_addr' does break strict-aliasing rules
> daemon.c:675: note: initialized from here
> daemon.c:691: warning: dereferencing pointer 'sin6_addr' does break strict-aliasing rules
> daemon.c:682: note: initialized from here
>

Yeah, I'm aware of these. I thought those warnings were already
present in the Linux build, but checking again I see that that's not
the case. Need to investigate.

> Otherwise it builds clean. The daemon running on Windows7 seems to be
> working fine for both ipv4 and ipv6 connections (I tried both).
>
> However, monitoring the resource usage in procexp it looks like there is
> a handle leak. Each 'git ls-remote' over ipv6 is gaining 16 handles that
> do not appear to be released. They're all process handles for dead
> processes it looks like, so possibly there is a missing waitpid() or
> something similar for the 'git daemon -serve' subprocess. Doing this
> over ipv4 leaks 2 handles per request.
>

Ah, thanks. For me it's leaking a variable amount of handles per
ls-remote, but if I apply the following patch it's down to one. Need
to find that one as well...

diff --git a/compat/mingw.c b/compat/mingw.c
index b780200..47e7d26 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -1519,8 +1519,10 @@ pid_t waitpid(pid_t pid, int *status, unsigned options)
 	}

 	if (pid > 0 && options & WNOHANG) {
-		if (WAIT_OBJECT_0 != WaitForSingleObject((HANDLE)pid, 0))
+		if (WAIT_OBJECT_0 != WaitForSingleObject((HANDLE)pid, 0)) {
+			CloseHandle(h);
 			return 0;
+		}
 		options &= ~WNOHANG;
 	}

Re: [PATCH v6 00/16] daemon-win32

[Erik Faye-Lund]

On Wed, Nov 3, 2010 at 11:18 PM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
> Ah, thanks. For me it's leaking a variable amount of handles per
> ls-remote, but if I apply the following patch it's down to one. Need
> to find that one as well...
>
> diff --git a/compat/mingw.c b/compat/mingw.c
> index b780200..47e7d26 100644
> --- a/compat/mingw.c
> +++ b/compat/mingw.c
> @@ -1519,8 +1519,10 @@ pid_t waitpid(pid_t pid, int *status, unsigned options)
>        }
>
>        if (pid > 0 && options & WNOHANG) {
> -               if (WAIT_OBJECT_0 != WaitForSingleObject((HANDLE)pid, 0))
> +               if (WAIT_OBJECT_0 != WaitForSingleObject((HANDLE)pid, 0)) {

AAAND the last one is right here as well:
-		if (WAIT_OBJECT_0 != WaitForSingleObject((HANDLE)pid, 0))
+		if (WAIT_OBJECT_0 != WaitForSingleObject(h, 0)) {

Re: [PATCH v6 00/16] daemon-win32

[Pat Thoyts]

Erik Faye-Lund <kusmabite@gmail.com> writes:

>On Wed, Nov 3, 2010 at 11:18 PM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
>> Ah, thanks. For me it's leaking a variable amount of handles per
>> ls-remote, but if I apply the following patch it's down to one. Need
>> to find that one as well...
>>
>> diff --git a/compat/mingw.c b/compat/mingw.c
>> index b780200..47e7d26 100644
>> --- a/compat/mingw.c
>> +++ b/compat/mingw.c
>> @@ -1519,8 +1519,10 @@ pid_t waitpid(pid_t pid, int *status, unsigned options)
>>        }
>>
>>        if (pid > 0 && options & WNOHANG) {
>> -               if (WAIT_OBJECT_0 != WaitForSingleObject((HANDLE)pid, 0))
>> +               if (WAIT_OBJECT_0 != WaitForSingleObject((HANDLE)pid, 0)) {
>
>AAAND the last one is right here as well:
>-		if (WAIT_OBJECT_0 != WaitForSingleObject((HANDLE)pid, 0))
>+		if (WAIT_OBJECT_0 != WaitForSingleObject(h, 0)) {
>

Applying both of these doesn't fix the handle leak when I test
this. Looking further I believe it is due to the use of a reallocated
array. When you remove a pinfo structure you realloc to the one you
found, potentially freeing items you still require.

Attached is a patch that switches to a linked list for this
instead. Using this I no longer accumulate leaked handles.

----
From e0c05f8f9ed8729648eea92cf654f357fa884e40 Mon Sep 17 00:00:00 2001
From: Pat Thoyts <patthoyts@users.sourceforge.net>
Date: Thu, 4 Nov 2010 00:23:08 +0000
Subject: [PATCH] win32-daemon: fix handle leaks

The use of an array of pinfo structures and the realloc used when cleaning
up a closed child can free structures that are still in use. Use a linked
list instead.
This stops the leaking of handles in the win32-daemon.

Signed-off-by: Pat Thoyts <patthoyts@users.sourceforge.net>
---
 compat/mingw.c |   43 ++++++++++++++++++++++++-------------------
 1 files changed, 24 insertions(+), 19 deletions(-)

diff --git a/compat/mingw.c b/compat/mingw.c
index b780200..29f4036 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -637,11 +637,12 @@ static int env_compare(const void *a, const void *b)
 	return strcasecmp(*ea, *eb);
 }
 
-struct {
+struct pinfo_t {
+	struct pinfo_t *next;
 	pid_t pid;
 	HANDLE proc;
-} *pinfo;
-static int num_pinfo;
+} pinfo_t;
+struct pinfo_t *pinfo = NULL;
 CRITICAL_SECTION pinfo_cs;
 
 static pid_t mingw_spawnve_fd(const char *cmd, const char **argv, char **env,
@@ -746,10 +747,13 @@ static pid_t mingw_spawnve_fd(const char *cmd, const char **argv, char **env,
 	 * Keep the handle in a list for waitpid.
 	 */
 	EnterCriticalSection(&pinfo_cs);
-	num_pinfo++;
-	pinfo = xrealloc(pinfo, sizeof(*pinfo) * num_pinfo);
-	pinfo[num_pinfo - 1].pid = pi.dwProcessId;
-	pinfo[num_pinfo - 1].proc = pi.hProcess;
+	{
+		struct pinfo_t *info = xmalloc(sizeof(struct pinfo_t));
+		info->pid = pi.dwProcessId;
+		info->proc = pi.hProcess;
+		info->next = pinfo;
+		pinfo = info;
+	}
 	LeaveCriticalSection(&pinfo_cs);
 
 	return (pid_t)pi.dwProcessId;
@@ -1519,13 +1523,15 @@ pid_t waitpid(pid_t pid, int *status, unsigned options)
 	}
 
 	if (pid > 0 && options & WNOHANG) {
-		if (WAIT_OBJECT_0 != WaitForSingleObject((HANDLE)pid, 0))
+		if (WAIT_OBJECT_0 != WaitForSingleObject(h, 0)) {
+			CloseHandle(h);
 			return 0;
+		}
 		options &= ~WNOHANG;
 	}
 
 	if (options == 0) {
-		int i;
+		struct pinfo_t **ppinfo;
 		if (WaitForSingleObject(h, INFINITE) != WAIT_OBJECT_0) {
 			CloseHandle(h);
 			return 0;
@@ -1536,17 +1542,16 @@ pid_t waitpid(pid_t pid, int *status, unsigned options)
 
 		EnterCriticalSection(&pinfo_cs);
 
-		for (i = 0; i < num_pinfo; ++i)
-			if (pinfo[i].pid == pid)
+		ppinfo = &pinfo;
+		while (*ppinfo) {
+			struct pinfo_t *info = *ppinfo;
+			if (info->pid == pid) {
+				CloseHandle(info->proc);
+				*ppinfo = info->next;
+				free(info);
 				break;
-
-		if (i < num_pinfo) {
-			CloseHandle(pinfo[i].proc);
-			memmove(pinfo + i, pinfo + i + 1,
-			    sizeof(*pinfo) * (num_pinfo - i - 1));
-			num_pinfo--;
-			pinfo = xrealloc(pinfo,
-			    sizeof(*pinfo) * num_pinfo);
+			}
+			ppinfo = &info->next;
 		}
 
 		LeaveCriticalSection(&pinfo_cs);
-- 
1.7.3.1.210.g3526b.dirty

Re: [PATCH v6 00/16] daemon-win32

[Erik Faye-Lund]

On Thu, Nov 4, 2010 at 1:28 AM, Pat Thoyts
<patthoyts@users.sourceforge.net> wrote:
> Erik Faye-Lund <kusmabite@gmail.com> writes:
>>On Wed, Nov 3, 2010 at 11:18 PM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
>>> diff --git a/compat/mingw.c b/compat/mingw.c
>>> index b780200..47e7d26 100644
>>> --- a/compat/mingw.c
>>> +++ b/compat/mingw.c
>>> @@ -1519,8 +1519,10 @@ pid_t waitpid(pid_t pid, int *status, unsigned options)
>>>        }
>>>
>>>        if (pid > 0 && options & WNOHANG) {
>>> -               if (WAIT_OBJECT_0 != WaitForSingleObject((HANDLE)pid, 0))
>>> +               if (WAIT_OBJECT_0 != WaitForSingleObject((HANDLE)pid, 0)) {
>>
>>AAAND the last one is right here as well:
>>-              if (WAIT_OBJECT_0 != WaitForSingleObject((HANDLE)pid, 0))
>>+              if (WAIT_OBJECT_0 != WaitForSingleObject(h, 0)) {
>>
>
> Applying both of these doesn't fix the handle leak when I test
> this. Looking further I believe it is due to the use of a reallocated
> array. When you remove a pinfo structure you realloc to the one you
> found, potentially freeing items you still require.
>

If this is the reason, then that's a bug. The code TRIES to do the
right thing by memmove'ing the the entro to be deleted to the end.
Looking over the code, I still don't see anyhing obviously wrong.
But...

> Attached is a patch that switches to a linked list for this
> instead. Using this I no longer accumulate leaked handles.
>
> ----
> From e0c05f8f9ed8729648eea92cf654f357fa884e40 Mon Sep 17 00:00:00 2001
> From: Pat Thoyts <patthoyts@users.sourceforge.net>
> Date: Thu, 4 Nov 2010 00:23:08 +0000
> Subject: [PATCH] win32-daemon: fix handle leaks
>
> The use of an array of pinfo structures and the realloc used when cleaning
> up a closed child can free structures that are still in use. Use a linked
> list instead.
> This stops the leaking of handles in the win32-daemon.
>
> Signed-off-by: Pat Thoyts <patthoyts@users.sourceforge.net>
> ---
>  compat/mingw.c |   43 ++++++++++++++++++++++++-------------------
>  1 files changed, 24 insertions(+), 19 deletions(-)
>
> diff --git a/compat/mingw.c b/compat/mingw.c
> index b780200..29f4036 100644
> --- a/compat/mingw.c
> +++ b/compat/mingw.c
> @@ -637,11 +637,12 @@ static int env_compare(const void *a, const void *b)
>        return strcasecmp(*ea, *eb);
>  }
>
> -struct {
> +struct pinfo_t {
> +       struct pinfo_t *next;
>        pid_t pid;
>        HANDLE proc;
> -} *pinfo;
> -static int num_pinfo;
> +} pinfo_t;
> +struct pinfo_t *pinfo = NULL;
>  CRITICAL_SECTION pinfo_cs;
>
>  static pid_t mingw_spawnve_fd(const char *cmd, const char **argv, char **env,
> @@ -746,10 +747,13 @@ static pid_t mingw_spawnve_fd(const char *cmd, const char **argv, char **env,
>         * Keep the handle in a list for waitpid.
>         */
>        EnterCriticalSection(&pinfo_cs);
> -       num_pinfo++;
> -       pinfo = xrealloc(pinfo, sizeof(*pinfo) * num_pinfo);
> -       pinfo[num_pinfo - 1].pid = pi.dwProcessId;
> -       pinfo[num_pinfo - 1].proc = pi.hProcess;
> +       {
> +               struct pinfo_t *info = xmalloc(sizeof(struct pinfo_t));
> +               info->pid = pi.dwProcessId;
> +               info->proc = pi.hProcess;
> +               info->next = pinfo;
> +               pinfo = info;
> +       }
>        LeaveCriticalSection(&pinfo_cs);
>
>        return (pid_t)pi.dwProcessId;
> @@ -1519,13 +1523,15 @@ pid_t waitpid(pid_t pid, int *status, unsigned options)
>        }
>
>        if (pid > 0 && options & WNOHANG) {
> -               if (WAIT_OBJECT_0 != WaitForSingleObject((HANDLE)pid, 0))
> +               if (WAIT_OBJECT_0 != WaitForSingleObject(h, 0)) {
> +                       CloseHandle(h);
>                        return 0;
> +               }
>                options &= ~WNOHANG;
>        }
>
>        if (options == 0) {
> -               int i;
> +               struct pinfo_t **ppinfo;
>                if (WaitForSingleObject(h, INFINITE) != WAIT_OBJECT_0) {
>                        CloseHandle(h);
>                        return 0;
> @@ -1536,17 +1542,16 @@ pid_t waitpid(pid_t pid, int *status, unsigned options)
>
>                EnterCriticalSection(&pinfo_cs);
>
> -               for (i = 0; i < num_pinfo; ++i)
> -                       if (pinfo[i].pid == pid)
> +               ppinfo = &pinfo;
> +               while (*ppinfo) {
> +                       struct pinfo_t *info = *ppinfo;
> +                       if (info->pid == pid) {
> +                               CloseHandle(info->proc);
> +                               *ppinfo = info->next;
> +                               free(info);
>                                break;
> -
> -               if (i < num_pinfo) {
> -                       CloseHandle(pinfo[i].proc);
> -                       memmove(pinfo + i, pinfo + i + 1,
> -                           sizeof(*pinfo) * (num_pinfo - i - 1));
> -                       num_pinfo--;
> -                       pinfo = xrealloc(pinfo,
> -                           sizeof(*pinfo) * num_pinfo);
> +                       }
> +                       ppinfo = &info->next;
>                }
>
>                LeaveCriticalSection(&pinfo_cs);

...yeah, using a linked list is more elegant. Do you mind if I snatch
your code and leave a comment in the commit message?

Re: [PATCH v6 00/16] daemon-win32

[Pat Thoyts]

Erik Faye-Lund <kusmabite@gmail.com> writes:

>On Thu, Nov 4, 2010 at 1:28 AM, Pat Thoyts
><patthoyts@users.sourceforge.net> wrote:
>> Erik Faye-Lund <kusmabite@gmail.com> writes:
>>>On Wed, Nov 3, 2010 at 11:18 PM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
>>>> diff --git a/compat/mingw.c b/compat/mingw.c
>>>> index b780200..47e7d26 100644
>>>> --- a/compat/mingw.c
>>>> +++ b/compat/mingw.c
>>>> @@ -1519,8 +1519,10 @@ pid_t waitpid(pid_t pid, int *status, unsigned options)
>>>>        }
>>>>
>>>>        if (pid > 0 && options & WNOHANG) {
>>>> -               if (WAIT_OBJECT_0 != WaitForSingleObject((HANDLE)pid, 0))
>>>> +               if (WAIT_OBJECT_0 != WaitForSingleObject((HANDLE)pid, 0)) {
>>>
>>>AAAND the last one is right here as well:
>>>-              if (WAIT_OBJECT_0 != WaitForSingleObject((HANDLE)pid, 0))
>>>+              if (WAIT_OBJECT_0 != WaitForSingleObject(h, 0)) {
>>>
>>
>> Applying both of these doesn't fix the handle leak when I test
>> this. Looking further I believe it is due to the use of a reallocated
>> array. When you remove a pinfo structure you realloc to the one you
>> found, potentially freeing items you still require.
>>
>
>If this is the reason, then that's a bug. The code TRIES to do the
>right thing by memmove'ing the the entro to be deleted to the end.
>Looking over the code, I still don't see anyhing obviously wrong.
>But...
>
>> Attached is a patch that switches to a linked list for this
>> instead. Using this I no longer accumulate leaked handles.
>>
>> ----
>> From e0c05f8f9ed8729648eea92cf654f357fa884e40 Mon Sep 17 00:00:00 2001
>> From: Pat Thoyts <patthoyts@users.sourceforge.net>
>> Date: Thu, 4 Nov 2010 00:23:08 +0000
>> Subject: [PATCH] win32-daemon: fix handle leaks
>>
>> The use of an array of pinfo structures and the realloc used when cleaning
>> up a closed child can free structures that are still in use. Use a linked
>> list instead.
>> This stops the leaking of handles in the win32-daemon.
>>
>> Signed-off-by: Pat Thoyts <patthoyts@users.sourceforge.net>
>> ---
>>  compat/mingw.c |   43 ++++++++++++++++++++++++-------------------
>>  1 files changed, 24 insertions(+), 19 deletions(-)
>>
>> diff --git a/compat/mingw.c b/compat/mingw.c
>> index b780200..29f4036 100644
>> --- a/compat/mingw.c
>> +++ b/compat/mingw.c
>> @@ -637,11 +637,12 @@ static int env_compare(const void *a, const void *b)
>>        return strcasecmp(*ea, *eb);
>>  }
>>
>> -struct {
>> +struct pinfo_t {
>> +       struct pinfo_t *next;
>>        pid_t pid;
>>        HANDLE proc;
>> -} *pinfo;
>> -static int num_pinfo;
>> +} pinfo_t;
>> +struct pinfo_t *pinfo = NULL;
>>  CRITICAL_SECTION pinfo_cs;
>>
>>  static pid_t mingw_spawnve_fd(const char *cmd, const char **argv, char **env,
>> @@ -746,10 +747,13 @@ static pid_t mingw_spawnve_fd(const char *cmd, const char **argv, char **env,
>>         * Keep the handle in a list for waitpid.
>>         */
>>        EnterCriticalSection(&pinfo_cs);
>> -       num_pinfo++;
>> -       pinfo = xrealloc(pinfo, sizeof(*pinfo) * num_pinfo);
>> -       pinfo[num_pinfo - 1].pid = pi.dwProcessId;
>> -       pinfo[num_pinfo - 1].proc = pi.hProcess;
>> +       {
>> +               struct pinfo_t *info = xmalloc(sizeof(struct pinfo_t));
>> +               info->pid = pi.dwProcessId;
>> +               info->proc = pi.hProcess;
>> +               info->next = pinfo;
>> +               pinfo = info;
>> +       }
>>        LeaveCriticalSection(&pinfo_cs);
>>
>>        return (pid_t)pi.dwProcessId;
>> @@ -1519,13 +1523,15 @@ pid_t waitpid(pid_t pid, int *status, unsigned options)
>>        }
>>
>>        if (pid > 0 && options & WNOHANG) {
>> -               if (WAIT_OBJECT_0 != WaitForSingleObject((HANDLE)pid, 0))
>> +               if (WAIT_OBJECT_0 != WaitForSingleObject(h, 0)) {
>> +                       CloseHandle(h);
>>                        return 0;
>> +               }
>>                options &= ~WNOHANG;
>>        }
>>
>>        if (options == 0) {
>> -               int i;
>> +               struct pinfo_t **ppinfo;
>>                if (WaitForSingleObject(h, INFINITE) != WAIT_OBJECT_0) {
>>                        CloseHandle(h);
>>                        return 0;
>> @@ -1536,17 +1542,16 @@ pid_t waitpid(pid_t pid, int *status, unsigned options)
>>
>>                EnterCriticalSection(&pinfo_cs);
>>
>> -               for (i = 0; i < num_pinfo; ++i)
>> -                       if (pinfo[i].pid == pid)
>> +               ppinfo = &pinfo;
>> +               while (*ppinfo) {
>> +                       struct pinfo_t *info = *ppinfo;
>> +                       if (info->pid == pid) {
>> +                               CloseHandle(info->proc);
>> +                               *ppinfo = info->next;
>> +                               free(info);
>>                                break;
>> -
>> -               if (i < num_pinfo) {
>> -                       CloseHandle(pinfo[i].proc);
>> -                       memmove(pinfo + i, pinfo + i + 1,
>> -                           sizeof(*pinfo) * (num_pinfo - i - 1));
>> -                       num_pinfo--;
>> -                       pinfo = xrealloc(pinfo,
>> -                           sizeof(*pinfo) * num_pinfo);
>> +                       }
>> +                       ppinfo = &info->next;
>>                }
>>
>>                LeaveCriticalSection(&pinfo_cs);
>
>...yeah, using a linked list is more elegant. Do you mind if I snatch
>your code and leave a comment in the commit message?
>

Sure. When I wrote the comment I'd forgotten about the memmove. So
hmmm. However, I just pulled you win32-daemon branch and applied this
patch on top. Now there are no more warnings so your union fix is
good. As I understand it thats the way to handle this particular
warning. ipv6 and ipv4 both still work fine as well.

With my patch a while true; do git ls-remote; done shows no more leaking
handles or memory. So the use of a list is fixing something. Looking
great now.

-- 
Pat Thoyts                            http://www.patthoyts.tk/
PGP fingerprint 2C 6E 98 07 2C 59 C8 97  10 CE 11 E6 04 E0 B9 DD

Re: [PATCH v6 00/16] daemon-win32

[Erik Faye-Lund]

On Wed, Nov 3, 2010 at 11:18 PM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
> On Wed, Nov 3, 2010 at 10:11 PM, Pat Thoyts
> <patthoyts@users.sourceforge.net> wrote:
>> Erik Faye-Lund <kusmabite@gmail.com> writes:
>>
>>>Here's hopefully the last iteration of this series. The previous version
>>>only got a single complain about a typo in the subject of patch 14/15, so
>>>it seems like most controversies have been settled.
>>
>> I pulled this win32-daemon branch into my msysgit build tree and built
>> it. I get the following warnings:
>>
>>    CC daemon.o
>> daemon.c: In function 'service_loop':
>> daemon.c:674: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
>> daemon.c:676: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
>> daemon.c:681: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
>> daemon.c:919: note: initialized from here
>> daemon.c:679: warning: dereferencing pointer 'sin_addr' does break strict-aliasing rules
>> daemon.c:675: note: initialized from here
>> daemon.c:691: warning: dereferencing pointer 'sin6_addr' does break strict-aliasing rules
>> daemon.c:682: note: initialized from here
>>
>
> Yeah, I'm aware of these. I thought those warnings were already
> present in the Linux build, but checking again I see that that's not
> the case. Need to investigate.
>

OK, it's the patch "daemon: use run-command api for async serving"
that introduce the warning. But looking closer at the patch it doesn't
seem the patch actually introduce the strict-aliasing violation, it's
there already. The patch only seems to change the code enough for GCC
to start realize there's a problem. Unless I'm misunderstanding
something vital, that is.

Anyway, here's a patch that makes it go away, I guess I'll squash it
into the next round.

diff --git a/daemon.c b/daemon.c
index 6eee570..d636446 100644
--- a/daemon.c
+++ b/daemon.c
@@ -1159,14 +1159,13 @@ int main(int argc, char **argv)
 	}

 	if (inetd_mode || serve_mode) {
-		struct sockaddr_storage ss;
-		struct sockaddr *peer = (struct sockaddr *)&ss;
-		socklen_t slen = sizeof(ss);
+		struct sockaddr sa;
+		socklen_t slen = sizeof(sa);

-		if (getpeername(0, peer, &slen))
-			peer = NULL;
-
-		return execute(peer);
+		if (getpeername(0, &sa, &slen))
+			return execute(NULL);
+		else
+			return execute(&sa);
 	}

 	if (detach) {

Re: [PATCH v6 00/16] daemon-win32

[Erik Faye-Lund]

On Wed, Nov 3, 2010 at 11:58 PM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
> On Wed, Nov 3, 2010 at 11:18 PM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
>> On Wed, Nov 3, 2010 at 10:11 PM, Pat Thoyts
>> <patthoyts@users.sourceforge.net> wrote:
>>> Erik Faye-Lund <kusmabite@gmail.com> writes:
>>>
>>>>Here's hopefully the last iteration of this series. The previous version
>>>>only got a single complain about a typo in the subject of patch 14/15, so
>>>>it seems like most controversies have been settled.
>>>
>>> I pulled this win32-daemon branch into my msysgit build tree and built
>>> it. I get the following warnings:
>>>
>>>    CC daemon.o
>>> daemon.c: In function 'service_loop':
>>> daemon.c:674: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
>>> daemon.c:676: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
>>> daemon.c:681: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
>>> daemon.c:919: note: initialized from here
>>> daemon.c:679: warning: dereferencing pointer 'sin_addr' does break strict-aliasing rules
>>> daemon.c:675: note: initialized from here
>>> daemon.c:691: warning: dereferencing pointer 'sin6_addr' does break strict-aliasing rules
>>> daemon.c:682: note: initialized from here
>>>
>>
>> Yeah, I'm aware of these. I thought those warnings were already
>> present in the Linux build, but checking again I see that that's not
>> the case. Need to investigate.
>>
>
> OK, it's the patch "daemon: use run-command api for async serving"
> that introduce the warning. But looking closer at the patch it doesn't
> seem the patch actually introduce the strict-aliasing violation, it's
> there already. The patch only seems to change the code enough for GCC
> to start realize there's a problem. Unless I'm misunderstanding
> something vital, that is.
>
> Anyway, here's a patch that makes it go away, I guess I'll squash it
> into the next round.
>

I also of course need to update "daemon: get remote host address from
root-process" as well, as it introduces a new such code-path (which is
actually the one complained about here). And I guess I should use
sockaddr_in instead of sockaddr.

But my luck stops there. The resulting git-daemon.exe leaves me with a
very bizarre error:

error: unable to make a socket file descriptor: Bad file descriptor
fatal: accept returned: Bad file descriptor

This is triggered by the call to accept() in mingw_accept returning -1.

What is even stranger is that if I change the code at the error-point like this
-				struct sockaddr_in sa;
+				struct sockaddr_in sa[2];
 				socklen_t salen = sizeof(sa);
-				int incoming = accept(pfd[i].fd, (struct sockaddr *)&sa, &salen);
+				int incoming = accept(pfd[i].fd, (struct sockaddr *)sa, &salen);

the error goes away. Similarly, if I change mingw_accept's call to
Winsock's accept(), like this:
-	SOCKET s2 = accept(s1, sa, sz);
+	SOCKET s2 = accept(s1, sa, NULL);

So it seems accept() somehow reacts to the value of the variable
pointed at by sz, which is 16. Strange, huh?

Perhaps it isn't -- the sockaddr_storage change seems to have been
introduced for IPv6 reasons. I'm trying to connect over IPv6, and IPv6
has a new sockaddr_in6 struct. So yeah.

Stuffing all of sockaddr, sockaddr_in and sockaddr_in6 (when built
with IPv6 support) in a union and passing that around instead does
seem to fix the issue completely. I don't find it very elegant, but
some google-searches on the issue seems to reveal that this is the
only way of getting rid of this. Any other suggestions, people?

Re: [PATCH v6 00/16] daemon-win32

[Erik Faye-Lund]

On Thu, Nov 4, 2010 at 1:06 AM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
> On Wed, Nov 3, 2010 at 11:58 PM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
>> On Wed, Nov 3, 2010 at 11:18 PM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
>>> On Wed, Nov 3, 2010 at 10:11 PM, Pat Thoyts
>>> <patthoyts@users.sourceforge.net> wrote:
>>>> Erik Faye-Lund <kusmabite@gmail.com> writes:
>>>>
>>>>>Here's hopefully the last iteration of this series. The previous version
>>>>>only got a single complain about a typo in the subject of patch 14/15, so
>>>>>it seems like most controversies have been settled.
>>>>
>>>> I pulled this win32-daemon branch into my msysgit build tree and built
>>>> it. I get the following warnings:
>>>>
>>>>    CC daemon.o
>>>> daemon.c: In function 'service_loop':
>>>> daemon.c:674: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
>>>> daemon.c:676: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
>>>> daemon.c:681: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
>>>> daemon.c:919: note: initialized from here
>>>> daemon.c:679: warning: dereferencing pointer 'sin_addr' does break strict-aliasing rules
>>>> daemon.c:675: note: initialized from here
>>>> daemon.c:691: warning: dereferencing pointer 'sin6_addr' does break strict-aliasing rules
>>>> daemon.c:682: note: initialized from here
>>>>
>>>
>>> Yeah, I'm aware of these. I thought those warnings were already
>>> present in the Linux build, but checking again I see that that's not
>>> the case. Need to investigate.
>>>
>>
>> OK, it's the patch "daemon: use run-command api for async serving"
>> that introduce the warning. But looking closer at the patch it doesn't
>> seem the patch actually introduce the strict-aliasing violation, it's
>> there already. The patch only seems to change the code enough for GCC
>> to start realize there's a problem. Unless I'm misunderstanding
>> something vital, that is.
>>
>> Anyway, here's a patch that makes it go away, I guess I'll squash it
>> into the next round.
>>
>
> Stuffing all of sockaddr, sockaddr_in and sockaddr_in6 (when built
> with IPv6 support) in a union and passing that around instead does
> seem to fix the issue completely. I don't find it very elegant, but
> some google-searches on the issue seems to reveal that this is the
> only way of getting rid of this. Any other suggestions, people?
>

Just for reference, this is the patch that fixes it. What do you think?

diff --git a/daemon.c b/daemon.c
index 941c095..8162f10 100644
--- a/daemon.c
+++ b/daemon.c
@@ -902,9 +903,15 @@ static int service_loop(struct socketlist *socklist)

 		for (i = 0; i < socklist->nr; i++) {
 			if (pfd[i].revents & POLLIN) {
-				struct sockaddr_storage ss;
+				union {
+					struct sockaddr sa;
+					struct sockaddr_in sai;
+#ifndef NO_IPV6
+					struct sockaddr_in6 sai6;
+#endif
+				} ss;
 				unsigned int sslen = sizeof(ss);
-				int incoming = accept(pfd[i].fd, (struct sockaddr *)&ss, &sslen);
+				int incoming = accept(pfd[i].fd, &ss.sa, &sslen);
 				if (incoming < 0) {
 					switch (errno) {
 					case EAGAIN:
@@ -915,7 +922,7 @@ static int service_loop(struct socketlist *socklist)
 						die_errno("accept returned");
 					}
 				}
-				handle(incoming, (struct sockaddr *)&ss, sslen);
+				handle(incoming, &ss.sa, sslen);
 			}
 		}
 	}

Re: [PATCH v6 00/16] daemon-win32

[Martin Storsjö]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 4697 bytes --]

On Thu, 4 Nov 2010, Erik Faye-Lund wrote:

> On Thu, Nov 4, 2010 at 1:06 AM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
> > On Wed, Nov 3, 2010 at 11:58 PM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
> >> On Wed, Nov 3, 2010 at 11:18 PM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
> >>> On Wed, Nov 3, 2010 at 10:11 PM, Pat Thoyts
> >>> <patthoyts@users.sourceforge.net> wrote:
> >>>> Erik Faye-Lund <kusmabite@gmail.com> writes:
> >>>>
> >>>>>Here's hopefully the last iteration of this series. The previous version
> >>>>>only got a single complain about a typo in the subject of patch 14/15, so
> >>>>>it seems like most controversies have been settled.
> >>>>
> >>>> I pulled this win32-daemon branch into my msysgit build tree and built
> >>>> it. I get the following warnings:
> >>>>
> >>>>    CC daemon.o
> >>>> daemon.c: In function 'service_loop':
> >>>> daemon.c:674: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
> >>>> daemon.c:676: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
> >>>> daemon.c:681: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
> >>>> daemon.c:919: note: initialized from here
> >>>> daemon.c:679: warning: dereferencing pointer 'sin_addr' does break strict-aliasing rules
> >>>> daemon.c:675: note: initialized from here
> >>>> daemon.c:691: warning: dereferencing pointer 'sin6_addr' does break strict-aliasing rules
> >>>> daemon.c:682: note: initialized from here
> >>>>
> >>>
> >>> Yeah, I'm aware of these. I thought those warnings were already
> >>> present in the Linux build, but checking again I see that that's not
> >>> the case. Need to investigate.
> >>>
> >>
> >> OK, it's the patch "daemon: use run-command api for async serving"
> >> that introduce the warning. But looking closer at the patch it doesn't
> >> seem the patch actually introduce the strict-aliasing violation, it's
> >> there already. The patch only seems to change the code enough for GCC
> >> to start realize there's a problem. Unless I'm misunderstanding
> >> something vital, that is.
> >>
> >> Anyway, here's a patch that makes it go away, I guess I'll squash it
> >> into the next round.
> >>
> >
> > Stuffing all of sockaddr, sockaddr_in and sockaddr_in6 (when built
> > with IPv6 support) in a union and passing that around instead does
> > seem to fix the issue completely. I don't find it very elegant, but
> > some google-searches on the issue seems to reveal that this is the
> > only way of getting rid of this. Any other suggestions, people?
> >
> 
> Just for reference, this is the patch that fixes it. What do you think?
> 
> diff --git a/daemon.c b/daemon.c
> index 941c095..8162f10 100644
> --- a/daemon.c
> +++ b/daemon.c
> @@ -902,9 +903,15 @@ static int service_loop(struct socketlist *socklist)
> 
>  		for (i = 0; i < socklist->nr; i++) {
>  			if (pfd[i].revents & POLLIN) {
> -				struct sockaddr_storage ss;
> +				union {
> +					struct sockaddr sa;
> +					struct sockaddr_in sai;
> +#ifndef NO_IPV6
> +					struct sockaddr_in6 sai6;
> +#endif
> +				} ss;
>  				unsigned int sslen = sizeof(ss);
> -				int incoming = accept(pfd[i].fd, (struct sockaddr *)&ss, &sslen);
> +				int incoming = accept(pfd[i].fd, &ss.sa, &sslen);
>  				if (incoming < 0) {
>  					switch (errno) {
>  					case EAGAIN:
> @@ -915,7 +922,7 @@ static int service_loop(struct socketlist *socklist)
>  						die_errno("accept returned");
>  					}
>  				}
> -				handle(incoming, (struct sockaddr *)&ss, sslen);
> +				handle(incoming, &ss.sa, sslen);
>  			}
>  		}
>  	}

As you say yourself, it's not elegant at all - sockaddr_storage is 
intended to be just that, an struct large enough to fit all the sockaddrs 
you'll encounter on this platform, with all fields aligned in the same way 
as all the other sockaddr structs. You're supposed to be able to cast the 
sockaddr struct pointers like currently is done, although I'm not familiar 
with the strict aliasing stuff well enough to know if anything else would 
be required somewhere.

I didn't see any of these hacks in the v7 patchset - did the warning go 
away by itself there?

FWIW, the actual warning itself isn't directly related to any of the code 
worked on here, gcc just happens to realize it after some of these 
changes. I'm able to trigger the same warnings on the current master, by 
simply doing this change:

diff --git a/daemon.c b/daemon.c
index 5783e24..467cea2 100644
--- a/daemon.c
+++ b/daemon.c
@@ -670,7 +670,6 @@ static void handle(int incoming, struct sockaddr 
*addr, int 
        dup2(incoming, 1);
        close(incoming);
 
-       exit(execute(addr));
 }
 
 static void child_handler(int signo)


// Martin

Re: [PATCH v6 00/16] daemon-win32

[Erik Faye-Lund]

On Thu, Nov 4, 2010 at 9:58 AM, Martin Storsjö <martin@martin.st> wrote:
> On Thu, 4 Nov 2010, Erik Faye-Lund wrote:
>
>> On Thu, Nov 4, 2010 at 1:06 AM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
>> > On Wed, Nov 3, 2010 at 11:58 PM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
>> >> On Wed, Nov 3, 2010 at 11:18 PM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
>> >>> On Wed, Nov 3, 2010 at 10:11 PM, Pat Thoyts
>> >>> <patthoyts@users.sourceforge.net> wrote:
>> >>>> Erik Faye-Lund <kusmabite@gmail.com> writes:
>> >>>>
>> >>>>>Here's hopefully the last iteration of this series. The previous version
>> >>>>>only got a single complain about a typo in the subject of patch 14/15, so
>> >>>>>it seems like most controversies have been settled.
>> >>>>
>> >>>> I pulled this win32-daemon branch into my msysgit build tree and built
>> >>>> it. I get the following warnings:
>> >>>>
>> >>>>    CC daemon.o
>> >>>> daemon.c: In function 'service_loop':
>> >>>> daemon.c:674: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
>> >>>> daemon.c:676: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
>> >>>> daemon.c:681: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
>> >>>> daemon.c:919: note: initialized from here
>> >>>> daemon.c:679: warning: dereferencing pointer 'sin_addr' does break strict-aliasing rules
>> >>>> daemon.c:675: note: initialized from here
>> >>>> daemon.c:691: warning: dereferencing pointer 'sin6_addr' does break strict-aliasing rules
>> >>>> daemon.c:682: note: initialized from here
>> >>>>
>> >>>
>> >>> Yeah, I'm aware of these. I thought those warnings were already
>> >>> present in the Linux build, but checking again I see that that's not
>> >>> the case. Need to investigate.
>> >>>
>> >>
>> >> OK, it's the patch "daemon: use run-command api for async serving"
>> >> that introduce the warning. But looking closer at the patch it doesn't
>> >> seem the patch actually introduce the strict-aliasing violation, it's
>> >> there already. The patch only seems to change the code enough for GCC
>> >> to start realize there's a problem. Unless I'm misunderstanding
>> >> something vital, that is.
>> >>
>> >> Anyway, here's a patch that makes it go away, I guess I'll squash it
>> >> into the next round.
>> >>
>> >
>> > Stuffing all of sockaddr, sockaddr_in and sockaddr_in6 (when built
>> > with IPv6 support) in a union and passing that around instead does
>> > seem to fix the issue completely. I don't find it very elegant, but
>> > some google-searches on the issue seems to reveal that this is the
>> > only way of getting rid of this. Any other suggestions, people?
>> >
>>
>> Just for reference, this is the patch that fixes it. What do you think?
>>
>> diff --git a/daemon.c b/daemon.c
>> index 941c095..8162f10 100644
>> --- a/daemon.c
>> +++ b/daemon.c
>> @@ -902,9 +903,15 @@ static int service_loop(struct socketlist *socklist)
>>
>>               for (i = 0; i < socklist->nr; i++) {
>>                       if (pfd[i].revents & POLLIN) {
>> -                             struct sockaddr_storage ss;
>> +                             union {
>> +                                     struct sockaddr sa;
>> +                                     struct sockaddr_in sai;
>> +#ifndef NO_IPV6
>> +                                     struct sockaddr_in6 sai6;
>> +#endif
>> +                             } ss;
>>                               unsigned int sslen = sizeof(ss);
>> -                             int incoming = accept(pfd[i].fd, (struct sockaddr *)&ss, &sslen);
>> +                             int incoming = accept(pfd[i].fd, &ss.sa, &sslen);
>>                               if (incoming < 0) {
>>                                       switch (errno) {
>>                                       case EAGAIN:
>> @@ -915,7 +922,7 @@ static int service_loop(struct socketlist *socklist)
>>                                               die_errno("accept returned");
>>                                       }
>>                               }
>> -                             handle(incoming, (struct sockaddr *)&ss, sslen);
>> +                             handle(incoming, &ss.sa, sslen);
>>                       }
>>               }
>>       }
>
> As you say yourself, it's not elegant at all - sockaddr_storage is
> intended to be just that, an struct large enough to fit all the sockaddrs
> you'll encounter on this platform, with all fields aligned in the same way
> as all the other sockaddr structs. You're supposed to be able to cast the
> sockaddr struct pointers like currently is done, although I'm not familiar
> with the strict aliasing stuff well enough to know if anything else would
> be required somewhere.
>

Strict aliasing isn't exactly about the structure being large enough
or not, it's only being able to access a particular piece of memory
through one type only (unless specificly marked with "union").
sockaddr_storage is an attempt at fixing the storage-problem without
addressing the type punning problem, which doesn't help us much.

> I didn't see any of these hacks in the v7 patchset - did the warning go
> away by itself there?
>

The strict aliasing problem was (as I described earlier, and you
pointed out later in your email) already present in the code. All the
patch did, was modify the code enough to make GCC realize it.

Since the code is removed by a later patch ("daemon: get remote host
address from root-process"), I figured adding a union just to remove
it was just noisy. So instead I changed the code enough for the
warning to go away again. It turned out that it was the assignment of
NULL to "peer" that triggered the warning, so I made two calls to
execute() instead, one that pass NULL and one that pass "peer".

Then in "daemon: get remote host address from root-process" which
causes a similar strict-aliasing issue (this time I'm the one who
introduced it, since the handle() code-path doesn't derefence "addr"),
I fixed it with a union.

> FWIW, the actual warning itself isn't directly related to any of the code
> worked on here, gcc just happens to realize it after some of these
> changes. I'm able to trigger the same warnings on the current master, by
> simply doing this change:
>
> diff --git a/daemon.c b/daemon.c
> index 5783e24..467cea2 100644
> --- a/daemon.c
> +++ b/daemon.c
> @@ -670,7 +670,6 @@ static void handle(int incoming, struct sockaddr
> *addr, int
>        dup2(incoming, 1);
>        close(incoming);
>
> -       exit(execute(addr));
>  }
>
>  static void child_handler(int signo)
>
>
> // Martin

Yeah, I already figured that one out, but thanks for making it clearer. :)

So a nice end-result of v7 is that we're good with strict aliasing,
which means that it's not safe(er) to compile git-daemon on GCC with
-O3.

Re: [PATCH v6 00/16] daemon-win32

[Martin Storsjö]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 4960 bytes --]

On Thu, 4 Nov 2010, Erik Faye-Lund wrote:

> On Thu, Nov 4, 2010 at 9:58 AM, Martin Storsjö <martin@martin.st> wrote:
> > On Thu, 4 Nov 2010, Erik Faye-Lund wrote:
> >
> >> On Thu, Nov 4, 2010 at 1:06 AM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
> >> >
> >> > Stuffing all of sockaddr, sockaddr_in and sockaddr_in6 (when built
> >> > with IPv6 support) in a union and passing that around instead does
> >> > seem to fix the issue completely. I don't find it very elegant, but
> >> > some google-searches on the issue seems to reveal that this is the
> >> > only way of getting rid of this. Any other suggestions, people?
> >> >
> >>
> >> Just for reference, this is the patch that fixes it. What do you think?
> >>
> >> diff --git a/daemon.c b/daemon.c
> >> index 941c095..8162f10 100644
> >> --- a/daemon.c
> >> +++ b/daemon.c
> >> @@ -902,9 +903,15 @@ static int service_loop(struct socketlist *socklist)
> >>
> >>               for (i = 0; i < socklist->nr; i++) {
> >>                       if (pfd[i].revents & POLLIN) {
> >> -                             struct sockaddr_storage ss;
> >> +                             union {
> >> +                                     struct sockaddr sa;
> >> +                                     struct sockaddr_in sai;
> >> +#ifndef NO_IPV6
> >> +                                     struct sockaddr_in6 sai6;
> >> +#endif
> >> +                             } ss;
> >>                               unsigned int sslen = sizeof(ss);
> >> -                             int incoming = accept(pfd[i].fd, (struct sockaddr *)&ss, &sslen);
> >> +                             int incoming = accept(pfd[i].fd, &ss.sa, &sslen);
> >>                               if (incoming < 0) {
> >>                                       switch (errno) {
> >>                                       case EAGAIN:
> >> @@ -915,7 +922,7 @@ static int service_loop(struct socketlist *socklist)
> >>                                               die_errno("accept returned");
> >>                                       }
> >>                               }
> >> -                             handle(incoming, (struct sockaddr *)&ss, sslen);
> >> +                             handle(incoming, &ss.sa, sslen);
> >>                       }
> >>               }
> >>       }
> >
> > As you say yourself, it's not elegant at all - sockaddr_storage is
> > intended to be just that, an struct large enough to fit all the sockaddrs
> > you'll encounter on this platform, with all fields aligned in the same way
> > as all the other sockaddr structs. You're supposed to be able to cast the
> > sockaddr struct pointers like currently is done, although I'm not familiar
> > with the strict aliasing stuff well enough to know if anything else would
> > be required somewhere.
> >
> 
> Strict aliasing isn't exactly about the structure being large enough
> or not, it's only being able to access a particular piece of memory
> through one type only (unless specificly marked with "union").
> sockaddr_storage is an attempt at fixing the storage-problem without
> addressing the type punning problem, which doesn't help us much.

Given this, does that mean that all code that uses sockaddr_storage 
directly without a union, and casting pointers to this struct into 
sockaddr, sockaddr_in and sockaddr_in6 is incorrect with regards to strict 
aliasing?

That is, even this example from RFC 2553 is faulty:

      struct sockaddr_storage __ss;
      struct sockaddr_in6 *sin6;
      sin6 = (struct sockaddr_in6 *) &__ss;

> > I didn't see any of these hacks in the v7 patchset - did the warning go
> > away by itself there?
> >
> 
> The strict aliasing problem was (as I described earlier, and you
> pointed out later in your email) already present in the code. All the
> patch did, was modify the code enough to make GCC realize it.
> 
> Since the code is removed by a later patch ("daemon: get remote host
> address from root-process"), I figured adding a union just to remove
> it was just noisy. So instead I changed the code enough for the
> warning to go away again. It turned out that it was the assignment of
> NULL to "peer" that triggered the warning, so I made two calls to
> execute() instead, one that pass NULL and one that pass "peer".
> 
> Then in "daemon: get remote host address from root-process" which
> causes a similar strict-aliasing issue (this time I'm the one who
> introduced it, since the handle() code-path doesn't derefence "addr"),
> I fixed it with a union.

Ah, I didn't see that one. Ok, that explains it.

The union solution is correct although not pretty, while the one changing 
it to a bare sockaddr was wrong, which was what caught my attention. :-)

> So a nice end-result of v7 is that we're good with strict aliasing,
> which means that it's not safe(er) to compile git-daemon on GCC with
> -O3.

Actually, couldn't there still be similar strict aliasing violations left 
that GCC hasn't realized yet?

// Martin

Re: [PATCH v6 00/16] daemon-win32

[Erik Faye-Lund]

On Thu, Nov 4, 2010 at 10:35 AM, Martin Storsjö <martin@martin.st> wrote:
> On Thu, 4 Nov 2010, Erik Faye-Lund wrote:
>
>> On Thu, Nov 4, 2010 at 9:58 AM, Martin Storsjö <martin@martin.st> wrote:
>> > On Thu, 4 Nov 2010, Erik Faye-Lund wrote:
>> >
>> >> On Thu, Nov 4, 2010 at 1:06 AM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
>> >> >
>> >> > Stuffing all of sockaddr, sockaddr_in and sockaddr_in6 (when built
>> >> > with IPv6 support) in a union and passing that around instead does
>> >> > seem to fix the issue completely. I don't find it very elegant, but
>> >> > some google-searches on the issue seems to reveal that this is the
>> >> > only way of getting rid of this. Any other suggestions, people?
>> >> >
>> >>
>> >> Just for reference, this is the patch that fixes it. What do you think?
>> >>
>> >> diff --git a/daemon.c b/daemon.c
>> >> index 941c095..8162f10 100644
>> >> --- a/daemon.c
>> >> +++ b/daemon.c
>> >> @@ -902,9 +903,15 @@ static int service_loop(struct socketlist *socklist)
>> >>
>> >>               for (i = 0; i < socklist->nr; i++) {
>> >>                       if (pfd[i].revents & POLLIN) {
>> >> -                             struct sockaddr_storage ss;
>> >> +                             union {
>> >> +                                     struct sockaddr sa;
>> >> +                                     struct sockaddr_in sai;
>> >> +#ifndef NO_IPV6
>> >> +                                     struct sockaddr_in6 sai6;
>> >> +#endif
>> >> +                             } ss;
>> >>                               unsigned int sslen = sizeof(ss);
>> >> -                             int incoming = accept(pfd[i].fd, (struct sockaddr *)&ss, &sslen);
>> >> +                             int incoming = accept(pfd[i].fd, &ss.sa, &sslen);
>> >>                               if (incoming < 0) {
>> >>                                       switch (errno) {
>> >>                                       case EAGAIN:
>> >> @@ -915,7 +922,7 @@ static int service_loop(struct socketlist *socklist)
>> >>                                               die_errno("accept returned");
>> >>                                       }
>> >>                               }
>> >> -                             handle(incoming, (struct sockaddr *)&ss, sslen);
>> >> +                             handle(incoming, &ss.sa, sslen);
>> >>                       }
>> >>               }
>> >>       }
>> >
>> > As you say yourself, it's not elegant at all - sockaddr_storage is
>> > intended to be just that, an struct large enough to fit all the sockaddrs
>> > you'll encounter on this platform, with all fields aligned in the same way
>> > as all the other sockaddr structs. You're supposed to be able to cast the
>> > sockaddr struct pointers like currently is done, although I'm not familiar
>> > with the strict aliasing stuff well enough to know if anything else would
>> > be required somewhere.
>> >
>>
>> Strict aliasing isn't exactly about the structure being large enough
>> or not, it's only being able to access a particular piece of memory
>> through one type only (unless specificly marked with "union").
>> sockaddr_storage is an attempt at fixing the storage-problem without
>> addressing the type punning problem, which doesn't help us much.
>
> Given this, does that mean that all code that uses sockaddr_storage
> directly without a union, and casting pointers to this struct into
> sockaddr, sockaddr_in and sockaddr_in6 is incorrect with regards to strict
> aliasing?
>
> That is, even this example from RFC 2553 is faulty:
>
>      struct sockaddr_storage __ss;
>      struct sockaddr_in6 *sin6;
>      sin6 = (struct sockaddr_in6 *) &__ss;
>

Yes, at least with C99. Section 6.5, paragraph 7 of the C99 specification says:

"An object shall have its stored value accessed only by an lvalue
expression that has one of
the following types:

- a type compatible with the effective type of the object,
- a qualified version of a type compatible with the effective type of the object,
- a type that is the signed or unsigned type corresponding to the
effective type of the
object,
- a type that is the signed or unsigned type corresponding to a
qualified version of the
effective type of the object,
- an aggregate or union type that includes one of the aforementioned
types among its
members (including, recursively,amember of a subaggregate or contained
union), or
- a character type."

A type punned pointer does not meet any of those requirements.

For pre-C99 I don't have any reference other than Wikpedia, which
indicates that it's illegal in the ISO version of the standard (which
I assume must be C90):

"To enable such optimizations in a predictable manner, the ISO
standard for the C programming language (including its newer C99
edition, see section 6.5, paragraph 7) specifies that it is illegal
(with some exceptions) for pointers of different types to reference
the same memory location."

ref: http://en.wikipedia.org/wiki/Aliasing_(computing)

But the code above is incorrect in another sense: it's declaring a
symbol with a name starting with double underscore, something that is
reserved for the compiler. So I don't think I would trust the person
who wrote it to care much about standards compliance.

Re: [PATCH v6 00/16] daemon-win32

[Martin Storsjö]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2824 bytes --]

On Wed, 3 Nov 2010, Erik Faye-Lund wrote:

> On Wed, Nov 3, 2010 at 11:18 PM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
> > On Wed, Nov 3, 2010 at 10:11 PM, Pat Thoyts
> > <patthoyts@users.sourceforge.net> wrote:
> >> Erik Faye-Lund <kusmabite@gmail.com> writes:
> >>
> >>>Here's hopefully the last iteration of this series. The previous version
> >>>only got a single complain about a typo in the subject of patch 14/15, so
> >>>it seems like most controversies have been settled.
> >>
> >> I pulled this win32-daemon branch into my msysgit build tree and built
> >> it. I get the following warnings:
> >>
> >>    CC daemon.o
> >> daemon.c: In function 'service_loop':
> >> daemon.c:674: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
> >> daemon.c:676: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
> >> daemon.c:681: warning: dereferencing pointer 'ss.124' does break strict-aliasing rules
> >> daemon.c:919: note: initialized from here
> >> daemon.c:679: warning: dereferencing pointer 'sin_addr' does break strict-aliasing rules
> >> daemon.c:675: note: initialized from here
> >> daemon.c:691: warning: dereferencing pointer 'sin6_addr' does break strict-aliasing rules
> >> daemon.c:682: note: initialized from here
> >>
> >
> > Yeah, I'm aware of these. I thought those warnings were already
> > present in the Linux build, but checking again I see that that's not
> > the case. Need to investigate.
> >
> 
> OK, it's the patch "daemon: use run-command api for async serving"
> that introduce the warning. But looking closer at the patch it doesn't
> seem the patch actually introduce the strict-aliasing violation, it's
> there already. The patch only seems to change the code enough for GCC
> to start realize there's a problem. Unless I'm misunderstanding
> something vital, that is.
> 
> Anyway, here's a patch that makes it go away, I guess I'll squash it
> into the next round.
> 
> diff --git a/daemon.c b/daemon.c
> index 6eee570..d636446 100644
> --- a/daemon.c
> +++ b/daemon.c
> @@ -1159,14 +1159,13 @@ int main(int argc, char **argv)
>  	}
> 
>  	if (inetd_mode || serve_mode) {
> -		struct sockaddr_storage ss;
> -		struct sockaddr *peer = (struct sockaddr *)&ss;
> -		socklen_t slen = sizeof(ss);
> +		struct sockaddr sa;
> +		socklen_t slen = sizeof(sa);
> 
> -		if (getpeername(0, peer, &slen))
> -			peer = NULL;
> -
> -		return execute(peer);
> +		if (getpeername(0, &sa, &slen))
> +			return execute(NULL);
> +		else
> +			return execute(&sa);
>  	}
> 
>  	if (detach) {

As you noticed later yourself, this is not right. You can get any kind of 
sockaddr back from getpeername. I'm not sure if any spec mandates that a 
"normal" sockaddr_in should fit into the base sockaddr, or if that just 
happens by coincidence or sheer luck.

// Martin