[PATCH 0/1] mingw: fix isatty() after dup2()

[PATCH 0/1] mingw: fix isatty() after dup2()

[Johannes Schindelin via GitGitGadget]

This patch has been looong in the waiting (well over a year) for being sent
to the Git mailing list; it is a fixup for a change of isatty() on Windows
that had long made it into git.git's master branch.

Johannes Schindelin (1):
  mingw: fix isatty() after dup2()

 compat/mingw.h   |  3 +++
 compat/winansi.c | 12 ++++++++++++
 2 files changed, 15 insertions(+)


base-commit: 4ede3d42dfb57f9a41ac96a1f216c62eb7566cc2
Published-As: https://github.com/gitgitgadget/git/releases/tags/pr-61%2Fdscho%2Fmingw-isatty-and-dup2-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-61/dscho/mingw-isatty-and-dup2-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/61
-- 
gitgitgadget

[PATCH 1/1] mingw: fix isatty() after dup2()

[Johannes Schindelin via GitGitGadget]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

Since a9b8a09c3c30 (mingw: replace isatty() hack, 2016-12-22), we handle
isatty() by special-casing the stdin/stdout/stderr file descriptors,
caching the return value. However, we missed the case where dup2()
overrides the respective file descriptor.

That poses a problem e.g. where the `show` builtin asks for a pager very
early, the `setup_pager()` function sets the pager depending on the
return value of `isatty()` and then redirects stdout. Subsequently,
`cmd_log_init_finish()` calls `setup_pager()` *again*. What should
happen now is that `isatty()` reports that stdout is *not* a TTY and
consequently stdout should be left alone.

Let's override dup2() to handle this appropriately.

This fixes https://github.com/git-for-windows/git/issues/1077

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 compat/mingw.h   |  3 +++
 compat/winansi.c | 12 ++++++++++++
 2 files changed, 15 insertions(+)

diff --git a/compat/mingw.h b/compat/mingw.h
index f31dcff2b..b04556ce0 100644
--- a/compat/mingw.h
+++ b/compat/mingw.h
@@ -390,6 +390,9 @@ int mingw_raise(int sig);
 int winansi_isatty(int fd);
 #define isatty winansi_isatty
 
+int winansi_dup2(int oldfd, int newfd);
+#define dup2 winansi_dup2
+
 void winansi_init(void);
 HANDLE winansi_get_osfhandle(int fd);
 
diff --git a/compat/winansi.c b/compat/winansi.c
index a11a0f16d..f4f08237f 100644
--- a/compat/winansi.c
+++ b/compat/winansi.c
@@ -474,6 +474,18 @@ static void die_lasterr(const char *fmt, ...)
 	va_end(params);
 }
 
+#undef dup2
+int winansi_dup2(int oldfd, int newfd)
+{
+	int ret = dup2(oldfd, newfd);
+
+	if (!ret && newfd >= 0 && newfd <= 2)
+		fd_is_interactive[newfd] = oldfd < 0 || oldfd > 2 ?
+			0 : fd_is_interactive[oldfd];
+
+	return ret;
+}
+
 static HANDLE duplicate_handle(HANDLE hnd)
 {
 	HANDLE hresult, hproc = GetCurrentProcess();
-- 
gitgitgadget